mirror of
https://git.datalinker.icu/vllm-project/vllm.git
synced 2026-03-16 15:37:13 +08:00
[Doc] mention fpdb for multiprocess breakpoints (#24452)
Signed-off-by: Mickael Seznec <mickael@mistral.ai>
This commit is contained in:
parent
0cdd213641
commit
ed16d0f26f
@ -40,6 +40,34 @@ If other strategies don't solve the problem, it's likely that the vLLM instance
|
||||
- `export NCCL_DEBUG=TRACE` to turn on more logging for NCCL.
|
||||
- `export VLLM_TRACE_FUNCTION=1` to record all function calls for inspection in the log files to tell which function crashes or hangs. Do not use this flag unless absolutely needed for debugging, it will cause significant delays in startup time.
|
||||
|
||||
## Breakpoints
|
||||
|
||||
Setting normal `pdb` breakpoints may not work in vLLM's codebase if they are executed in a subprocess. You will experience something like:
|
||||
|
||||
``` text
|
||||
File "/usr/local/uv/cpython-3.12.11-linux-x86_64-gnu/lib/python3.12/bdb.py", line 100, in trace_dispatch
|
||||
return self.dispatch_line(frame)
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
File "/usr/local/uv/cpython-3.12.11-linux-x86_64-gnu/lib/python3.12/bdb.py", line 125, in dispatch_line
|
||||
if self.quitting: raise BdbQuit
|
||||
^^^^^^^^^^^^^
|
||||
bdb.BdbQuit
|
||||
```
|
||||
|
||||
One solution is using [forked-pdb](https://github.com/Lightning-AI/forked-pdb). Install with `pip install fpdb` and set a breakpoint with something like:
|
||||
|
||||
``` python
|
||||
__import__('fpdb').ForkedPdb().set_trace()
|
||||
```
|
||||
|
||||
Another option is to disable multiprocessing entirely, with the `VLLM_ENABLE_V1_MULTIPROCESSING` environment variable.
|
||||
This keeps the scheduler in the same process, so you can use stock `pdb` breakpoints:
|
||||
|
||||
``` python
|
||||
import os
|
||||
os.environ["VLLM_ENABLE_V1_MULTIPROCESSING"] = "0"
|
||||
```
|
||||
|
||||
## Incorrect network setup
|
||||
|
||||
The vLLM instance cannot get the correct IP address if you have a complicated network config. You can find a log such as `DEBUG 06-10 21:32:17 parallel_state.py:88] world_size=8 rank=0 local_rank=0 distributed_init_method=tcp://xxx.xxx.xxx.xxx:54641 backend=nccl` and the IP address should be the correct one.
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user