mirror of
https://git.datalinker.icu/vllm-project/vllm.git
synced 2026-06-08 03:29:06 +08:00
[Misc] Inline Molmo requirements (#17190)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
This commit is contained in:
parent
df5c879527
commit
9d98ab5ec6
@ -1112,7 +1112,33 @@ To use `TIGER-Lab/Mantis-8B-siglip-llama3`, you have to pass `--hf_overrides '{"
|
|||||||
:::
|
:::
|
||||||
|
|
||||||
:::{warning}
|
:::{warning}
|
||||||
For improved output quality of `AllenAI/Molmo-7B-D-0924` (especially in object localization tasks), we recommend using the pinned dependency versions listed in <gh-file:requirements/molmo.txt> (including `vllm==0.7.0`). These versions match the environment that achieved consistent results on both A10 and L40 GPUs.
|
The output quality of `AllenAI/Molmo-7B-D-0924` (especially in object localization tasks) has deteriorated in recent updates.
|
||||||
|
|
||||||
|
For the best results, we recommend using the following dependency versions (tested on A10 and L40):
|
||||||
|
|
||||||
|
```text
|
||||||
|
# Core vLLM-compatible dependencies with Molmo accuracy setup (tested on L40)
|
||||||
|
torch==2.5.1
|
||||||
|
torchvision==0.20.1
|
||||||
|
transformers==4.48.1
|
||||||
|
tokenizers==0.21.0
|
||||||
|
tiktoken==0.7.0
|
||||||
|
vllm==0.7.0
|
||||||
|
|
||||||
|
# Optional but recommended for improved performance and stability
|
||||||
|
triton==3.1.0
|
||||||
|
xformers==0.0.28.post3
|
||||||
|
uvloop==0.21.0
|
||||||
|
protobuf==5.29.3
|
||||||
|
openai==1.60.2
|
||||||
|
opencv-python-headless==4.11.0.86
|
||||||
|
pillow==10.4.0
|
||||||
|
|
||||||
|
# Installed FlashAttention (for float16 only)
|
||||||
|
flash-attn>=2.5.6 # Not used in float32, but should be documented
|
||||||
|
```
|
||||||
|
|
||||||
|
**Note:** Make sure you understand the security implications of using outdated packages.
|
||||||
:::
|
:::
|
||||||
|
|
||||||
:::{note}
|
:::{note}
|
||||||
|
|||||||
@ -1,20 +0,0 @@
|
|||||||
# Core vLLM-compatible dependencies with Molmo accuracy setup (tested on L40)
|
|
||||||
torch==2.5.1
|
|
||||||
torchvision==0.20.1
|
|
||||||
transformers==4.48.1
|
|
||||||
tokenizers==0.21.0
|
|
||||||
tiktoken==0.7.0
|
|
||||||
vllm==0.7.0
|
|
||||||
|
|
||||||
# Optional but recommended for improved performance and stability
|
|
||||||
triton==3.1.0
|
|
||||||
xformers==0.0.28.post3
|
|
||||||
uvloop==0.21.0
|
|
||||||
protobuf==5.29.3
|
|
||||||
openai==1.60.2
|
|
||||||
opencv-python-headless==4.11.0.86
|
|
||||||
pillow==10.4.0
|
|
||||||
|
|
||||||
# Installed FlashAttention (for float16 only)
|
|
||||||
flash-attn>=2.5.6 # Not used in float32, but should be documented
|
|
||||||
|
|
||||||
Loading…
x
Reference in New Issue
Block a user