xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2025-12-26 05:01:49 +08:00
Code Issues Packages Projects Releases Wiki Activity

[Bugfix][Harmless] Fix float16 dtype for model_is_embedding (#7566)

Browse Source
This commit is contained in:
Michael Goin 2024-08-15 21:26:19 -04:00 committed by GitHub
parent 21313e09e3
commit 9c8e2d1161
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
vllm/entrypoints/openai/api_server.py
View File

@ -66,7 +66,7 @@ def model_is_embedding(model_name: str, trust_remote_code: bool) -> bool:
tokenizer_mode="auto",
trust_remote_code=trust_remote_code,
seed=0,
dtype="float16").embedding_mode
dtype="auto").embedding_mode
@asynccontextmanager