xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2026-01-02 10:20:54 +08:00
Code Issues Packages Projects Releases Wiki Activity

Fix passing SpeculativeConfig from the CLI (#22652)

Browse Source 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
This commit is contained in:
Harry Mellor 2025-08-12 06:13:32 +01:00 committed by GitHub
parent ad344ef552
commit 4fbd8bb597
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 4 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
vllm/engine/arg_utils.py
View File

@ -826,6 +826,10 @@ class EngineArgs:
title="VllmConfig",
description=VllmConfig.__doc__,
)
# We construct SpeculativeConfig using fields from other configs in
# create_engine_config. So we set the type to a JSON string here to
# delay the Pydantic validation that comes with SpeculativeConfig.
vllm_kwargs["speculative_config"]["type"] = optional_type(json.loads)
vllm_group.add_argument("--speculative-config",
**vllm_kwargs["speculative_config"])
vllm_group.add_argument("--kv-transfer-config",