xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2026-05-25 05:24:25 +08:00
Code Issues Packages Projects Releases Wiki Activity

[Doc] Added unmentioned required option "method" in the usage of EAGLE-3 based models (#21737)

Browse Source 
Signed-off-by: Dilute-l <dilu2333@163.com>
Co-authored-by: Dilute-l <dilu2333@163.com>
This commit is contained in:
Hongsheng Liu 2025-08-12 15:14:51 +08:00 committed by GitHub
parent 4fbd8bb597
commit 3a7e3bbdd2
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 4 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
docs/features/spec_decode.md
View File

@ -203,6 +203,7 @@ an [EAGLE (Extrapolation Algorithm for Greater Language-model Efficiency)](https
"model": "yuhuili/EAGLE-LLaMA3-Instruct-8B", "model": "yuhuili/EAGLE-LLaMA3-Instruct-8B",
"draft_tensor_parallel_size": 1, "draft_tensor_parallel_size": 1,
"num_speculative_tokens": 2, "num_speculative_tokens": 2,
"method": "eagle",
}, },
) )
@ -231,6 +232,9 @@ A few important things to consider when using the EAGLE based draft models:
reported in the reference implementation [here](https://github.com/SafeAILab/EAGLE). This issue is under reported in the reference implementation [here](https://github.com/SafeAILab/EAGLE). This issue is under
investigation and tracked here: <gh-issue:9565>. investigation and tracked here: <gh-issue:9565>.
4. When using EAGLE-3 based draft model, option "method" must be set to "eagle3".
That is, to specify `"method": "eagle3"` in `speculative_config`.
A variety of EAGLE draft models are available on the Hugging Face hub: A variety of EAGLE draft models are available on the Hugging Face hub:
| Base Model | EAGLE on Hugging Face | # EAGLE Parameters | | Base Model | EAGLE on Hugging Face | # EAGLE Parameters |