Cody Yu b2c620230a

[Spec Decode] Introduce DraftModelRunner (#5799)

2024-06-28 09:17:51 -07:00

..

__init__.py

[Bugfix] Add __init__.py files for vllm/core/block/ and vllm/spec_decode/ (#3798)

2024-04-02 12:35:31 -07:00

batch_expansion.py

[Model] MLPSpeculator speculative decoding support (#4947)

2024-06-20 20:23:12 -04:00

draft_model_runner.py

[Spec Decode] Introduce DraftModelRunner (#5799)

2024-06-28 09:17:51 -07:00

interfaces.py

[Model] MLPSpeculator speculative decoding support (#4947)

2024-06-20 20:23:12 -04:00

metrics.py

[Speculative decoding 7/9] Speculative decoding end-to-end correctness tests. (#3951)

2024-04-23 08:02:36 +00:00

mlp_speculator_worker.py

[Core] Refactor Worker and ModelRunner to consolidate control plane communication (#5408)

2024-06-25 20:30:03 -07:00

multi_step_worker.py

[Spec Decode] Introduce DraftModelRunner (#5799)

2024-06-28 09:17:51 -07:00

ngram_worker.py

[mypy] Enable type checking for test directory (#5017)

2024-06-15 04:45:31 +00:00

proposer_worker_base.py

[Speculative Decoding] Support draft model on different tensor-parallel size than target model (#5414)

2024-06-25 09:56:06 +00:00

smaller_tp_proposer_worker.py

[Speculative Decoding] Support draft model on different tensor-parallel size than target model (#5414)

2024-06-25 09:56:06 +00:00

spec_decode_worker.py

[Spec Decode] Introduce DraftModelRunner (#5799)

2024-06-28 09:17:51 -07:00

top1_proposer.py

[Model] MLPSpeculator speculative decoding support (#4947)

2024-06-20 20:23:12 -04:00

util.py

[Model] MLPSpeculator speculative decoding support (#4947)

2024-06-20 20:23:12 -04:00