Ekagra Ranjan 418d2f8bfb

[V1][Spec Decode] Share input embedding of target model with EAGLE draft model to free ~1GB for llama 3 model (#17326) ...

Co-authored-by: root <root@ekagra-8xh100.us-east5-a.c.serving-efficiency-poc.internal>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

2025-05-14 12:31:46 -07:00

..

__init__.py

[V1][BugFix] Add __init__.py to v1/spec_decode/ (#13359)

2025-02-16 09:39:08 -08:00

eagle.py

[V1][Spec Decode] Share input embedding of target model with EAGLE draft model to free ~1GB for llama 3 model (#17326)

2025-05-14 12:31:46 -07:00

metadata.py

[V1][Spec Decode] Optimize Rejection Sampler with Triton Kernels (#14930)

2025-03-18 14:31:54 -07:00

metrics.py

[V1][Spec Decoding] Log accumulated metrics after system goes idle (#17913)

2025-05-09 18:23:07 -07:00

ngram_proposer.py

[V1][Spec Decode] Handle draft tokens beyond max_model_len (#16087)

2025-04-21 12:38:50 -07:00

utils.py

[V1][Spec Decode] Enable spec decode for top-p & top-k sampling (#15063)

2025-03-24 17:16:46 -07:00