xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2025-12-10 10:55:45 +08:00
Code Issues Packages Projects Releases Wiki Activity
3,469 Commits 155 Branches 89 Tags
Commit Graph

3 Commits

Author SHA1 Message Date
Kuntai Du
81ede99ca4
[Core] Deprecating block manager v1 and make block manager v2 default (#8704) 
Removing the block manager v1. This is the initial piece of prefix-caching-centric design. In order to achieve prefix-caching-centric design, we need to simplify the code path so that we only use v2 block manager (which has much higher performance on prefix caching).
 2024-10-17 11:38:15 -05:00
Nick Hill
2110557dab
[BugFix] Fix cuda graph for MLPSpeculator (#5875) 
Co-authored-by: Abhinav Goyal <abhinav.goyal@flipkart.com>
 2024-06-27 04:12:10 +00:00
Joshua Rosenkranz
b12518d3cf
[Model] MLPSpeculator speculative decoding support (#4947) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>

Co-authored-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: Davis Wertheimer <Davis.Wertheimer@ibm.com>
 2024-06-20 20:23:12 -04:00