xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2025-12-28 08:05:16 +08:00
Code Issues Packages Projects Releases Wiki Activity
9,542 Commits 155 Branches 89 Tags
Commit Graph

491 Commits

Author SHA1 Message Date
Zhuohan Li
6c47f6bfa4
[Core] Remove tokenizer group in vLLM (#24078) 
Signed-off-by: Zhuohan Li <zhuohan123@gmail.com>
 2025-09-17 08:42:59 +00:00
Nick Hill
eeb135eb87
[Core] Use CpuGpuBuffer for block table tensors (#24795) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-09-16 19:18:06 -07:00
Sage Moore
567939953b
[Core/DBO][1/N] Add Dual-Batch Overlap mechanism to VLLM (#23693) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Sage Moore <sage@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2025-09-16 12:21:48 -04:00
Woosuk Kwon
759ef49b15
Remove V0 Encoder-Decoder Support (#24907) 
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
 2025-09-15 21:17:14 -07:00
Mickaël Seznec
45bfa49cb8
[Tests] fix initialization of kv hash in tests (#24273) 
Signed-off-by: Mickael Seznec <mickael@mistral.ai>
 2025-09-15 21:48:27 +00:00
Harry Mellor
c4afdb69cc
Move MultiModalConfig from config/__init__.py to config/multimodal.py (#24659) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-09-15 17:43:16 +00:00
Ning Xie
bc0f6059a2
[UT] enhance free kv cache block queue popleft_n (#24220) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2025-09-15 10:04:37 +00:00
Ning Xie
3f3313981c
[kv cache] update num_free_blocks in the end (#24228) 
Signed-off-by: Andy Xie <andy.xning@gmail.com>
 2025-09-15 05:15:12 +00:00
Chen Zhang
8e5cdcda4e
[Hybrid Allocator] Support Pipeline Parallel (#23974) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-09-14 15:55:17 -07:00
Wenlong Wang
cfa3234a5b
[CI][Spec Decode] Adjust threshold for flaky ngram spec decoding test again (#24771) 
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
 2025-09-13 15:45:11 +08:00
Nick Hill
4fdd6f5cbf
[Core] Support async scheduling with uniproc executor (#24219) 
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: Ronald1995 <ronaldautomobile@163.com>
Co-authored-by: Ronald1995 <ronaldautomobile@163.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2025-09-12 16:34:28 -07:00
Matthew Bonanni
5fe643fc26
Add FLASHINFER_MLA to backend selector test (#24753) 
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
 2025-09-12 22:30:07 +00:00
Matthew Bonanni
7ba32aa60b
[Attention][FlashInfer] Enable FP8 FlashInfer (TRTLLM) MLA decode (#24705) 
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
 2025-09-12 15:45:53 -06:00
Flora Feng
0377802c20
[Multimodal] Remove legacy multimodal fields in favor of MultiModalFeatureSpec (#24548) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2025-09-12 21:42:23 +08:00
Chen Zhang
561a0baee0
[CI] Fix flaky test v1/worker/test_gpu_model_runner.py::test_kv_cache_stride_order (#24640) 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
 2025-09-12 07:49:09 +00:00
RichardoMu
40b6c9122b
[V1] feat:add engine v1 tracing (#20372) 
Signed-off-by: Mu Huai <tianbowen.tbw@antgroup.com>
Signed-off-by: Ye Zhang <zhysishu@gmail.com>
Signed-off-by: RichardoMu <44485717+RichardoMrMu@users.noreply.github.com>
Signed-off-by: simon-mo <simon.mo@hey.com>
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
Co-authored-by: Mu Huai <tianbowen.tbw@antgroup.com>
Co-authored-by: Ye Zhang <zhysishu@gmail.com>
Co-authored-by: Benjamin Bartels <benjamin@bartels.dev>
Co-authored-by: simon-mo <simon.mo@hey.com>
Co-authored-by: 瑜琮 <ly186375@antfin.com>
Co-authored-by: Aaron Pham <contact@aarnphm.xyz>
Co-authored-by: 22quinn <33176974+22quinn@users.noreply.github.com>
 2025-09-11 17:10:39 -07:00
Russell Bryant
37e8182bfe
[v1] Add Whisper model support (encoder-decoder) (#21088) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: NickLucche <nlucches@redhat.com>
 2025-09-10 13:53:35 -07:00
co63oc
3144d90217
fix some typos (#24167) 
Signed-off-by: co63oc <co63oc@users.noreply.github.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
 2025-09-10 06:21:23 -07:00
Harry Mellor
f36355abfd
Move LoadConfig from config/__init__.py to config/load.py (#24566) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-09-10 06:14:18 -07:00
Chenheli Hua
009d689b0c
[Core] Simplify and unify mm uuid handling & auto-generated mm hash overrides processing. (#24271) 
Signed-off-by: Chenheli Hua <huachenheli@outlook.com>
 2025-09-09 21:36:09 -07:00
Wenlong Wang
53b42f4102
[BugFix][Spec Decode] Fix out-of-range index triggered by eagle3; re-enable test for LlamaForCausalLMEagle3 (#24392) 
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
 2025-09-09 21:24:23 -07:00
Nick Hill
83dd28aae4
[CI] Adjust threshold for flaky ngram spec decoding test (#24528) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-09-09 21:07:33 -07:00
Flora Feng
15cb047e25
Extend renderer with embedding support and integrate completion endpoint (#24405) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2025-09-10 01:46:46 +08:00
Zebing Lin
82dfb12e52
[Core] Use sha256 bytes instead of BlockHash to reduce GC overhead (#23673) 
Signed-off-by: linzebing <linzebing1995@gmail.com>
 2025-09-08 21:34:37 -07:00
Matthew Bonanni
620db1fc58
[Attention] FlashAttention MLA cudagraph support (#23958) 
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2025-09-08 22:05:26 +00:00
22quinn
3a3e91bdfe
[CI/Build] Disable flaky test_structured_output tests (#24404) 
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
 2025-09-08 02:51:59 +00:00
Xingyu Liu
b3d7e3c845
[Sampler] Support returning all prompt logprobs (#23868) 
Signed-off-by: Xingyu Liu <charlotteliu12x@gmail.com>
Co-authored-by: 22quinn <33176974+22quinn@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-09-07 19:34:31 -07:00
Woosuk Kwon
105d3d62ef
[TPU] Remove TopKTopPSampler dependency for TPU sampler (#24391) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-09-07 01:12:36 -07:00
Aaron Pham
e67597545b
[CI][Fix] deterministic seed for flaky CI runs on structured outputs (#24380) 
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
 2025-09-07 11:10:40 +08:00
Roger Wang
b121ca22ad
[CI] Disable flaky structured output test from CI (#24366) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2025-09-06 13:31:56 +00:00
Seiji Eicher
60b755cbcb
[Misc] Have AsyncLLM custom_stat_loggers extend default logger list (#20952) 
Signed-off-by: Seiji Eicher <seiji@anyscale.com>
Signed-off-by: Seiji Eicher <58963096+eicherseiji@users.noreply.github.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-09-04 14:25:30 -07:00
Kebe
8f423e5f43
[Feature][Response API] Add streaming support for non-harmony (#23741) 
Signed-off-by: Kebe <mail@kebe7jun.com>
 2025-09-04 17:49:06 +08:00
Lucas Wilkinson
402759d472
[Attention] FlashAttn MLA (#14258) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
Co-authored-by: Matthew Bonanni <mbonanni001@gmail.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-09-04 02:47:59 -07:00
dsinghvi
70549c1245
[CI/Build] Serve images used by multimodal tests through local HTTP Server (#23907) 
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-09-03 16:13:11 +08:00
Didier Durand
d7e1e59972
[Doc]: fix typos in Python comments (#24093) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
 2025-09-02 21:05:45 -07:00
co63oc
1bd007f234
fix some typos (#24071) 
Signed-off-by: co63oc <co63oc@users.noreply.github.com>
 2025-09-02 20:44:50 -07:00
afeldman-nm
136d853e65
[V1] Wrapper which plumbs request-level logits processors into vLLM batch-level logits processing (#23656) 
Signed-off-by: Andrew Feldman <afeldman@redhat.com>
 2025-09-03 02:52:51 +00:00
Didier Durand
fad73be1a5
[Doc]: fix typos in Python comments (#24077) 
Signed-off-by: Didier Durand <durand.didier@gmail.com>
 2025-09-02 02:38:55 -07:00
Roger Wang
749be00a98
[Core][Multimodal] Allow passing multi_modal_uuids as multimodal identifiers. (#23394) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2025-08-30 18:01:22 -07:00
Ning Xie
5490d633ce
[UT] fix unify_kv_cache_configs when kv cache config needs sort (#23843) 2025-08-30 11:22:14 +00:00
Yong Hoon Shin
8c3e199998
Revert gemma3n fast prefill changes (#23897) 
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
 2025-08-29 12:16:57 -07:00
Nick Hill
d90d8eb674
[BugFix] Async scheduling and PP compatibility with DP (#23770) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-08-29 08:17:27 -07:00
Flora Feng
69f46359dd
[Multimodal] Consolidate mm inputs into MultiModalFeatureSpec (#23779) 
Signed-off-by: sfeng33 <4florafeng@gmail.com>
 2025-08-29 18:36:57 +08:00
Yong Hoon Shin
cb293f6a79
[V1] Enable prefill optimization for Gemma3n (#22628) 
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
 2025-08-28 14:54:30 -07:00
Hanchenli
5da4f5d857
[Bugfix] Fix for V1 priority scheduling crashes at preemption (#23713) 
Signed-off-by: Hanchenli <lihanc2002@gmail.com>
 2025-08-28 00:44:52 +00:00
Nick Hill
3ce8285d6d
[LogitsProcs] Deduplicate built-in LP implementation logic (#23362) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-08-27 23:11:33 +08:00
Roger Wang
b5d34af328
[Bugfix] Fix scheduling when repeated images in one request (#23544) 
Signed-off-by: Roger Wang <hey@rogerw.me>
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Roger Wang <hey@rogerw.me>
Co-authored-by: knlnguyen1802 <knlnguyen1802@gmail.com>
 2025-08-26 09:46:28 +00:00
Driss Guessous
e0329ed4b4
Updates to Flex + VLLm integration (#21416) 
Signed-off-by: drisspg <drisspguessous@gmail.com>
 2025-08-25 09:32:42 -04:00
Ayush Satyam
5c4b6e66fe
[Attention] Unify mamba and attention backend selection (#23171) 
Signed-off-by: Ayush Satyam <ayushsatyam146@gmail.com>
 2025-08-25 09:09:36 +00:00
Chenguang Zheng
d765cf01fe
[Core][Multimodal] Track encode cache entries by mm_hash and enable embedding sharing between requests (#22711) 
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com>
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: knlnguyen1802 <knlnguyen1802@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2025-08-25 00:41:17 -07:00