xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2025-12-09 17:45:19 +08:00
Code Issues Packages Projects Releases Wiki Activity
7,081 Commits 155 Branches 89 Tags
Commit Graph

7081 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Pavani Majety
8058c91108
[HOT-FIX] Add kv_sharing_target_layer_name argument to cutlass_mla backend (#19374) 
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
 2025-06-09 19:00:07 -04:00
Siyuan Liu
7d44c469fe
[TPU]Fix KV cache sharing tests (#19371) 2025-06-09 18:38:15 -04:00
liusiqian-tal
31f58be96a
[Frontend] Make TIMEOUT_KEEP_ALIVE configurable through env var (#18472) 
Signed-off-by: liusiqian <liusiqian@tal.com>
 2025-06-09 21:41:21 +00:00
Kyle Sayers
ebb2f383b8
[Quantization] Bump compressed-tensors version (#19295) 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
 2025-06-09 14:33:15 -07:00
22quinn
c1c7dbbeeb
[Bugfix][Core] Prevent token lengths exceeding max_model_len in V0 (#19348) 
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
 2025-06-09 23:01:29 +08:00
Varun Sundar Rabindranath
5cf2daea9a
[Misc] Fixes and Optimizations for DeepEP + DeepGEMM combination. (#19298) 
Signed-off-by: Varun <vsundarr@redhat.com>
Co-authored-by: Varun <vsundarr@redhat.com>
 2025-06-09 10:50:39 -04:00
Isotr0py
b8089195b4
[v1] Add fp32 support to v1 engine through flex attn (#19319) 
Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-06-09 22:10:44 +08:00
Yinghai Lu
770e5dcdb8
[full_graph] Fix query_start_loc padding (#19321) 
Signed-off-by: Yinghai Lu <yinghai@thinkingmachines.ai>
 2025-06-09 21:32:56 +08:00
Michael Yao
c57c9415b1
[Docs] Fix a bullet list in usage/security.md (#19358) 
Signed-off-by: windsonsea <haifeng.yao@daocloud.io>
 2025-06-09 13:28:51 +00:00
Lu Fang
01810f9236
[CI] Introduce rules for llama auto-label (#19323) 
Signed-off-by: Lu Fang <lufang@fb.com>
 2025-06-09 20:05:42 +08:00
Conroy Cheers
59abbd84f9
[Fix] Allow kernel compilation for CUDA capability 8.7 (#19328) 
Signed-off-by: Conroy Cheers <conroy@corncheese.org>
 2025-06-09 02:57:23 -07:00
Jee Jee Li
95a6568b5c
[CI/Build] Fix LoRA test (#19350) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-06-09 09:52:10 +00:00
Se7en
0eca5eacd0
[Doc] Fix description in the Automatic Prefix Caching design doc (#19333) 
Signed-off-by: cr7258 <chengzw258@163.com>
 2025-06-09 17:30:02 +08:00
Reid
12e5829221
[doc] improve ci doc (#19307) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
 2025-06-09 07:26:12 +00:00
Richard Zou
3a4d417707
[Misc] Cleanup compilation tests (#19343) 
Signed-off-by: rzou <zou3519@gmail.com>
 2025-06-09 15:05:44 +08:00
Kseniya Parkhamchuk
8335667c22
[Frontend] Remove unreachable code from llm.py (#19288) 
Signed-off-by: KsuParkhamchuk <k.parkhamchuk@gmail.com>
 2025-06-09 10:22:10 +08:00
Isotr0py
e1c4380d4c
[Misc] Add documentation update reminder to PR template (#19289) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-06-09 10:20:53 +08:00
Cyrus Leung
e31ae3de36
[Deprecation] Remove inputs arg fallback in Engine classes (#18799) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-06-09 10:19:56 +08:00
wang.yuqi
2ffb9b6e07
[Bugfix] model_max_length should consider max_model_len in tokenizer_config (#19201) 2025-06-08 07:17:53 -07:00
jennyyyyzhen
cda10fa3e2
[Multi Modal] Add an env var for message queue max chunk bytes (#19242) 
Signed-off-by: yZhen <yZhen@fb.com>
Co-authored-by: yZhen <yZhen@fb.com>
 2025-06-08 21:39:12 +08:00
Dipika Sikka
c123bc33f9
[Quantization] Add compressed-tensors NVFP4 support (#18312) 2025-06-08 09:05:55 -04:00
Akash kaothalkar
b9a1791e2c
[Hardware][POWER] Add IBM POWER11 Support to CPU Extension Detection (#19082) 
Signed-off-by: Akash Kaothalkar <akash.kaothalkar@ibm.com>
Co-authored-by: Akash Kaothalkar <akash.kaothalkar@ibm.com>
 2025-06-08 09:17:14 +00:00
Xu Wenqing
989dcee981
Add H20-3e fused MoE kernel tuning configs for Qwen3-235B-A22B (#19315) 
Signed-off-by: Xu Wenqing <xuwq1993@qq.com>
 2025-06-08 16:07:02 +08:00
Richard Zou
3d64d366e0
[Misc] Change tests/compile to use VLLM_V1 by default (#19302) 
Signed-off-by: rzou <zou3519@gmail.com>
 2025-06-08 16:06:48 +08:00
Richard Zou
eaa2e51088
[Bugfix] Re-enable use_cudagraph in vLLM v1 (#19299) 
Signed-off-by: Richard Zou <zou3519@gmail.com>
 2025-06-08 08:56:12 +08:00
Chauncey
d77f7fb871
[Bugfix]: Fix TypeError: 'float' object cannot be interpreted as an integer (#19283) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-06-08 08:16:31 +08:00
Luka Govedič
2d8476e465
[BugFix][V1] Fix memory profiling bug (#18974) 
Signed-off-by: luka <luka@neuralmagic.com>
 2025-06-07 10:34:51 -07:00
pramenku
88be823d57
[AMD] Update compatible packaging version (#19309) 
Signed-off-by: pramkuma <Pramendra.Kumar@amd.com>
 2025-06-07 20:55:09 +08:00
Lifans
4e4f63ad45
[Nit][Benchmark]Fix example in benchmark_serving_structured_output.py (#19311) 
Signed-off-by: Lifan Shen <lifans@meta.com>
 2025-06-07 18:25:38 +08:00
Isotr0py
d2f0e7e615
[CI/Build] Improve Llama GGUF test robustness (#19287) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-06-07 17:23:28 +08:00
Reid
122cdca5f6
[Misc] refactor context extension (#19246) 
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
 2025-06-07 05:13:21 +00:00
Driss Guessous
cf02f9b283
Add FlexAttention to V1 (#16078) 
Signed-off-by: drisspg <drisspguessous@gmail.com>
 2025-06-06 21:58:55 -07:00
Aaruni Aggarwal
c4296b1a27
[CI][PowerPC] Use a more appropriate way to select testcase in tests/models/language/pooling/test_embedding.py (#19253) 
Signed-off-by: Aaruni Aggarwal <aaruniagg@gmail.com>
 2025-06-07 11:52:52 +08:00
QiliangCui
66c508b137
[TPU][Test] Add script to run benchmark on TPU for buildkite (#19039) 
Signed-off-by: Qiliang Cui <derrhein@gmail.com>
 2025-06-06 20:10:24 -07:00
ElizaWszola
84166fee97
[Kernel] Integrate CUTLASS MoE kernel with PPLX (#18762) 
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-06-06 18:26:11 -07:00
Lu Fang
6e0cd10f72
[Easy][Test] Simplify test_function_tool_use with multiple parametrizes (#19269) 
Signed-off-by: Lu Fang <lufang@fb.com>
 2025-06-07 09:19:09 +08:00
Alexei-V-Ivanov-AMD
e010688f50
[Build][ROCm] Update Dockerfile.rocm (#19296) 
Signed-off-by: Alexei V. Ivanov <alexei.ivanov@amd.com>
 2025-06-06 19:35:16 -04:00
Chenyaaang
441b65d8c7
[Misc][Tools][Benchmark] Fix and improve auto tune script (#19163) 
Signed-off-by: Chenyaaang <chenyangli@google.com>
 2025-06-06 23:31:19 +00:00
Nick Hill
46ecc57973
[BugFix] Fix tpu_model_runner block_id concatenation (#19228) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-06-06 16:28:17 -07:00
Nicolò Lucchesi
b6a3a9f76d
[Core] Fix abrupt request abort (#18485) 
Signed-off-by: nicklucche <nlucches@redhat.com>
Signed-off-by: Nick Hill <nhill@redhat.com>

Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-06-06 16:27:59 -07:00
Adolfo Victoria
ca27f0f9c1
[Bugfix][Core] Update cancellation logic in generate() to handle Generator exits (#19225) 
Co-authored-by: Adolfo Victoria <adovi@meta.com>
 2025-06-06 20:17:54 +00:00
Nick Hill
aad30bd306
[BugFix] Fix MultiConnector test after HMA changes (#19291) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-06-06 20:16:24 +00:00
Nishidha
94ecee6282
Fixed ppc build when it runs on non-RHEL based linux distros (#18422) 
Signed-off-by: Nishidha Panpaliya <nishidha.panpaliya@partner.ibm.com>
Signed-off-by: Md. Shafi Hussain <Md.Shafi.Hussain@ibm.com>
Signed-off-by: npanpaliya <nishidha.panpaliya@partner.ibm.com>
Co-authored-by: Md. Shafi Hussain <Md.Shafi.Hussain@ibm.com>
 2025-06-06 11:54:26 -07:00
Yu Guo
8267f9916f
improve logits bias (#19041) 2025-06-06 19:59:25 +08:00
jmswen
7353492a47
[Core] Raise when non-multi-instance DP clients target a DP rank (#19227) 
Signed-off-by: Jon Swenson <jmswen@gmail.com>
 2025-06-06 19:03:01 +08:00
Jee Jee Li
7661e92ef8
[Model] Optimize nemotron_h implementation (#19249) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-06-06 10:05:14 +00:00
Siqi Yan
f168b85725
Unit Test for run_dp_sharded_vision_model (#19103) 
Signed-off-by: Siqi Yan <siqi@meta.com>
Co-authored-by: Siqi Yan <siqi@meta.com>
 2025-06-06 16:24:02 +08:00
Richard Zou
da511d54d8
Fix CompilationConfig repr (#19091) 
Signed-off-by: rzou <zou3519@gmail.com>
 2025-06-06 16:23:35 +08:00
Nick Hill
65c69444b1
[Docs] Improve V1 KVConnector interface documentation (#19172) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-06-06 16:22:45 +08:00
Dipika Sikka
94870359cd
[Quantization] Bump compressed-tensors version; update NVFP4A16 test model (#19224) 
Signed-off-by: Dipika Sikka <dipikasikka1@gmail.com>
 2025-06-06 01:21:54 -07:00