xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2025-12-21 05:15:33 +08:00
Code Issues Packages Projects Releases Wiki Activity
8,002 Commits 155 Branches 89 Tags
Commit Graph

8002 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
QiliangCui
07d80d7b0e
[TPU][TEST] HF_HUB_DISABLE_XET=1 the test 3. (#21539) 
Signed-off-by: Qiliang Cui <derrhein@gmail.com>
 2025-07-24 15:33:04 -07:00
weiliang
2dd72d23d9
update flashinfer to v0.2.9rc1 (#21485) 
Signed-off-by: Weiliang Liu <weiliangl@nvidia.com>
 2025-07-24 14:06:11 -07:00
Simon Mo
a6c7fb8cff
[Docs] Add Expert Parallelism Initial Documentation (#21373) 
Signed-off-by: simon-mo <simon.mo@hey.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-07-24 12:36:06 -07:00
Ricardo Decal
a7272c23d0
[Docs][minor] Fix broken gh-file link in distributed serving docs (#21543) 
Signed-off-by: Ricardo Decal <rdecal@anyscale.com>
 2025-07-24 10:36:56 -07:00
Juncheng Gu
6066284914
[P/D] Support CPU Transfer in NixlConnector (#18293) 
Signed-off-by: Juncheng Gu <juncgu@gmail.com>
Signed-off-by: Richard Liu <ricliu@google.com>
Co-authored-by: Richard Liu <39319471+richardsliu@users.noreply.github.com>
Co-authored-by: Richard Liu <ricliu@google.com>
 2025-07-24 17:58:42 +01:00
Rui Qiao
1e9ea8e69d
[P/D] Move FakeNixlWrapper to test dir (#21328) 
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
 2025-07-24 08:53:45 -07:00
Chaojun Zhang
d9f9a3fd96
[XPU] Conditionally import CUDA-specific passes to avoid import errors on xpu platform (#21036) 
Signed-off-by: chzhang <chaojun.zhang@intel.com>
 2025-07-24 23:23:36 +08:00
Shu Wang
1b25f1fe75
Update flashinfer CUTLASS MoE Kernel (#21408) 
Signed-off-by: Shu Wang. <shuw@nvidia.com>
 2025-07-24 08:13:31 -07:00
Wentao Ye
e8cb0d0495
[Bug] Fix Compressed Tensor NVFP4 cutlass_fp4_group_mm illegal memory access (#21465) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-07-24 08:13:24 -07:00
Ricardo Decal
684174115d
[Docs] Rewrite Distributed Inference and Serving guide (#20593) 
Signed-off-by: Ricardo Decal <rdecal@anyscale.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-07-24 08:13:05 -07:00
Sanger Steel
cdb79ee63d
[Docs] Update Tensorizer usage documentation (#21190) 
Signed-off-by: Sanger Steel <sangersteel@gmail.com>
Signed-off-by: William Goldby <willgoldby@gmail.com>
Co-authored-by: William Goldby <willgoldby@gmail.com>
 2025-07-24 06:56:18 -07:00
elvischenv
5a19a6c670
[Fix] Update mamba_ssm to 2.2.5 (#21421) 
Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
 2025-07-24 03:25:41 -07:00
Ming Yang
2ded067fd2
[Bugfix] Fix CUDA arch flags for MoE permute (#21426) 
Signed-off-by: Ming Yang <minos.future@gmail.com>
 2025-07-24 03:23:59 -07:00
Harry Mellor
13abd0eaf9
[Model] Officially support Emu3 with Transformers backend (#21319) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-07-24 03:22:12 -07:00
Lucas Wilkinson
61b8cea3b4
[Attention] Optimize FlashInfer MetadataBuilder Build call (#21137) 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
 2025-07-24 03:21:46 -07:00
cjackal
526078a96c
bump flashinfer to v0.2.8 (#21385) 
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
 2025-07-24 03:20:38 -07:00
Chauncey
6da0078523
[Feat] Allow custom naming of vLLM processes (#21445) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-07-24 03:15:23 -07:00
Rui Qiao
73e3949d07
[Misc] Improve comment for DPEngineCoreActor._set_cuda_visible_devices() (#21501) 
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
 2025-07-24 03:13:40 -07:00
Shintarou Okada
6eca337ce0
Replace --expand-tools-even-if-tool-choice-none with --exclude-tools-when-tool-choice-none for v0.10.0 (#20544) 
Signed-off-by: okada <kokuzen@gmail.com>
Signed-off-by: okada shintarou <okada@preferred.jp>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-07-24 02:56:36 -07:00
Yuxuan Zhang
85bda9e7d0
remove GLM-4.5 quantization wrong Code (#21435) 2025-07-24 01:52:43 -07:00
22quinn
610852a423
[Core] Support model loader plugins (#21067) 
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
 2025-07-24 01:49:44 -07:00
Nick Hill
f0f4de8f26
[Misc] Fix duplicate FusedMoEConfig debug messages (#21455) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-07-24 01:27:30 -07:00
Zhou Fang
fc5f756db4
[v1][Core] Clean up usages of SpecializedManager (#21407) 
Signed-off-by: Zhou Fang <fang.github@gmail.com>
 2025-07-24 00:40:11 -07:00
Chengji Yao
e74bfc70e4
[TPU][Bugfix] fix moe layer (#21340) 
Signed-off-by: Chengji Yao <chengjiyao@google.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
 2025-07-24 00:38:39 -07:00
Gregory Shtrasberg
90eeea8f85
[Bugfix][ROCm] Fix for warp_size uses on host (#21205) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
 2025-07-24 00:37:19 -07:00
Harry Mellor
dde295a934
Deduplicate Transformers backend code using inheritance (#21461) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-07-24 00:16:23 -07:00
Julien Denize
6d8d0a24c0
Add think chunk (#21333) 
Signed-off-by: Julien Denize <julien.denize@mistral.ai>
v0.10.0rc2 v0.10.0 		2025-07-23 21:51:32 -07:00
Yinghai Lu
11ef7a611e
[BugFix] Set CUDA_VISIBLE_DEVICES before spawning the subprocesses (#21211) 
Signed-off-by: Yinghai Lu <yinghai@thinkingmachines.ai>
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Rui Qiao <ruisearch42@gmail.com>
 2025-07-23 21:44:04 -07:00
Woosuk Kwon
dc2f159f8a
Dump input metadata on crash for async scheduling (#21258) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-07-23 21:10:30 -07:00
Robert Shaw
d5b981f8b1
[DP] Internal Load Balancing Per Node [one-pod-per-node] (#21238) 
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
 2025-07-23 20:57:32 -07:00
Nick Hill
eec6942014
[BugFix] Fix KVConnector TP worker aggregation (#21473) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-07-23 20:56:49 -07:00
KazusatoOoko
fd48d99ffd
[BugFix]: Batch generation from prompt_embeds fails for long prompts (#21390) 
Signed-off-by: KazusatoOko <kazusto.oko@sakana.ai>
Co-authored-by: KazusatoOko <kazusto.oko@sakana.ai>
 2025-07-23 20:43:17 -07:00
WeiQing Chen
f8c15c4efb
[Bugfix] Fix example disagg_example_p2p_nccl_xpyd.sh zombie process (#21437) 
Signed-off-by: David Chen <530634352@qq.com>
 2025-07-23 20:42:11 -07:00
Matthew Bonanni
aa08a954f9
[Bugfix] Fix casing warning (#21468) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-07-23 20:41:23 -07:00
Liangliang Ma
13e4ee1dc3
[XPU][UT] increase intel xpu CI test scope (#21492) 
Signed-off-by: Ma, Liangliang <liangliang.ma@intel.com>
 2025-07-23 20:24:04 -07:00
Ming Yang
772ce5af97
[Misc] Add dummy maverick test to CI (#21324) 
Signed-off-by: Ming Yang <minos.future@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-07-23 20:22:42 -07:00
deven-labovitch
63d92abb7c
[Frontend] Set MAX_AUDIO_CLIP_FILESIZE_MB via env var instead of hardcoding (#21374) 
Signed-off-by: Deven Labovitch <deven@videa.ai>
 2025-07-23 20:22:19 -07:00
Hardik Gupta
11599b0e1f
feat(gguf_loader): accept HF repo paths & URLs for GGUF (#20793) 
Signed-off-by: Hardik <hardikgupta1999@gmail.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-07-23 20:21:02 -07:00
Michael Goin
f3137cdd81
[Core] Freeze gc during cuda graph capture to speed up init (#21146) 
Signed-off-by: Codex <codex@openai.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-07-23 17:20:14 -07:00
Michael Goin
82ec66f514
[V0 Deprecation] Remove Prompt Adapters (#20588) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-07-23 16:36:48 -07:00
Yong Hoon Shin
78c13e30e1
[V1] Fix local chunked attention always disabled (#21419) 
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
 2025-07-23 15:59:30 -07:00
22quinn
5c9b807b34
[Core] Add reload_weights RPC method (#20096) 
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
 2025-07-23 14:24:52 -07:00
QiliangCui
14bf19e39f
[TPU][TEST] Fix the downloading issue in TPU v1 test 11. (#21418) 
Signed-off-by: Qiliang Cui <derrhein@gmail.com>
 2025-07-23 11:29:36 -07:00
Yong Hoon Shin
4ac7713e32
Add test case for compiling multiple graphs (#21044) 
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
 2025-07-23 11:00:47 -07:00
Christian Pinto
8560a5b258
[Core][Model] PrithviMAE Enablement on vLLM v1 engine (#20577) 
Signed-off-by: Christian Pinto <christian.pinto@ibm.com>
 2025-07-23 11:00:23 -07:00
Nick Hill
316b1bf706
[Tests] Add tests for headless internal DP LB (#21450) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-07-23 07:49:25 -07:00
Tao He
7c734ee09b
[Bugfix][Qwen][DCA] fixes bug in dual-chunk-flash-attn backend for qwen 1m models. (#21364) 
Signed-off-by: Tao He <linzhu.ht@alibaba-inc.com>
 2025-07-23 06:34:37 -07:00
Cyrus Leung
f59ec35b7f
[V1] Check all pooling tasks during profiling (#21299) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-07-23 05:53:26 -07:00
Asher
2671334d45
[Model] add Hunyuan V1 Dense Model support. (#21368) 
Signed-off-by: Asher Zhang <asherszhang@tencent.com>
 2025-07-23 03:54:08 -07:00
Michael Yao
2cc5016a19
[Docs] Clean up v1/metrics.md (#21449) 
Signed-off-by: windsonsea <haifeng.yao@daocloud.io>
 2025-07-23 03:37:25 -07:00