xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2025-12-09 23:15:31 +08:00
Code Issues Packages Projects Releases Wiki Activity
7,391 Commits 155 Branches 89 Tags
Commit Graph

2208 Commits

Author SHA1 Message Date
Li, Jiang
6cc1e7d96d
[CPU] Update custom ops for the CPU backend (#20255) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-07-01 07:25:03 +00:00
czhu-cohere
9909726d2a
Enable ZP Support for Machete (#20268) 
Signed-off-by: czhu-cohere <conway.zhu@cohere.com>
 2025-07-01 07:12:20 +00:00
Alex Kogan
27949354fa
[Feature] A calibration-free RTN-based quantization for accurate and accelerated INT4/INT8 inference (#18768) 
Signed-off-by: Alex Kogan <alex.kogan@oracle.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-07-01 05:44:38 +00:00
fyuan1316
e28533a16f
[Bugfix] Fix include prompt in stream response when echo=true (#15233) 
Signed-off-by: Yuan Fang <yuanfang@alauda.io>
 2025-07-01 01:30:14 +00:00
Luka Govedič
6d42ce8315
[CLI] Improve CLI arg parsing for -O/--compilation-config (#20156) 
Signed-off-by: luka <luka@neuralmagic.com>
 2025-07-01 01:03:13 +00:00
Kyle Sayers
d8cf819a9a
[Core] [Bugfix] [Multimodal] Fix multimodal profiling and generation for SFT/PTQed models (#20058) 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
 2025-06-30 17:26:49 +00:00
Wentao Ye
551ef1631a
[Unit Test] Add unit test for deep gemm (#20090) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-06-30 10:26:42 -06:00
Woosuk Kwon
2863befce3
[Optimization] Use Shared CachedRequestData Instance Across All Requests (#20232) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-06-30 09:07:50 -07:00
redmoe-moutain
65b1cbb138
[Model] support dots1 (#18254) 
Signed-off-by: redmoe-moutain <agiredmoe@gmail.com>
 2025-06-29 19:34:36 -07:00
Dipika Sikka
6f2f53a82d
[Quantization] Add compressed-tensors NVFP4 MoE Support (#19990) 
Signed-off-by: Dipika Sikka <dipikasikka1@gmail.com>
Signed-off-by: Dipika <dipikasikka1@gmail.com>
 2025-06-29 22:05:40 +00:00
Michael Goin
7b1895e6ce
[CI Fix] Try fixing eagle e2e test OOM by reducing block allocation (#20213) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-06-29 10:31:37 +08:00
Wentao Ye
4d36693687
[Refactor] Create a function util and cache the results for has_deepgemm, has_deepep, has_pplx (#20187) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-06-28 22:06:38 +00:00
Stan Wozniak
daec9dea6e
[Bugfix] Correct behavior of GraniteMoeHybrid for TensorParallel execution (#20137) 
Signed-off-by: Stanislaw Wozniak <stw@zurich.ibm.com>
 2025-06-28 08:16:41 -07:00
Thomas Parnell
8615d9776f
[CI/Build] Add new CI job to validate Hybrid Models for every PR (#20147) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
 2025-06-27 23:00:25 -07:00
Chales Xu
e53be6f00a
[Misc] Add type assertion of request_id for LLMEngine.add_request (#19700) 
Signed-off-by: n2ptr <xuzhanchaomail@163.com>
 2025-06-27 22:47:36 -07:00
Michael Goin
c329ceca6d
[CI Fix] Pin tests/models/registry.py MiniMaxText01ForCausalLM to revision due to model changes (#20199) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-06-28 13:43:06 +08:00
Luka Govedič
aafabaa0d5
[Fix][torch.compile] Enable custom ops by default when Inductor off (#20102) 
Signed-off-by: luka <luka@neuralmagic.com>
 2025-06-27 09:00:42 -06:00
Robert Shaw
d1c956dc0f
Gemma3n (Text-only) (#20134) 
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Signed-off-by: Roger Wang <hey@rogerw.me>
Co-authored-by: Roger Wang <hey@rogerw.me>
 2025-06-27 07:16:26 +00:00
Yazan Sharaya
6e244ae091
[Perf][Frontend] eliminate api_key and x_request_id headers middleware overhead (#19946) 
Signed-off-by: Yazan-Sharaya <yazan.sharaya.yes@gmail.com>
 2025-06-27 00:44:14 -04:00
wang.yuqi
cd4cfee689
[Model][1/N] Automatic conversion of CrossEncoding model (#20012) 
Signed-off-by: wang.yuqi <noooop@126.com>
 2025-06-26 21:10:04 -07:00
Thomas Parnell
e110930680
[Fix] Fix gemma CI test failing on main (#20124) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
 2025-06-26 21:06:59 -07:00
Yang Wang
8b64c895c0
[CI] Sync test dependency with test.in for torch nightly (#19632) 
Signed-off-by: Yang Wang <elainewy@meta.com>
Signed-off-by: Yida Wu <yidawu@alumni.cmu.edu>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Concurrensee <yida.wu@amd.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-06-26 20:55:25 -07:00
li haoyang
0740e29b66
[Feature] add quick all reduce (#19744) 
Signed-off-by: ilmarkov <imarkov@redhat.com>
Signed-off-by: Haoyang Li <Haoyang.Li@amd.com>
Co-authored-by: ilmarkov <imarkov@redhat.com>
 2025-06-26 20:54:24 -07:00
Michael Goin
71799fd005
[CI Failure] Fix OOM with test_oot_registration_embedding (#20144) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-06-27 11:21:04 +08:00
Bowen Wang
e9fd658a73
[Feature] Expert Parallelism Load Balancer (EPLB) (#18343) 
Signed-off-by: Bowen Wang <abmfy@icloud.com>
 2025-06-26 15:30:21 -07:00
Wentao Ye
562308816c
[Refactor] Rename commnication utils (#20091) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-06-26 22:19:32 +00:00
Chengji Yao
04e1642e32
[TPU] add kv cache update kernel (#19928) 
Signed-off-by: Chengji Yao <chengjiyao@google.com>
 2025-06-26 10:01:37 -07:00
Wentao Ye
c894c5dc1f
[Bug Fix] Fix address/port already in use error for deep_ep test (#20094) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-06-26 22:33:13 +08:00
Li, Jiang
0567c8249f
[CPU] Fix torch version in x86 CPU backend (#19258) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-06-26 03:34:47 -07:00
Seiji Eicher
65397e40f5
[Bugfix] Allow CUDA_VISIBLE_DEVICES='' in Platform.device_id_to_physical_device_id (#18979) 
Signed-off-by: Seiji Eicher <seiji@anyscale.com>
 2025-06-26 00:01:57 -07:00
Nicolò Lucchesi
2582683566
[PD] Skip tp_size exchange with rank0 (#19413) 
Signed-off-by: NickLucche <nlucches@redhat.com>
 2025-06-25 20:04:39 -07:00
Michael Goin
754b00edb3
[Bugfix] Fix Mistral tool-parser regex for nested JSON (#20093) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-06-26 01:01:17 +00:00
Chenyaaang
2d7620c3eb
[TPU] Add TPU specific var VLLM_TPU_MOST_MODEL_LEN (#19919) 
Signed-off-by: Chenyaaang <chenyangli@google.com>
 2025-06-25 15:51:02 -07:00
Nick Hill
c40692bf9a
[Misc] Add parallel state node_count function (#20045) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-06-25 13:38:53 -07:00
lkchen
4734704b30
[PD] let toy proxy handle /chat/completions (#19730) 
Signed-off-by: Linkun <github@lkchen.net>
 2025-06-25 15:17:45 -04:00
Nicolò Lucchesi
e795d723ed
[Frontend] Add /v1/audio/translations OpenAI API endpoint (#19615) 
Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
 2025-06-25 17:54:14 +00:00
bnellnm
015fab8c2f
[Kernels][Bugfix] Use torch op for all kernels in FusedMoE forward. Add additional testing for cudagraphs. (#19717) 
Signed-off-by: Bill Nell <bnell@redhat.com>
 2025-06-24 23:22:58 -07:00
Wentao Ye
879f69bed3
[Refactor] Remove duplicate ceil_div (#20023) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-06-25 05:19:09 +00:00
Wentao Ye
a6c4b87fbc
Revert "[Feature] Integrate new deepgemm (#19820)" (#20049) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-06-24 19:45:22 -07:00
Boyuan Feng
c01d1c5aba
use .dev for version comparison with pytorch nightly release (#20031) 
Signed-off-by: Boyuan Feng <boyuan@meta.com>
 2025-06-24 21:52:16 +00:00
Wentao Ye
c6e3bba8e6
[Feature] Integrate new deepgemm (#19820) 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
 2025-06-24 12:51:56 -07:00
lkchen
91f7d9d0b6
[P/D] Asynchronously do _nixl_handshake (#19836) 
Signed-off-by: Linkun Chen <github@lkchen.net>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-06-24 12:46:10 -07:00
amit
981eeca41a
[Fix][V1] Remove --scheduling-policy oracle (#20010) 
Signed-off-by: amit <amit.man@gmail.com>
 2025-06-24 09:52:15 -07:00
Li, Jiang
53da4cd397
[Bugfix][CPU] Fix InputBatch for pooling models in the CPU v1 (#20014) 
Signed-off-by: jiang1.li <jiang1.li@intel.com>
 2025-06-24 13:20:04 +00:00
Chenyaaang
33d5e29be9
[TPU] Fix tpu model runner test (#19995) 
Signed-off-by: Chenyaaang <chenyangli@google.com>
 2025-06-23 16:04:28 -07:00
cascade
e6327c9b3e
[Feature] Support sequence parallelism for static fp8 quantization (#19181) 
Signed-off-by: cascade812 <cascade812@outlook.com>
 2025-06-23 16:09:02 -04:00
Isotr0py
61f4fc5dc6
[Bugfix][v1] Fix step pooler implementation and step pooling usage in v1 (#19956) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-06-23 18:38:06 +00:00
Tyler Michael Smith
68aaeb3749
[EP+DP] Optimize the little operations in the DeepGEMM + DeepEP low latency case (#19885) 
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Signed-off-by: Tyler Michael Smith <tysmith@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
 2025-06-23 11:07:47 -07:00
Jee Jee Li
a6e6604d32
[Bugfix] Fix CI bitsandbytes failure (#19969) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-06-23 21:30:55 +08:00
lkchen
1bcd15edc7
[BugFix][P/D] Fix for cases where _recving_transfers can be cleaned up when *all* transfer done (#19874) 
Signed-off-by: Linkun Chen <github@lkchen.net>
 2025-06-22 22:41:53 -07:00