xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2025-12-10 05:45:00 +08:00
Code Issues Packages Projects Releases Wiki Activity
11,846 Commits 155 Branches 89 Tags
Commit Graph

1707 Commits

Author SHA1 Message Date
Shengqi Chen
4b612664fd
[CI] Renovation of nightly wheel build & generation (take 2) (#29838) 
Signed-off-by: Shengqi Chen <harry-chen@outlook.com>
 2025-12-01 22:17:10 -08:00
Kevin H. Luu
1336a1ea24
Revert #29787 and #29690 (#29815) 2025-12-01 13:42:03 -08:00
Finbarr Timbers
38caf7fa1a
Update FAQ on interleaving sliding windows support (#29796) 
Signed-off-by: Finbarr Timbers <finbarrtimbers@gmail.com>
 2025-12-01 19:15:19 +00:00
shivampr
cabc77cc86
[Core][Observability] Add KV cache residency metrics (#27793) 
Introduces three new Prometheus histograms for fine-grained observability of KV cache residency behavior:

vllm:kv_block_lifetime_seconds — total lifetime from allocation to free
vllm:kv_block_idle_before_evict_seconds — idle duration before eviction
vllm:kv_block_reuse_gap_seconds — time between consecutive reuses of the same block

These metrics help operators analyze KV cache efficiency, reuse patterns, and eviction timing beyond simple utilization rates.

Implementation uses monotonic timestamps for accuracy, 1% sampling for minimal overhead (~48 bytes/block), and is fully thread-safe with zero runtime cost when disabled.

Two new runtime flags are introduced:

--kv-cache-metrics – enable KV cache residency metrics
--kv-cache-metrics-sample – control sampling ratio (default: 0.01)

Signed-off-by: Shivam <shivamprasad91@gmail.com>
 2025-12-01 18:27:53 +00:00
sangbumlikeagod
092bb73b8a
[Frontend] add 'verbose_json' and 'timestamp' feature on Whisper Transcription/Translation (#24209) 
Signed-off-by: sangbumlikeagod <oironese@naver.com>
Signed-off-by: sangbumlikeagod <98077576+sangbumlikeagod@users.noreply.github.com>
 2025-12-01 18:19:17 +01:00
Shengqi Chen
36db0a35e4
[CI] Renovation of nightly wheel build & generation (#29690) 
Signed-off-by: Shengqi Chen <harry-chen@outlook.com>
 2025-12-01 21:25:39 +08:00
wang.yuqi
62de4f4257
[Frontend] Resettle pooling entrypoints (#29634) 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
 2025-12-01 15:30:43 +08:00
Yifei Zhang
1ab8fc8197
Make PyTorch profiler gzip and CUDA time dump configurable (#29568) 
Signed-off-by: Yifei Zhang <yifei.zhang1992@outlook.com>
 2025-12-01 04:30:46 +00:00
Cyrus Leung
2afcec4dec
[Misc] Update TokenizerLike interface and move get_cached_tokenizer (#29730) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-11-30 14:59:47 +08:00
Jinzhen Lin
1656ad3704
[Kernel][Quantization] add w4a8 support for marlin kernel (#24722) 
Signed-off-by: Jinzhen Lin <jinzhen.ljz@antgroup.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin@redhat.com>
 2025-11-29 07:19:33 -08:00
dublc
f4341f45d3
[Doc]: fix code block rendering (#29728) 
Signed-off-by: dublc <jdublc0x@gmail.com>
 2025-11-29 13:46:48 +00:00
Cyrus Leung
34a984274e
[Misc] Refactor tokenizer interface (#29693) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-11-29 04:02:21 -08:00
Yanan Cao
3461e7efd8
[Frontend] Remap -O to -cc commandline flag (#29557) 
Signed-off-by: Yanan Cao <gmagogsfm@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2025-11-28 21:51:12 +00:00
Harry Mellor
4332955602
[Docs] Add CLI reference doc for vllm bench sweep plot_pareto (#29689) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-11-28 08:10:08 -09:00
Wilson Wu
5c2b5cb422
[Docs] Add SPLADE and Ultravox models to supported models documentation (#29659) 
Signed-off-by: Wilson Wu <iwilsonwu@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-11-28 01:29:28 -09:00
Cyrus Leung
ccbdf51bd5
[Doc] Reorganize benchmark docs (#29658) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-11-28 17:19:25 +08:00
rongfu.leng
480598958e
[Feature][Bench] Add pareto visualization (#29477) 
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
 2025-11-27 23:53:20 -08:00
Wilson Wu
18523b87f6
[Docs] Update supported models for Olmo 3 in tool calling documentation (#29411) 
Signed-off-by: Wilson Wu <iwilsonwu@gmail.com>
 2025-11-28 02:53:55 +00:00
Morrison Turnansky
0838b52e2e
[Frontend][torch.compile] CompilationConfig Overhaul (#20283): Set up -O infrastructure (#26847) 
Signed-off-by: morrison-turnansky <mturnans@redhat.com>
Signed-off-by: adabeyta <aabeyta@redhat.com>
Signed-off-by: Morrison Turnansky <mturnans@redhat.com>
Co-authored-by: adabeyta <aabeyta@redhat.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-11-27 01:55:58 -08:00
TJian
da8e1a1bf9
[DOC] Add vLLM Bangkok Meetup info (#29561) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2025-11-27 04:42:50 +00:00
Louie Tsai
9bb33c8919
add xpu supported model and model id for cpu (#29380) 
Signed-off-by: Tsai, Louie <louie.tsai@intel.com>
 2025-11-27 11:30:50 +08:00
Lucas Wilkinson
56539cddac
[Core] Refactor padding logic and pad for CUDA graphs before attention metadata building (#28579) 2025-11-26 14:07:13 -05:00
Matthew Bonanni
430dd4d9eb
[Attention] Remove imports from vllm/attention/__init__.py (#29342) 
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
 2025-11-26 10:53:15 -07:00
yxt
3650a74ed8
Optimize the wording of the document and unify the terminology and th… (#29491) 2025-11-26 05:16:12 -08:00
Michael Goin
e502098643
[Kernel] Add NVFP4 MoE CUTLASS support for SM120 (#29242) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
 2025-11-25 06:59:07 -08:00
Ben Browning
e1dd706cd1
[Frontend] Respect Chat Completion parallel_tool_calls param (#26233) 
Signed-off-by: Ben Browning <bbrownin@redhat.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2025-11-25 09:56:15 +00:00
Harry Mellor
316c8492bf
Scheduled removal of guided_* config fields (#29326) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-11-25 05:24:05 +00:00
Isotr0py
92effb07a4
[Model] Add HunyuanOCR support (#29327) 
Signed-off-by: manayang <jackmanayang@gmail.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: sergeywang <sergeywang@tencent.com>
Co-authored-by: manayang <jackmanayang@gmail.com>
Co-authored-by: manayang <manayang@tencent.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2025-11-25 03:28:51 +00:00
Tyler Michael Smith
4dd42db566
Remove VLLM_SKIP_WARMUP tip (#29331) 
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
 2025-11-24 22:16:05 +00:00
Benjamin Bartels
4d6afcaddc
[CI/Build] Moves to cuda-base runtime image while retaining minimal JIT dependencies (#29270) 
Signed-off-by: bbartels <benjamin@bartels.dev>
Signed-off-by: Benjamin Bartels <benjamin@bartels.dev>
 2025-11-24 11:40:54 -08:00
Laith Sakka
7a228b5305
Add option to use unbacked, and backed size obl dynamic shapes for more sounds compilation. (#26199) 
Signed-off-by: Laith Sakka <lsakka@meta.com>
 2025-11-24 10:12:41 -05:00
Yuan Tang
f716a15372
Update KServe guide link in documentation (#29258) 
Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
 2025-11-24 14:40:05 +00:00
Roger Wang
0ff70821c9
[Core] Deprecate xformers (#29262) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2025-11-24 04:18:55 +00:00
Zero
30854783ad
[Model] Add OpenCUA-7B support (#29068) 
Signed-off-by: lim4349 <rockmanzero@naver.com>
Signed-off-by: Zero <rockmanzero@naver.com>
Co-authored-by: Cloud User <ubuntu@a100-80g-4.novalocal>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-11-24 10:27:55 +08:00
Cyrus Leung
389aa1b2eb
[Doc] Update more docs with respect to V1 (#29188) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-11-23 10:58:48 +08:00
Michael Act
3ed767ec06
docs: fixes distributed executor backend config for multi-node vllm (#29173) 
Signed-off-by: Michael Act <michael.a.c.tulenan@gdplabs.id>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
 2025-11-23 10:58:28 +08:00
Benjamin Bartels
eb5352a770
[CI/build] Removes source compilation from runtime image (#26966) 
Signed-off-by: bbartels <benjamin@bartels.dev>
 2025-11-22 10:23:09 -08:00
Angela Yi
d5dbdbfcb2
[docs] Fix cudagraph mode config (#29170) 
Signed-off-by: angelayi <yiangela7@gmail.com>
 2025-11-21 17:10:27 -08:00
Julien Denize
57430fc95c
Default model load/config/tokenizer to mistral format if relevant files exist (#28659) 
Signed-off-by: Julien Denize <julien.denize@mistral.ai>
Signed-off-by: Julien Denize <40604584+juliendenize@users.noreply.github.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-11-21 13:58:59 -08:00
wangxiyuan
4050bae417
[Doc] Update plugin doc (#28532) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2025-11-21 14:57:26 +00:00
Cyrus Leung
9452863088
Revert "Revert #28875 (#29159)" (#29179) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-11-21 04:27:43 -08:00
Cyrus Leung
aab0102a26
[V0 deprecation] Remove more V0 references (#29088) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-11-21 11:56:59 +00:00
Cyrus Leung
4d7231e774
Revert #28875 (#29159) 2025-11-21 01:40:17 -08:00
Cyrus Leung
56e96b37e4
[V0 Deprecation] Remove best_of (#29090) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-11-21 11:40:40 +08:00
Qidong Su
698024ecce
[Doc] update installation guide regarding aarch64+cuda pytorch build (#28875) 
Signed-off-by: Qidong Su <soodoshll@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-11-20 19:40:25 -08:00
jeremyteboul
0730414999
[Core] Add audio_embeds support to chat completions (#29059) 
Signed-off-by: Jeremy Teboul <jeremyteboul@fb.com>
Co-authored-by: Jeremy Teboul <jeremyteboul@fb.com>
 2025-11-21 11:39:47 +08:00
Michael Goin
87cbbdff63
Update model references for OLMo3 (#29099) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-11-21 09:16:52 +08:00
Rob Mulla
dd39f91edb
[Doc] cleanup TPU documentation and remove outdated examples (#29048) 
Signed-off-by: Rob Mulla <rob.mulla@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-11-21 00:05:59 +00:00
Shinichi Hemmi
c9e093116c
[MODEL] Implement plamo3 (#28834) 
Signed-off-by: Shinichi Hemmi <50256998+Alnusjaponica@users.noreply.github.com>
 2025-11-20 03:00:19 -08:00
Shanshan Shen
d44e9df7d4
[Model][Mamba] Add selector for mamba attention backend and make it pluggable for other device (#26487) 
Signed-off-by: shen-shanshan <467638484@qq.com>
 2025-11-19 16:24:55 +00:00