xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2026-07-24 07:07:10 +08:00
Code Issues Packages Projects Releases Wiki Activity
6,345 Commits 155 Branches 89 Tags
Commit Graph

4295 Commits

Author SHA1 Message Date
Cyrus Leung
19dcc02a72
[Bugfix] Fix mistral model tests (#17181) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-04-25 06:03:34 -07:00
Alex Brooks
7feae92c1f
[Doc] Move todo out of beam search docstring (#17183) 
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
 2025-04-25 04:44:58 -07:00
Lu Fang
fc966e9cc6
Only turn on FastIncrementalDetokenizer when tokenizers >= 0.21.1 (#17158) 2025-04-25 17:10:32 +08:00
rasmith
a41351f363
[Quantization][FP8] Add support for FP8 models with input_scale for output projection and QK quantization (#15734) 
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>
 2025-04-25 00:45:02 -07:00
Sangyeon Cho
6aae216b4e
[Bugfix] remove fallback in guided_json (int range, patterns) (#16725) 
Signed-off-by: csy1204 <josang1204@gmail.com>
Co-authored-by: 조상연[플레이스 AI] <sang-yeon.cho@navercorp.com>
 2025-04-25 06:54:43 +00:00
yexin(叶鑫)
b22980a1dc
[Perf]Optimize rotary_emb implementation to use Triton operator for improved inference performance (#16457) 
Signed-off-by: cynthieye <yexin93@qq.com>
Co-authored-by: MagnetoWang <magnetowang@outlook.com>
 2025-04-25 14:52:28 +08:00
Mengqing Cao
2f54045508
[Bugfix][Misc] Use TritonPlaceholderModule to defensively import triton (#15099) 
Signed-off-by: Mengqing Cao <cmq0113@163.com>
 2025-04-24 22:51:02 -07:00
Lifu Huang
5aa6efb9a5
[Misc] Clean up redundant code in uniproc_executor.py (#16762) 
Signed-off-by: Lifu Huang <lifu.hlf@gmail.com>
 2025-04-24 22:49:30 -07:00
Harry Mellor
6ca0234478
Move missed SchedulerConfig args into scheduler config group in EngineArgs (#17131) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-04-24 22:48:53 -07:00
Zaida Zhou
69bff9bc89
fix float16 support for kimi-vl (#17156) 
Co-authored-by: zhouzaida <zhouzaida@msh.team>
 2025-04-24 20:16:32 -07:00
vllmellm
eef364723c
[FEAT] [ROCm]: AITER Fused MOE V1 Support (#16752) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2025-04-25 11:06:50 +08:00
jglaser
0d6e187e88
Use custom address for listening socket (#15988) 
Signed-off-by: Jens Glaser <glaserj@ornl.gov>
 2025-04-25 01:57:16 +00:00
Michael Goin
9420a1fc30
Better error message for missing mistral params.json (#17132) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-04-24 23:43:08 +00:00
Maximilien de Bayser
05e1fbfc52
Add chat template for Llama 4 models (#16428) 
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
 2025-04-24 20:19:36 +00:00
Yinghai Lu
fe92176321
Add collective_rpc to llm engine (#16999) 
Signed-off-by: Yinghai Lu <yinghai@thinkingmachines.ai>
 2025-04-24 20:16:52 +00:00
Harry Mellor
0fa939e2d1
Improve configs - LoRAConfig + PromptAdapterConfig (#16980) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-04-24 10:29:34 -07:00
Mark McLoughlin
340d7b1b21
[V1][Spec Decoding] Add num_drafts and num_accepted_tokens_per_position metrics (#16665) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2025-04-24 08:57:40 -07:00
wang.yuqi
67309a1cb5
[Frontend] Using matryoshka_dimensions control the allowed output dimensions. (#16970) 2025-04-24 07:06:28 -07:00
Shanshan Shen
b724afe343
[V1][Structured Output] Clear xgrammar compiler object when engine core shut down to avoid nanobind leaked warning (#16954) 
Signed-off-by: shen-shanshan <467638484@qq.com>
 2025-04-24 06:15:03 -07:00
Harry Mellor
21f4f1c9a4
Improve static type checking in LoRAModelRunnerMixin (#17104) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-04-24 06:14:47 -07:00
Isotr0py
b0c1f6202d
[Misc] Remove OLMo2 config copy (#17066) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-04-24 06:14:32 -07:00
Rui Qiao
c0dfd97519
[V1][PP] Optimization: continue scheduling prefill chunks (#17080) 
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
 2025-04-24 05:27:08 -07:00
Harry Mellor
0a05ed57e6
Simplify TokenizerGroup (#16790) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-04-24 04:43:56 -07:00
Woosuk Kwon
b411418ff0
[Chore] Remove Sampler from Model Code (#17084) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-04-24 02:49:33 -07:00
张宇
6167c0e5d2
[Bugfix][Core] add seq_id_to_seq_group clearing to avoid memory leak when s… (#16472) 
Signed-off-by: 开哲 <kaizhe.zy@alibaba-inc.com>
Co-authored-by: 开哲 <kaizhe.zy@alibaba-inc.com>
 2025-04-24 11:25:37 +08:00
Areeb Syed
ed2e464653
Addendum Fix to support FIPS enabled machines with MD5 hashing (#17043) 
Signed-off-by: sydarb <areebsyed237@gmail.com>
 2025-04-23 19:55:00 -07:00
Harry Mellor
2c8ed8ee48
More informative error when using Transformers backend (#16988) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-04-23 19:54:03 -07:00
Michael Goin
ed50f46641
[Bugfix] Enable V1 usage stats (#16986) 
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
 2025-04-23 19:54:00 -07:00
Woosuk Kwon
46e678bcff
[Minor] Use larger batch sizes for A100/B100/B200/MI300x (#17073) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-04-23 19:18:59 -07:00
Chen Xia
6b2427f995
[Quantization]add prefix for commandA quantized model (#17017) 2025-04-23 17:32:40 -07:00
Woosuk Kwon
41fb013d29
[V1][Spec Decode] Always use argmax for sampling draft tokens (#16899) 
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-04-23 14:57:43 -07:00
Yong Hoon Shin
32d4b669d0
[BugFix][V1] Fix int32 token index overflow when preparing input ids (#16806) 2025-04-23 12:12:35 -07:00
Travis Johnson
3cde34a4a4
[Frontend] Support guidance:no-additional-properties for compatibility with xgrammar (#15949) 
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
 2025-04-23 18:34:41 +00:00
Harry Mellor
bdb3660312
Use @property and private field for data_parallel_rank_local (#17053) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-04-23 08:50:08 -07:00
Harry Mellor
f3a21e9c68
CacheConfig.block_size should always be int when used (#17052) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-04-23 08:50:05 -07:00
Harry Mellor
8e630d680e
Improve Transformers backend model loading QoL (#17039) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-04-23 07:33:51 -07:00
Harry Mellor
53c0fa1e25
Ensure that pid passed to kill_process_tree is int for mypy (#17051) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-04-23 07:32:26 -07:00
Michael Goin
aa72d9a4ea
Mistral-format support for compressed-tensors (#16803) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-04-23 08:46:23 -04:00
Chauncey
8c87a9ad46
[Bugfix] Fix AssertionError: skip_special_tokens=False is not supported for Mistral tokenizers (#16964) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-04-23 07:24:09 +00:00
huafeng
ec69124eb4
[Misc] Improve readability of get_open_port function. (#17024) 
Signed-off-by: gitover22 <qidizou88@gmail.com>
 2025-04-23 06:16:53 +00:00
Lucas Wilkinson
d0da99fb70
[BugFix] llama4 fa3 fix - RuntimeError: scheduler_metadata must have shape (metadata_size) (#16998) 
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
 2025-04-22 21:49:24 -07:00
Nick Hill
b2f195c429
[V1] Avoid socket errors during shutdown when requests are in in-flight (#16807) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-04-23 12:36:29 +08:00
vllmellm
047797ef90
[Bugfix] Triton FA function takes no keyword arguments (#16902) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
 2025-04-22 21:35:24 -07:00
Chendi.Xue
56a735261c
[INTEL-HPU][v0] Port delayed sampling to upstream (#16949) 
Signed-off-by: Michal Adamczyk <michal.adamczyk@intel.com>
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
Co-authored-by: Michal Adamczyk <madamczyk@habana.ai>
 2025-04-22 20:14:11 -07:00
youkaichao
e1cf90e099
[misc] tune some env vars for GB200 (#16992) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-04-23 10:59:48 +08:00
Chauncey
6bc1e30ef9
Revert "[Misc] Add S3 environment variables for better support of MinIO." (#17021) 2025-04-22 19:22:29 -07:00
vllmellm
7e081ba7ca
[BugFix] Revert ROCm Custom Paged Attention Env Flag Check (#17022) 
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
 2025-04-22 19:17:48 -07:00
Nick Hill
1e013fa388
[V1][DP] More robust DP/EP dummy request coordination (#16277) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-04-22 19:12:15 -07:00
Aleksandr Malyshev
bc7c4d206b
[Kernel][ROCM] Upstream prefix prefill speed up for vLLM V1 (#13305) 
Signed-off-by: Sage Moore <sage@neuralmagic.com>
Signed-off-by: root <root@banff-cyxtera-s73-5.ctr.dcgpu>
Signed-off-by: Aleksandr Malyshev <maleksan@amd.com>
Signed-off-by: root <root@banff-cyxtera-s65-4.amd.com>
Signed-off-by: maleksan85 <maleksan@amd.com>
Signed-off-by: <>
Co-authored-by: Sage Moore <sage@neuralmagic.com>
Co-authored-by: root <root@banff-cyxtera-s73-5.ctr.dcgpu>
Co-authored-by: Aleksandr Malyshev <maleksan@amd.com>
Co-authored-by: qli88 <qiang.li2@amd.com>
Co-authored-by: root <root@banff-cyxtera-s65-4.amd.com>
 2025-04-22 19:11:56 -07:00
Guillaume Calmettes
36fe78769f
[Bugfix] validate urls object for multimodal content parts (#16990) 
Signed-off-by: Guillaume Calmettes <gcalmettes@scaleway.com>
 2025-04-23 09:43:06 +08:00