xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2026-10-31 20:49:08 +08:00
Code Issues Packages Projects Releases Wiki Activity
6,698 Commits 155 Branches 89 Tags
Commit Graph

1841 Commits

Author SHA1 Message Date
Pavani Majety
f2036734fb
[ModelOpt] Introduce VLLM_MAX_TOKENS_PER_EXPERT_FP4_MOE env var to control blockscale tensor allocation (#18160) 
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
 2025-05-23 15:52:20 -07:00
Michael Goin
0ddf88e16e
[CI] Enable test_initialization to run on V1 (#16736) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-05-23 15:09:44 -07:00
Jiayi Yao
2628a69e35
[V1] Support Deepseek MTP (#18435) 
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
Signed-off-by: YaoJiayi <120040070@link.cuhk.edu.cn>
Co-authored-by: Rui Qiao <ruisearch42@gmail.com>
 2025-05-23 10:26:28 -07:00
youkaichao
6a7988c55b
Refactor pplx init logic to make it modular (prepare for deepep) (#18200) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-05-23 23:43:43 +08:00
Harry Mellor
d4c2919760
Include private attributes in API documentation (#18614) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-05-23 06:18:31 -07:00
Tristan Leclercq
6220f3c6b0
[Bugfix] Fix transformers model impl ignored for mixtral quant (#18602) 
Signed-off-by: Tristan Leclercq <tristanleclercq@gmail.com>
 2025-05-23 05:54:13 -07:00
Harry Mellor
2edb533af2
Replace {func} with mkdocs style links (#18610) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-05-23 05:51:38 -07:00
Kay Yan
7ab056c273
[Hardware][CPU] Update intel_extension_for_pytorch 2.7.0 and move to requirements/cpu.txt (#18542) 
Signed-off-by: Kay Yan <kay.yan@daocloud.io>
 2025-05-23 04:38:42 -07:00
Madeesh Kannan
e493e48524
[V0][Bugfix] Fix parallel sampling performance regression when guided decoding is enabled (#17731) 
Signed-off-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
 2025-05-23 03:38:23 -07:00
Mengqing Cao
4ce64e2df4
[Bugfix][Model] Fix baichuan model loader for tp (#18597) 
Signed-off-by: Mengqing Cao <cmq0113@163.com>
 2025-05-23 02:39:05 -07:00
Harry Mellor
a1fe24d961
Migrate docs from Sphinx to MkDocs (#18145) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-05-23 02:09:53 -07:00
Benjamin Chislett
583507d130
[Spec Decode] Make EAGLE3 draft token ID mapping optional (#18488) 
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
 2025-05-22 20:17:39 -07:00
Harry Mellor
4b0da7b60e
Enable hybrid attention models for Transformers backend (#18494) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-05-23 10:12:08 +08:00
Mark McLoughlin
c6b636f9fb
[V1][Spec Decoding] Use model_loader.get_model() to load models (#18273) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2025-05-23 02:05:44 +00:00
Chenheli Hua
04eb88dc80
Re-submit: Fix: Proper RGBA -> RGB conversion for PIL images. (#18569) 
Signed-off-by: Chenheli Hua <huachenheli@outlook.com>
 2025-05-23 01:59:18 +00:00
Sanger Steel
c32e249a23
[Frontend] [Core] Add Tensorizer support for V1, LoRA adapter serialization and deserialization (#17926) 
Signed-off-by: Sanger Steel <sangersteel@gmail.com>
 2025-05-22 18:44:18 -07:00
Tyler Michael Smith
6e588da0f4
[Build/CI] Fix CUDA 11.8 build (#17679) 
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Tyler Michael Smith <tysmith@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
 2025-05-22 12:13:54 -07:00
aws-elaineyz
fa72f9a812
Order sequence ids + config update to support specifying custom quantization layers (#18279) 
Signed-off-by: Elaine Zhao <elaineyz@amazon.com>
Co-authored-by: Tailin Pan <tailinpa@amazon.com>
Co-authored-by: Rishabh Rajesh <rishyraj@amazon.com>
Co-authored-by: Yishan McNabb <yishanm@amazon.com>
Co-authored-by: Patrick Lange <patlange@amazon.com>
Co-authored-by: Maxwell Goldberg <mgld@amazon.com>
Co-authored-by: Aakash Shetty <sheaak@amazon.com>
 2025-05-22 02:20:36 -07:00
aws-elaineyz
ebed81fbf5
Update default neuron config for speculation (#18274) 
Signed-off-by: Elaine Zhao <elaineyz@amazon.com>
Co-authored-by: Shashwat Srijan <sssrijan@amazon.com>
Co-authored-by: Aakash Shetty <sheaak@amazon.com>
 2025-05-22 02:18:55 -07:00
Shane A
51797775c3
[Bugfix][Model] Make Olmo2Model weight loading return loaded weights (#18504) 
Signed-off-by: Shane A <shanea@allenai.org>
 2025-05-21 21:17:03 -07:00
youngrok cha
d022115cc6
[Bugfix] Inconsistent token calculation compared to HF in llava family (#18479) 
Signed-off-by: jaycha <jaycha@ncsoft.com>
 2025-05-21 20:21:47 -07:00
Dhia Eddine Rhaiem
20bd6f4d2e
[FalconH1] Fix output dtype in RMSNorm fallback path for Falcon-H1 (e.g. 0.5B) (#18500) 
Signed-off-by: dhia.rhaiem <dhia.rhaiem@tii.ae>
Co-authored-by: younesbelkada <younesbelkada@gmail.com>
Co-authored-by: Ilyas Chahed <ilyas.chahed@tii.ae>
Co-authored-by: Jingwei Zuo <jingwei.zuo@tii.ae>
 2025-05-21 19:23:59 -07:00
GiantCroc
c154d89306
[Doc] fix arg docstring in linear layers (#18410) 
Signed-off-by: giantcroc <1204449533@qq.com>
 2025-05-21 06:45:57 -07:00
Dhia Eddine Rhaiem
eca18691d2
[MODEL] FalconH1 (#18406) 
Signed-off-by: dhia.rhaiem <dhia.rhaiem@tii.ae>
Co-authored-by: younesbelkada <younesbelkada@gmail.com>
Co-authored-by: Ilyas Chahed <ilyas.chahed@tii.ae>
Co-authored-by: Jingwei Zuo <jingwei.zuo@tii.ae>
 2025-05-21 04:59:06 -07:00
Gregory Shtrasberg
0c15c2e486
[Bugfix] config.head_dim is now explicitly set to None (#18432) 
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
 2025-05-20 21:04:33 -07:00
Michael Goin
3b17ea26e4
[TPU] Re-enable the Pallas MoE kernel (#18025) 
Signed-off-by: Michael Goin <mgoin64@gmail.com>
 2025-05-20 19:52:27 -07:00
Calvin Chen
e1f5a71ed7
[Model] use AutoWeightsLoader for bloom (#18300) 
Signed-off-by: calvin chen <120380290@qq.com>
 2025-05-20 09:40:05 -07:00
Michael Goin
f4a8a37465
[Minor] Rename quantization nvfp4 to modelopt_fp4 (#18356) 
Signed-off-by: mgoin <mgoin64@gmail.com>
 2025-05-20 09:08:37 -07:00
Random Fly
bca55b556f
[Bugfix] fix adding bias twice in ipex GPTQ quantization (#18363) 
Signed-off-by: rand-fly <randfly@outlook.com>
 2025-05-20 00:54:33 -07:00
Isotr0py
f07a673eb2
[Misc] Allow AutoWeightsLoader to skip loading weights with specific substr in name (#18358) 
Signed-off-by: Isotr0py <2037008807@qq.com>
 2025-05-19 20:20:12 -07:00
Satyajith Chilappagari
dc1440cf9f
Neuron up mistral (#18222) 
Signed-off-by: Satyajith Chilappagari <satchill@amazon.com>
 2025-05-19 09:54:47 -07:00
sunyicode0012
7937c2fd52
Add files via uploadAdd fused MoE kernel tuning configs (fp8_w8a8) for DeepSeek V3/R1 on a single-node 8x NVIDIA H20 96GB setup (#18337) 2025-05-19 09:49:57 -07:00
Wenhua Cheng
e2ee1e8e9e
[Feature]Add support for models quantized with AutoRound (#17850) 
Signed-off-by: wenhuach21 <wenhua.cheng@intel.com>
 2025-05-19 09:38:53 -07:00
Jee Jee Li
6781af5608
[Quantization] Pool model support bitsandbytes (#18087) 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
 2025-05-19 09:03:43 -07:00
Shaoyu Yang
d637b96099
[BugFix] [Vul] Add missing usedforsecurity=False in MD5 hashing to enable FIPS (#18319) 
Signed-off-by: cascade812 <cascade812@outlook.com>
Signed-off-by: shaoyuyoung <shaoyuyoung@gmail.com>
Co-authored-by: cascade <cascade812@outlook.com>
 2025-05-19 01:31:23 -07:00
CYJiang
275c5daeb0
fix: Add type specifications for CLI arguments in tensorizer options (#18314) 2025-05-18 23:42:17 -07:00
wwl2755
9da1095daf
[Spec Decode][V0] Fix spec decode correctness test in V0 eagle/medusa (#18175) 
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
 2025-05-18 19:49:46 -07:00
Lifu Huang
4fb349f66a
Fix copy-paste error in phi4mm image processing (#18315) 
Signed-off-by: Lifu Huang <lifu.hlf@gmail.com>
 2025-05-18 07:00:12 -07:00
22quinn
908733aca7
[Model] Use sigmoid for single-label classification (#18313) 
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
 2025-05-18 07:00:09 -07:00
rongfu.leng
9214e60631
[Model] use AutoWeightsLoader for solar (#18113) 2025-05-17 00:24:17 -07:00
Bowen Wang
7fdfa01530
[Sampler] Adapt to FlashInfer 0.2.3 sampler API (#15777) 
Signed-off-by: Bowen Wang <abmfy@icloud.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
 2025-05-16 15:14:03 -07:00
learner0810
87d871470d
[Model] Use autoweightloader for dbrx (#18251) 
Signed-off-by: learner0810 <zhongjun.li@daocloud.io>
 2025-05-16 07:54:13 -07:00
Lain
e23564cb70
use ceil_div in cutlass block scaling shape check (#17918) 2025-05-16 03:02:58 -07:00
Vadim Gimpelson
67da5720d4
[PERF] Speed up Qwen2.5-VL model by speed up rotary position embedding (#17973) 
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@centml.ai>
 2025-05-15 23:31:02 -07:00
Sky Lee
f4937a51c1
[Model] vLLM v1 supports Medusa (#17956) 
Signed-off-by: lisiqi23 <lisiqi23@xiaomi.com>
Signed-off-by: skylee-01 <497627264@qq.com>
Co-authored-by: lisiqi23 <lisiqi23@xiaomi.com>
 2025-05-15 21:05:31 -07:00
TJian
92540529c0
[Bugfix] [ROCm]: Remove assertion logic when using AITER fused moe in unquantizedMethod to reenable LLama4 BF16 (#18205) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2025-05-15 09:53:18 -07:00
Harry Mellor
07ad27121f
Update deprecated type hinting in model_loader (#18130) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-05-15 04:00:21 -07:00
omahs
a9944aabfa
fix: typos (#18151) 
Signed-off-by: omahs <73983677+omahs@users.noreply.github.com>
 2025-05-15 02:16:15 -07:00
Mengqing Cao
70f8b96724
[Bugfix] Fix FusedMoEPrepareAndFinalize for cuda-disalike backends (#18178) 
Signed-off-by: Mengqing Cao <cmq0113@163.com>
 2025-05-14 23:16:31 -07:00
inkcherry
dd2a94596a
[Model] Allow the use of sliding window in Qwen2 (#17772) 
Signed-off-by: inkcherry <mingzhi.liu@intel.com>
 2025-05-14 22:29:38 -07:00