xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2026-01-07 11:09:47 +08:00
Code Issues Packages Projects Releases Wiki Activity
11,255 Commits 155 Branches 89 Tags
Commit Graph

1062 Commits

Author SHA1 Message Date
baonudesifeizhai
c428e8d80b
Fix io processor pooling #28273 (#28484) 
Signed-off-by: baonudesifeizhai <baonudesifeizhai@gmail.com>
 2025-11-13 11:34:14 +00:00
Chauncey
5c9ad138d5
[Frontend] supports interleaved thinking (#28531) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-11-13 16:14:13 +08:00
Andrew Xia
1a0b157a2e
[Frontend][responsesAPI][1/n] convert responses API tool input to chat completions tool format (#28231) 
Signed-off-by: Andrew Xia <axia@fb.com>
Co-authored-by: Andrew Xia <axia@fb.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2025-11-13 04:47:22 +00:00
Andrew Xia
7c38ed0f1c
[Frontend] split append tool output (#28333) 
Signed-off-by: Andrew Xia <axia@fb.com>
Co-authored-by: Andrew Xia <axia@fb.com>
 2025-11-13 04:03:23 +00:00
Yanan Cao
48c879369f
[Frontend] Change CompilationMode to a proper Enum (#28165) 
Signed-off-by: Yanan Cao <gmagogsfm@gmail.com>
 2025-11-11 19:46:18 -05:00
Zuyi Zhao
bca74e32b7
[Frontend] Add sagemaker_standards dynamic lora adapter and stateful session management decorators to vLLM OpenAI API server (#27892) 
Signed-off-by: Zuyi Zhao <zhaozuy@amazon.com>
Signed-off-by: Shen Teng <sheteng@amazon.com>
Co-authored-by: Shen Teng <sheteng@amazon.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
 2025-11-11 04:57:01 +00:00
Jialin Ouyang
b30372cbd0
[Perf] Move gc.freeze logic from EngineCoreProc to EngineCore for better coverage (#27896) 
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
 2025-11-10 15:34:18 -08:00
Andrew Xia
4b94ed8f92
[Frontend][2/n] remove empty content from _parse_tool_calls_from_content (#28331) 
Signed-off-by: Andrew Xia <axia@fb.com>
Co-authored-by: Andrew Xia <axia@fb.com>
 2025-11-10 14:07:49 -08:00
Benjamin Chislett
975676d174
[Feat] Drop-in Torch CUDA Profiler (#27841) 
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
 2025-11-08 14:07:37 -08:00
Harry Mellor
d9ab1ad9d1
reasoning_content -> reasoning (#27752) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-11-08 12:15:08 +00:00
Iceber Gu
e0d6b4a867
[CLI] add --max-tokens to vllm complete (#28109) 
Signed-off-by: Iceber Gu <caiwei95@hotmail.com>
 2025-11-07 12:21:40 +00:00
baonudesifeizhai
9da9208b20
[Bug] Fix missing token_ids for reasoning parser models in chat completions #28246 (#28256) 2025-11-07 07:31:58 +00:00
Vico Chu
d4aa65c998
[Chore] eliminate duplicated and unconditional object serialization in anthropic messages api (#27792) 
Signed-off-by: Vico Chu <vico24826@gmail.com>
 2025-11-06 19:09:19 +00:00
Roy Wang
d1dd5f53e4
[Frontend] Fix logging format when enable response logging (#28049) 
Signed-off-by: esmeetu <jasonailu87@gmail.com>
 2025-11-06 16:25:39 +00:00
Chauncey
59a50afa08
[Frontend] OpenAI Responses API supports Tool/Function calling - non-harmony (#26874) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-11-06 10:40:03 +00:00
Chauncey
07d614511f
[Misc] Remove the duplicate code (#28111) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-11-05 21:07:47 -05:00
Walter Beller-Morales
752ddeacaa
[Core] add support for reasoning parser plugins (#28075) 
Signed-off-by: walter beller-morales <walter.beller.morales@gmail.com>
 2025-11-06 01:15:06 +08:00
Isotr0py
3f5a4b6473
[Bugfix] Validate custom logits processor xargs for online serving (#27560) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-11-05 16:53:33 +00:00
Ilya Markov
e50c454672
[BugFix] Support EP/DP + EPLB with MTP (#25311) 
Signed-off-by: ilmarkov <markovilya197@gmail.com>
Signed-off-by: Sage Moore <sage@neuralmagic.com>
Co-authored-by: Sage Moore <sage@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
 2025-11-05 15:22:17 +00:00
Eric Yue
b57789b62b
Fix excessive logging noise by reducing the log level of the MinimaxM2ToolParser import success message (#27635) 
Signed-off-by: minatoaquaMK2 <jiacheng.yue@foxmail.com>
 2025-11-05 19:03:51 +08:00
Chauncey
0976711f3b
[Refactor] to simplify and extract the shared logic between chat completion and responses (#27961) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-11-05 15:46:39 +08:00
Chauncey
e261d37c9a
[Refactor] Lazy-loaded reasoning_parser (#28092) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-11-05 15:37:02 +08:00
Alex Brooks
b7cbc25416
[Model, Core] Support Granite Speech & LoRA for STT (#24455) 2025-11-05 08:33:48 +01:00
wangxiyuan
428bc7bf1c
[V0 deprecation] Remove VLLM_USE_V1 usage in most modules (#27955) 
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
 2025-11-04 20:51:16 -08:00
Chenheli Hua
1fb4217a05
[Multimodal] Make MediaConnector extensible. (#27759) 
Signed-off-by: Chenheli Hua <huachenheli@outlook.com>
 2025-11-04 18:28:01 +00:00
Chauncey
c02fccdbd2
[Refactor] Lazy import tool_parser (#27974) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-11-04 10:10:10 +08:00
Misha Efimov
ba464e6ae2
Add ORCA endpoint load metrics support (#24905) 
Signed-off-by: Misha Efimov <mef@google.com>
 2025-11-03 08:21:31 +00:00
Sungyoon Jeong
470ad118b6
[Frontend] Align finish_reason when tool is called with OpenAI (#25054) 
Signed-off-by: Sungyoon Jeong <sungyoon.jeong@furiosa.ai>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
 2025-11-03 04:21:18 +00:00
Vensen
0ce743f4e1
Fix(llm): Abort orphaned requests when llm.chat() batch fails Fixes #26081 (#27420) 
Signed-off-by: vensenmu <vensenmu@gmail.com>
 2025-11-02 16:24:01 +00:00
Benjamin Bartels
1e88fb751b
Adds anthropic /v1/messages endpoint to openai api_server (#27882) 
Signed-off-by: bbartels <benjamin@bartels.dev>
Signed-off-by: Benjamin Bartels <benjamin@bartels.dev>
 2025-11-01 12:45:42 -07:00
wenxindongwork
af6e19f50f
[Core][TPU] Support TPU Data Parallalism (#27365) 
Signed-off-by: wenxindongwork <wenxindong@google.com>
 2025-11-01 17:14:44 +00:00
Vinay R Damodaran
5e8862e9e0
[Feature] Pydantic validation for scheduler.py and structured_outputs.py (#26519) 
Signed-off-by: Vinay Damodaran <vrdn@hey.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-31 18:05:50 +00:00
Nick Hill
9e5bd3076e
[Cleanup] Remove no-longer-used SpeculativeConfig.enable_chunked_prefill (#27826) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-10-31 10:57:45 -07:00
Chenguang Zheng
103a468bbf
[bugfix] Missing cached item in beam search (#27874) 
Signed-off-by: fake0fan <645327136@qq.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
 2025-10-31 17:34:27 +00:00
cong-meta
a2981c4272
[EP/DP][API Server] Enable DP-aware routing in OpenAI API requests (#24945) 
Co-authored-by: Cong Chen <prowindy@gmail.com>
 2025-10-30 12:10:16 -07:00
wang.yuqi
4464723f22
[Frontend][Doc][5/N] Improve all pooling task | Polish encode (pooling) api & Document. (#25524) 
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-10-30 12:13:05 +00:00
Cyrus Leung
ecca3fee76
[Frontend] Add vllm bench sweep to CLI (#27639) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-10-29 05:59:48 -07:00
Alec S
ab2eb27b74
[Frontend] [gpt-oss] Mcp type bug (#27689) 
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
Signed-off-by: Alec Solder <alecs@fb.com>
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
Co-authored-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
Co-authored-by: Alec Solder <alecs@fb.com>
Co-authored-by: Ye (Charlotte) Qi <yeq@meta.com>
 2025-10-29 10:01:32 +00:00
Alec S
3c7fefdeba
[Frontend] [gpt-oss] Tool json call parsing error retry (#27675) 
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
Signed-off-by: Alec Solder <alecs@fb.com>
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
Co-authored-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
Co-authored-by: Alec Solder <alecs@fb.com>
Co-authored-by: Ye (Charlotte) Qi <yeq@meta.com>
 2025-10-29 09:42:44 +00:00
Junpu Fan
b186149e8e
[Bugfix][Frontend] validate arg priority in frontend LLM class before add request (#27596) 
Signed-off-by: Junpu Fan <junpufan@gmail.com>
 2025-10-28 14:02:43 +00:00
wangln19
446912d1cb
fix: allow HuggingFace standard chat template params via **kwargs (#27622) 
Signed-off-by: wangln19 <wanglinian@dev.wanglinian.msh-dev.svc.cluster.local>
Signed-off-by: wangln19 <96399074+wangln19@users.noreply.github.com>
Co-authored-by: wangln19 <wanglinian@dev.wanglinian.msh-dev.svc.cluster.local>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
 2025-10-28 21:12:34 +08:00
Cyrus Leung
f58d9b6404
[Misc] Separate out utils.counter and move utils.Device to engine (#27588) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-28 12:20:46 +00:00
Nick Hill
7a865f2325
[V0 Deprecation] Remove vestigial V0 logits_processors.py file (#27601) 
Signed-off-by: Nick Hill <nhill@redhat.com>
 2025-10-28 19:17:45 +08:00
wangln19
2fa90bda27
Fix a robust parsing issue in KimiK2ToolParser that causes IndexError (#27565) 
Signed-off-by: wangln19 <wanglinian@dev.wanglinian.msh-dev.svc.cluster.local>
Co-authored-by: wangln19 <wanglinian@dev.wanglinian.msh-dev.svc.cluster.local>
 2025-10-28 11:11:50 +00:00
Andrew Xia
53a56e658b
[gpt-oss][2/N] Support input_messages in responsesRequest (#26962) 
Signed-off-by: Andrew Xia <axia@fb.com>
Co-authored-by: Andrew Xia <axia@fb.com>
 2025-10-27 23:15:49 +00:00
Cyrus Leung
6ebffafbb6
[Misc] Clean up more utils (#27567) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-27 15:30:38 +00:00
Chauncey
a4fc21895e
[Bugfix] Fixed when return_token_ids=False, the first event still contains prompt_token_ids. (#27561) 
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
 2025-10-27 11:06:43 +00:00
Cyrus Leung
7c2bdb83dc
[Misc] Clean up utils (#27552) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-10-27 09:05:40 +00:00
Roger Young
720af6ab79
[Model][MiniMax-M2] Support MiniMax-M2 Model (#27535) 
Signed-off-by: xuebi <xuebi@minimaxi.com>
Co-authored-by: xuebi <xuebi@minimaxi.com>
 2025-10-27 00:59:11 +08:00
Yeshwanth N
71b1c8b667
[Chore]:Extract math and argparse utilities to separate modules (#27188) 
Signed-off-by: Yeshwanth Surya <yeshsurya@gmail.com>
Signed-off-by: Yeshwanth N <yeshsurya@gmail.com>
Signed-off-by: yeshsurya <yeshsurya@gmail.com>
 2025-10-26 04:03:32 -07:00