Robert Shaw
|
0c0c2015c5
|
Update openai_compatible_server.md (#11536)
Co-authored-by: Simon Mo <simon.mo@hey.com>
|
2024-12-26 16:26:18 -08:00 |
|
Simon Mo
|
82d24f7aac
|
[Docs] Document Deepseek V3 support (#11535)
Signed-off-by: simon-mo <simon.mo@hey.com>
|
2024-12-26 16:21:56 -08:00 |
|
Simon Mo
|
f49777ba62
|
Deepseek v3 (#11502)
Signed-off-by: mgoin <michael@neuralmagic.com>
Co-authored-by: mgoin <michael@neuralmagic.com>
Co-authored-by: robertgshaw2-neuralmagic <rshaw@neuralmagic.com>
v0.6.6
|
2024-12-26 16:09:44 -08:00 |
|
Robert Shaw
|
55fb97f7bd
|
[2/N] API Server: Avoid ulimit footgun (#11530)
|
2024-12-26 23:43:05 +00:00 |
|
Michael Goin
|
2072924d14
|
[Model] [Quantization] Support deepseek_v3 w8a8 fp8 block-wise quantization (#11523)
Signed-off-by: mgoin <michael@neuralmagic.com>
Signed-off-by: simon-mo <simon.mo@hey.com>
Signed-off-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: simon-mo <simon.mo@hey.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: HandH1998 <1335248067@qq.com>
|
2024-12-26 15:33:30 -08:00 |
|
Robert Shaw
|
720b10fdc6
|
[1/N] API Server (Remove Proxy) (#11529)
|
2024-12-26 23:03:43 +00:00 |
|
Isotr0py
|
b85a977822
|
[Doc] Add video example to openai client for multimodal (#11521)
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
|
2024-12-26 17:31:29 +00:00 |
|
Cyrus Leung
|
eec906d811
|
[Misc] Add placeholder module (#11501)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2024-12-26 13:12:51 +00:00 |
|
Jee Jee Li
|
f57ee5650d
|
[Model] Modify MolmoForCausalLM MLP (#11510)
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
|
2024-12-26 13:12:05 +00:00 |
|
sroy745
|
dcb1a944d4
|
[V1] Adding min tokens/repetition/presence/frequence penalties to V1 sampler (#10681)
Signed-off-by: Sourashis Roy <sroy@roblox.com>
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2024-12-26 19:02:58 +09:00 |
|
Roger Wang
|
7492a36207
|
[Doc] Add QVQ and QwQ to the list of supported models (#11509)
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
|
2024-12-26 09:44:32 +00:00 |
|
Jee Jee Li
|
aa25985bd1
|
[Misc][LoRA] Fix LoRA weight mapper (#11495)
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
|
2024-12-26 15:52:48 +08:00 |
|
Lucas Tucker
|
dbeac95dbb
|
Mypy checking for vllm/compilation (#11496)
Signed-off-by: lucast2021 <lucast2021@headroyce.org>
Co-authored-by: lucast2021 <lucast2021@headroyce.org>
|
2024-12-26 05:04:07 +00:00 |
|
Cyrus Leung
|
51a624bf02
|
[Misc] Move some multimodal utils to modality-specific modules (#11494)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2024-12-26 04:23:20 +00:00 |
|
Cyrus Leung
|
6ad909fdda
|
[Doc] Improve GitHub links (#11491)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2024-12-25 14:49:26 -08:00 |
|
Cyrus Leung
|
b689ada91e
|
[Frontend] Enable decord to load video from base64 (#11492)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2024-12-25 16:33:55 +00:00 |
|
Jiaxin Shan
|
fc601665eb
|
[Misc] Update disaggregation benchmark scripts and test logs (#11456)
Signed-off-by: Jiaxin Shan <seedjeffwan@gmail.com>
|
2024-12-25 06:58:48 +00:00 |
|
Rui Qiao
|
9832e5572a
|
[V1] Unify VLLM_ENABLE_V1_MULTIPROCESSING handling in RayExecutor (#11472)
|
2024-12-24 19:49:46 -08:00 |
|
Cyrus Leung
|
3f3e92e1f2
|
[Model] Automatic conversion of classification and reward models (#11469)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2024-12-24 18:22:22 +00:00 |
|
Yuan Tang
|
409475a827
|
[Bugfix] Fix issues in CPU build Dockerfile. Fixes #9182 (#11435)
Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
|
2024-12-24 16:53:28 +00:00 |
|
Jee Jee Li
|
196c34b0ac
|
[Misc] Move weights mapper (#11443)
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
|
2024-12-24 13:05:25 +00:00 |
|
Mengqing Cao
|
5c7963249d
|
[attn][tiny fix] fix attn backend in MultiHeadAttention (#11463)
Signed-off-by: Mengqing Cao <cmq0113@163.com>
|
2024-12-24 12:39:36 +00:00 |
|
Ilya Lavrenov
|
461cde2080
|
[OpenVINO] Fixed installation conflicts (#11458)
Signed-off-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
|
2024-12-24 11:38:21 +00:00 |
|
Isotr0py
|
7a5286cc04
|
[Bugfix][Hardware][CPU] Fix CPU input_positions creation for text-only inputs with mrope (#11434)
Signed-off-by: Isotr0py <2037008807@qq.com>
|
2024-12-24 17:59:51 +08:00 |
|
Jee Jee Li
|
b1b1038fbd
|
[Bugfix] Fix Qwen2-VL LoRA weight loading (#11430)
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
|
2024-12-24 09:56:10 +00:00 |
|
Cyrus Leung
|
9edca6bf8f
|
[Frontend] Online Pooling API (#11457)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2024-12-24 17:54:30 +08:00 |
|
dpxa
|
4f074fbf53
|
[Misc]Suppress irrelevant exception stack trace information when CUDA… (#11438)
Co-authored-by: shiquan <shiquan>
|
2024-12-24 08:43:39 +00:00 |
|
Rui Qiao
|
a491d6f535
|
[V1] TP Ray executor (#11107)
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
|
2024-12-23 23:00:12 +00:00 |
|
Rafael Vasquez
|
32aa2059ad
|
[Docs] Convert rST to MyST (Markdown) (#11145)
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
|
2024-12-23 22:35:38 +00:00 |
|
yansh97
|
94d545a1a1
|
[Doc] Fix typo in the help message of '--guided-decoding-backend' (#11440)
|
2024-12-23 20:20:44 +00:00 |
|
Michael Goin
|
60fb4f3bcf
|
[Bugfix] Add kv cache scales to gemma2.py (#11269)
|
2024-12-23 19:30:45 +00:00 |
|
Michael Goin
|
63afbe9215
|
[CI] Expand OpenAI test_chat.py guided decoding tests (#11048)
Signed-off-by: mgoin <michael@neuralmagic.com>
|
2024-12-23 18:35:38 +00:00 |
|
Dipika Sikka
|
8cef6e02dc
|
[Misc] add w8a8 asym models (#11075)
|
2024-12-23 13:33:20 -05:00 |
|
Dipika Sikka
|
b866cdbd05
|
[Misc] Add assertion and helpful message for marlin24 compressed models (#11388)
|
2024-12-24 02:23:38 +08:00 |
|
Yuan Tang
|
2e726680b3
|
[Bugfix] torch nightly version in ROCm installation guide (#11423)
Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
|
2024-12-23 17:20:22 +00:00 |
|
Michael Goin
|
5bfb30a529
|
[Bugfix] Fix CFGGuide and use outlines for grammars that can't convert to GBNF (#11389)
Signed-off-by: mgoin <michael@neuralmagic.com>
|
2024-12-23 23:06:20 +08:00 |
|
Lucas Tucker
|
e51719ae72
|
mypy type checking for vllm/worker (#11418)
Signed-off-by: lucast2021 <lucast2021@headroyce.org>
Co-authored-by: lucast2021 <lucast2021@headroyce.org>
|
2024-12-23 13:55:49 +00:00 |
|
youkaichao
|
f30581c518
|
[misc][perf] remove old code (#11425)
Signed-off-by: youkaichao <youkaichao@gmail.com>
|
2024-12-23 08:01:08 +00:00 |
|
Simon Mo
|
048fc57a0f
|
[CI] Unboock H100 Benchmark (#11419)
Signed-off-by: simon-mo <simon.mo@hey.com>
|
2024-12-22 14:17:43 -08:00 |
|
Jason T. Greene
|
f1d1bf6288
|
[Bugfix] Fix fully sharded LoRAs with Mixtral (#11390)
Signed-off-by: Jason Greene <jason.greene@redhat.com>
|
2024-12-22 23:25:10 +08:00 |
|
youkaichao
|
72d9c316d3
|
[cd][release] fix race conditions (#11407)
Signed-off-by: youkaichao <youkaichao@gmail.com>
|
2024-12-22 00:39:11 -08:00 |
|
youkaichao
|
4a9139780a
|
[cd][release] add pypi index for every commit and nightly build (#11404)
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
|
2024-12-21 23:53:44 -08:00 |
|
Roger Wang
|
29c748930e
|
[CI] Fix flaky entrypoint tests (#11403)
Signed-off-by: Roger Wang <ywang@roblox.com>
|
2024-12-21 21:08:44 -08:00 |
|
Roger Wang
|
c2d1b075ba
|
[Bugfix] Fix issues for Pixtral-Large-Instruct-2411 (#11393)
Signed-off-by: ywang96 <ywang@example.com>
Co-authored-by: ywang96 <ywang@example.com>
|
2024-12-21 10:15:03 +00:00 |
|
Ricky Xu
|
584f0ae40d
|
[V1] Make AsyncLLMEngine v1-v0 opaque (#11383)
Signed-off-by: Ricky Xu <xuchen727@hotmail.com>
|
2024-12-21 15:14:08 +08:00 |
|
George
|
51ff216d85
|
[Bugfix] update should_ignore_layer (#11354)
Signed-off-by: George Ohashi <george@neuralmagic.com>
|
2024-12-21 06:36:23 +00:00 |
|
Woosuk Kwon
|
dd2b5633dd
|
[V1][Bugfix] Skip hashing empty or None mm_data (#11386)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
2024-12-21 14:22:21 +09:00 |
|
Jiaxin Shan
|
47a0b615b4
|
Add ray[default] to wget to run distributed inference out of box (#11265)
Signed-off-by: Jiaxin Shan <seedjeffwan@gmail.com>
|
2024-12-20 13:54:55 -08:00 |
|
youkaichao
|
5d2248d81a
|
[doc] explain nccl requirements for rlhf (#11381)
Signed-off-by: youkaichao <youkaichao@gmail.com>
|
2024-12-20 13:00:56 -08:00 |
|
Michael Goin
|
d573aeadcc
|
[Bugfix] Don't log OpenAI field aliases as ignored (#11378)
Signed-off-by: mgoin <michael@neuralmagic.com>
|
2024-12-20 19:03:50 +00:00 |
|