xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2026-03-18 14:57:21 +08:00
Code Issues Packages Projects Releases Wiki Activity

[Doc]: Improve feature tables (#13224)

Browse Source 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
This commit is contained in:
Harry Mellor 2025-02-18 10:52:39 +00:00 committed by GitHub
parent 8cf97f8661
commit 2358ca527b
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
5 changed files with 143 additions and 129 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
docs/source/_static/custom.css Normal file
View File

@ -0,0 +1,8 @@
.vertical-table-header th.head:not(.stub) {
writing-mode: sideways-lr;
white-space: nowrap;
max-width: 0;
p {
margin: 0;
}
}

9
docs/source/conf.py
View File

@ -78,8 +78,12 @@ html_theme_options = {
'use_repository_button': True,
'use_edit_page_button': True,
}
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ["_static"]
html_js_files = ["custom.js"]
html_css_files = ["custom.css"]
myst_url_schemes = {
'http': None,
@ -121,11 +125,6 @@ if READTHEDOCS_VERSION_TYPE == "tag":
if os.path.exists(header_file):
os.remove(header_file)
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
# html_static_path = ['_static']
# Generate additional rst documentation here.
def setup(app):

153
docs/source/features/compatibility_matrix.md
View File

@ -4,8 +4,14 @@
The tables below show mutually exclusive features and the support on some hardware.
The symbols used have the following meanings:
- ✅ = Full compatibility
- 🟠 = Partial compatibility
- ❌ = No compatibility
:::{note}
Check the '✗' with links to see tracking issue for unsupported feature/hardware combination.
Check the ❌ or 🟠 with links to see tracking issue for unsupported feature/hardware combination.
:::
## Feature x Feature
@ -29,6 +35,7 @@ Check the '✗' with links to see tracking issue for unsupported feature/hardwar
:header-rows: 1
:stub-columns: 1
:widths: auto
:class: vertical-table-header
- * Feature
* [CP](#chunked-prefill)
@ -48,7 +55,7 @@ Check the '✗' with links to see tracking issue for unsupported feature/hardwar
* beam-search
* <abbr title="Guided Decoding">guided dec</abbr>
- * [CP](#chunked-prefill)
*
* ✅
*
*
*
@ -66,7 +73,7 @@ Check the '✗' with links to see tracking issue for unsupported feature/hardwar
*
- * [APC](#automatic-prefix-caching)
* ✅
*
* ✅
*
*
*
@ -82,9 +89,9 @@ Check the '✗' with links to see tracking issue for unsupported feature/hardwar
*
*
- * [LoRA](#lora-adapter)
* [✗](gh-pr:9057)
* ✅
*
* ✅
* ✅
*
*
*
@ -102,7 +109,7 @@ Check the '✗' with links to see tracking issue for unsupported feature/hardwar
* ✅
* ✅
* ✅
*
* ✅
*
*
*
@ -118,9 +125,9 @@ Check the '✗' with links to see tracking issue for unsupported feature/hardwar
- * [SD](#spec_decode)
* ✅
* ✅
* ✗
* ❌
* ✅
* ✅
*
*
*
*
@ -138,7 +145,7 @@ Check the '✗' with links to see tracking issue for unsupported feature/hardwar
* ✅
* ✅
* ✅
*
* ✅
*
*
*
@ -150,13 +157,13 @@ Check the '✗' with links to see tracking issue for unsupported feature/hardwar
*
*
- * <abbr title="Pooling Models">pooling</abbr>
* ✗
* ✗
* ✗
* ✗
* ✗
* ✗
*
* ❌
* ❌
* ❌
* ❌
* ❌
* ❌
* ✅
*
*
*
@ -167,14 +174,14 @@ Check the '✗' with links to see tracking issue for unsupported feature/hardwar
*
*
- * <abbr title="Encoder-Decoder Models">enc-dec</abbr>
* ✗
* [✗](gh-issue:7366)
* ✗
* ✗
* [✗](gh-issue:7366)
* ❌
* [❌](gh-issue:7366)
* ❌
* ❌
* [❌](gh-issue:7366)
* ✅
* ✅
* ✅
*
*
*
*
@ -190,9 +197,9 @@ Check the '✗' with links to see tracking issue for unsupported feature/hardwar
* ✅
* ✅
* ✅
* ✗
* ❌
* ✅
* ✅
*
*
*
*
@ -205,12 +212,12 @@ Check the '✗' with links to see tracking issue for unsupported feature/hardwar
* ✅
* ✅
* ✅
* [✗](gh-pr:8199)
* ✅
* ✗
* ✅
* ✅
*
* ❌
* ✅
* ✅
* ✅
*
*
*
@ -222,49 +229,49 @@ Check the '✗' with links to see tracking issue for unsupported feature/hardwar
* ✅
* ✅
* ✅
* ✗
* ❌
* ✅
* ❌
* ❌
* ✅
* ✗
* ✗
* ✅
* ✅
*
*
*
*
*
*
- * multi-step
* ✗
* ❌
* ✅
* ✗
* ❌
* ✅
* ❌
* ✅
* ❌
* ❌
* ✅
* ✗
* ✅
* ✗
* ✗
* ✅
* [✗](gh-issue:8198)
* ✅
*
*
*
*
*
- * <abbr title="Multimodal Inputs">mm</abbr>
* ✅
* [✗](gh-pr:8348)
* [✗](gh-pr:7199)
* ?
* ?
* [🟠](gh-pr:8348)
* [🟠](gh-pr:4194)
* ❔
* ❔
* ✅
* ✅
* ✅
* ✅
* ✅
* ✅
* ?
*
* ❔
* ✅
*
*
*
@ -273,16 +280,16 @@ Check the '✗' with links to see tracking issue for unsupported feature/hardwar
* ✅
* ✅
* ✅
* [✗](gh-issue:6137)
* [❌](gh-issue:6137)
* ✅
* ✗
* ❌
* ✅
* ✅
* ✅
* ?
* [✗](gh-issue:7968)
* ❔
* [❌](gh-issue:7968)
* ✅
* ✅
*
*
*
- * beam-search
@ -290,35 +297,35 @@ Check the '✗' with links to see tracking issue for unsupported feature/hardwar
* ✅
* ✅
* ✅
* [✗](gh-issue:6137)
* [❌](gh-issue:6137)
* ✅
* ✗
* ❌
* ✅
* ✅
* ✅
* ?
* [✗](gh-issue:7968)
* ?
* ❔
* [❌](gh-issue:7968)
* ❔
* ✅
* ✅
*
*
- * <abbr title="Guided Decoding">guided dec</abbr>
* ✅
* ✅
* ?
* ?
* [✗](gh-issue:11484)
* ❔
* ❔
* [❌](gh-issue:11484)
* ✅
* ✗
* ?
* ❌
* ❔
* ✅
* ✅
* ✅
* [✗](gh-issue:9893)
* ?
* [❌](gh-issue:9893)
* ❔
* ✅
* ✅
* ✅
*
:::
(feature-x-hardware)=
@ -339,7 +346,7 @@ Check the '✗' with links to see tracking issue for unsupported feature/hardwar
* CPU
* AMD
- * [CP](#chunked-prefill)
* [✗](gh-issue:2729)
* [❌](gh-issue:2729)
* ✅
* ✅
* ✅
@ -347,7 +354,7 @@ Check the '✗' with links to see tracking issue for unsupported feature/hardwar
* ✅
* ✅
- * [APC](#automatic-prefix-caching)
* [✗](gh-issue:3687)
* [❌](gh-issue:3687)
* ✅
* ✅
* ✅
@ -368,7 +375,7 @@ Check the '✗' with links to see tracking issue for unsupported feature/hardwar
* ✅
* ✅
* ✅
* [✗](gh-issue:8475)
* [❌](gh-issue:8475)
* ✅
- * [SD](#spec_decode)
* ✅
@ -384,7 +391,7 @@ Check the '✗' with links to see tracking issue for unsupported feature/hardwar
* ✅
* ✅
* ✅
* ✗
* ❌
* ✅
- * <abbr title="Pooling Models">pooling</abbr>
* ✅
@ -393,7 +400,7 @@ Check the '✗' with links to see tracking issue for unsupported feature/hardwar
* ✅
* ✅
* ✅
* ?
* ❔
- * <abbr title="Encoder-Decoder Models">enc-dec</abbr>
* ✅
* ✅
@ -401,7 +408,7 @@ Check the '✗' with links to see tracking issue for unsupported feature/hardwar
* ✅
* ✅
* ✅
* ✗
* ❌
- * <abbr title="Multimodal Inputs">mm</abbr>
* ✅
* ✅
@ -432,15 +439,15 @@ Check the '✗' with links to see tracking issue for unsupported feature/hardwar
* ✅
* ✅
* ✅
* ✗
* ✗
* ❌
* ❌
- * multi-step
* ✅
* ✅
* ✅
* ✅
* ✅
* [✗](gh-issue:8477)
* [❌](gh-issue:8477)
* ✅
- * best-of
* ✅

94
docs/source/features/quantization/supported_hardware.md
View File

@ -20,93 +20,93 @@ The table below shows the compatibility of various quantization implementations
* AWS Inferentia
* Google TPU
- * AWQ
* ✗
* ❌
* ✅︎
* ✅︎
* ✅︎
* ✅︎
* ✗
* ❌
* ✅︎
* ✅︎
* ✗
* ✗
* ❌
* ❌
- * GPTQ
* ✅︎
* ✅︎
* ✅︎
* ✅︎
* ✅︎
* ✗
* ❌
* ✅︎
* ✅︎
* ✗
* ✗
* ❌
* ❌
- * Marlin (GPTQ/AWQ/FP8)
* ✗
* ✗
* ❌
* ❌
* ✅︎
* ✅︎
* ✅︎
* ✗
* ✗
* ✗
* ✗
* ✗
* ❌
* ❌
* ❌
* ❌
* ❌
- * INT8 (W8A8)
* ✗
* ❌
* ✅︎
* ✅︎
* ✅︎
* ✅︎
* ✗
* ✗
* ❌
* ❌
* ✅︎
* ✗
* ✗
* ❌
* ❌
- * FP8 (W8A8)
* ✗
* ✗
* ✗
* ❌
* ❌
* ❌
* ✅︎
* ✅︎
* ✅︎
* ✗
* ✗
* ✗
* ✗
* ❌
* ❌
* ❌
* ❌
- * AQLM
* ✅︎
* ✅︎
* ✅︎
* ✅︎
* ✅︎
* ✗
* ✗
* ✗
* ✗
* ✗
* ❌
* ❌
* ❌
* ❌
* ❌
- * bitsandbytes
* ✅︎
* ✅︎
* ✅︎
* ✅︎
* ✅︎
* ✗
* ✗
* ✗
* ✗
* ✗
* ❌
* ❌
* ❌
* ❌
* ❌
- * DeepSpeedFP
* ✅︎
* ✅︎
* ✅︎
* ✅︎
* ✅︎
* ✗
* ✗
* ✗
* ✗
* ✗
* ❌
* ❌
* ❌
* ❌
* ❌
- * GGUF
* ✅︎
* ✅︎
@ -114,16 +114,16 @@ The table below shows the compatibility of various quantization implementations
* ✅︎
* ✅︎
* ✅︎
* ✗
* ✗
* ✗
* ✗
* ❌
* ❌
* ❌
* ❌
:::
- Volta refers to SM 7.0, Turing to SM 7.5, Ampere to SM 8.0/8.6, Ada to SM 8.9, and Hopper to SM 9.0.
- "✅︎" indicates that the quantization method is supported on the specified hardware.
- "✗" indicates that the quantization method is not supported on the specified hardware.
- ✅︎ indicates that the quantization method is supported on the specified hardware.
- ❌ indicates that the quantization method is not supported on the specified hardware.
:::{note}
This compatibility chart is subject to change as vLLM continues to evolve and expand its support for different hardware platforms and quantization methods.

8
docs/source/models/pooling_models.md
View File

@ -28,10 +28,10 @@ The selected option sets the default pooler used to extract the final hidden sta
- * Embedding (`embed`)
* `LAST`
* ✅︎
* ✗
* ❌
- * Classification (`classify`)
* `LAST`
* ✗
* ❌
* ✅︎
- * Sentence Pair Scoring (`score`)
* \*
@ -39,8 +39,8 @@ The selected option sets the default pooler used to extract the final hidden sta
* \*
- * Reward Modeling (`reward`)
* `ALL`
* ✗
* ✗
* ❌
* ❌
:::
\*The default pooler is always defined by the model.