xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2026-05-16 09:49:08 +08:00
Code Issues Packages Projects Releases Wiki Activity

[Doc] Link to RFC for pooling optimizations (#21806)

Browse Source 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
This commit is contained in:
Cyrus Leung 2025-07-29 14:53:18 +08:00 committed by GitHub
parent 7234fe2685
commit a2480251ec
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 3 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
docs/models/pooling_models.md
View File

@ -7,9 +7,9 @@ These models use a [Pooler][vllm.model_executor.layers.pooler.Pooler] to extract
before returning them. before returning them.
!!! note !!! note
We currently support pooling models primarily as a matter of convenience. We currently support pooling models primarily as a matter of convenience. This is not guaranteed to have any performance improvement over using HF Transformers / Sentence Transformers directly.
As shown in the [Compatibility Matrix](../features/compatibility_matrix.md), most vLLM features are not applicable to
pooling models as they only work on the generation or decode stage, so performance may not improve as much. We are now planning to optimize pooling models in vLLM. Please comment on <gh-issue:21796> if you have any suggestions!
## Configuration ## Configuration