xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2025-12-09 05:34:55 +08:00
Code Issues Packages Projects Releases Wiki Activity
vllm/examples/online_serving/pooling/README.md
wang.yuqi 4464723f22
[Frontend][Doc][5/N] Improve all pooling task | Polish encode (pooling) api & Document. (#25524) 
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
2025-10-30 12:13:05 +00:00

2.1 KiB
Raw Blame History

Pooling models

Cohere rerank usage

# vllm serve BAAI/bge-reranker-base
python examples/online_serving/pooling/cohere_rerank_client.py

Embedding requests base64 encoding_format usage

# vllm serve intfloat/e5-small
python examples/online_serving/pooling/embedding_requests_base64_client.py

Embedding requests bytes encoding_format usage

# vllm serve intfloat/e5-small
python examples/online_serving/pooling/embedding_requests_bytes_client.py

Jinaai rerank usage

# vllm serve BAAI/bge-reranker-base
python examples/online_serving/pooling/jinaai_rerank_client.py

Multi vector retrieval usage

# vllm serve BAAI/bge-m3
python examples/online_serving/pooling/multi_vector_retrieval_client.py

Named Entity Recognition (NER) usage

# vllm serve boltuix/NeuroBERT-NER
python examples/online_serving/pooling/ner_client.py

OpenAI chat embedding for multimodal usage

python examples/online_serving/pooling/openai_chat_embedding_client_for_multimodal.py

OpenAI classification usage

# vllm serve jason9693/Qwen2.5-1.5B-apeach
python examples/online_serving/pooling/openai_classification_client.py

OpenAI cross_encoder score usage

# vllm serve BAAI/bge-reranker-v2-m3
python examples/online_serving/pooling/openai_cross_encoder_score.py

OpenAI cross_encoder score for multimodal usage

# vllm serve jinaai/jina-reranker-m0
python examples/online_serving/pooling/openai_cross_encoder_score_for_multimodal.py

OpenAI embedding usage

# vllm serve intfloat/e5-small
python examples/online_serving/pooling/openai_embedding_client.py

OpenAI embedding matryoshka dimensions usage

# vllm serve jinaai/jina-embeddings-v3 --trust-remote-code
python examples/online_serving/pooling/openai_embedding_matryoshka_fy.py

OpenAI pooling usage

# vllm serve internlm/internlm2-1_8b-reward --trust-remote-code
python examples/online_serving/pooling/openai_pooling_client.py

Online Prithvi Geospatial MAE usage

python examples/online_serving/pooling/prithvi_geospatial_mae.py