diff --git a/docs/contributing/README.md b/docs/contributing/README.md index 65ae9cc963676..10c50e0072434 100644 --- a/docs/contributing/README.md +++ b/docs/contributing/README.md @@ -130,7 +130,7 @@ pytest -s -v tests/test_logger.py If you encounter a bug or have a feature request, please [search existing issues](https://github.com/vllm-project/vllm/issues?q=is%3Aissue) first to see if it has already been reported. If not, please [file a new issue](https://github.com/vllm-project/vllm/issues/new/choose), providing as much relevant information as possible. -!!! warning +!!! important If you discover a security vulnerability, please follow the instructions [here](gh-file:SECURITY.md#reporting-a-vulnerability). ## Pull Requests & Code Reviews diff --git a/docs/contributing/model/multimodal.md b/docs/contributing/model/multimodal.md index 892ab9098407c..bed6d4e653d65 100644 --- a/docs/contributing/model/multimodal.md +++ b/docs/contributing/model/multimodal.md @@ -48,8 +48,8 @@ Further update the model as follows: return vision_embeddings ``` -!!! warning - The returned `multimodal_embeddings` must be either a **3D [torch.Tensor][]** of shape `(num_items, feature_size, hidden_size)`, or a **list / tuple of 2D [torch.Tensor][]'s** of shape `(feature_size, hidden_size)`, so that `multimodal_embeddings[i]` retrieves the embeddings generated from the `i`-th multimodal data item (e.g, image) of the request. +!!! important + The returned `multimodal_embeddings` must be either a **3D [torch.Tensor][]** of shape `(num_items, feature_size, hidden_size)`, or a **list / tuple of 2D [torch.Tensor][]'s** of shape `(feature_size, hidden_size)`, so that `multimodal_embeddings[i]` retrieves the embeddings generated from the `i`-th multimodal data item (e.g, image) of the request. - Implement [get_input_embeddings][vllm.model_executor.models.interfaces.SupportsMultiModal.get_input_embeddings] to merge `multimodal_embeddings` with text embeddings from the `input_ids`. If input processing for the model is implemented correctly (see sections below), then you can leverage the utility function we provide to easily merge the embeddings. @@ -100,8 +100,8 @@ Further update the model as follows: ``` !!! note - The model class does not have to be named `*ForCausalLM`. - Check out [the HuggingFace Transformers documentation](https://huggingface.co/docs/transformers/model_doc/auto#multimodal) for some examples. + The model class does not have to be named `*ForCausalLM`. + Check out [the HuggingFace Transformers documentation](https://huggingface.co/docs/transformers/model_doc/auto#multimodal) for some examples. ## 2. Specify processing information diff --git a/docs/contributing/model/registration.md b/docs/contributing/model/registration.md index 7a7bd79140585..a6dc1e32dfb95 100644 --- a/docs/contributing/model/registration.md +++ b/docs/contributing/model/registration.md @@ -18,7 +18,7 @@ After you have implemented your model (see [tutorial][new-model-basic]), put it Then, add your model class to `_VLLM_MODELS` in so that it is automatically registered upon importing vLLM. Finally, update our [list of supported models][supported-models] to promote your model! -!!! warning +!!! important The list of models in each section should be maintained in alphabetical order. ## Out-of-tree models @@ -49,6 +49,6 @@ def register(): ) ``` -!!! warning +!!! important If your model is a multimodal model, ensure the model class implements the [SupportsMultiModal][vllm.model_executor.models.interfaces.SupportsMultiModal] interface. Read more about that [here][supports-multimodal]. diff --git a/docs/contributing/model/tests.md b/docs/contributing/model/tests.md index 67f8eda61dc54..a8cb457453b91 100644 --- a/docs/contributing/model/tests.md +++ b/docs/contributing/model/tests.md @@ -15,7 +15,7 @@ Without them, the CI for your PR will fail. Include an example HuggingFace repository for your model in . This enables a unit test that loads dummy weights to ensure that the model can be initialized in vLLM. -!!! warning +!!! important The list of models in each section should be maintained in alphabetical order. !!! tip diff --git a/docs/design/multiprocessing.md b/docs/design/multiprocessing.md index 4d58fae20f06c..d2d8f73aed332 100644 --- a/docs/design/multiprocessing.md +++ b/docs/design/multiprocessing.md @@ -7,7 +7,7 @@ page for information on known issues and how to solve them. ## Introduction -!!! warning +!!! important The source code references are to the state of the code at the time of writing in December, 2024. The use of Python multiprocessing in vLLM is complicated by: diff --git a/docs/features/multimodal_inputs.md b/docs/features/multimodal_inputs.md index 19b6681729028..afb9a6d4df9ae 100644 --- a/docs/features/multimodal_inputs.md +++ b/docs/features/multimodal_inputs.md @@ -211,7 +211,7 @@ for o in outputs: Our OpenAI-compatible server accepts multi-modal data via the [Chat Completions API](https://platform.openai.com/docs/api-reference/chat). -!!! warning +!!! important A chat template is **required** to use Chat Completions API. For HF format models, the default chat template is defined inside `chat_template.json` or `tokenizer_config.json`. diff --git a/docs/getting_started/quickstart.md b/docs/getting_started/quickstart.md index d24e75e8141d8..38fc9925eb51c 100644 --- a/docs/getting_started/quickstart.md +++ b/docs/getting_started/quickstart.md @@ -61,7 +61,8 @@ from vllm import LLM, SamplingParams ``` The next section defines a list of input prompts and sampling parameters for text generation. The [sampling temperature](https://arxiv.org/html/2402.05201v1) is set to `0.8` and the [nucleus sampling probability](https://en.wikipedia.org/wiki/Top-p_sampling) is set to `0.95`. You can find more information about the sampling parameters [here][sampling-params]. -!!! warning + +!!! important By default, vLLM will use sampling parameters recommended by model creator by applying the `generation_config.json` from the Hugging Face model repository if it exists. In most cases, this will provide you with the best results by default if [SamplingParams][vllm.SamplingParams] is not specified. However, if vLLM's default sampling parameters are preferred, please set `generation_config="vllm"` when creating the [LLM][vllm.LLM] instance. @@ -116,7 +117,7 @@ vllm serve Qwen/Qwen2.5-1.5B-Instruct !!! note By default, the server uses a predefined chat template stored in the tokenizer. You can learn about overriding it [here][chat-template]. -!!! warning +!!! important By default, the server applies `generation_config.json` from the huggingface model repository if it exists. This means the default values of certain sampling parameters can be overridden by those recommended by the model creator. To disable this behavior, please pass `--generation-config vllm` when launching the server. diff --git a/docs/mkdocs/stylesheets/extra.css b/docs/mkdocs/stylesheets/extra.css index 088143ed59563..6f30d459d5f42 100644 --- a/docs/mkdocs/stylesheets/extra.css +++ b/docs/mkdocs/stylesheets/extra.css @@ -34,3 +34,40 @@ body[data-md-color-scheme="slate"] .md-nav__item--section > label.md-nav__link . color: rgba(255, 255, 255, 0.75) !important; font-weight: 700; } + +/* Custom admonitions */ +:root { + --md-admonition-icon--announcement: url('data:image/svg+xml;charset=utf-8,'); + --md-admonition-icon--important: url('data:image/svg+xml;charset=utf-8,'); +} + +.md-typeset .admonition.announcement, +.md-typeset details.announcement { + border-color: rgb(255, 110, 66); +} +.md-typeset .admonition.important, +.md-typeset details.important { + border-color: rgb(239, 85, 82); +} + +.md-typeset .announcement > .admonition-title, +.md-typeset .announcement > summary { + background-color: rgb(255, 110, 66, 0.1); +} +.md-typeset .important > .admonition-title, +.md-typeset .important > summary { + background-color: rgb(239, 85, 82, 0.1); +} + +.md-typeset .announcement > .admonition-title::before, +.md-typeset .announcement > summary::before { + background-color: rgb(239, 85, 82); + -webkit-mask-image: var(--md-admonition-icon--announcement); + mask-image: var(--md-admonition-icon--announcement); +} +.md-typeset .important > .admonition-title::before, +.md-typeset .important > summary::before { + background-color: rgb(239, 85, 82); + -webkit-mask-image: var(--md-admonition-icon--important); + mask-image: var(--md-admonition-icon--important); +} diff --git a/docs/models/generative_models.md b/docs/models/generative_models.md index 566b1c29fca9f..e52c5ae01cb8a 100644 --- a/docs/models/generative_models.md +++ b/docs/models/generative_models.md @@ -51,7 +51,7 @@ for output in outputs: print(f"Prompt: {prompt!r}, Generated text: {generated_text!r}") ``` -!!! warning +!!! important By default, vLLM will use sampling parameters recommended by model creator by applying the `generation_config.json` from the huggingface model repository if it exists. In most cases, this will provide you with the best results by default if [SamplingParams][vllm.SamplingParams] is not specified. However, if vLLM's default sampling parameters are preferred, please pass `generation_config="vllm"` when creating the [LLM][vllm.LLM] instance. @@ -81,7 +81,7 @@ The [chat][vllm.LLM.chat] method implements chat functionality on top of [genera In particular, it accepts input similar to [OpenAI Chat Completions API](https://platform.openai.com/docs/api-reference/chat) and automatically applies the model's [chat template](https://huggingface.co/docs/transformers/en/chat_templating) to format the prompt. -!!! warning +!!! important In general, only instruction-tuned models have a chat template. Base models may perform poorly as they are not trained to respond to the chat conversation. diff --git a/docs/models/supported_models.md b/docs/models/supported_models.md index a66199a539762..69122b664ba7e 100644 --- a/docs/models/supported_models.md +++ b/docs/models/supported_models.md @@ -379,7 +379,7 @@ Specified using `--task generate`. See [this page](./pooling_models.md) for more information on how to use pooling models. -!!! warning +!!! important Since some model architectures support both generative and pooling tasks, you should explicitly specify the task type to ensure that the model is used in pooling mode instead of generative mode. @@ -432,7 +432,7 @@ Specified using `--task reward`. If your model is not in the above list, we will try to automatically convert the model using [as_reward_model][vllm.model_executor.models.adapters.as_reward_model]. By default, we return the hidden states of each token directly. -!!! warning +!!! important For process-supervised reward models such as `peiyi9979/math-shepherd-mistral-7b-prm`, the pooling config should be set explicitly, e.g.: `--override-pooler-config '{"pooling_type": "STEP", "step_tag_id": 123, "returned_token_ids": [456, 789]}'`. @@ -485,7 +485,7 @@ On the other hand, modalities separated by `/` are mutually exclusive. See [this page][multimodal-inputs] on how to pass multi-modal inputs to the model. -!!! warning +!!! important **To enable multiple multi-modal items per text prompt in vLLM V0**, you have to set `limit_mm_per_prompt` (offline inference) or `--limit-mm-per-prompt` (online serving). For example, to enable passing up to 4 images per text prompt: @@ -640,7 +640,7 @@ Specified using `--task generate`. See [this page](./pooling_models.md) for more information on how to use pooling models. -!!! warning +!!! important Since some model architectures support both generative and pooling tasks, you should explicitly specify the task type to ensure that the model is used in pooling mode instead of generative mode. diff --git a/docs/serving/openai_compatible_server.md b/docs/serving/openai_compatible_server.md index c2e39d029dd5a..3002b2f92e4d5 100644 --- a/docs/serving/openai_compatible_server.md +++ b/docs/serving/openai_compatible_server.md @@ -36,7 +36,7 @@ print(completion.choices[0].message) vLLM supports some parameters that are not supported by OpenAI, `top_k` for example. You can pass these parameters to vLLM using the OpenAI client in the `extra_body` parameter of your requests, i.e. `extra_body={"top_k": 50}` for `top_k`. -!!! warning +!!! important By default, the server applies `generation_config.json` from the Hugging Face model repository if it exists. This means the default values of certain sampling parameters can be overridden by those recommended by the model creator. To disable this behavior, please pass `--generation-config vllm` when launching the server. @@ -250,7 +250,7 @@ and passing a list of `messages` in the request. Refer to the examples below for --chat-template examples/template_vlm2vec.jinja ``` - !!! warning + !!! important Since VLM2Vec has the same model architecture as Phi-3.5-Vision, we have to explicitly pass `--task embed` to run this model in embedding mode instead of text generation mode. @@ -294,13 +294,13 @@ and passing a list of `messages` in the request. Refer to the examples below for --chat-template examples/template_dse_qwen2_vl.jinja ``` - !!! warning + !!! important Like with VLM2Vec, we have to explicitly pass `--task embed`. Additionally, `MrLight/dse-qwen2-2b-mrl-v1` requires an EOS token for embeddings, which is handled by a custom chat template: - !!! warning + !!! important `MrLight/dse-qwen2-2b-mrl-v1` requires a placeholder image of the minimum image size for text query embeddings. See the full code example below for details. diff --git a/docs/usage/v1_guide.md b/docs/usage/v1_guide.md index ca27f4dfc8ef7..c89c21d8575ab 100644 --- a/docs/usage/v1_guide.md +++ b/docs/usage/v1_guide.md @@ -1,6 +1,6 @@ # vLLM V1 -!!! important +!!! announcement We have started the process of deprecating V0. Please read [RFC #18571](https://github.com/vllm-project/vllm/issues/18571) for more details.