Lingfan Yu 33170081f1

[Neuron][Kernel] Vectorize KV cache load in FlashPagedAttention to maximize DMA bandwidth (#13245) ...

Signed-off-by: Lingfan Yu <lingfany@amazon.com>

2025-02-20 17:45:45 -08:00

..

async_engine

[Misc] Add SPDX-License-Identifier headers to python source files (#12628)

2025-02-02 11:58:18 -08:00

basic_correctness

[2/n][ci] S3: Use full model path (#13564)

2025-02-20 01:20:15 -08:00

compile

Consolidate Llama model usage in tests (#13094)

2025-02-13 22:18:03 -08:00

core

[Core] Reduce TTFT with concurrent partial prefills (#10235)

2025-02-14 15:36:07 -08:00

data

[Bugfix] Fix load config when using bools (#9533)

2024-10-27 13:46:41 -04:00

distributed

[Bugfix] Fix 2 Node and Spec Decode tests (#13341)

2025-02-16 22:20:22 +08:00

encoder_decoder

[Misc] Add SPDX-License-Identifier headers to python source files (#12628)

2025-02-02 11:58:18 -08:00

engine

[2/n][ci] S3: Use full model path (#13564)

2025-02-20 01:20:15 -08:00

entrypoints

[Frontend] Add backend-specific options for guided decoding (#13505)

2025-02-20 15:07:58 -05:00

kernels

[Bugfix] Fix invalid rotary embedding unit test (#13431)

2025-02-18 11:52:03 +00:00

kv_transfer

Consolidate Llama model usage in tests (#13094)

2025-02-13 22:18:03 -08:00

lora

[Misc] Qwen2.5 VL support LoRA (#13261)

2025-02-19 18:37:55 -08:00

metrics

[1/n][CI] Load models in CI from S3 instead of HF (#13205)

2025-02-19 07:34:59 +00:00

mistral_tool_use

[Frontend] Generate valid tool call IDs when using tokenizer-mode=mistral (#12332)

2025-02-12 08:29:56 -08:00

model_executor

[Frontend] Add backend-specific options for guided decoding (#13505)

2025-02-20 15:07:58 -05:00

models

[VLM][Bugfix] Pass processor kwargs properly on init (#13516)

2025-02-19 13:13:50 +00:00

mq_llm_engine

[1/n][CI] Load models in CI from S3 instead of HF (#13205)

2025-02-19 07:34:59 +00:00

multi_step

[Misc] Add SPDX-License-Identifier headers to python source files (#12628)

2025-02-02 11:58:18 -08:00

multimodal

[VLM][Bugfix] Pass processor kwargs properly on init (#13516)

2025-02-19 13:13:50 +00:00

neuron

[Neuron][Kernel] Vectorize KV cache load in FlashPagedAttention to maximize DMA bandwidth (#13245)

2025-02-20 17:45:45 -08:00

plugins

[Misc] Add SPDX-License-Identifier headers to python source files (#12628)

2025-02-02 11:58:18 -08:00

plugins_tests

Merge similar examples in offline_inference into single basic example (#12737)

2025-02-20 04:53:51 -08:00

prefix_caching

[Misc] Add SPDX-License-Identifier headers to python source files (#12628)

2025-02-02 11:58:18 -08:00

prompt_adapter

[Misc] Add SPDX-License-Identifier headers to python source files (#12628)

2025-02-02 11:58:18 -08:00

prompts

[BugFix] Fix input positions for long context with sliding window (#2088)

2023-12-13 12:28:13 -08:00

quantization

Consolidate Llama model usage in tests (#13094)

2025-02-13 22:18:03 -08:00

runai_model_streamer_test

[1/n][CI] Load models in CI from S3 instead of HF (#13205)

2025-02-19 07:34:59 +00:00

samplers

[1/n][CI] Load models in CI from S3 instead of HF (#13205)

2025-02-19 07:34:59 +00:00

spec_decode

[Model][Speculative Decoding] DeepSeek MTP spec decode (#12755)

2025-02-19 17:06:23 +08:00

standalone_tests

make sure mistral_common not imported for non-mistral models (#12669)

2025-02-03 13:40:25 +08:00

system_messages

[V1] Implement Cascade Attention (#11635)

2025-01-01 21:56:46 +09:00

tensorizer_loader

[Misc] Add SPDX-License-Identifier headers to python source files (#12628)

2025-02-02 11:58:18 -08:00

tokenization

Consolidate Llama model usage in tests (#13094)

2025-02-13 22:18:03 -08:00

tool_use

[Misc] Add SPDX-License-Identifier headers to python source files (#12628)

2025-02-02 11:58:18 -08:00

tpu

[Misc] Add SPDX-License-Identifier headers to python source files (#12628)

2025-02-02 11:58:18 -08:00

tracing

[Misc] Add SPDX-License-Identifier headers to python source files (#12628)

2025-02-02 11:58:18 -08:00

v1

[V1][Core] Generic mechanism for handling engine utility (#13060)

2025-02-19 17:09:22 +08:00

vllm_test_utils

[Misc] Add SPDX-License-Identifier headers to python source files (#12628)

2025-02-02 11:58:18 -08:00

weight_loading

[Quant][Perf] Use moe_wna16 kernel by default for MoEs with many experts (#13236)

2025-02-14 12:53:42 -08:00

worker

[1/n][CI] Load models in CI from S3 instead of HF (#13205)

2025-02-19 07:34:59 +00:00

__init__.py

[Small] Formatter only checks lints in changed files (#1528)

2023-10-31 15:39:38 -07:00

conftest.py

[2/n][ci] S3: Use full model path (#13564)

2025-02-20 01:20:15 -08:00

test_cache_block_hashing.py

[Misc] Add SPDX-License-Identifier headers to python source files (#12628)

2025-02-02 11:58:18 -08:00

test_config.py

[2/n][ci] S3: Use full model path (#13564)

2025-02-20 01:20:15 -08:00

test_embedded_commit.py

[Misc] Add SPDX-License-Identifier headers to python source files (#12628)

2025-02-02 11:58:18 -08:00

test_inputs.py

[Misc] Add SPDX-License-Identifier headers to python source files (#12628)

2025-02-02 11:58:18 -08:00

test_logger.py

[Misc] Add SPDX-License-Identifier headers to python source files (#12628)

2025-02-02 11:58:18 -08:00

test_logits_processor.py

[Misc] Add SPDX-License-Identifier headers to python source files (#12628)

2025-02-02 11:58:18 -08:00

test_regression.py

[2/n][ci] S3: Use full model path (#13564)

2025-02-20 01:20:15 -08:00

test_sampling_params.py

[Misc] Add SPDX-License-Identifier headers to python source files (#12628)

2025-02-02 11:58:18 -08:00

test_scalartype.py

[NVIDIA] Support nvfp4 quantization (#12784)

2025-02-12 19:51:51 -08:00

test_seed_behavior.py

[Bugfix] fix flaky test (#13089)

2025-02-11 14:41:20 +00:00

test_sequence.py

[Misc] Add SPDX-License-Identifier headers to python source files (#12628)

2025-02-02 11:58:18 -08:00

test_sharded_state_loader.py

Consolidate Llama model usage in tests (#13094)

2025-02-13 22:18:03 -08:00

test_utils.py

[Misc] Add SPDX-License-Identifier headers to python source files (#12628)

2025-02-02 11:58:18 -08:00

utils.py

[core] port pynvml into vllm codebase (#12963)

2025-02-09 15:00:00 +08:00