Shawn Du f8ece6e17f

[Core][v1] Unify allocating slots in prefill and decode in KV cache manager (#12608) ...

As mentioned in RFC https://github.com/vllm-project/vllm/issues/12254,
this PR achieves the task: combine allocate_slots and append_slots.

There should be no functionality change, except that in decode, also
raise exception when num_tokens is zero (like prefill), and change the
unit test case accordingly.

@comaniac @rickyyx @WoosukKwon @youkaichao @heheda12345 @simon-mo

---------

Signed-off-by: Shawn Du <shawnd200@outlook.com>

2025-02-02 16:40:58 +08:00

..

test_kv_cache_utils.py

[v1][Bugfix] Add extra_keys to block_hash for prefix caching (#12603)

2025-01-31 13:13:04 -08:00

test_prefix_caching.py

[Core][v1] Unify allocating slots in prefill and decode in KV cache manager (#12608)

2025-02-02 16:40:58 +08:00