Chen Zhang a8da78eac9

[Bugfix] Max concurrency estimation and check_enough_kv_cache_memory for models with sliding window layers (#19029) ...

Signed-off-by: Chen Zhang <zhangch99@outlook.com>

2025-06-04 00:14:06 +00:00

..

core

[Bugfix] Max concurrency estimation and check_enough_kv_cache_memory for models with sliding window layers (#19029)

2025-06-04 00:14:06 +00:00

e2e

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

engine

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

entrypoints

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

kv_connector

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

metrics

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

sample

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

shutdown

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

spec_decode

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

structured_output

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

tpu

[V1] Support cross-layer KV sharing (#18212)

2025-06-03 20:33:07 +00:00

worker

[v1] Re-init input batch for multiple kv cache groups (#18654)

2025-06-03 21:41:36 +00:00

__init__.py

[V1] AsyncLLM Implementation (#9826)

2024-11-11 23:05:38 +00:00

test_async_llm_dp.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

test_metrics_reader.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

test_oracle.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

test_serial_utils.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00

test_utils.py

[Misc] Add SPDX-FileCopyrightText (#19100)

2025-06-03 11:20:17 -07:00