Carol Zheng fba02e3bd1

[Bugfix][TPU] Fix tpu model runner testcase failure (#18810) ...

Signed-off-by: Carol Zheng <cazheng@google.com>

2025-05-30 18:04:03 +08:00

..

attention

[ROCm] Remove unnecessary assertion of max_model_len in ROCM_AITER_MLA attention backend. (#18938)

2025-05-29 22:33:17 -07:00

core

[BugFix][Spec Decode] Improve Prefix Caching Logic in Speculative Decoding (#18668)

2025-05-24 17:33:46 -07:00

engine

[P/D] NixlConnector DP fixes (#18903)

2025-05-29 18:08:40 +00:00

executor

[BugFix] Increase TP execute_model timeout (#18558)

2025-05-23 10:22:11 +08:00

metrics

[V1][Metrics] Add API for accessing in-memory Prometheus metrics (#17010)

2025-05-27 09:37:06 +00:00

sample

[V1][Sampler] Improve performance of FlashInfer sampling by sampling logits instead of probs (#18608)

2025-05-26 11:49:36 -04:00

spec_decode

[V1] [Bugfix] eagle bugfix and enable correct lm_head for multimodal (2) (#18781)

2025-05-28 05:09:14 +00:00

structured_output

[Bugfix] Migrate to REGEX Library to prevent catastrophic backtracking (#18454)

2025-05-23 16:16:26 -07:00

worker

[Bugfix][TPU] Fix tpu model runner testcase failure (#18810)

2025-05-30 18:04:03 +08:00

__init__.py

[V1] AsyncLLM Implementation (#9826)

2024-11-11 23:05:38 +00:00

kv_cache_interface.py

[v1] Redo "Support multiple KV cache groups in GPU model runner (#17945)" (#18593)

2025-05-23 09:39:47 -07:00

outputs.py

[P/D] NIXL Integration (#17751)

2025-05-12 09:46:16 -07:00

request.py

[Feature][V1]: suupports cached_tokens in response usage (#18149)

2025-05-23 01:41:03 -07:00

serial_utils.py

[Bugfix] Ensure tensors are contiguous during serialisation (#18860)

2025-05-29 03:05:20 -07:00

utils.py

[V1] DP scale-out (2/N): Decouple engine process management and comms (#15977)

2025-05-13 10:48:21 -07:00