Will Eaton 64eaf5fe05

[P/D] NixlConnector DP fixes (#18903) ...

Signed-off-by: Will Eaton <weaton@redhat.com>

2025-05-29 18:08:40 +00:00

..

__init__.py

[Feature][V1]: suupports cached_tokens in response usage (#18149)

2025-05-23 01:41:03 -07:00

async_llm.py

[V1] Fix _pickle.PicklingError: Can't pickle <class 'transformers_modules.deepseek-ai.DeepSeek-V2-Lite... (#18640)

2025-05-24 20:07:20 -07:00

core_client.py

[BugFix] Make DP work with connector-delayed new requests (#18559)

2025-05-29 18:04:18 +00:00

core.py

[P/D] NixlConnector DP fixes (#18903)

2025-05-29 18:08:40 +00:00

detokenizer.py

Only turn on FastIncrementalDetokenizer when tokenizers >= 0.21.1 (#17158)

2025-04-25 17:10:32 +08:00

exceptions.py

[V1][Frontend] Improve Shutdown And Logs (#11737)

2025-04-16 19:48:34 -07:00

llm_engine.py

[V1][Metrics] Add API for accessing in-memory Prometheus metrics (#17010)

2025-05-27 09:37:06 +00:00

logprobs.py

[V1] Aggregate chunked prompt logprobs in model runner (#14875)

2025-03-24 12:27:57 -04:00

mm_input_cache.py

[Bugfix] Avoid repeatedly creating dummy data during engine startup (#17935)

2025-05-12 22:40:19 -07:00

output_processor.py

[Feature][V1]: suupports cached_tokens in response usage (#18149)

2025-05-23 01:41:03 -07:00

parallel_sampling.py

[V1] Avoid redundant input processing in n>1 case (#14985)

2025-03-20 22:24:10 -07:00

processor.py

[Bugfix] Avoid repeatedly creating dummy data during engine startup (#17935)

2025-05-12 22:40:19 -07:00