Nick Hill 7df331c66b

[BugFix] Fix chunked prompt logprobs + preemption (#29071)

2025-11-22 16:07:18 -05:00

..

attention

[perf][cpu] Accelerate paged attention GEMMs (QK, PV) on Arm CPUs with NEON (#29193)

2025-11-22 09:04:36 -08:00

core

[Misc] Further clean up chunked prefill and prefix caching init (#29186)

2025-11-22 19:34:15 +08:00

engine

Default model load/config/tokenizer to mistral format if relevant files exist (#28659)

2025-11-21 13:58:59 -08:00

executor

[Feature] Prefill Context Parallel (PCP) basic support (#28718)

2025-11-19 15:52:44 -05:00

kv_offload

[KVConnector][Core] Support cross-layer KV blocks (#27743)

2025-11-20 19:09:59 +01:00

metrics

[Doc] Update plugin doc (#28532)

2025-11-21 14:57:26 +00:00

pool

…

sample

[BugFix] Fix returned logprobs with spec decode + prefill chunking (#29216)

2025-11-22 22:41:25 +08:00

spec_decode

[BugFix] Fix Eagle IndexError: list index out of range for even num_speculative_tokens (#29102)

2025-11-22 00:47:05 +00:00

structured_output

[Bugfix] Fix llguidance backend, rollback when EOS was encountered (#25905)

2025-11-10 14:53:32 -05:00

worker

[BugFix] Fix chunked prompt logprobs + preemption (#29071)

2025-11-22 16:07:18 -05:00

__init__.py

…

cudagraph_dispatcher.py

…

kv_cache_interface.py

[Feature] Prefill Context Parallel (PCP) basic support (#28718)

2025-11-19 15:52:44 -05:00

outputs.py

Revert "[Redo] #26368 (#28771)" (#29121)

2025-11-20 21:27:45 -08:00

request.py

[KV Connector] Fix async connector prefix cache metrics (#28585)

2025-11-21 17:45:00 -05:00

serial_utils.py

[DisaggEverything] Tokens in<>out /generate endpoint (#24261)

2025-11-14 09:58:01 -07:00

utils.py

[BugFix] Fix async scheduling + chunked prefill + preemption (#28787)

2025-11-17 06:46:46 +08:00