Jialin Ouyang 3319a493fc

[Core] Reuse created spec tokens lists to mitigate GC cost (#28917) ...

Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>

2025-11-19 19:20:22 +00:00

..

attention

[BugFix] Fix async-scheduling + FlashAttn MLA (#28990)

2025-11-19 10:04:07 -05:00

core

[Misc] Fix wrong comment in scheduler (#28880)

2025-11-17 15:32:22 -08:00

engine

[Core] Switch Flat logprob control from environment variable to SamplingParams (#28914)

2025-11-19 02:10:02 +00:00

executor

[BugFix] Fix PP/async scheduling with pooling models (#28899)

2025-11-18 00:20:45 -08:00

kv_offload

Replace torch.cuda.Event with torch.Event for better hardware compatibility (#26985)

2025-11-18 11:34:36 -08:00

metrics

[Metrics] Fix KV cache usage percent metric multiproc (#28792)

2025-11-17 09:54:15 +00:00

pool

…

sample

[FEAT] [AITER] [ROCm] integrate aiter sampling ops (#26084)

2025-11-18 17:28:34 +00:00

spec_decode

[Core] Async Scheduling X Spec Decoding Compatibility (#24799)

2025-11-17 12:16:20 -08:00

structured_output

[Bugfix] Fix llguidance backend, rollback when EOS was encountered (#25905)

2025-11-10 14:53:32 -05:00

worker

[Core] Reuse created spec tokens lists to mitigate GC cost (#28917)

2025-11-19 19:20:22 +00:00

__init__.py

…

cudagraph_dispatcher.py

…

kv_cache_interface.py

…

outputs.py

[Redo] #26368 (#28771)

2025-11-14 22:47:41 -08:00

request.py

[Model] Allow users to control skip reading cache per request. (#28194)

2025-11-16 00:04:50 -08:00

serial_utils.py

[DisaggEverything] Tokens in<>out /generate endpoint (#24261)

2025-11-14 09:58:01 -07:00

utils.py

[BugFix] Fix async scheduling + chunked prefill + preemption (#28787)

2025-11-17 06:46:46 +08:00