xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2025-12-10 01:55:01 +08:00
Code Issues Packages Projects Releases Wiki Activity

[Misc] Fixed the abnormally high TTFT issue in the PD disaggregation example (#18644)

Browse Source 
Signed-off-by: zhaohaidao <zhaohaidao2008@hotmail.com>
Signed-off-by: zhaohaiyuan <zhaohaiyuan@xiaohongshu.com>
Co-authored-by: zhaohaiyuan <zhaohaiyuan@xiaohongshu.com>
This commit is contained in:
AlexZhao 2025-05-26 13:51:27 +08:00 committed by GitHub
parent fba0642704
commit 8820821b59
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 2 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
examples/lmcache/disagg_prefill_lmcache_v1/disagg_proxy_server.py
View File

@ -135,7 +135,7 @@ async def handle_completions(request: Request):
yield chunk
return StreamingResponse(generate_stream(),
media_type="application/json")
media_type="text/event-stream")
except Exception as e:
import sys
@ -172,7 +172,7 @@ async def handle_chat_completions(request: Request):
yield chunk
return StreamingResponse(generate_stream(),
media_type="application/json")
media_type="text/event-stream")
except Exception as e:
import sys