Lucas Wilkinson
|
7b31e8a8ff
|
wip seperate comm and compute threads
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
|
2025-05-27 16:51:27 +00:00 |
|
Lucas Wilkinson
|
2f3920638c
|
add comment
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
|
2025-05-27 14:45:02 +00:00 |
|
Sage Moore
|
020d9b05bc
|
fix dp=2 tp=2 hang
Signed-off-by: Sage Moore <sage@neuralmagic.com>
|
2025-05-26 18:37:03 +00:00 |
|
Lucas Wilkinson
|
37bdf9f324
|
better logging
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
|
2025-05-23 18:34:08 +00:00 |
|
Lucas Wilkinson
|
e4419df256
|
better debug utils
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
|
2025-05-23 18:23:29 +00:00 |
|
Lucas Wilkinson
|
952f3c5c1e
|
tone down prints
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
|
2025-05-23 18:18:05 +00:00 |
|
Lucas Wilkinson
|
9edd08231b
|
debugging hang
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
|
2025-05-23 15:22:50 +00:00 |
|
Lucas Wilkinson
|
2dc3b8b0a2
|
wip
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
|
2025-05-23 03:32:25 +00:00 |
|
Lucas Wilkinson
|
00f526f55b
|
seperate gpu wait
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
|
2025-05-22 21:52:27 +00:00 |
|
Sage Moore
|
2a7f25fbe2
|
fix hang
|
2025-05-22 20:51:36 +00:00 |
|
Lucas Wilkinson
|
9c60a6299d
|
tp1 working multistream tp > 1 broken
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
|
2025-05-22 20:51:36 +00:00 |
|
Lucas Wilkinson
|
2259b47951
|
use vllm current_stream
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
|
2025-05-22 20:51:36 +00:00 |
|
Lucas Wilkinson
|
04f11d97a0
|
working but only on the same stream
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
|
2025-05-22 20:51:36 +00:00 |
|
Lucas Wilkinson
|
ffb740ae95
|
manually manage stream
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
|
2025-05-22 20:51:36 +00:00 |
|
Lucas Wilkinson
|
9ccfd094ff
|
fix dummy mode
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
|
2025-05-22 20:51:35 +00:00 |
|
Lucas Wilkinson
|
8293182c8c
|
wip
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
|
2025-05-22 20:51:35 +00:00 |
|