6 Commits

Author	SHA1	Message	Date
youkaichao	63575bc2e1	[Core][Optimization] change python dict to pytorch tensor (#4607)	2024-05-06 21:30:27 -07:00
youkaichao	344a5d0c33	[Core][Distributed] enable allreduce for multiple tp groups (#4566)	2024-05-02 17:32:33 -07:00
youkaichao	f4f921b7f1	[Core][Distributed] use cpu group to broadcast metadata in cpu (#4444)	2024-04-29 13:52:22 -07:00
SangBin Cho	09473ee41c	[mypy] Add mypy type annotation part 1 (#4006)	2024-04-12 14:35:50 -07:00
SangBin Cho	67b4221a61	[Core][5/N] Fully working chunked prefill e2e (#3884)	2024-04-10 17:56:48 -07:00
youkaichao	63e7176f26	[Core][Refactor] move parallel_utils into vllm/distributed (#3950) ... [WIP][Core][Refactor] move vllm/model_executor/parallel_utils into vllm/distributed and vllm/device_communicators (#3950)	2024-04-10 15:33:30 -07:00