12 Commits

Author	SHA1	Message	Date
Woosuk Kwon	84eee24e20	Collect system stats in scheduler & Add scripts for experiments (#30)	2023-04-12 15:03:49 -07:00
Woosuk Kwon	7a7929abe8	Implement preemption via recomputation & Refactor scheduling logic (#12)	2023-03-30 14:51:46 -07:00
Woosuk Kwon	d359cda5fa	Minor	2023-03-26 08:00:39 +00:00
Zhuohan Li	2f49f15585	Support tensor parallel (#2)	2023-03-21 13:45:42 -07:00
Woosuk Kwon	1a7eb7da61	Support beam search & parallel generation (#7)	2023-03-10 09:58:21 -08:00
Woosuk Kwon	b39f149a08	Add is_finished	2023-02-24 11:44:21 +00:00
Woosuk Kwon	af16c05074	Add get_len	2023-02-23 05:58:04 +00:00
Woosuk Kwon	d094512296	Move max_context_len	2023-02-23 04:57:46 +00:00
Woosuk Kwon	3363c27d19	Add __repr__	2023-02-14 09:34:07 +00:00
Woosuk Kwon	0961f5a49a	Add find method to sequence group	2023-02-13 02:39:12 +00:00
Woosuk Kwon	a2a9869cb7	SERVING -> RUNNING	2023-02-12 08:25:05 +00:00
Woosuk Kwon	d904350a2c	Add sequence	2023-02-09 11:26:35 +00:00