11 Commits

Author	SHA1	Message	Date
Woosuk Kwon	0f40557af6	Implement block copy kernel to optimize beam search (#32)	2023-04-07 17:45:07 -07:00
Woosuk Kwon	a90c97d727	Use FP32 for log probabilities (#19)	2023-03-31 23:33:43 -07:00
Woosuk Kwon	88c0268a18	Implement custom kernel for LLaMA rotary embedding (#14)	2023-03-30 11:04:21 -07:00
Woosuk Kwon	80a2f812f1	Implement LLaMA (#9) ... Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>	2023-03-30 12:25:32 +08:00
Woosuk Kwon	d359cda5fa	Minor	2023-03-26 08:00:39 +00:00
Zhuohan Li	2f49f15585	Support tensor parallel (#2)	2023-03-21 13:45:42 -07:00
Woosuk Kwon	cfae35b861	Add miscellaneous updates (#8)	2023-03-13 13:48:38 -07:00
Woosuk Kwon	1a7eb7da61	Support beam search & parallel generation (#7)	2023-03-10 09:58:21 -08:00
Woosuk Kwon	de0fabbc5c	Fix sampler	2023-02-23 20:30:12 +00:00
Woosuk Kwon	fdd0f2f472	Minor	2023-02-23 20:23:47 +00:00
Woosuk Kwon	b56b6ca0d6	Add greedy sampler	2023-02-23 09:26:09 +00:00