2 Commits

Author SHA1 Message Date
comfyanonymous
f5ebfaba43 Allow SPieceTokenizer to load model from a byte string. 2024-07-23 14:17:42 -04:00
comfyanonymous
a7f1068957 Rename LLAMATokenizer to SPieceTokenizer. 2024-07-22 12:21:45 -04:00