xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2026-08-04 06:37:00 +08:00
Code Issues Packages Projects Releases Wiki Activity
vllm/tests/tokenization
History
Nick Hill 49782fcb76
[Misc] Some minor simplifications to detokenization logic (#3670) 
Some simplifications made for clarity.

Also moves detokenization-related functions from tokenizer.py to detokenizer.py.
2024-04-01 13:22:06 -07:00
..
__init__.py
Asynchronous tokenization (#2879)
2024-03-15 23:37:01 +00:00
test_cached_tokenizer.py
[CI] Try introducing isort. (#3495)
2024-03-25 07:59:47 -07:00
test_detokenize.py
[Misc] Some minor simplifications to detokenization logic (#3670)
2024-04-01 13:22:06 -07:00
test_tokenizer_group.py
[CI] Try introducing isort. (#3495)
2024-03-25 07:59:47 -07:00