xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2026-07-08 18:37:12 +08:00
Code Issues Packages Projects Releases Wiki Activity

Fix typo in tokenizer.py (#750)

Browse Source 
conjuction -> conjunction
This commit is contained in:
Ikko Eltociear Ashimine 2023-08-15 14:26:36 +09:00 committed by GitHub
parent 1b151ed181
commit 805de738f6
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
vllm/transformers_utils/tokenizer.py
View File

@ -72,7 +72,7 @@ def detokenize_incrementally(
new_token_id: int, new_token_id: int,
skip_special_tokens: bool, skip_special_tokens: bool,
) -> Tuple[str, str]: ) -> Tuple[str, str]:
"""Detokenizes the new token in conjuction with the previous output tokens. """Detokenizes the new token in conjunction with the previous output tokens.
NOTE: This function does not update prev_output_tokens. NOTE: This function does not update prev_output_tokens.