xinyun/DeepSeek-V3
1
0
Fork 0
mirror of https://git.datalinker.icu/deepseek-ai/DeepSeek-V3.git synced 2025-12-09 04:44:28 +08:00
Code Issues Packages Projects Releases Wiki Activity

Merge 6d7aeee7debbbab18d909e44e6ddcf52c7bd7f84 into 9b4e9788e4a3a731f7567338ed15d3ec549ce03b

Browse Source
This commit is contained in:
Luca Lowndes 2025-10-29 19:10:45 +11:00 committed by GitHub
commit b99f4bf504
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
README.md
View File

@ -229,7 +229,7 @@ We also provide OpenAI-Compatible API at DeepSeek Platform: [platform.deepseek.c
DeepSeek-V3 can be deployed locally using the following hardware and open-source community software:
1. **DeepSeek-Infer Demo**: We provide a simple and lightweight demo for FP8 and BF16 inference.
2. **SGLang**: Fully support the DeepSeek-V3 model in both BF16 and FP8 inference modes, with Multi-Token Prediction [coming soon](https://github.com/sgl-project/sglang/issues/2591).
2. **SGLang**: Fully support the DeepSeek-V3 model in both BF16 and FP8 inference modes, with Multi-Token Prediction [details here](https://lmsys.org/blog/2025-07-17-mtp/).
3. **LMDeploy**: Enables efficient FP8 and BF16 inference for local and cloud deployment.
4. **TensorRT-LLM**: Currently supports BF16 inference and INT4/8 quantization, with FP8 support coming soon.
5. **vLLM**: Support DeepSeek-V3 model with FP8 and BF16 modes for tensor parallelism and pipeline parallelism.