01github.com
deepseek-ai/DeepInfer
DeepInfer is an open-source, high-performance offline inference framework for serving DeepSeek models at large scale, currently supporting DeepSeek-V4.1-Flash. It provides optimized kernels and runtimes, Attention DP, balanced CP, MoE EP + EPLB, DSpark verification, SWA bounded replay, asynchronous scheduling, batch-invariant and deterministic inference, and structured outputs. It runs on NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs, but a complete service still requires KV cache, disaggregated execution, and replica load balancing.