大模型网摘DeepInfer Goes Open Source: DeepSeek's Production Inference Framework Released
On October 20, 2026, DeepSeek open-sourced DeepInfer, its in-house inference framework, at the GitHub repository deepseek-ai/DeepInfer. The README describes it as "the high-performance inference framework that powers DeepSeek's large-scale production services," with this release providing offline inference and the core features needed for serving at scale. The launch quickly gathered more than 2,300 points on Hacker News, and Bloomberg covered it as an open-source move to help cut AI operating costs.
What is in the release
According to the documentation, the release centers on the inference engine itself: optimized kernels (FlashMLA, DeepGEMM, DeepEP, TileKernels and others), Attention DP / balanced CP parallelism, and MoE EP + EPLB strategies. For decoding speed there is DSpark speculative decoding with confidence-scheduled verification (enabled via an environment variable, default draft length of 5). The framework also ships SWA bounded replay, asynchronous batch scheduling, batch-invariant inference and deterministic sampling (NVIDIA platforms), plus structured outputs constrained by regex, JSON Schema, or EBNF grammars.
Supported models and hardware
The current release supports DeepSeek-V4.1-Flash, with the default checkpoint being V4.1-Flash-1017 from October 17, available on Hugging Face. Hardware support covers both NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs. Requirements include Linux, GCC 13 or newer, Python 3.12, and CUDA 13 (NVIDIA) or CANN 9.2 (Ascend).
The boundary: this is an offline-inference release
The README explicitly lists what is not included: a KV cache system, prefill/decode disaggregation, and load balancing across data-parallel replicas. In other words, a complete online serving stack still has to be assembled by the user. We analyze who should take that path in a separate piece on self-hosting scenarios.
Takeaway
This continues DeepSeek's run of infrastructure open-sourcing after FlashMLA, DeepEP and DeepGEMM, and this time the production framework itself is on the table. For most developers the official API remains the low-friction route (cache-hit input as low as $0.003 per million tokens); for teams with self-hosting requirements, DeepInfer moves the practical path one step closer. For the training side of the new model, see our RLSF deep dive.