大模型网摘After DeepInfer: Scenarios and Barriers for Self-Hosting DeepSeek Inference
DeepInfer, open-sourced on October 20, raises the starting point for "running DeepSeek models yourself": what you get is the main body of the inference framework behind the company's own production services. Equally important is what the release does not include. This piece walks through enterprise scenarios: who should take it on, and who should not.
Draw the boundary first: what you get, what is missing
You get production-grade kernels (FlashMLA, DeepGEMM, DeepEP and friends), DSpark speculative decoding with confidence-scheduled verification, asynchronous batch scheduling, batch-invariant inference, deterministic sampling, and structured outputs, supporting V4.1-Flash weights. You do not get a KV cache system, prefill/decode disaggregation, or load balancing across replicas — the stretch of engineering between "offline inference" and "complete online service" is yours to build or wait for. Hardware means NVIDIA SM100 or Huawei Ascend 950; software requires Python 3.12 plus CUDA 13 or CANN 9.2.
Four scenarios worth evaluating
1. Data sovereignty and on-prem compliance: model weights inside the intranet with data never leaving the domain is a hard constraint in finance, healthcare and government. Worth noting: the US DHS exclusion order of October 20 does not name open weights or self-hosting, but compliance interpretation belongs to your own legal team. 2. Regulated and scientific workloads needing reproducible outputs: batch-invariant inference plus deterministic sampling with a fixed seed turns "same input, same output" into an engineering promise — real value for audits and experiment replication. 3. High-throughput cost control: Bloomberg framed the release as a tool to cut AI operating costs — self-hosting starts to make sense when monthly API bills approach GPU amortization; model specs in the V4.1-Flash profile. 4. Offline and weak-network environments: local weights plus a local framework, no public-internet dependency.
Barriers, and a more realistic hybrid
For most small and mid-sized teams the direct recommendation is a hybrid: steady-state traffic through the official API (cache-hit input at $0.003–0.006 per million tokens, output halved off-peak — see the cost checklist), with sensitive data or burst batches self-hosted. When evaluating, count all three costs: GPU purchase or amortization, engineering labor to complete the online serving stack, and version-tracking overhead. For concurrency and rate-limit parameters on the API side, see the rate-limit explainer — self-hosting does not automatically mean unlimited concurrency.