大模型网摘Full 510GB V4.1-Flash on 12GB VRAM: Promethevs compress-then-correct and local inference tools
A project called Promethevs surfaced on Hacker News on November 6, claiming to run the full 510GB of DeepSeek-V4.1-Flash weights on a five-year-old gaming PC: not a distilled copy, the original weights. The reference setup is a Ryzen 9 3950X, 64GB of DDR4, an RTX 3060 with 12GB, and two NVMe drives. The engine is open source under Apache-2.0.
Compress-then-correct: compress first, correct when unsure
The method, devised by Ryan Flavell, works like this: every word is computed on a 2-bit copy of the expert weights; when the model is unsure (its top two candidates are close), the word is recomputed with the original weights, with the correction data stored as a correction plane and rebuilt on demand. The result runs at near-compressed speed while answering close to the full model. The installer downloads DeepSeek's original weights and builds locally, checked against the published hashes.
The quality and speed ledger
Per the project's own figures: on 960 audited words, the output matches the full model's word choice 98.3% of the time, versus 97.4% for the 2-bit copy alone. On speed, code prompts with a warm cache reach up to 7.4 tokens/s in the best case, against 2.14 tokens/s for exact full-model inference on the same machine; server runs measure 5 to 8 tokens/s on code and 4.5 to 5.4 on reasoning and agent prompts. Only the first read of a long prompt is slow; follow-up messages in a conversation start in about 2 seconds. The project exposes an OpenAI-compatible endpoint on localhost with streaming and tool calls, so existing coding agents can use it directly.
Why V4.1-Flash suits this approach
V4.1-Flash is a 552B-parameter MoE with the CED architecture (8B active per input token, 16B per output token). Per the official release, its KV cache cuts HBM requirements to a quarter and SSD requirements to an eighth of the previous generation. A small cache keeps the pressure of streaming expert weights from disk manageable, which is precisely what makes the "weights in RAM, compute in VRAM" approach viable. The official bar for large-scale deployment remains a 2,000-GPU cluster with storage; individuals take the community route, and server-grade deployments can follow the official vLLM recipe.
Slow model swaps? llama-sleep cuts 10 seconds to 2
Published by Robert Zaage on November 5, llama-sleep addresses the neighboring nuisance: swapping models on a single GPU. A hard swap between two GPU-filling models via llama-swap takes 10.3 seconds and discards the cached prompts; llama-sleep instead parks the GPU memory in host RAM (building on cuda-checkpoint), finishing in 2.1 seconds with the cache intact; enabling multi-size transparent huge pages brings it to 1.7 seconds. The author's conclusion: the bottleneck is copying 4KB pages, not PCIe bandwidth.
To try or not to try
Two notes for tinkerers: downloading weights and building locally takes time and disk, and the first successful run requires patience; 5 to 8 tokens/s suits overnight batch jobs and low-frequency chat, not high-concurrency serving. For ecosystem background, see our DeepSeek Harness v0.2.1 tools roundup and the V4.1-Flash tech report explainer; for enterprise self-hosting routes, see DeepInfer self-hosting scenarios.