Kimi K3 on 8GB RAM: How the C99 Demo Works and Why It's So Slow
September 25, 2026 · kimi k3 8gb ram · kimi k3 run locally · kimi-k3-in-c · kimi k3 cpu inference · kimi k3 ssd streaming · kimi k3 gguf · run kimi k3 without gpu · moonshot ai
Yes, Kimi K3 can run on 8GB of RAM, and no, you should not use it that way. The viral demo is a real open-source C99 engine called kimi-k3-in-c that runs the full 2.8-trillion-parameter checkpoint at a measured 8.24GB peak RSS by keeping 93% of the model on an NVMe drive. The price is speed: 26.5 to 32.7 seconds per token, roughly 0.03 tokens per second, measured on a 124-core server rather than a laptop.
This is our breakdown as of September 25, 2026: who built it, the exact trick that shrinks 1.56TB into 8GB, what each token actually costs in disk reads, the fine print the viral posts skipped, and the local setups that are genuinely usable. If you only want the verdict on hardware in general, our Kimi K3 VRAM requirements guide has the full GPU math; this article is about the “no GPU at all” end of the spectrum.
| Quick facts | |
|---|---|
| Project | kimi-k3-in-c by Fareed Khan (Apache-2.0), published August 1, 2026 |
| Engine | Portable C99, 176KB binary, no BLAS, no framework, no GPU path |
| Peak RAM (measured) | 8.24GB at the smallest ordinary preset |
| Speed at 8GB | 32.69 s/token (launch capture), 26.5 s/token (v1.0.0 README) |
| Speed at 128GB+ | 5.6 to 10.7 s/token depending on version and run |
| Disk required | ~1.7TB free (1.56TB checkpoint + 109GB packed trunk) |
| CPU required | x86-64 with AVX2 + FMA (Linux reference; macOS and Windows build) |
| Test machine | 2x AMD EPYC 7763, 124 cores, 228GB RAM, 3.2TB NVMe, memory capped by cgroup |
| Usable for chat? | No. A 500-token answer takes ~3.7 hours at the 8GB setting |
Where the “Kimi K3 on 8GB” claim comes from
On August 1, 2026, five days after Moonshot’s open-weights release, Fareed Khan pushed kimi-k3-in-c to GitHub with the tagline “A 2.78-trillion-parameter model. One CPU. 8 GB of RAM.” (2.78T is the repo’s exact count; Moonshot and most coverage round it to 2.8T.) It hit GitHub’s trending page within days, according to a review on andrew.ooo, and at the time of writing the repository shows roughly 8.5k stars and 1.4k forks.
The headline capture is honest about itself: prompted with “The capital of France is”, the engine answered ” Paris.” and produced 8 tokens in 261.5 seconds (32.69 s/token) at a peak RSS of 8.24GB.
A v1.0.0 release on August 7 added faster prefill, conversation resume and speculative decoding, which is why the current README headline reads 26.5 s/token at 8GB. Later commits added an official-format chat mode and a native Windows build.
To be clear: this is not a Moonshot product, not a “lite” K3 and not a pruned model. It is the same checkpoint you download from Hugging Face, read by a tiny program that is very deliberate about where each byte lives.
How a 1.56TB model fits in 8GB: the four reductions
The trick rests on K3 being a Mixture-of-Experts model: each of its 92 MoE layers holds 896 routed experts, and the router picks only 16 per token. (Our Kimi K3 specifications page covers the architecture in detail.)
The README walks the memory requirement down in four steps:
| Step | Memory needed | What changed |
|---|---|---|
| Every parameter at BF16 | 5,560GB | The naive starting point |
| Checkpoint as shipped | 1,560GB | Moonshot already ships the experts in MXFP4, about half a byte per weight |
| Resident set only | 113.49GB | Routed experts never load into RAM; only the always-on “trunk” does |
| Trunk streamed layer by layer | 8.24GB (measured) | The 93 dense layers are read from disk as needed instead of held |
The byte census is what makes it work. The repository counts 82,432 routed experts (896 x 92 layers), each exactly 17,547,264 bytes, for a total of 1.447TB. That is 93% of the checkpoint. Everything else (attention, routers, norms, the two shared experts that run on every token, embeddings and the output head) comes to 113.49GB at BF16: 108.81GB of per-layer trunk plus 4.70GB of embeddings and output head.
From there, three engineering decisions do the rest:
- Experts are read straight off NVMe with
O_DIRECT, bypassing the page cache (on the test machine, direct reads measured faster than buffered ones: 3.2 vs 2.3 GB/s). - The trunk is repacked once into a single 109GB file (about four minutes) so each layer can be fetched in one read. The memory budget then becomes a dial: pin as many trunk layers in RAM as you can afford, stream the rest.
- Nothing is re-quantized. The experts are multiplied directly from their shipped 4-bit form and the trunk stays in BF16. The author measured what int4 would do to the trunk and found about 17% mean reconstruction error (worst rows up to 65%), versus about 1% for int8, so the engine offers no bit-width knob at all.
The payoff: across twelve memory budgets from 8GB to 224GB, the generated token IDs were byte-identical. More RAM changes the clock, never the answer.
What each token actually costs
The engine publishes a full memory ladder, measured on one server with a hard cgroup memory cap and swap disabled:
| Memory budget | Trunk layers pinned | Seconds per token | Tokens per second | Peak RSS |
|---|---|---|---|---|
| 8GB | 0 of 93 | 32.69 | 0.031 | 8.24GB |
| 16GB | 3 | 32.21 | 0.031 | 16.00GB |
| 32GB | 11 | 31.44 | 0.032 | 31.90GB |
| 64GB | 27 | 28.60 | 0.035 | 63.71GB |
| 96GB | 43 | 24.40 | 0.041 | 95.51GB |
| 192GB | 90 | 21.32 | 0.047 | 191.83GB |
| 224GB | 90 | 19.21 | 0.052 | 223.82GB |
Source: the memory ladder in the kimi-k3-in-c README. These are eight-token runs that include the first-token prefill, so the steady-state rate at the top rungs is better than shown: sustained generation runs at the server preset settled at about 8.9 seconds per step, 10.7 to 11.8 s/token averaged over 24 to 32 tokens at 127.9GB of RAM. The newer headline figures (26.5 / 24.2 / 19.8 / 5.6 s/token at 8 / 32 / 64 / 128GB+) reflect v1.0.0.
Why so slow? At the 8GB budget, every token reads 25.83GB of routed-expert weights plus the unpinned trunk, which the README describes as 108.81GB re-read in full each token when nothing is pinned. That is well over 100GB of disk traffic per token. At the 5.4 to 6.1 GB/s the engine sustained on that server’s NVMe, the arithmetic lands close to where the measurements do. The README’s own summary: 28 times the memory buys 1.70 times the speed. Going from 8GB to 64GB buys about 14%.
To put that in human terms:
| Task | kimi-k3-in-c at 8GB (26.5 s/token) | At 128GB+ (5.6 s/token) | Official API (~34 tok/s) |
|---|---|---|---|
| 8-token completion | ~3.5 minutes | ~45 seconds | under 1 second |
| 150-token short chat reply (with thinking) | ~66 minutes | ~14 minutes | ~4.5 seconds |
| 500-token answer | ~3 hours 40 minutes | ~47 minutes | ~15 seconds |
| 1M tokens of output | ~307 days | ~65 days | ~8 hours, $15 |
The chat row matters more than it looks. K3 thinks before it answers, and the engine’s chat mode defaults to maximum thinking effort to match the official tokenizer. In one real-checkpoint run the author reports that 119 of the 150 tokens of a five-word answer were the think block. The --no-think flag is, in the README’s words, “the largest speed lever there is.”
The fine print the viral posts skipped
The README is unusually candid, and an independent audit by Kingy AI (August 3) filled in the rest. Six caveats matter:
1. “Laptop” is a memory budget, not a laptop. Every number came from one dual-socket EPYC 7763 server with 124 cores, 228GB of RAM and fast local NVMe, capped to smaller budgets with a cgroup. Kingy’s audit puts it plainly: the 8GB figure is a process ceiling on a 228GB server, not a demonstrated laptop run. A real 8-core laptop should be slower on compute, and much slower if its SSD can’t sustain multi-GB/s random reads.
2. Storage is the whole game. Disk waits consumed 41 to 61% of wall-clock time at every budget. The same configuration measured twice read the identical 374.99GB but got 2,709 MB/s in one run and 5,874 MB/s in the other, which turned into 29.40 vs 18.37 s/token.
3. A third of any timing is noise. Three back-to-back identical runs came in at 14.78, 14.67 and 20.14 s/token, a 33.1% spread.
4. More RAM for experts barely helps. Below about 36GB of cache, the expert cache retained literally nothing between steps (0.0% true resident hit rate). K3’s load balancing spreads work evenly across experts, so there is no “hot” subset to keep. Give memory to the trunk first.
5. Full-model correctness rests on the author’s logs. The repo has CI-checked tests against a tiny 13-layer reference model, and the author’s logs show all 93 real layers passing per-layer checks. Kingy’s audit notes that full-checkpoint parity had not been independently reproduced when it reviewed the code.
6. It is a narrow tool. Text only (MoonViT vision is unimplemented), no tool calling, no HTTP server, a 32,768-token prompt ceiling with no chunked prefill, and about 2.37MB of KV cache per context position. K3’s 1M-token window is a memory fact the engine does not try to reach.
None of this makes the project less impressive. It makes it a research artifact, not a product.
Disk and SSD requirements
If you want to reproduce the demo, the real gate is storage, not RAM:
| Requirement | Figure | Notes |
|---|---|---|
| Official checkpoint | 1.56TB (1,560,936,091,448 bytes, 96 shards) | Download is “hours rather than minutes”; the script verifies every shard’s size |
| Packed trunk | 109GB | One-time, about 4 minutes; put it on your fastest disk |
| Total free space | ~1.7TB | In practice, a 2TB+ local NVMe drive (our estimate) |
| Read bandwidth | 5.4 to 6.1 GB/s sustained in the author’s runs | Throughput scales with the drive, not the CPU |
| Storage type | Local NVMe | Network volumes measured ~2.2x slower |
SSD wear is mostly a non-issue: NAND endurance is consumed by writes, and after the one-time ~1.7TB of download and packing, inference is read-only. The real risk is a partial download, which per the README silently produces wrong tokens instead of failing, so do not skip the script’s per-shard verification.
The 1.56TB figure matches our measurement from the open-weights release. The weights themselves fall under the Kimi K3 license, separate from the engine’s Apache-2.0 code.
Real local options, from 8GB to datacenter
The C99 engine is one of several ways people run K3 outside the cloud, on a spectrum from “exact but glacial” to “fast but lossy” to “fast and expensive”:
| Setup | Memory | Disk / model size | Reported generation speed | Fidelity |
|---|---|---|---|---|
| kimi-k3-in-c, 8GB budget | 8.24GB RSS | ~1.7TB | 26.5 to 32.7 s/token (~0.03 tok/s) | Bit-exact to release |
| kimi-k3-in-c, server preset | ~128GB | ~1.7TB | 5.6 to 10.7 s/token | Bit-exact to release |
| Deltafin, 64GB M1 Max MacBook Pro | 64GB | 1.7TB full, or 215GB start in stream mode | 0.08 to 0.29 tok/s (3.4 to 12.5 s/token) | Full experts; best figure uses an int8 resident spine plus draft models |
| Unsloth UD-IQ1_S GGUF on llama.cpp, 4x A100 40GB + 2TB RAM | ~594GB | 594GB | 0.10 tok/s (per ComputingForGeeks) | 78.9% top-1 agreement |
| 4x Mac Studio (512GB each), MLX over Thunderbolt 5 | 2TB unified | ~817GB quant | ~14.7 tok/s, ~238 tok/s prefill (as reported by MindStudio) | Quantized, all 896 experts |
| 8x B300 or 8x MI355X server | ~2.3TB HBM | 1.56TB | Production-class | Native MXFP4 |
| Official API or hosted provider | None | None | ~34 tok/s standard | Full model |
A few honest notes on that table:
- Deltafin is a separate MIT-licensed project that also streams experts from disk but adds draft-model speculative decoding (K3 verifies every token). Its own performance notes say the 0.27 to 0.29 tok/s figures come from a favorable short completion with an int8 resident spine; a creative-chat check ran at 0.081 to 0.095 tok/s. It started at 0.0141 tok/s on July 27, so these projects are improving fast.
- The Unsloth GGUF path is where most “run it locally” guides point. Unsloth’s Kimi-K3-GGUF repo lists these builds:
| Unsloth quant | File size | Top-1 accuracy vs full | Recommended RAM + VRAM |
|---|---|---|---|
| UD-Q1_0 | 466GB | not published | not published |
| UD-IQ1_S | 594GB | 78.9% | 610GB |
| UD-IQ1_M | 649GB | 81.2% | 665GB |
| UD-IQ2_XXS | 711GB | 84.1% | 726GB |
| UD-Q2_K_XL | 861GB | 90.4% | 880GB |
| UD-Q8_K_XL (lossless) | 1.56TB | full | 1.6TB |
Figures from the Unsloth Kimi K3 docs. The trap: the 1-bit build still needs about 610GB of combined memory, and when ComputingForGeeks ran it mostly from system RAM, generation was 0.10 tok/s, “not a usable chat model” in their words, with memory bandwidth as the bottleneck. And you give up over a fifth of top-1 agreement to get there.
- The Mac Studio cluster is the only reported sub-datacenter setup at a conversational pace, but we found it only secondhand (MindStudio credits hardware reviewer Alex Ziskind), and at roughly $16,000 per machine it is a $60,000+ build. Treat 14.7 tok/s as reported, not verified.
Which should you pick?
| You are… | Best path | Why |
|---|---|---|
| Curious how MoE inference works | Clone kimi-k3-in-c and run make test | The weightless test suite runs in about a minute with no checkpoint; the README is a genuinely good MoE tutorial |
| Owner of a 2TB NVMe and patience | Run kimi-k3-in-c on a short prompt with --no-think | Real K3 output on your own machine, as a one-off experiment |
| Hoping for a private local chatbot | Pick a smaller model, or use K3 through a provider with zero data retention | At 0.03 tok/s, a single reply takes an hour or more |
| Local-LLM enthusiast with 600GB+ of RAM + VRAM | Unsloth GGUF, and accept the accuracy loss | The most mature tooling, but still sub-1 tok/s unless it all fits in fast memory |
| Team that needs air-gapped K3 | 8x B300 / 8x MI355X-class node | The only local path at production speed; see our VRAM guide |
| Everyone else | API or hosted provider | $0.30 / $3 / $15 per million tokens at ~34 tok/s |
The cost comparison is not close: a 500-token answer costs $0.0075 in API output tokens, versus nearly four hours of a machine running its NVMe flat out. As one Hacker News commenter put it about a similar project, generating a million tokens locally would take years, against a few dollars on a commercial API. If you just want to talk to K3, our PC download guide covers the app and web options.
FAQ
Can Kimi K3 really run on 8GB of RAM? Yes, technically. The open-source kimi-k3-in-c engine by Fareed Khan ran the full released Kimi K3 checkpoint, with no extra quantization, at a measured peak of 8.24GB of RAM. It does this by leaving the 1.447TB of routed experts on an NVMe drive and streaming the dense trunk layer by layer. The catch is speed: the launch capture took 261.5 seconds to produce 8 tokens (32.69 s/token), and the v1.0.0 README lists 26.5 s/token at the 8GB setting.
How fast is Kimi K3 on 8GB RAM in tokens per second? About 0.03 to 0.04 tokens per second. The engine’s published numbers are 26.5 to 32.69 seconds per token at the 8GB budget, 24.2 s at 32GB, 19.8 s at 64GB and 5.6 s at 128GB and above. A 500-token answer at 26.5 s/token takes roughly 3 hours 40 minutes. The official API streams at around 34 tokens per second.
Was the 8GB result measured on a laptop? No. Every figure in the repository comes from one server: a dual-socket AMD EPYC 7763 with 124 cores, 228GB of RAM and 3.2TB of NVMe, with memory capped by a Linux cgroup to simulate smaller machines. The rows labelled laptop and desktop are memory budgets on that server, not runs on consumer hardware. A real laptop with fewer cores and a slower SSD should be slower.
How much disk space do I need to run Kimi K3 locally? About 1.7TB free for kimi-k3-in-c: the 1.56TB official checkpoint (1,560,936,091,448 bytes across 96 shards) plus a 109GB packed trunk file. In practice that means a 2TB or larger local NVMe drive. Network storage roughly halves throughput; the author measured 31.71 s/token on local NVMe against 70.62 s/token on a network volume.
Does streaming Kimi K3 from an SSD wear out the drive? Not in the way people fear. Flash endurance is consumed by writes, and this workload is read-dominated: about 25.83GB of expert reads per token, plus trunk reads, with no weight rewriting after the one-time download and trunk pack. The bigger costs are the one-time 1.7TB of writes and the fact that disk I/O takes 41 to 61 percent of every run’s wall-clock time.
Is there a GGUF version of Kimi K3 for llama.cpp? Yes. Unsloth publishes Kimi-K3-GGUF with builds from 466GB (UD-Q1_0) and 594GB (UD-IQ1_S) up to 861GB (UD-Q2_K_XL) and 1.56TB (UD-Q8_K_XL, lossless). Unsloth recommends at least 610GB of combined RAM and VRAM for the 594GB build. It trades accuracy for size: the 1-bit build keeps 78.9% top-1 agreement with the full model, the 2-bit XL 90.4%.
What is the cheapest way to run Kimi K3 at a usable speed? For almost everyone, the API. Moonshot charges $0.30 per million cache-hit input tokens, $3 input and $15 output, so a 500-token reply costs under one cent. The cheapest reported local setup at a chat-usable pace is a four-Mac-Studio cluster (4 x 512GB, roughly $60,000+) generating about 14.7 tokens per second on an ~817GB quant. Full-precision datacenter serving starts at 8 x B300 or 8 x MI355X.
The bottom line
The Kimi K3 8GB RAM demo is real, carefully engineered and practically unusable, and all three are true at once. kimi-k3-in-c proves that a 2.8-trillion-parameter MoE does not need 1.56TB of memory to run correctly: leave the 1.447TB of experts on NVMe, stream the trunk, and the same tokens come out at 8GB as at 224GB. What it cannot escape is physics: over 100GB of disk reads per token means half a minute per token, and that was on a 124-core server, not a laptop.
Read it as a research milestone and a superb MoE tutorial, not a way to chat with K3. Usable local K3 means a 600GB+ GGUF with accuracy loss, a $60,000+ Mac cluster or a datacenter node. Everyone else should use the API at $0.30 / $3 / $15: hundreds of times faster, for under a cent per answer.
Keep reading: the Kimi K3 VRAM requirements breakdown, every Kimi K3 API provider compared, the open-weights release, the full Kimi K3 specifications and our complete Kimi K3 guide.