infosave/Kimi-Linear-48B-A3B-Code-CMF

Kimi-Linear-48B-A3B — CMF code specialist (17.7 GB, runs on a 24 GB MacBook)

A code-calibrated specialist build of moonshotai/Kimi-Linear-48B-A3B-Instruct in the CMF format — one file, mmap-served, no Python at inference:

	original bf16	CMF q4t (full)	this file
Size	98 GB	27.7 GB	17.7 GB
Held-out code ppl	—	7.11	7.30 (+2.7%)
Decode, M4 MacBook 24 GB	—	4.8 tok/s (pages)	11.1 tok/s
Decode, RTX A5000 24 GB (CUDA)	—	—	164 tok/s
Decode, RTX A5000 24 GB (Vulkan)	—	—	97 tok/s

The speedup is structural: the full 27.7 GB file does not fit a 24 GB page cache and pages on every token; the specialist does fit, so the same machine decodes ×2.3 faster.

How it was made

1. Convert (pure Rust, streamed — one shard on disk at a time, so a 98 GB checkpoint converts on a laptop): cortiq convert --model moonshotai/Kimi-Linear-48B-A3B-Instruct --quant q4t --output kimi48-q4t.cmf The engine executes Kimi's KDA (Kimi Delta Attention: delta rule with per-channel decay, per-projection short convolutions, sigmoid-gated output norm), NoPE MLA full-attention layers, and the sigmoid MoE router with its selection bias. The tiktoken rank table becomes a standard tokenizer.json at convert time.
2. Calibrate: run a representative code corpus once with CMF_MOE_STATS=stats.json — the engine records per-layer expert routing frequencies. On code, the top 64 of 256 experts carry 73% of the routing mass.
3. Defrag: cortiq moe-defrag kimi48-q4t.cmf --stats stats.json --cover 0.95 --output kimi48-code.cmf Per layer, the smallest expert set covering 95% of the recorded routing mass is kept (~160 of 256); experts are renumbered into a dense prefix and the router rows AND the noaux selection bias are sliced to match. Runtime semantics equal the runtime expert mask — the ppl of the cut file is bit-identical to masking the full file.

The expert restriction is task-shaped: this file is at its best on code and technical text. For general-purpose use, convert the full model yourself with the command above (30.7 GB of free disk is enough).

Speed on NVIDIA (RTX A5000 24 GB, CUDA, cortiq 0.8.98)

cortiq bench, CMF_GPU=cuda on a build with the CUDA backend (cargo install cortiq-cli --features cuda, the NVIDIA driver and the CUDA 12.x libraries). Kimi Delta Attention and MLA run on the card:

Mode	Speed
decode	162.6–164.1 tok/s
4096-token prompt	2,210–2,255 tok/s (first token 1.82–1.85 s)
decode after the 4096-token prompt	134.4–136.1 tok/s

Speed on Vulkan (RTX A5000 24 GB, cortiq 0.8.98)

cortiq bench on the default build (Vulkan / DX12). Kimi Delta Attention, MLA and the experts run on the card:

Mode	Speed
decode	96.6–97.6 tok/s
4096-token prompt	507 tok/s
decode after the 4096-token prompt	67.7–72.0 tok/s

Run it

cargo install cortiq-cli          # pure Rust, no Python
hf download infosave/Kimi-Linear-48B-A3B-Code-CMF kimi48-code.cmf --local-dir .

cortiq run kimi48-code.cmf --prompt "Write a Python function that returns the n-th Fibonacci number iteratively." --max-tokens 120
cortiq serve kimi48-code.cmf      # OpenAI-compatible API

License

MIT, inherited from the base model. Weights © Moonshot AI; this repackaging only changes the storage format and the served expert set.

Ecosystem

• Engine / converter / format spec: https://github.com/infosave2007/cmf — the pure-Rust runtime this file targets (cargo install cortiq-cli).
• CMF Mobile — a Flutter app on the same runtime: local on-device chat, or the phone as an OpenAI-compatible server. This 17.7 GB specialist wants a desktop's RAM; on phones pick a smaller CMF build (e.g. Bonsai-1.7B or Nanbeige 4.2 3B).