Qwen3.5 9B (MNN)
Alibaba's Qwen3.5 9B as taobao-mnn's 4-bit MNN bundle, mirrored with an optional MTP head, for chatting with text and images on your phone in TokForge.
Published by TokForge for on-device inference.
| What it is | Vision chat model (text + images) |
| Format | MNN bundle, 4-bit weights (block 64, HQQ), bf16 embeddings, with an optional MTP head |
| Runs on | Android phones (CPU) |
| Phone class | 12 GB RAM or more (TokForge catalog) |
| Download | 7.51 GB (13 files) |
| Base model | Qwen/Qwen3.5-9B |
| License | Apache-2.0 |
TokForge
- Website: https://tokforge.ai
- Discord: https://discord.gg/Acv3CBtfVm
- Google Play: https://play.google.com/store/apps/details?id=dev.tokforge
- iOS TestFlight: https://testflight.apple.com/join/jnufjzRr
Runs on-device in the TokForge app.
Use it in TokForge
- Install TokForge from Google Play, or join the iOS TestFlight.
- On Android, open Models > Recommended > Chat and download Qwen3.5 9B. TokForge lists it on phones with 12 GB RAM or more.
- Load it and start a chat. The model runs on the phone; once it is downloaded, chatting needs no connection.
On phones with a Snapdragon 8 Elite Gen 5 and TokForge 1.3.9.6 or later, TokForge can also add an NPU prompt pack for this model, darkmaniac7/Qwen3.5-9B-NPU-catalogpair: the NPU reads long prompts and the CPU writes the reply with this bundle. The pack needs about 6.6 GB more memory while the model is loaded, so in practice a 24 GB phone. TokForge downloads the pack only when you ask for it: tap Get under NPU packs for your phone in Models > Browse, or turn on Faster long prompts (NPU) in the model's settings.
Collections: TokForge Chat Models (MNN), TokForge Qwen Models (MNN), TokForge Vision Models.
Files
| File | Size | What it is |
|---|---|---|
config.json |
677 B | MNN runtime settings (taobao-mnn's, plus dynamic_option: 2) |
embeddings_bf16.bin |
2.03 GB | Token embeddings (bf16) |
export_args.json |
1.04 KB | Export settings used for the conversion |
llm.mnn |
3.65 MB | Model graph |
llm.mnn.json |
9.17 MB | Model graph as JSON (reference copy) |
llm.mnn.weight |
4.96 GB | Model weights |
llm_config.json |
8.58 KB | Model settings and chat template |
mtp-v32k.mnn |
34.0 KB | MTP draft head graph (optional) |
mtp-v32k.mnn.vocab |
132 KB | MTP draft head vocabulary (optional) |
mtp-v32k.mnn.weight |
237 MB | MTP draft head weights (optional) |
tokenizer.txt |
6.47 MB | Tokenizer |
visual.mnn |
548 KB | Vision encoder graph |
visual.mnn.weight |
259 MB | Vision encoder weights |
Every file except config.json and the three mtp-v32k.* files is taobao-mnn's original, unchanged (see "Trunk files" below).
About the model
Qwen3.5 9B is a mid-size model in Alibaba's Qwen3.5 family. Qwen3.5 models read text and images, mix Gated DeltaNet (linear attention) layers with full-attention layers, cover 201 languages and dialects, and think before they answer by default. TokForge runs Qwen3.5 on the CPU by default.
| Architecture | Qwen3.5 (32 layers: 24 Gated DeltaNet + 8 full attention), with vision encoder |
| Parameters | 9B (4-bit quantized) |
| Quantization | 4-bit weights, block size 64, HQQ (taobao-mnn's export settings); embeddings in bf16 |
| Vocab | 248,320 tokens |
| Context | 262,144 tokens in the original model |
Trunk files (same as taobao-mnn, except config.json)
This repo mirrors taobao-mnn/Qwen3.5-9B-MNN at commit
149e1e052267ffa377b87e3bb0ddb75f46217a9a and adds an optional multi-token-prediction (MTP) head next to the unchanged
trunk, used by TokForge on the MNN CPU route. Every trunk file except config.json is byte-identical to the original
(same sha256); the MNN conversion and all credit for it belong to the taobao-mnn team, the model to the Qwen team.
config.json adds one runtime setting, "dynamic_option": 2 (since 2026-10-14); nothing else in it changed. The
original is 652 bytes, sha256 92853033efe602f95efca3e1c05cd8b108f973c8beed417843a9671f8147ed8d. The setting turns on
per-block activation quantization in MNN's tiled int8 CPU path, which phones without Arm's i8mm instructions use for
the model's layers. TokForge picks this option per phone at each load, so the value in the file matters when you run
the bundle with MNN yourself.
| file | bytes | sha256 |
|---|---|---|
config.json |
677 | b402f08907a899e658450a66069a746def7e108ec029f775f4e93c24e92752ee |
embeddings_bf16.bin |
2,034,237,440 | d87d444a38822588c6c1ed07dec4404ccd8d526f57d58565cbbbf60acc758afe |
export_args.json |
1,041 | 5d9cb1fc15d189e86b3117576f0e7a6875295a16628f6dc9d9ad4fda9ef73d19 |
llm.mnn |
3,651,096 | fe9101e1b1754c304bef2091114c260cffbbac0ca6180110af0a04efd7f24c13 |
llm.mnn.json |
9,172,377 | 7d87ce4a07e74366d4a2126369ac8f8ac753612bb3b1a9cb16e062e438d816ee |
llm.mnn.weight |
4,961,957,226 | 896bf5a7b7e164186f1c0d065cb6d92306c9aa9c3cb62047531e24fa08c033af |
llm_config.json |
8,582 | 6c7547f03faf34af1a916e4cd700bf3c5be7e14ef75da75a04f88bc1ea49e86b |
tokenizer.txt |
6,465,727 | 7e75de1f279a10b65bd9dc1a5207205cb8993823861c4c42bbbd74e48e1c23a4 |
visual.mnn |
547,632 | f85e3ea5a2b51f267d8d8a857cc05da4ea2ae1f5fc1f98ec676d8130f31df748 |
visual.mnn.weight |
258,831,244 | 3c9988e012c35b1d22e03412f316b2dc24e993ac8b149b3ad2b4095917f3c90c |
MTP head (optional add-on)
The three mtp-v32k.* files are a multi-token-prediction (MTP) head for the trunk above. The head drafts the next token
and the trunk verifies every drafted token, so output quality is the trunk's own. TokForge uses it only on the MNN CPU
route: it downloads the head with the model (or later, for a model that is already installed), checks each file's size
and sha256, and decides per phone whether to use it.
Exported from Qwen/Qwen3.5-9B @ c202236235762e1c871ad0ccb60c8ee5ba337b9a with TokForge's fork of MNN, llmexport --mtp --mtp_only --quant_bit 4 --quant_block 64
and a 32k draft vocabulary (mtp-v32k.mnn.vocab, int32 token ids). To use it with MNN yourself, load with
speculative_type: "mtp", mtp_model: "mtp-v32k.mnn", draft_predict_length: 1.
| file | bytes | sha256 |
|---|---|---|
mtp-v32k.mnn.vocab |
132,160 | 80af0c499240994c092bb3e79fe4d32a4e9e8dbc82fbbf8d728158f11fbae1b3 |
mtp-v32k.mnn.weight |
236,794,054 | 20606e4785e6c724b1f9038e2835c65ba6608ce995988a855b2590afc2b5c700 |
mtp-v32k.mnn |
34,024 | 66b26ccdc76d324423dd5ff5ba50c78276c98ade011b772f69231a4c395ce5f3 |
How it was made
- Trunk: taobao-mnn's MNN export of Qwen/Qwen3.5-9B (4-bit, block 64, HQQ, per its
export_args.json), copied unchanged except for the one added line inconfig.json. - MTP head: exported by TokForge from the original Qwen checkpoint, as described above.
Limitations
- Intended for TokForge / MNN on-device inference
- This is a runtime bundle, not a standard Transformers checkpoint
- Quantization (Q4) may slightly reduce quality compared to the full-precision original
License
License: Apache-2.0, as the original model and conversion.
Credits
- Qwen3.5-9B by the Qwen team (Alibaba Cloud) (Hugging Face). Used under the Apache-2.0 license.
- The MNN conversion (every trunk file here except
config.json, which adds one runtime setting, is byte-identical to it) by the taobao-mnn team (Hugging Face). Used under the Apache-2.0 license. - The MTP head is exported from Qwen/Qwen3.5-9B (Apache-2.0) by TokForge with llmexport from TokForge's fork of MNN.
- Downloads last month
- 246