Qwen3.5 9B (MNN)

Alibaba's Qwen3.5 9B as taobao-mnn's 4-bit MNN bundle, mirrored with an optional MTP head, for chatting with text and images on your phone in TokForge.

Published by TokForge for on-device inference.

What it is Vision chat model (text + images)
Format MNN bundle, 4-bit weights (block 64, HQQ), bf16 embeddings, with an optional MTP head
Runs on Android phones (CPU)
Phone class 12 GB RAM or more (TokForge catalog)
Download 7.51 GB (13 files)
Base model Qwen/Qwen3.5-9B
License Apache-2.0

TokForge

Runs on-device in the TokForge app.

Use it in TokForge

  1. Install TokForge from Google Play, or join the iOS TestFlight.
  2. On Android, open Models > Recommended > Chat and download Qwen3.5 9B. TokForge lists it on phones with 12 GB RAM or more.
  3. Load it and start a chat. The model runs on the phone; once it is downloaded, chatting needs no connection.

On phones with a Snapdragon 8 Elite Gen 5 and TokForge 1.3.9.6 or later, TokForge can also add an NPU prompt pack for this model, darkmaniac7/Qwen3.5-9B-NPU-catalogpair: the NPU reads long prompts and the CPU writes the reply with this bundle. The pack needs about 6.6 GB more memory while the model is loaded, so in practice a 24 GB phone. TokForge downloads the pack only when you ask for it: tap Get under NPU packs for your phone in Models > Browse, or turn on Faster long prompts (NPU) in the model's settings.

Collections: TokForge Chat Models (MNN), TokForge Qwen Models (MNN), TokForge Vision Models.

Files

File Size What it is
config.json 677 B MNN runtime settings (taobao-mnn's, plus dynamic_option: 2)
embeddings_bf16.bin 2.03 GB Token embeddings (bf16)
export_args.json 1.04 KB Export settings used for the conversion
llm.mnn 3.65 MB Model graph
llm.mnn.json 9.17 MB Model graph as JSON (reference copy)
llm.mnn.weight 4.96 GB Model weights
llm_config.json 8.58 KB Model settings and chat template
mtp-v32k.mnn 34.0 KB MTP draft head graph (optional)
mtp-v32k.mnn.vocab 132 KB MTP draft head vocabulary (optional)
mtp-v32k.mnn.weight 237 MB MTP draft head weights (optional)
tokenizer.txt 6.47 MB Tokenizer
visual.mnn 548 KB Vision encoder graph
visual.mnn.weight 259 MB Vision encoder weights

Every file except config.json and the three mtp-v32k.* files is taobao-mnn's original, unchanged (see "Trunk files" below).

About the model

Qwen3.5 9B is a mid-size model in Alibaba's Qwen3.5 family. Qwen3.5 models read text and images, mix Gated DeltaNet (linear attention) layers with full-attention layers, cover 201 languages and dialects, and think before they answer by default. TokForge runs Qwen3.5 on the CPU by default.

Architecture Qwen3.5 (32 layers: 24 Gated DeltaNet + 8 full attention), with vision encoder
Parameters 9B (4-bit quantized)
Quantization 4-bit weights, block size 64, HQQ (taobao-mnn's export settings); embeddings in bf16
Vocab 248,320 tokens
Context 262,144 tokens in the original model

Trunk files (same as taobao-mnn, except config.json)

This repo mirrors taobao-mnn/Qwen3.5-9B-MNN at commit 149e1e052267ffa377b87e3bb0ddb75f46217a9a and adds an optional multi-token-prediction (MTP) head next to the unchanged trunk, used by TokForge on the MNN CPU route. Every trunk file except config.json is byte-identical to the original (same sha256); the MNN conversion and all credit for it belong to the taobao-mnn team, the model to the Qwen team.

config.json adds one runtime setting, "dynamic_option": 2 (since 2026-10-14); nothing else in it changed. The original is 652 bytes, sha256 92853033efe602f95efca3e1c05cd8b108f973c8beed417843a9671f8147ed8d. The setting turns on per-block activation quantization in MNN's tiled int8 CPU path, which phones without Arm's i8mm instructions use for the model's layers. TokForge picks this option per phone at each load, so the value in the file matters when you run the bundle with MNN yourself.

file bytes sha256
config.json 677 b402f08907a899e658450a66069a746def7e108ec029f775f4e93c24e92752ee
embeddings_bf16.bin 2,034,237,440 d87d444a38822588c6c1ed07dec4404ccd8d526f57d58565cbbbf60acc758afe
export_args.json 1,041 5d9cb1fc15d189e86b3117576f0e7a6875295a16628f6dc9d9ad4fda9ef73d19
llm.mnn 3,651,096 fe9101e1b1754c304bef2091114c260cffbbac0ca6180110af0a04efd7f24c13
llm.mnn.json 9,172,377 7d87ce4a07e74366d4a2126369ac8f8ac753612bb3b1a9cb16e062e438d816ee
llm.mnn.weight 4,961,957,226 896bf5a7b7e164186f1c0d065cb6d92306c9aa9c3cb62047531e24fa08c033af
llm_config.json 8,582 6c7547f03faf34af1a916e4cd700bf3c5be7e14ef75da75a04f88bc1ea49e86b
tokenizer.txt 6,465,727 7e75de1f279a10b65bd9dc1a5207205cb8993823861c4c42bbbd74e48e1c23a4
visual.mnn 547,632 f85e3ea5a2b51f267d8d8a857cc05da4ea2ae1f5fc1f98ec676d8130f31df748
visual.mnn.weight 258,831,244 3c9988e012c35b1d22e03412f316b2dc24e993ac8b149b3ad2b4095917f3c90c

MTP head (optional add-on)

The three mtp-v32k.* files are a multi-token-prediction (MTP) head for the trunk above. The head drafts the next token and the trunk verifies every drafted token, so output quality is the trunk's own. TokForge uses it only on the MNN CPU route: it downloads the head with the model (or later, for a model that is already installed), checks each file's size and sha256, and decides per phone whether to use it.

Exported from Qwen/Qwen3.5-9B @ c202236235762e1c871ad0ccb60c8ee5ba337b9a with TokForge's fork of MNN, llmexport --mtp --mtp_only --quant_bit 4 --quant_block 64 and a 32k draft vocabulary (mtp-v32k.mnn.vocab, int32 token ids). To use it with MNN yourself, load with speculative_type: "mtp", mtp_model: "mtp-v32k.mnn", draft_predict_length: 1.

file bytes sha256
mtp-v32k.mnn.vocab 132,160 80af0c499240994c092bb3e79fe4d32a4e9e8dbc82fbbf8d728158f11fbae1b3
mtp-v32k.mnn.weight 236,794,054 20606e4785e6c724b1f9038e2835c65ba6608ce995988a855b2590afc2b5c700
mtp-v32k.mnn 34,024 66b26ccdc76d324423dd5ff5ba50c78276c98ade011b772f69231a4c395ce5f3

How it was made

  • Trunk: taobao-mnn's MNN export of Qwen/Qwen3.5-9B (4-bit, block 64, HQQ, per its export_args.json), copied unchanged except for the one added line in config.json.
  • MTP head: exported by TokForge from the original Qwen checkpoint, as described above.

Limitations

  • Intended for TokForge / MNN on-device inference
  • This is a runtime bundle, not a standard Transformers checkpoint
  • Quantization (Q4) may slightly reduce quality compared to the full-precision original

License

License: Apache-2.0, as the original model and conversion.

Credits

  • Qwen3.5-9B by the Qwen team (Alibaba Cloud) (Hugging Face). Used under the Apache-2.0 license.
  • The MNN conversion (every trunk file here except config.json, which adds one runtime setting, is byte-identical to it) by the taobao-mnn team (Hugging Face). Used under the Apache-2.0 license.
  • The MTP head is exported from Qwen/Qwen3.5-9B (Apache-2.0) by TokForge with llmexport from TokForge's fork of MNN.
Downloads last month
246
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for darkmaniac7/Qwen3.5-9B-MNN

Finetuned
Qwen/Qwen3.5-9B
Quantized
(733)
this model

Collections including darkmaniac7/Qwen3.5-9B-MNN