darkmaniac7/TokForge-LLM-QNN-NPU

Qwen3 0.6B, 1.7B and 4B Abliterated (QNN NPU packs)

Four NPU packs for Qwen3 models in TokForge: the NPU of a Snapdragon phone reads long prompts, and the CPU writes every reply with the model you already downloaded.

Published by TokForge for on-device inference.

What it is	NPU prompt pack (four packs for Qwen3 0.6B, 1.7B and 4B Abliterated)
Format	QNN context binaries for the Hexagon NPU plus a small MNN graph, built with QAIRT 2.40; one folder per model and chip, 0.62 to 4.09 GB each
Runs on	Snapdragon phones (Hexagon NPU), with the paired MNN model on the CPU
Phone class	Snapdragon 8 Gen 3 or Snapdragon 8 Elite Gen 5
Download	16.4 GB (59 files)
Base model	Qwen/Qwen3-0.6B, Qwen/Qwen3-1.7B, huihui-ai/Huihui-Qwen3-4B-abliterated-v2
License	Apache-2.0

TokForge

• Website: https://tokforge.ai
• Discord: https://discord.gg/Acv3CBtfVm
• Google Play: https://play.google.com/store/apps/details?id=dev.tokforge
• iOS TestFlight: https://testflight.apple.com/join/jnufjzRr

Runs on-device in the TokForge app.

Use it in TokForge

1. Install TokForge from Google Play.
2. On Android, open Models > Recommended > Chat and download Qwen3 1.7B (listed on phones with 6 GB RAM or more) or Qwen3 4B Abliterated (8 GB RAM or more). The Qwen3 0.6B pack pairs with taobao-mnn/Qwen3-0.6B-MNN at commit 34dfccda. The current app does not list that model under Recommended; if it is on your phone, TokForge can use it for background work such as summaries and offers its pack.
3. Open the model's settings and turn on Faster long prompts (NPU), or tap Get next to the model under NPU packs for your phone in Models > Browse. TokForge downloads the pack for your chip, checks every file by sha256 and asks for a restart before first use.
4. Load the model and chat as usual. Long prompts are read on the NPU; replies are written on the CPU.

TokForge 1.3.8 and later know these packs and download them from this repo at a fixed commit. The Qualcomm runtime libraries are not in this repo; TokForge 1.3.9 and later ship them inside the app (TokForge 1.3.8 fetched them from darkmaniac7/TokForge-SD15-QNN-NPU, qnn_runtime/2.40).

Collections: TokForge NPU Packs.

Files

File	Size	What it is
Qwen3-0.6B-MNN-catalogpair/	1.24 GB in 11 files	Qwen3 0.6B pack: MANIFEST.json plus v75/ and v81/, each with 2 QNN context binaries, qnn/llm.mnn, qnn/llm_config.json and config_qnn.json
Qwen3-1.7B-MNN-catalogpair/	3.51 GB in 13 files	Qwen3 1.7B pack for the earlier taobao-mnn download: MANIFEST.json plus v75/ and v81/, each with 3 QNN context binaries and the same small files
Qwen3-1.7B-MNN-HQQ-catalogpair/	3.51 GB in 13 files	Qwen3 1.7B pack for the HQQ download: MANIFEST.json plus v75/ and v81/, each with 3 QNN context binaries and the same small files
Qwen3-4B-abliterated-MNN-catalogpair/	8.18 GB in 19 files	Qwen3 4B Abliterated pack: MANIFEST.json plus v75/ and v81/, each with 6 QNN context binaries and the same small files
ATTRIBUTION.txt	1.42 KB	Source models and paired bundles of each pack, with commits and licences
LICENSE	11.3 KB	Apache License 2.0 text (as distributed with Qwen/Qwen3-1.7B)
MODIFICATIONS.txt	1.49 KB	What the NPU packaging changed

Each pack folder holds MANIFEST.json (every file per chip with sha256 and size, and the exact download it pairs with) and one folder per chip (v75/, v81/) with config_qnn.json (runtime config for the NPU session), qnn/graph*.bin (the Qualcomm QNN context binaries: the model body, compiled for that NPU), qnn/llm.mnn (the MNN graph that drives them) and qnn/llm_config.json.

What you get

• Sooner first replies on long prompts, depending on the model and chip.
• Replies still come from your model: the CPU writes every reply with the model you downloaded; the pack only reads the prompt.
• Checked before use: every file is verified by sha256 on download, and TokForge runs a quick on-device check before it uses the NPU. If the check does not pass, prompts are read on the CPU as before.
• Download only what your phone needs: one folder per model and chip, fetched only when you turn the setting on or tap Get.

The original models are by the Qwen team. The Qwen3 4B pack is built from huihui-ai/Huihui-Qwen3-4B-abliterated-v2, an abliterated variant of Qwen3 4B. These packs only change the runtime format for the phone's NPU.

Supported phones

Chip	NPU	Folder
Snapdragon 8 Gen 3 (SM8650)	Hexagon V75	v75/
Snapdragon 8 Elite Gen 5 (SM8850)	Hexagon V81	v81/

Packs

Folder	Model in TokForge	Works with this exact download	Size per chip
Qwen3-1.7B-MNN-HQQ-catalogpair	Qwen3 1.7B	darkmaniac7/Qwen3-1.7B-MNN-HQQ @ 15ab6d1d	1.75 GB
Qwen3-1.7B-MNN-catalogpair	Qwen3 1.7B (earlier download)	taobao-mnn/Qwen3-1.7B-MNN @ 1b12fe7d	1.75 GB
Qwen3-4B-abliterated-MNN-catalogpair	Qwen3 4B Abliterated	darkmaniac7/Qwen3-4B-abliterated-MNN @ edba0fdb	4.09 GB
Qwen3-0.6B-MNN-catalogpair	Qwen3 0.6B (worker model)	taobao-mnn/Qwen3-0.6B-MNN @ 34dfccda	0.62 GB

A pack is matched to its model download by file hashes. With any other download it is not used.

How it was made

Each pack is exported from its paired download's own weights: MNN's loader dequantizes them, and MNN llmexport writes 8-bit per-channel weights for the NPU graph. Rotary position tables are fed from fp32 tables, as on the CPU. The body is compiled offline into QNN context binaries with Qualcomm AI Runtime 2.40. Source training weights were not edited. See MODIFICATIONS.txt.

Limitations

• Snapdragon 8 Gen 3 and 8 Elite Gen 5 only. On other chips TokForge reads prompts on the CPU.
• Long prompts only: the NPU reads prompts of about 512 to 2,000 tokens; shorter prompts stay on the CPU.
• Runtime files for TokForge and MNN, not a Transformers checkpoint.
• The models are small: they can still get facts wrong. Check important answers.

License

Apache License 2.0 (LICENSE). The original models are by the Qwen team; the Qwen3 4B pack builds on huihui-ai's abliterated variant. All source authors keep their copyrights; no endorsement is claimed. Details in ATTRIBUTION.txt and MODIFICATIONS.txt.

The context binaries were compiled with Qualcomm's QAIRT SDK. The Qualcomm runtime libraries that run them are not in this repo; TokForge ships them inside the app under the Qualcomm AI Stack License.

Credits

• Qwen3-0.6B by the Qwen team (Alibaba Cloud) (Hugging Face). Used under the Apache-2.0 license.
• Qwen3-1.7B by the Qwen team (Alibaba Cloud) (Hugging Face). Used under the Apache-2.0 license.
• Huihui-Qwen3-4B-abliterated-v2 by huihui-ai (Hugging Face). Used under the Apache-2.0 license.
• Qwen3-4B by the Qwen team (Alibaba Cloud) (Hugging Face). Used under the Apache-2.0 license.
• The paired MNN bundles taobao-mnn/Qwen3-0.6B-MNN and taobao-mnn/Qwen3-1.7B-MNN by the taobao-mnn team. Used under the Apache-2.0 license.
• Converted with llmexport from MNN by Alibaba (Apache-2.0).
• Compiled for the Hexagon NPU with Qualcomm AI Runtime (QAIRT) 2.40 by Qualcomm.