Instructions to use FINAL-Bench/POCKET-Qwen3.8-Flash-Next-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use FINAL-Bench/POCKET-Qwen3.8-Flash-Next-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf FINAL-Bench/POCKET-Qwen3.8-Flash-Next-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf FINAL-Bench/POCKET-Qwen3.8-Flash-Next-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf FINAL-Bench/POCKET-Qwen3.8-Flash-Next-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf FINAL-Bench/POCKET-Qwen3.8-Flash-Next-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf FINAL-Bench/POCKET-Qwen3.8-Flash-Next-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf FINAL-Bench/POCKET-Qwen3.8-Flash-Next-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf FINAL-Bench/POCKET-Qwen3.8-Flash-Next-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf FINAL-Bench/POCKET-Qwen3.8-Flash-Next-GGUF:Q4_K_M

Use Docker

docker model run hf.co/FINAL-Bench/POCKET-Qwen3.8-Flash-Next-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use FINAL-Bench/POCKET-Qwen3.8-Flash-Next-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "FINAL-Bench/POCKET-Qwen3.8-Flash-Next-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FINAL-Bench/POCKET-Qwen3.8-Flash-Next-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/FINAL-Bench/POCKET-Qwen3.8-Flash-Next-GGUF:Q4_K_M

Ollama
How to use FINAL-Bench/POCKET-Qwen3.8-Flash-Next-GGUF with Ollama:
```
ollama run hf.co/FINAL-Bench/POCKET-Qwen3.8-Flash-Next-GGUF:Q4_K_M
```
Unsloth Desktop

How to use FINAL-Bench/POCKET-Qwen3.8-Flash-Next-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf FINAL-Bench/POCKET-Qwen3.8-Flash-Next-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "FINAL-Bench/POCKET-Qwen3.8-Flash-Next-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use FINAL-Bench/POCKET-Qwen3.8-Flash-Next-GGUF with Docker Model Runner:
```
docker model run hf.co/FINAL-Bench/POCKET-Qwen3.8-Flash-Next-GGUF:Q4_K_M
```

Lemonade

How to use FINAL-Bench/POCKET-Qwen3.8-Flash-Next-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull FINAL-Bench/POCKET-Qwen3.8-Flash-Next-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.POCKET-Qwen3.8-Flash-Next-GGUF-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use FINAL-Bench/POCKET-Qwen3.8-Flash-Next-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf FINAL-Bench/POCKET-Qwen3.8-Flash-Next-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default FINAL-Bench/POCKET-Qwen3.8-Flash-Next-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use FINAL-Bench/POCKET-Qwen3.8-Flash-Next-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf FINAL-Bench/POCKET-Qwen3.8-Flash-Next-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "FINAL-Bench/POCKET-Qwen3.8-Flash-Next-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

🆕 POCKET-Zimage-CPU — photoreal images in 46 s on a CPU only. No GPU, no CUDA, no Python.

📚 Collections

▶ POCKET Models — this family (on-device, no GPU) Darwin Family · Aether Foundation · VKAE Accelerated

POCKET-Qwen3.8-Flash-Next-GGUF

A 180B model on a laptop with an 8 GB GPU and 32 GB RAM. Measured on a real consumer laptop, not simulated: 4.17 ± 0.17 tok/s.

The file is 111 GiB. That is 3.5× the laptop's RAM and 14× its VRAM, and it still runs, on stock llama.cpp, with no server and no cloud.

Pick your build →

💻 The headline: 180B on the laptop you already own

[measured — 2026-09-16, llama-bench, 2 repetitions each]

Machine	Samsung Galaxy Book6 Ultra (consumer laptop)
GPU	NVIDIA RTX 5060 Laptop, 8 GB VRAM
RAM	32 GB (31.4 GB usable)
CPU	Intel Core Ultra 7 356H, 16 cores
Storage	NVMe SSD, 1.6 GB/s sequential read
Model	Qwen3.8-Flash-Next, 180B total / 6B active, Q4_K_M, 111 GiB

Build	Generate (tg256)	Prompt (pp64)	VRAM used
`Q4_K_M` · 8 threads	4.17 ± 0.17 tok/s	1.02 ± 0.12 tok/s	4.1 GB
`Q4_K_M` · 4 threads	2.80 ± 0.56	1.18 ± 0.06	4.1 GB
`Q4_K_M-v2` · 8 threads	2.93 ± 1.92	1.21 ± 0.03	3.7 GB
`Q4_K_M-v2` · 4 threads	2.55 ± 0.14	2.32 ± 0.27	3.7 GB

How a 111 GiB model runs in 32 GB of RAM. Only 6B of the 180B parameters are active per token (10 of 512 experts). llama.cpp memory-maps the file: the always-used layers sit on the 8 GB GPU, the experts stay in RAM and on the NVMe SSD, and each token only touches the few experts it needs. RAM decides the speed, not whether it runs.

🔴 Counter-intuitive, measured: do not fill the 8 GB GPU. Pushing expert layers onto the GPU made it slower (4.1 GB → 7.9 GB VRAM: 3.10 → 1.75 tok/s). The driver starts paging VRAM to system RAM. The command below is already the sweet spot.

Honest scope. About 4 tokens per second is reading speed for short answers, not chat speed for long ones. The point is that a frontier-scale 180B model runs locally, privately, on a laptop. Prompt processing is slow (1–2 tok/s), so keep prompts short. On a 64 GB machine more of the model stays in RAM and it gets faster (not yet measured by us).

Quickstart (laptop, 8 GB GPU)

# recent llama.cpp with Qwen3.8-Flash-Next support; point -m at the first shard
llama-cli -m Q4_K_M/POCKET-Qwen3.8-Flash-Next-Q4_K_M-00001-of-00003.gguf \
  -ngl 99 -ot "exps=CPU" -t 8 -p "안녕하세요"
# reproduce our numbers (always use -r 2 or more; single runs swing a lot)
llama-bench -m Q4_K_M/POCKET-Qwen3.8-Flash-Next-Q4_K_M-00001-of-00003.gguf -ngl 99 -ot "exps=CPU" -t 8 -p 64 -n 256 -r 2

-ot "exps=CPU" keeps the experts off the GPU. That is what makes it fit in 8 GB. Use physical cores for -t.

Files in this repo

Folder	Size	Notes
`Q4_K_M/` ⭐	111 GiB (3 shards)	fastest generation on the laptop (4.17 tok/s)
`Q4_K_M-v2/`	93 GiB (3 shards)	16% smaller, 2× faster prompt processing at 4 threads; our proprietary bit allocation

Both builds keep every tensor at 4-bit or higher. Text only: the base model's vision encoder and 4B MTP head are not included (the GGUF holds 177B of the 180B).

Quality — measured, including where we do not win

[measured] Korean perplexity: Wikipedia-Korean (20231101.ko), exactly 80 chunks (40,960 tokens), -c 512, lower is better:

Build	Size	Korean PPL
POCKET `Q4_K_M`	111 GiB	6.030 ± 0.103
POCKET `Q4_K_M-v2`	93 GiB	6.080 ± 0.104
unsloth `UD-IQ4_XS` (reference)	88 GB	6.013 ± 0.103

All three are within error of each other: no measurable quality difference. We do not claim a better quantization. What we add is the laptop measurement itself: to our knowledge nobody else has published a measured 180B run on an 8 GB / 32 GB consumer laptop.

Where it fits in the POCKET lineup

Your hardware	Pick
Phone (8–12 GB)	POCKET-KR-MLX · POCKET-KR-GGUF · POCKET-EN-GGUF
Mini-PC / PC, no GPU	POCKET-35B-GGUF · POCKET-26B-GGUF
Laptop with 8 GB GPU + 32 GB RAM, want the biggest model	this repo, 180B

Lineage

Component	Origin
Base model	Qwen/Qwen3.8-Flash-Next (Qwen), 180B total, 6B active, 512 experts / 10 active
Quantization	GGUF `Q4_K_M` built by VIDRAFT from the BF16 weights; `-v2` uses our proprietary bit allocation
Runtime	upstream llama.cpp, unmodified
Measurement	VIDRAFT, on the laptop above

Limitations

~3–4 tok/s generation and 1–2 tok/s prompt processing on the 8 GB / 32 GB laptop. Usable for short, private questions, not for long chats.
93–111 GiB download; put it on a fast NVMe SSD (speed depends on it).
Text only (no vision encoder, no MTP head).
Needs a llama.cpp build new enough to support Qwen3.8-Flash-Next.

License

Derived from Qwen3.8-Flash-Next under the Qwen Community License 1.0 (included in this repo). Note clause 2: a Model-as-a-Service or AI Work Assistant business needs a separate license from Qwen for commercial use.

POCKET is a VIDRAFT model family. Big models, small hardware.

Learn more

On-device LLMs without a GPU — and how POCKET measures up: Can you run a large LLM without a GPU?
What model quantization is, and why a 4-bit model stays smart: What is model quantization?

🧩 The POCKET Family — On-device AI by VIDRAFT

Big models, small hardware. No GPU, no cloud.

Models

📦 POCKET-35B-GGUF — flagship, PC / server, no GPU
📦 POCKET-26B-GGUF — compact 26B
🇰🇷 POCKET-KR-GGUF — Korean, Android
🍎 POCKET-KR-MLX — Korean, iPhone / Mac
🌍 POCKET-EN-GGUF — English, phone / PC
💻 POCKET-Qwen3.8-Flash-Next-GGUF — 180B on a laptop (8 GB VRAM + 32 GB RAM)
🖼️ POCKET-Image-Zimage — character-perfect text in any image
🖥️ POCKET-Zimage-CPU — photoreal images on a CPU only

Demos & tools (Spaces)

🎨 POCKET-Image Studio — text-in-image, generate in-page
🖥️ POCKET-35B-CPU — 35B answering on a CPU
🖥️ POCKET-26B-CPU — 26B on a CPU
🖼️ POCKET-Zimage-CPU — image generation on a CPU

📚 Full POCKET collection

Downloads last month: 71

GGUF

Model size

177B params

Architecture

qwen4exp

Hardware compatibility

4-bit

Model tree for FINAL-Bench/POCKET-Qwen3.8-Flash-Next-GGUF

Base model

Qwen/Qwen3.8-Flash-Next

Quantized

(286)

this model

Collection including FINAL-Bench/POCKET-Qwen3.8-Flash-Next-GGUF

POCKET-MODELs

Collection

12 items • Updated 1 day ago • 42

FINAL-Bench
/

POCKET-Qwen3.8-Flash-Next-GGUF

🆕 POCKET-Zimage-CPU — photoreal images in 46 s on a CPU only. No GPU, no CUDA, no Python.

📚 Collections

POCKET-Qwen3.8-Flash-Next-GGUF

A 180B model on a laptop with an 8 GB GPU and 32 GB RAM. Measured on a real consumer laptop, not simulated: 4.17 ± 0.17 tok/s.

💻 The headline: 180B on the laptop you already own

Quickstart (laptop, 8 GB GPU)

Files in this repo

Quality — measured, including where we do not win

Where it fits in the POCKET lineup

Lineage

Limitations

License

Learn more

🧩 The POCKET Family — On-device AI by VIDRAFT

Model tree for FINAL-Bench/POCKET-Qwen3.8-Flash-Next-GGUF

Collection including FINAL-Bench/POCKET-Qwen3.8-Flash-Next-GGUF

POCKET-MODELs