Qwen3.5-2B-Opus46-Reasoning

Fine-tuned Qwen3.5-2B on the Opus-4.6-Reasoning-3300x dataset using LoRA (r=64) via Unsloth.

Benchmark

Benchmark	Base Qwen3.5-2B	This model
GSM8K	21.5%	58.0%

Training details

Parameter	Value
Base model	unsloth/Qwen3.5-2B
Dataset	crownelius/Opus-4.6-Reasoning-3300x (2160 examples)
LoRA rank	64
LoRA alpha	64
Epochs	3
Batch size	32 (2 × 16 grad accum)
Learning rate	1e-4 (cosine)
Sequence length	4096
Precision	bf16

Benchmark

Benchmark	Base Qwen3.5-2B	This model
GSM8K (200 samples)	21.5%	58.0%

Evaluated with greedy decoding (temperature=0). Answer extracted from the model's solution section (after </think>), prioritising **Answer:** lines and bold numbers.

Training

Fine-tuned using Unsloth for 2x faster training.

from unsloth import FastModel
from datasets import load_dataset
from trl import SFTTrainer, SFTConfig

model, tokenizer = FastModel.from_pretrained(
    model_name="unsloth/Qwen3.5-2B", max_seq_length=4096, load_in_16bit=True
)
model = FastModel.get_peft_model(model, r=64, lora_alpha=64,
    target_modules=["q_proj","k_proj","v_proj","o_proj","gate_proj","up_proj","down_proj"])

dataset = load_dataset("crownelius/Opus-4.6-Reasoning-3300x", split="train")

# format as native Qwen3.5 thinking format
def fmt(row):
    return (f"<|im_start|>user\n{row['problem']}<|im_end|>\n"
            f"<|im_start|>assistant\n<think>\n{row['thinking']}\n</think>\n"
            f"{row['solution']}<|im_end|>")

Usage

from unsloth import FastModel

model, tokenizer = FastModel.from_pretrained(
    model_name     = "supersamdev/Qwen3.5-2B-Opus46-Reasoning",
    max_seq_length = 4096,
    load_in_16bit  = True,
)
FastModel.for_inference(model)

messages = [{"role": "user", "content": "What is 25 × 48?"}]
prompt = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True, enable_thinking=True
)
inputs = tokenizer(text=prompt, return_tensors="pt").to("cuda")
out = model.generate(**inputs, max_new_tokens=512, temperature=0.6, do_sample=True)
print(tokenizer.decode(out[0], skip_special_tokens=True))

GGUF variants

Quantized versions available at supersamdev/Qwen3.5-2B-Opus46-Reasoning-GGUF.