empero-ai/Qwen3.5-9B-Claude-Code

CodeClawd — Qwen3.5-9B Claude Opus 4.6 Distill (SFT on DataClaw)

A supervised fine-tune of empero-ai/Qwen3.5-9B-Claude-Opus-4.6-Distill on real Claude Code / Codex agentic sessions from the DataClaw dataset collection.

The model is trained to behave like a capable coding agent: thinking through problems, calling tools, interpreting results, and responding clearly — mirroring the full agentic loop of a real Claude Code session.

---

Model Details

Property	Value
Base model	empero-ai/Qwen3.5-9B-Claude-Opus-4.6-Distill
Fine-tune type	SFT (LoRA, merged)
LoRA rank / alpha	r=64, alpha=128
Training data	DataClaw (Claude Code + Codex traces)
Sequence length	4096 tokens
Quantization (train)	4-bit NF4 (bitsandbytes)
Optimizer	paged_adamw_8bit
LR scheduler	cosine
Learning rate	2e-4

---

Training Data

Trained on 29 DataClaw datasets containing real agentic coding sessions. Sessions were quality-filtered to require:

• At least 4 messages per session
• At least 2 assistant turns
• At least 1 tool use
• Formatted length between 200–100,000 characters

Datasets include contributions from: peteromallet, woctordho, zhiyaowang, parani01, vaynelee, Batman787, sunsun123new, GolienHzmsr, tillg, DJTRIXUK, emperorfutures, MRiabov, misterkerns, REXX-NEW, xuechengjiang, akenove, michaelwaves, introvoyz041, gutenbergpbc, and others.

---

Agentic Format

The model is trained on the full agentic loop:

user → <think>...</think> → <tool_call> → <tool_result> → response → repeat

Tool Call Format

<tool_call>
<tool>Read</tool>
<input>
{"file_path": "/path/to/file.py"}
</input>
</tool_call>

<tool_result>
<tool>Read</tool>
<output>
... file contents ...
</output>
</tool_result>

Thinking Format

The model reasons inside <think> tags before acting:

<think>
The user wants to refactor the auth module. I should first read the existing
implementation before suggesting changes.
</think>

System Prompt

You are an expert AI coding assistant. You help users with software engineering
tasks including writing code, debugging, refactoring, explaining code, and more.
You think through problems carefully inside <think> tags before acting.
You have access to tools like Read, Edit, Write, Bash, Grep, Glob, and others
to interact with the user's codebase. You use tools when needed and provide
clear, concise responses. You show your reasoning process and use tools
methodically to accomplish tasks.

---

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "empero-ai/Qwen3.5-9B-Claude-Code"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)

messages = [
    {
        "role": "system",
        "content": (
            "You are an expert AI coding assistant. You help users with software engineering "
            "tasks including writing code, debugging, refactoring, explaining code, and more. "
            "You think through problems carefully inside <think> tags before acting. "
            "You have access to tools like Read, Edit, Write, Bash, Grep, Glob, and others "
            "to interact with the user's codebase. You use tools when needed and provide "
            "clear, concise responses. You show your reasoning process and use tools "
            "methodically to accomplish tasks."
        ),
    },
    {"role": "user", "content": "Read the file main.py and explain what it does."},
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=1024,
        temperature=0.7,
        top_p=0.9,
        do_sample=True,
    )

print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

---

Intended Use

• Agentic coding assistants and IDE integrations
• Tool-use / function-calling capable code models
• Research into agentic SFT from real human-AI coding sessions
• Local deployment of a capable coding agent (fits on 24GB+ GPU in fp16)

Limitations

• Trained on traces from Claude Code / Codex sessions — tool format is XML-based, not OpenAI function-call JSON
• Quality of outputs reflects quality of DataClaw contributors' sessions
• Not RLHF'd or DPO'd — raw SFT only
• May hallucinate tool outputs if not connected to an actual tool executor

---

License

Inherit's the base model's license. Please check empero-ai/Qwen3.5-9B-Claude-Opus-4.6-Distill and the underlying Qwen3.5 license before commercial use.

---

Citation

If you use this model or the DataClaw datasets in your work, please credit the DataClaw contributors and the original Claude Code trace collection effort.