echoforest/yt-chat-qwen3.5-9b-screen-heads-instruct

YouTube live-chat market classifier: Qwen3.5-9B, screening arm heads-instruct

Screening checkpoint, not a production model. This is one of four arms trained on the same 15,000-turn sample for one epoch to choose the configuration for the full training run. Scores are on a fixed 4,000-turn validation sample, before calibration. The model classifies one author's turn in Korean bitcoin-futures YouTube live chat into eight labels.

• Base model: Qwen/Qwen3.5-9B (instruct checkpoint), text decoder only.
• Arm: LoRA adapter on the text decoder plus eight linear heads (heads.safetensors) on the final-norm hidden state of the last token. One forward pass per turn.
• Labels: stored reference labels from an LLM labeller. Every score measures agreement with those labels, not ground truth.

This arm won the Qwen screen and was retrained on all 71,437 turns for 2 epochs, then calibrated. That full model is echoforest/yt-chat-classifier-qwen3.5-9b and is the one in production; use it rather than this screening checkpoint.

Screening results

Correct direction views at the lowest gate τ with ≥80% val precision, and mean field agreement, on the same 4,000 val turns. Both backbones were screened with the same recipe and data; the production model is the Qwen3.5-9B heads-instruct arm retrained in full (echoforest/yt-chat-classifier-qwen3.5-9b).

Arm	Qwen3.5-9B (this screen): correct views / agreement	Gemma 4 12B: correct views / agreement
heads-base	707 / 86.4%	936 / 88.3%
heads-instruct (this model)	717 / 86.2%	0 / 68.9%
gen-base	705 / 86.6%	903 / 88.6%
gen-instruct	624 / 86.1%	876 / 88.5%

This arm: mean field agreement 86.2%; all eight fields identical on 39.3% of turns; direction views: 717 correct of 896 accepted (precision 80.0%, recall 52.3%) at τ 0.59.

Per-field val agreement: relevance 94.3 · direction 77.8 · direction_basis 81.4 · target 88.9 · own_position_or_trade 93.4 · pnl_state 96.7 · expressed_emotion 71.2 · sarcasm 86.0.

Fields

Field	Options (in order)
relevance	market, off_topic, unclear
direction	bullish, bearish, rangebound, mixed, none, unclear
direction_basis	explicit_forecast, own_position, recommendation_to_others, wish_or_hope, reaction_to_move, not_applicable
target	bitcoin, ethereum, named_altcoin, altcoins_general, crypto_market, traditional, multiple, none
own_position_or_trade	long, short, took_profit, stopped_or_liquidated, hedged, flat_waiting, none
pnl_state	profit, loss, liquidated_or_large_loss, none
expressed_emotion	fear, fomo, greed_euphoria, enthusiasm, frustration, regret, mockery, uncertainty, calm, mixed, none
sarcasm	likely, not_evident, unclear

Input format

Transcripts must be rendered exactly as in training (the project's N1 template): a stream preamble with a slang glossary, recent earlier chat, and the target author's lines fenced as <<<...>>>. Qwen has no BOS token; tokenize without special tokens.

Usage

import json, torch
from huggingface_hub import snapshot_download
from peft import PeftModel
from safetensors.torch import load_file
from transformers import AutoConfig, AutoTokenizer, Qwen3_5ForCausalLM

repo = snapshot_download("echoforest/yt-chat-qwen3.5-9b-screen-heads-instruct")
base = "Qwen/Qwen3.5-9B"
lm = Qwen3_5ForCausalLM.from_pretrained(  # text decoder only; vision and MTP weights are ignored
    base, config=AutoConfig.from_pretrained(base).text_config, dtype=torch.bfloat16, device_map="cuda")
text = PeftModel.from_pretrained(lm.model, repo).eval()
labels = json.load(open(f"{repo}/labels.json"))
heads = {k: v.cuda().float() for k, v in load_file(f"{repo}/heads.safetensors").items()}
tok = AutoTokenizer.from_pretrained(base)
piece = lambda s: tok(s, add_special_tokens=False)["input_ids"]

def classify(transcript):
    ids = torch.tensor([piece(transcript + "\n### LABELS")], device="cuda")  # Qwen has no BOS token
    with torch.no_grad():
        h = text(input_ids=ids).last_hidden_state[0, -1].float()
    return {f: dict(zip(labels["options"][f],
                     torch.softmax(h @ heads[f + ".weight"].T + heads[f + ".bias"], -1).tolist()))
            for f in labels["fields"]}

Needs transformers ≥ 5.20 and peft ≥ 0.21. Install flash-linear-attention and causal-conv1d for the fast Gated DeltaNet kernels; on Hopper with torch 2.8 (Triton 3.4), flash-linear-attention also needs tilelang. Probabilities are uncalibrated (screening runs fit no temperatures).

Training

• Data: export v20261016-1835 (splits by whole stream). A stream-proportional 15,000-turn train sample (seed 13), 1 epoch, max length 1024 tokens, nothing truncated. Turns that needed stream audio trained at half weight.
• Method: LoRA r 32 / α 32 / dropout 0.05 on attention q/k/v/o, the Gated DeltaNet projections (in_proj_qkv, in_proj_z, out_proj) and MLP gate/up/down; heads trained in full with dropout 0.1, cross-entropy summed over fields, direction ×2. LoRA LR 0.0002, heads LR 0.001, cosine schedule, 3% warmup, AdamW, weight decay 0.01, 32 sequences per step, bf16, gradient checkpointing.
• Compute: 1× H100 80GB, about 5,800 tokens/s.
• Versions: torch 2.8.0+cu128, transformers 5.20.1, peft 0.21.2.

Limitations

• Screening checkpoint: trained on about a fifth of the data for one epoch, uncalibrated, and evaluated only on a val sample from 2 streams. The test set is reserved for the final run.
• Narrow data: 15 train streams over 6 days (2026-10-10 to 10-15), all Korean crypto-futures chat.
• Chat text that imitates the transcript fences or gives instructions can move labels; the robustness suite is run on the final model only.