circuit-1.7b

v1.3 (2026-09-22). Trained on long documents and multi-step reading as well: merger agreements, contract clauses, whole government policy pages, and multi-hop questions over 10 to 20 paragraphs, at 4,096 tokens instead of 1,024. On held-out rows of those five sources it answers 82.2% (v1.2: 59.1%; Jev, which never trained on them: 74.9%). Unseen families rise from .722 to .733 with better calibration (ECE .147 to .120). The cost: a few small in-house production families drop, see Versions. revision="v1.2" gets the previous weights.

v1.2 (2026-09-22). The options of a choice question are encoded side by side, so the answer cannot depend on the order they are listed in: 0.5% of answers change under reordering, against 14.5% for v1.1 and 8% for Jev. It costs about 1.6 points of accuracy on unseen data and 8 on the one task where options must be compared against each other. revision="v1.1" gets the previous weights, which the hosted API still serves. See Versions.

v1.1 (2026-09-21). Same base, head and recipe as v1.0, with two question families added to the training data: is this claim supported by the evidence, and does this tool fit the request. See Versions. revision="v1.0" gets the original weights.

A System One decision model: typed questions in, calibrated probability distributions out, one forward pass, no text generation. It speaks TypeSafe's POST /v1/systemone contract and is the open-weights backend for decision-circuits.

circuit-1.7b is a LoRA adapter plus a pointer readout head on Qwen/Qwen3-1.7B-Base. A request packs a state and a question; each option is wrapped in delimiter tokens and the sequence ends with a decide token. The head scores every option's closing delimiter against the decide token and applies softmax. Those probabilities are the answer, trained with cross-entropy against outcome labels, so calibration is learned rather than derived.

Results

Same items for every model, human labels, accuracy / ECE (15 bins).

cold eval (1,200 items)	MultiNLI	SMS spam	Civil toxicity	CLINC 151-way
Jev (TypeSafe, API)	88% / 0.04	96% / 0.05	82% / 0.06	90% / 0.05
Bespoke-Nimble-9B	84% / 0.09	91% / 0.06	86% / 0.08	not supported (26-option cap)
kev-0.5b	46% / 0.28	50% / 0.30	62% / 0.16	62% / 0.17
circuit-1.7b	82% / 0.08	97% / 0.03	87% / 0.13	91% / 0.05

The four cold-eval tasks' train splits are in this model's training mix; they are presumably not in Jev's. Two checks nothing was trained on:

out of distribution	Jev	Nimble-9B	kev-0.5b	circuit-1.7b
100 water-utility calls, 11-way (Barney 2025)	98% / 0.02	93% / 0.05	80% / 0.13	93% / 0.06
generalization grid, 9 operations x 6 formats, code-labeled	95%	85%	48%	97%*
ClassicMiniDIY typesafe-bench, 546 production questions (agreement with Jev / ECE)	1.0 by definition	0.84 / 0.05	0.49 / 0.11	0.75 / 0.06

* the grid generator is ours, so this is held-out items, not held-out structure. Trained with an operation and a format withheld, the same recipe scores 91% on the unseen format and 57% on the unseen operation: layouts transfer, new kinds of judgment must be in the training data.

Known weakness shared with every model above, Jev included: on inputs built to be undecidable, mean confidence is 0.5 to 0.9 where it should be near 0. Calibration on ambiguity is the open problem.

Datasets nobody prepared for

300 items each from four public datasets that are in no model's training mix here. Accuracy / ECE / Brier score; lower is better for the last two. ChaosNLI's reference is the vote of 100 annotators per item, so its Brier measures distance from how people answered. (An earlier version of this card reported KL. Jev's API rounds to two decimals, which makes KL against it meaningless, so this table uses Brier.)

	tool relevance (BFCL)	groundedness (HaluEval QA)	human disagreement (ChaosNLI)	64 intents (HWU64)
Jev (TypeSafe, API)	.813 / .069 / .259	.910 / .029 / .140	.600 / .254 / .269	.800 / .087 / .276
Bespoke-Nimble-9B	.827 / .060 / .238	.840 / .085 / .236	.563 / .315 / .331	not supported (26-option cap)
circuit-1.7b v1.0	.580 / .228 / .539	.730 / .052 / .352	.523 / .242 / .322	.723 / .107 / .381
circuit-1.7b v1.1	.817 / .112 / .287	.767 / .126 / .325	.587 / .238 / .292	.780 / .065 / .322

v1.0 was near a coin flip on tool relevance because nothing in its training asked that question. Groundedness is still 14 points behind Jev at this size. Reproduce all of it with one command: REPRODUCE.md.

Training

• Base: Qwen/Qwen3-1.7B-Base (Apache 2.0), frozen. LoRA rank 16 on all attention and MLP projections (17.4M params). Pointer head: two 2048 x 256 linear maps.
• Data, 19,738 items, every label computed by code or by humans, every source permissive (CC BY / CC0 / MIT / Apache): the generalization grid (8,100 + 838 oversampled ambiguous items with soft 0.5 labels), a commercial-safe slice of public classification tasks (2,100), and CC-licensed real data (MNLI, civil_comments, sms_spam, CLINC; 5,700). New in v1.1, 3,000 items: claim and answer support from VitaminC and SQuAD v2, and tool relevance, tool choice and tool-call checks built from CLINC and SNIPS utterances against hand-written tool specs. BFCL and HaluEval supply nothing; they are the test. No teacher-model outputs. New in v1.3, 6,300 items of long documents and multi-hop reading: MAUD merger-agreement deal points, ShARC with each question's whole rules page as the policy, CUAD contract windows, MuSiQue (with wrong answers taken from the question's own intermediate hops, and unanswerable versions with one hop's paragraph removed) and 2WikiMultihopQA. Nothing from JevBench.
• 2 epochs, batch 4, max 1,024 tokens, lr 1e-4 (LoRA) / 1e-3 (head), soft-target cross-entropy. The kept checkpoint is the lowest validation ECE among those within two points of the best validation accuracy (step 4,400 of 8,892). 58 minutes on one H100.
• Training code, data generators, and the evaluation harness: github.com/Barneyjm/circuit.

Order stability

Ask any causal decoder a multiple-choice question and its answer depends on the order the options are written in, because each option is read having read the ones before it. Over 981 questions from 16 task families, each asked in 4 option orders, the top answer changed on 8.3% of items for Jev, 23% for a stock Qwen3.5-4B read through its answer-letter logits, and 14.5% for circuit-1.7b v1.1.

v1.2 encodes each option so that it attends to the state, the question and itself, never to another option, with positions that restart where the question ended; the decide token attends to all of them from one position past the longest. Hidden states and probabilities are then identical under any ordering, up to bf16 summation order (about 1e-3), in one forward pass. 0.5% of answers change on the same 981 items. The mechanism is about forty lines, s1proto/parallel.py in the repo, and config.json here carries "parallel_options": true so the repo's scorer applies it. Served without it, these weights lose about 5 points; the guarantee is the mask and the weights together.

Reproduce: uv run python scripts/eval_permutations.py lora:<this repo> --out out.json.

Versions

tag	date	what changed
v1.3	2026-09-22	+6,300 long-document and multi-hop items, 4,096 tokens, 1 epoch (kept step 5,200 of 5,864, 3 h on one A40). Hard-tier eval (650 held-out rows of the five new sources) .822 / ECE .040 against v1.2's .591 and Jev's .749. Unseen sets .733 / ECE .120 / Brier .303; water calls .900; order flips 0.9%. On the 981-item order set, accuracy .706 against v1.2's .729: the drop is in small in-house production families (30 items or fewer each), while every public held-out family held or improved. All numbers measured through the hosted endpoint.
v1.2	2026-09-22	Options encoded side by side: order-stable answers (0.5% flips). Unseen sets .722 / ECE .147 / Brier .327 against v1.1's .738 / .133 / .306; water calls .850 against .930; grid .961 against .970. Per-type temperatures (choice 1.3) in the config.
v1.1	2026-09-21	+3,000 training items in two new families (groundedness, tool calls). Tool relevance .58 to .82, groundedness .73 to .77, production questions .70 to .75; everything else within a point or two.
v1.0	2026-09	First release.

Pin one with revision="v1.2" in huggingface_hub, or hf download jbarney/circuit-1.7b --revision v1.2.

Use

Serve it with the circuit repo and point any System One client at it:

S1_MODEL=lora:runs/circuit-1.7b uv run python -m s1proto        # :8901

from decision_circuits import Circuit, Q, argmax
from decision_circuits.backends import SystemOne

c = Circuit()
c.noul("pii", "Does the message contain personal information?")
c.choice("dept", "Which team?", {"billing": None, "technical": None, "other": None})
c.gate("redact", Q("pii") >= 0.7, on_uncertain="escalate")
c.gate("route", argmax("dept", min_confidence=0.35))
out = c.run(SystemOne("http://localhost:8901/v1/systemone", api_key="x"), "Card charged twice, my card ends in 4412")

Files: adapter/ (PEFT LoRA), head.pt (pointer head, keys q.weight, k.weight), config.json (base, hidden size, head type, layout, training args).

Intended use and limits

Research and evaluation of calibrated decision models, and as a local backend for decision circuits. Not a production system for decisions that affect people. English only. Confidence on undecidable inputs is not yet reliable (see above); put thresholds inside an uncertainty band and route the band to a human, which is what decision-circuits does.

License

Adapter and head: Apache 2.0. Base model: Qwen license (Apache 2.0 for Qwen3).