QuantID Measurement Lab
š
Measuring AI, science and risk with reproducible methods
QuantID PRO
quantid
AI & ML interests
None yet
Recent Activity
updated a Space about 11 hours ago
quantid/measurement-lab liked a Space 1 day ago
masterset-ai/PhysicalEye-Decide-35B-demo liked a dataset 1 day ago
FINAL-Bench/oqc-dataOrganizations
None yet
updated a
Space about 11 hours ago
liked a
Space 1 day ago
liked a
dataset 1 day ago
liked a
Space 1 day ago
reacted to SeaWolf-AI's post with š„ 1 day ago
Post
3709
āļø The Open Quantum Challenge is live. Both seasons open today.
Quantum computers are expensive, queued, and reachable by only a few. So we removed the barrier. With a single GPU and a public harness, anyone can contribute to core quantum-computing problems with no quantum hardware at all. That is the point of this challenge: to move quantum research from a handful of labs to everyone.
ā¢ Season 1, Quantum simulation: reproduce a target quantum system on a GPU.
ā¢ Season 2, QEC decoder: correct errors on coded quantum states.
Answers are held privately and every submission is auto-scored against a frozen ground truth, so the ranking is reproducible and hardware-independent.
š Prize: 2,000 USD total (1,000 USD per season, š„600 / š„300 / š„100)
Getting started takes one step: copy the participation guide from the Space and paste it into Codex or Claude Code, then submit against the harness.
š Read the article: https://huggingface.co/blog/FINAL-Bench/the-open-quantum-challenge-quantum-simulation-and
š Challenge: FINAL-Bench/open-quantum-challenge
Quantum computers are expensive, queued, and reachable by only a few. So we removed the barrier. With a single GPU and a public harness, anyone can contribute to core quantum-computing problems with no quantum hardware at all. That is the point of this challenge: to move quantum research from a handful of labs to everyone.
ā¢ Season 1, Quantum simulation: reproduce a target quantum system on a GPU.
ā¢ Season 2, QEC decoder: correct errors on coded quantum states.
Answers are held privately and every submission is auto-scored against a frozen ground truth, so the ranking is reproducible and hardware-independent.
š Prize: 2,000 USD total (1,000 USD per season, š„600 / š„300 / š„100)
Getting started takes one step: copy the participation guide from the Space and paste it into Codex or Claude Code, then submit against the harness.
š Read the article: https://huggingface.co/blog/FINAL-Bench/the-open-quantum-challenge-quantum-simulation-and
š Challenge: FINAL-Bench/open-quantum-challenge
liked a
Space 1 day ago
liked a
Space 3 days ago
upvoted an article 4 days ago
Article
Leading the System One Mosaic Benchmark: What Darwin-27B-ZTC-v2's #1 Means
FINAL-Bench
ā¢ ā¢ 6 reacted to SeaWolf-AI's post with š„ 4 days ago
Post
4095
š Darwin-27B-ZTC-v2 just took #1 on the System One Mosaic Benchmark (S1MB).
S1MB compares 102 models across 137 specialized benchmarks, in three task types: Noul (assess a condition), Choice (select an option), Score (rate on a scale). Ranking is by overall Borda score.
š Top of the board
š„ Darwin ZTC v2 (FINAL-Bench) 89.58
š„ OpenJev-27B 87.50
š„ AutoJev-27B 87.07
4ļøā£ Eikos 27B 85.43
5ļøā£ Jev 1.13 85.05
š Ranks 2 to 5 are all the JEV family (TypeSafe AI's System One model, from ex-OpenAI researchers). S1MB exists to compare these System One judges, so leading it is the headline.
āļø Why a zero-token judge wins here
š¹ It does not generate. It reads the input and typed questions and returns a calibrated distribution in a single forward pass.
š¹ Zero generated tokens, no decoding loop, so latency and cost stay low.
š¹ Holds up out of distribution too: General Noul 96.00, General Choice 99.34.
It is also #1 on the typed-decisions leaderboard (0.743, zero-shot). Same message from both: a deterministic, calibrated judge at one forward pass per call.
š Model: FINAL-Bench/Darwin-27B-ZTC-v2
š Leaderboard: hotchpotch/S1MB-leaderboard
Standings move as new models are added. Numbers reflect the board at the time of writing. š
S1MB compares 102 models across 137 specialized benchmarks, in three task types: Noul (assess a condition), Choice (select an option), Score (rate on a scale). Ranking is by overall Borda score.
š Top of the board
š„ Darwin ZTC v2 (FINAL-Bench) 89.58
š„ OpenJev-27B 87.50
š„ AutoJev-27B 87.07
4ļøā£ Eikos 27B 85.43
5ļøā£ Jev 1.13 85.05
š Ranks 2 to 5 are all the JEV family (TypeSafe AI's System One model, from ex-OpenAI researchers). S1MB exists to compare these System One judges, so leading it is the headline.
āļø Why a zero-token judge wins here
š¹ It does not generate. It reads the input and typed questions and returns a calibrated distribution in a single forward pass.
š¹ Zero generated tokens, no decoding loop, so latency and cost stay low.
š¹ Holds up out of distribution too: General Noul 96.00, General Choice 99.34.
It is also #1 on the typed-decisions leaderboard (0.743, zero-shot). Same message from both: a deterministic, calibrated judge at one forward pass per call.
š Model: FINAL-Bench/Darwin-27B-ZTC-v2
š Leaderboard: hotchpotch/S1MB-leaderboard
Standings move as new models are added. Numbers reflect the board at the time of writing. š
liked a
model 4 days ago
liked a
Space 4 days ago
liked 9
datasets 5 days ago
Delores-Lin/MDPBench
Benchmark ā¢ Updated ā¢ 1.35k ā¢ 48
joelniklaus/LEXam-hard
Benchmark ā¢ Updated ā¢ 518 ā¢ 877 ā¢ 24
LiquidAI/ifstruct-v1.0
Benchmark ā¢ Updated ā¢ 2k ā¢ 633 ā¢ 103
MathArena/aime_2026
Benchmark ā¢ Updated ā¢ 30 ā¢ 30.5k ā¢ 86
MathArena/hmmt_feb_2026
Benchmark ā¢ Updated ā¢ 33 ā¢ 17.9k ā¢ 31
LEXam-Benchmark/LEXam
Benchmark ā¢ Updated ā¢ 7.54k ā¢ 1.77k ā¢ 71
llamaindex/ExtractBench
Benchmark ā¢ Updated ā¢ 370 ā¢ 15.6k ā¢ 58
MMMU/MMMU_Pro
Benchmark ā¢ Updated ā¢ 5.19k ā¢ 19.2k ā¢ 92
LocalLLaMA/typed-decisions
Benchmark ā¢ Updated ā¢ 3.2k ā¢ 40.3k ā¢ 171