Primary model
Claude Sonnet 5.5
· handles 53 of 60 benchmark exercises
Escalate7/6012% of exercises→
Specialist model
GPT-6 Luna
· used only where observed accuracy and weighted utility both improved
Decision experiments on benchmark evidence
Build a model system,
not just a ranking.
Compare models directly or let the observed 60-exercise panel recommend a primary model, a specialist and a retrospective routing strategy.
21 × 60 observed cellsPrimary + specialist ordered rolesBenchmark only recommendation evidence
01
Your requirements
Set priorities and scope
Weights define per-exercise utility; limits determine which retrospective strategies remain eligible.
Priority presets
Accuracy
Rewards higher observed exercise scores.
Latency
Rewards lower observed end-to-end time.
Cost
Rewards lower estimated API cost per benchmark exercise.
Total tokens
Rewards lower reported token use per benchmark exercise.
Decision weightsAccuracy 50%Latency 25%Cost 25%Tokens 0%Total: 100%
Operational limits
Define an acceptable system
All limits are evaluated as retrospective means across the selected subject.
Best evaluated routing: Claude Sonnet 5.5 as primary and GPT-6 Luna as specialist on 7 of 60 benchmark exercises.
02
Best evaluated routing · All subjects
Primary model plus specialist
Best among bounded candidates ranked by observed utility, accuracy, cost and latency.
Retrospective accuracy99.2%+0.5 pts vs primary only
Mean cost / exercise$0.050-$0.0064 vs primary only
Mean time / exercise49.5s+10.7s vs primary only
Mean tokens / exercise7,503P90 latency 72.8s
03
Observed replay
Retrospective simulation
Three strategies replayed on the same 60 exercises.
| Strategy | Accuracy | Mean cost | Mean time | Mean tokens |
|---|---|---|---|---|
| Claude Sonnet 5.5Primary only · 100% | 98.7% | $0.056 | 38.9s | 6,443 |
| Best evaluated retrospective routing7 escalated exercises | 99.2% | $0.050 | 49.5s | 7,503 |
| GPT-6 LunaSpecialist only · 100% | 98.5% | $0.0058 | 100.7s | 12,199 |
04
Decision evidence
Why this pair was selected
Every claim below is calculated from benchmark observations.
Exercises assigned to the specialist7 observed recovery opportunities+
| Exercise | Subject | Primary | Specialist | Observed gain |
|---|---|---|---|---|
| F2-26-P1-E2 | Physics II | 72.9% | 88.9% | +16.1 pts |
| A3-25-P4-E1 | Analysis III | 99.5% | 100.0% | +0.5 pts |
| A3-26-P2B-E2 | Analysis III | 98.5% | 100.0% | +1.5 pts |
| A3-25-P5-E2 | Analysis III | 99.5% | 100.0% | +0.5 pts |
| F2-25-P4-E3 | Physics II | 92.6% | 100.0% | +7.4 pts |
| F2-26-P4-E1 | Physics II | 99.2% | 100.0% | +0.8 pts |
| F2-26-P2-E2 | Physics II | 97.9% | 100.0% | +2.1 pts |
Next feasible strategies
Claude Sonnet 5.5 → GPT-6.1 Sol3/60 escalated · 99.0% · $0.058 · 47.0s
Claude Sonnet 5.5 → Claude Opus 5.52/60 escalated · 99.0% · $0.058 · 39.5s
Claude Sonnet 5.5 → DeepSeek V4 Flash3/60 escalated · 98.8% · $0.053 · 47.9s
Evidence boundary
The recommendation uses only accuracy, cost, end-to-end latency and token values observed in UniAIBench. It evaluates prefixes from four deterministic orderings; this bounded heuristic is not an exhaustive constrained optimum. Pair selection and allocation use the same panel, so this is an optimistic in-sample scenario, not a predictive router. Most model–exercise cells are single-run; no uncertainty interval is estimated. Token accounting differs by provider; latency includes generation and retries. Gemma 4 31B IT is excluded because its zero tariff is campaign-specific.
Review the benchmark evidence →