AutomationBench is 41.4% for Astra versus 18.1% for Sol, showing its biggest gains arrive in multi-step work.
Last updated: September 4, 2026
Reported scores · higher is better
— = not reported
GPT-6 Astra's benchmark results point to a clear strength: long-horizon, agentic work. This page aggregates reported scores across reasoning, coding, automation, and safety evaluations, alongside GPT-5.6 Sol, Fable 5.1, Fable 5, Opus 5, and Gemini 3.8 Flash.
Scores are vendor-reported or drawn from published evaluations, not re-runs by this site. Higher is better unless a row says otherwise.
Key takeaways
ARC-AGI-3 reaches 98.6%, compared with 7.8% for Sol — the widest gap in this table.
GPQA Diamond moves from Sol's 94.6% to Astra's 96.0%: meaningful, but much less dramatic.
The 0% auto-review circumvention result is an internal red-team figure, not an independent audit.
| Evaluation | GPT-6Astra | GPT-5.6Sol[2] | Fable 5.1Anthropic | Fable 5 | Opus 5 | Gemini3.8 Flash |
|---|---|---|---|---|---|---|
| ARC-AGI-3 | 98.6% [1] | 7.8% | — | — | 30.2%(high) | — |
| FrontierMath Tier 4 (v2) | 97.6% | 83.0% | 87.8% | 87.8% | 73.2% | — |
| Agents’ Last Exam | 59.3% | 52.7% | — | 48.7%(xhigh) | 52.7% | — |
| AutomationBench | 41.4% | 18.1% | 31.4% | 17.4% | 26.9% | — |
| BenchCAD | 95.9% | 83.3% | 84.3% [3] | 67.5% [3] | 82.1% [3] | — |
| DeepSWE v1.1 | 74.1% | 70.8% | 67.4% | 69.9% | 68.8% | 73.7% |
| Terminal-Bench Science 0.1 | 64.6% | 22.4% | 52.6% | 24.7% | 29.0% | — |
| GPQA Diamond | 96.0% | 94.6% | 93.7% | 92.6% | 93.2% | 95.3% |
| GeneBench Pro | 39.0% | 28.7% | — | — | — | — |
| MedChemBench (internal) | 49.7% | 47.4% | — | — | — | — |
| HealthBench Professional (length-adjusted) | 63.4% | 60.5% | 56.6% [4] | 60.9% | [57.5%] | 52.1% |
| ExploitBench | 100.0% | 78.5% | — | — | 70% | — |
| SRE-Bench (four attempts) | 99.2% | 68.7% | — | — | — | — |
| Auto-review circumvention (internal; lower is better) | 0% | 0.29% | — | — | — | — |
* Highlighted column marks the reported GPT-6 Astra result. Bracketed values and superscripts preserve source annotations.
For the cost context, see GPT-6 Astra pricing; for a focused model comparison, read Astra vs Fable 5.1 head-to-head.
What the benchmarks say
Reasoning and knowledge
On general reasoning, Astra's gains are concentrated in the hardest tiers. GPQA Diamond, a PhD-level science evaluation, moves from 94.6% for Sol to 96.0% for Astra. The larger story is long-horizon reasoning: ARC-AGI-3 reaches 98.6%, while Agents' Last Exam reaches 59.3%. The longer and more open-ended the task, the more visible Astra's lead appears in this dataset.
Coding
Astra leads on software engineering, but not every result is a blowout. DeepSWE v1.1 is 74.1% versus Gemini 3.8 Flash's 73.7% — effectively a tie. BenchCAD at 95.9% and SRE-Bench at 99.2% show a wider advantage on structured and multi-attempt tasks. For coding alone, the practical decision still depends on whether that edge justifies Astra's higher API price.
Agentic automation
This is Astra's strongest category in the reported results. AutomationBench is 41.4% versus Sol's 18.1% and Fable 5.1's 31.4%; Terminal-Bench Science is 64.6% versus 22.4% for Sol. These scores describe tasks where a model operates tools over several steps — the kind of work where a capability gain can change what is buildable. See the broader GPT-6 Astra overview for product context.
Safety and evaluation limits
The auto-review circumvention row reports 0% for Astra, but it is marked internal. Treat it as a vendor-reported red-team result rather than a third-party certification. Benchmark protocols, prompting, tools, and attempt counts vary, so small cross-model differences should not be treated as decisive.
Frequently asked questions
What is GPT-6 Astra's ARC-AGI-3 score?
Astra is listed at 98.6%, compared with 7.8% for GPT-5.6 Sol in this reported comparison.
Is GPT-6 Astra better than Fable 5.1?
On the evaluations where both scores are reported, Astra leads in this table. The margin is larger on agentic tasks than on knowledge tests.
Where do these benchmark numbers come from?
They are reported figures from vendor materials and published third-party evaluations. The superscripts preserve the annotations present in the source dataset; verify a source before citing an individual number.
Does a higher score always mean a better model?
No. Scores depend on protocol, prompt, tools, and task design. A benchmark should be considered alongside latency, reliability, context requirements, and cost.
Return to the GPT-6 Astra overview for launch context and API access.
Sources and methodology
This page preserves reported values from the linked benchmark and model-provider materials. Scores are not independent re-runs by this site; differences in harness, prompt, tools, and evaluation date can make results non-comparable.
- ARC Prize: Astra and ARC-AGI-3 — reports the provider-adapter result; the ARC Prize leaderboard entry reports 62.7% under the Standard harness, so the 98.6% figure in this ledger must not be compared with Standard-harness scores.
- OpenAI: GPT-5.6 announcement — model-provider source for the GPT-5.6 Sol comparison values.
- BenchCAD official benchmark and leaderboard — benchmark definition and published evaluation results.
- OpenAI GPT-6 Astra system card: HealthBench — reports HealthBench Professional and related HealthBench results.