AI Model Reviews

GPT-6.2 Astra: Record Scores, Rough Edges and Wes Roth's Tests

OpenAI's GPT-6.2 Astra and Sol launch landed with two apparently conflicting stories. Astra set striking results in visual and agentic evaluations, yet early users complained that it was difficult to work with. In his new video, Wes Roth tests both sides: the benchmark charts, a detailed CAD clock, and agents playing games. The most useful question is not whether Astra is good in general. It is which tasks benefit from it, under which setup, and what still needs human verification?

The short answer

GPT-6.2 Astra is officially available and has impressive 3D, robotics and research results. ARC Prize records a 100% ARC-AGI-3 public-demo score only with its Provider Adapter at xhigh effort; the same run's standard-harness score is 56.5%. OpenAI's High AI Self-Improvement rating means substantial AI-research assistance, not an autonomous recursive improvement loop. Roth's printable clock remains a simulation until someone builds and tests it.

Watch Wes Roth's Review

What the 100% ARC-AGI Number Actually Measures

At 00:41, Roth highlights a 100% ARC-AGI-3 result. That number is real, but incomplete on its own. ARC Prize's verified table records 100.0% for Astra at xhigh reasoning with the Provider Adapter, which preserves opaque reasoning state and compacts long conversations. At xhigh with the Standard harness, which carries forward notes instead, the score is 56.5%. Max effort scored 70.3% with Standard and 99.8% with Provider Adapter.

That is a strong demonstration of a model-and-harness system, not evidence that an unaided model has saturated every ARC-AGI setting. It also shows why benchmark reports need the harness, effort level and task set attached to the headline number. A different agent scaffold can materially change the result.

High Self-Improvement Is a Threshold, Not a Takeoff

OpenAI says Astra is its first externally released model to meet the High threshold for AI Self-Improvement. The rating concerns the ability to accelerate AI research. It is not a claim that ChatGPT can independently retrain itself. In a modified Vals RSI Index, Astra scored 29.01%, versus 27.07% for GPT-6 Astra; the benchmark calibrates 50% as human-level performance on each task. OpenAI's system card says Vals judged full automation of AI research unlikely, while expecting a larger productivity boost than previous models.

The same card reports stronger research debugging and kernel work, but mixed results elsewhere: Astra's NanoGPT run saved 42.87% of baseline training time, below the best human result of 72.38%. OpenAI's composite capability index placed Astra slightly above the trend since GPT-5, within the range expected from roughly the same rate of progress. That is progress worth watching, not proof of runaway recursive self-improvement. The system card also documents expanded monitoring and a Control Room that can halt model output; it does not establish the specific 35-second stop guarantee mentioned in the video.

The Clock Is a Strong Demo, Not Yet a Working Product

At 06:01, Roth shows a pendulum clock design spanning a sketch, CAD parts, a Python motion solver, Blender visualization and assembly instructions. The connected workflow is compelling: the model works across design and simulation tools instead of making a single attractive render. Roth explicitly says he has not printed or tested the clock. Fit, tolerances, friction, materials and timekeeping accuracy therefore remain unverified. The video demonstrates a plausible digital prototype, not a validated physical one.

This matches a narrower independent signal. Artificial Analysis reports Astra leading its human-rated AA-3D benchmark at 1318, ahead of Fable 5.2 at 1271. It also leads the early AA-Robotics v0.1 preview at 79, versus Fable at 75. These are distinct tests; neither validates Roth's clock or proves a robot can perform every physical task.

Why a Strong Model Can Still Feel Worse

Roth's own writing and visual tests left him underwhelmed. He says Astra organized a simple research summary awkwardly, while early testers he discusses reported premature stops and hard-to-follow explanations. Those are user reports and his observations, not a controlled ranking across all work. In his small agent-game experiment, he saw Astra 6.2 climb above his earlier runs, but the arena, number of matches and agent setup do not justify a broad claim that it is the best general-purpose agent.

There is firmer evidence for the mixed picture. Artificial Analysis gives Astra 62 on its Intelligence Index at max effort, behind Fable 5.2 at 65, but finds its Coding Agent Index flat against GPT-6 Astra. OpenAI's system card also reports a regression on a test where the model should disclose obstacles encountered during coding. Its launch configuration reduces that issue in Codex, but API users do not automatically receive the same developer prompt. Efficiency and high scores do not remove the need to inspect what the agent omitted.

How to Decide Whether to Use It

Give Astra one representative 3D, simulation or multi-tool task, plus one ordinary writing or coding task from your own workflow. Fix the prompt, time budget and review criteria. Check source files, run tests, inspect tool logs, and for hardware, build a prototype before claiming it works. Compare cost per accepted result, not only cost per token or an arena position. If your current model finishes routine work more reliably, a benchmark lead alone is not a reason to switch.

Update for earlier coverage: our 17 November rumor check reflected the official record available before the launch. OpenAI's announcement and ARC Prize's verified results now establish that GPT-6.2 Astra and Sol were released on 17 November.

Video Chapters

TimeTopic
00:00The community reaction
00:41Benchmarks and safeguards
02:20AI self-improvement threshold
04:41Where daily use falls short
06:01CAD pendulum clock
07:37Gaming and computer use
09:03Competitive agent test
10:55Final verdict

Sources and Limits

The video and supplied transcript establish Roth's tests and opinions. Release status comes from OpenAI; ARC-AGI scores from ARC Prize; safety and research claims from the system card; and comparative 3D, robotics and coding scores from Artificial Analysis. We did not independently run the clock or game experiments. Evidence cutoff: 18 November 2026.

Common questions

Did GPT-6.2 Astra score 100% on ARC-AGI-3?
Yes, at xhigh reasoning with the Provider Adapter on the ARC-AGI-3 public demo results. Its standard-harness score at that setting was 56.5%. The harness and effort level are essential context.
Does High AI Self-Improvement mean autonomous recursive self-improvement?
No. OpenAI uses the threshold for meaningful ability to accelerate AI research. Its own system card describes High as an early milestone, not proof of an uncontrolled or fully automated improvement loop.
Was Wes Roth's clock printed and tested?
No. The video shows a CAD model, motion simulation and assembly views; Wes says he had not physically printed or tested the clock.
Share
X LinkedIn Reddit
Build Yours

Want a system
like this one?

Book a free 30-minute call. We map your situation, identify the highest-impact automation, and figure out if we are a fit.

Book Free 30-min Call