AI Trends

ARC Prize announces evaluation results of GPT-6.2 Astra on ARC-AGI

ARC PrizeUpdated Nov 18 02:31

AI summary

ARC Prize announces the results of OpenAI GPT-6.2 Astra on the ARC-AGI series of benchmarks: ARC-AGI-3 reaches 100.0% (SOTA), with a cost of $4.1K using the provider adapter, and 70.3% at $24.2K with the standard approach; ARC-AGI-2 is 95.8% at $0.50/task, and ARC-AGI-1 is 98.5% at $0.17/task. ARC Prize states that this is the first model to achieve a 100% score on ARC-AGI-3. On the 25 public ARC-AGI-3 games under the max reasoning setting, GPT-6.2 Astra uses 64.1% fewer actions than the human baseline and 5.2% fewer than GPT-6 Astra; on ARC-AGI-2 semi-private tasks at the low to xhigh levels, its reasoning token usage is 51% lower than the previous generation, while at the max level it is 12% higher.

Why read this

SelectedAI score 73

ARC Prize gives GPT-6.2 Astra's scores and costs on the three ARC-AGI benchmarks, and compares the difference in reasoning token usage versus the previous generation.

Topics

Tags

Body

@OpenAI On the ARC-AGI-3 semi-private games with the provider adapter harness, GPT-6.2 Astra used reasoning on a smaller share of actions than GPT-6 Astra at every effort: https://t.co/SE31IIKn9o

Source

ARC Prize