AI Trends

ARC Prize publishes score and cost comparison of Sol and Astra on ARC-AGI-3

ARC PrizeUpdated Nov 18 02:31

AI summary

ARC Prize reports that Sol achieved a score of 98.2% on ARC-AGI-3 at the high reasoning setting, 1.8 percentage points lower than Astra's 100% at the highest reasoning setting, but at 64% lower cost under the provider adapter setup. Sol scored 85.8% on ARC-AGI-2 at $0.13 per task, and 97.5% on ARC-AGI-1 at $0.06 per task; under the ARC-AGI-3 standard setup it scored 54.9% at a cost of $5.5K.

Why read this

SelectedAI score 75

The original text provides a score and cost comparison of Sol and Astra on ARC-AGI-3, allowing readers to compare the cost-effectiveness of the two.

Topics

Tags

Body

Sol's best ARC-AGI-3 score of 98.2% at high reasoning was 1.8 percentage points behind Astra's 100% score at max reasoning, but at 64% lower cost with the provider adapter harness:

- ARC-AGI-3: 98.2%, $1.5K (provider adapter), 54.9%, $5.5K (standard) - ARC-AGI-2: 85.8%, $0.13/task - ARC-AGI-1: 97.5%, $0.06/task

Full results: https://t.co/lhIci4y1B8

Source

ARC Prize