Search “Meta”

ARC Prize publishes Meta Muse Spark Open's evaluation results on ARC-AGI

ARC Prize published the validation results of Meta's Muse Spark Open on ARC-AGI: ARC-AGI-1 at 89.0% with $0.29 per task, ARC-AGI-2 at 57.5% with $0.73 per task, and ARC-AGI-3 at 0.1% under the standard harness with $3.9K. ARC Prize states that its v1/2 scores are close to those of Haiku 5.5 and GPT-6 Luna, but the cost is about 9 times that of the latter; the model is priced on OpenRouter at $1.25 per million input tokens and $4.25 per million output tokens. As the reasoning tier increases, ARC-AGI-2 rises from 29.4% at medium (averaging 55k tokens/task) to 57.2% at xhigh (180k tokens/task).

ARC Prize tests Meta Spark Open: ARC-AGI-2 score 57.5%, cost about 9 times that of models with the same score

ARC Prize tested Meta Spark Open on ARC-AGI: it scored 89.0% on ARC-AGI-1, 57.5% on ARC-AGI-2, and only 0.4% on ARC-AGI-3. Its v1/v2 scores are close to those of Haiku 5.5 and GPT-6 Luna, but its cost is about 9 times theirs. Across providers on OpenRouter, the model costs $1.25/M for input and $4.25/M for output, while Haiku 5.5 and GPT-6 Luna both cost $0.10/M for input and $0.50/M for output. As the reasoning tier increases, ARC-AGI-2 rises from 29.4% at medium (averaging 55k tokens/task) to 57.2% at xhigh (180k tokens/task).

ARC Prize announces Meta Muse Spark Open's evaluation results on various versions of ARC-AGI

ARC Prize announced the validation results of Meta Muse Spark Open on ARC-AGI: ARC-AGI-1 is 89.0% at $0.29 per task, ARC-AGI-2 is 57.5% at $0.73 per task, and ARC-AGI-3 is 0.4% at $3.0K under the provider adapter harness and 0.1% at $3.9K under the standard harness. ARC Prize states that its scores on v1/v2 are similar to those of Haiku 5.5 and GPT-6 Luna, but the cost is about 9 times that of the latter; the model is priced on OpenRouter at $1.25 per million input tokens and $4.25 per million output tokens, while Haiku 5.5 and GPT-6 Luna are both $0.10 per million input tokens and $0.50 per million output tokens. On public tasks, as the reasoning scale increases, the ARC-AGI-2 score rises from an average of 55k tokens/task corresponding to 29.4% at the medium tier, to 150k corresponding to 51.0% at the high tier, and then to 180k corresponding to 57.2% at the xhigh tier; ARC-AGI-1 has already reached 92.8% at the high tier and rises to 94.0% at the xhigh tier.

ARC Prize publishes Muse Spark Open reasoning scaling test results on ARC-AGI

ARC Prize published the test results of Meta's Muse Spark Open on the public ARC-AGI tasks: ARC-AGI-2 improves with reasoning scale, with medium averaging 55k tokens per task for 29.4%, high at 150k tokens for 51.0%, and xhigh at 180k tokens for 57.2%. ARC-AGI-1 already reached 92.8% at high and rose to 94.0% at xhigh. The model scores 57.5% on ARC-AGI-2 at $0.73 per task, and 89.0% on ARC-AGI-1 at $0.29 per task; its scores are close to those of Haiku 5.5 and GPT-6 Luna, but its cost is about 9 times theirs.

Artificial Analysis measured Meta Muse Spark Open inference throughput

Artificial Analysis published preliminary test results on AA-AgentPerf for Meta's new open-weight model Muse Spark Open, an 816B-parameter, 42B-active MoE that scores 39 on the Artificial Analysis Intelligence Index. Tests show NVIDIA B300 and GB300 each reach about 150M tokens/s per megawatt under throughput-first configurations, with GB300 at about 108M and B300 at about 92M at 100 tok/s output speed; AMD MI355X reaches 70M tokens/s per megawatt at 65 tok/s. 46 of the model's 62 layers use 2,048-token sliding window attention, reducing KV cache requirements at 1M context by about 75% versus full attention; Meta released an NVFP4 checkpoint and AMD released an MXFP4 checkpoint.

English translation is being prepared.

English translation is being prepared.

MIT Course MAS.S62: Two Lectures on the Evolution of Harnesses and Multi-Agent Systems

Lectures 4-5 of the MIT course MAS.S62 "How to AI an AI" focus on Harness evolution, covering the evolution of harness components, memory, tools, and skills, as well as the evolution of planning, task decomposition, and multi-agent systems. The memory section uses a three-axis taxonomy to connect MemGPT, mem0, MEM1, and Reflexion/ExpeL, and uses CLM to unify context management as file operations; the tools section moves from Toolformer, ToolLLM, and CodeAct to Meta-Harness, which represents the entire harness as a file system and provides about 400 times the amount of feedback information of previous methods. The multi-agent section presents five empirical findings, including that MAS wins 81% on decomposable tasks and loses 70% on tasks with sequential dependencies, and that CORAL achieves new SOTA on 8 tasks, with multi-agent evolution improving over single-agent by 4%-20%.

English translation is being prepared.

Arena Alignment Index Update: Three New Models from Meta, Reflection, and StepFun Join the Leaderboard

The Arena Alignment Index has been updated, adding two new labs and three new models. Meta's Muse Wonder ranks 6th at 77.5%, the highest score among the three new models, with an unauthorized action rate of 2.09% and a deceptive completion rate of 14.41%, both the lowest of the three; Reflection's Beam-1.0-501B-A23B ranks 7th at 75.3%, with a false attribution rate of 4.50%, the best of the three; StepFun's Step 5 Preview ranks 8th at 75.1%, with an unauthorized action rate of 2.11%, close to that of Muse Wonder.

English translation is being prepared.

English translation is being prepared.

English translation is being prepared.

English translation is being prepared.

English translation is being prepared.

English translation is being prepared.

English translation is being prepared.

Beijing News tests Qwen Go! and Lingguang 2.0: personal agent competition shifts toward autonomous task handling

Beijing News Shell Finance reporters tested Alibaba's Qwen Go! and Ant's Lingguang 2.0. The former set up a daily 8 a.m. medication reminder with a single sentence and checked it item by item, while the latter, after authorization on Alipay, sorted out the past six months of bills. The article also reviews the layouts of Meta Muse, OpenAI dots, as well as ByteDance and Tencent, and points out that cross-ecosystem permissions are currently the hardest wall to break down, with ByteDance's SAEP published rejection list reaching 155 apps.

Stanford CS 329Z Lecture 9 Outlines the Selection of Agent System Optimization Methods

Lecture 9 of Stanford's CS 329Z course presents a selection diagram for Agent system optimization: the prerequisite for optimization is having data, a system, and evaluation metrics all in place; generally, context optimization (prompts and few-shots) is done first because it is cheap, fast, and sample-efficient; when context optimization is insufficient, or when you want to distill capabilities into a small model, or when prompts grow longer and longer, then you move on to changing the weights. The lecture notes also review reflective prompt optimization methods such as OPRO, MIPROv2, TextGrad, and GEPA; weight optimization paths such as SFT, RL (PPO vs. GRPO), and on-policy distillation; and approaches that let agents self-optimize, such as the Darwin Gödel Machine, Meta-Harness, and karpathy's autoresearch. They also point out that agents are already decent optimizers, making evaluation metrics the link that most needs to be controlled.

English translation is being prepared.