NSelected

  1. Artificial AnalysisAnalysis
    AI score71

    Artificial Analysis tests GitHub Copilot Fusion: $5.59 cost per task…

    In Coding Agent Index v2.0, Artificial Analysis benchmarked GitHub Copilot's Copilot Fusion, which costs $5.59 per task, compared with $11.42 for Claude Code with Opus 5.5 (max), while performing only 2 points lower. Copilot Fusion is a runtime orchestration system that selects and coordinates among Anthropic and OpenAI models such as Claude Opus 5.5, Claude Sonnet 5.5, GPT-6 Astra, and GPT-6 Sol. It leads on Terminal-Bench 4.0 with a score of 71, ranks third on FrontierCode with 51 points, and uses 7.2M tokens per task, about half that of Opus 5.5 in Claude Code.

    Why read thisArtificial Analysis benchmarked Copilot Fusion in the Coding Agent Index, reporting cost, token usage, and scores on three benchmarks, directly comparable with Claude Code and Devin Fusion.

  2. ArenaAnalysis
    AI score70

    Gemini 4 Carbon ranks 2nd on Agent Arena with a net improvement of 16.2%

    Arena.ai says Gemini 4 Carbon ranks 2nd in the real-task-based Agent Arena with a net improvement of 16.2%, and sits on the Pareto frontier for cost, time, and output tokens per task. Compared with Gemini 4 Argon, which ranks 7th with an 11% improvement, Carbon is 5 places higher and 5.2 percentage points better; the gains mainly come from a +13.2 percentage point increase in the positive-feedback-to-complaint ratio, +5.6 percentage points in confirmed success, and +4.8 percentage points in steerability. Arena says Carbon's availability will be announced by Google DeepMind.

    Why read thisArena's empirical data breaks down Carbon's improvement over Argon into three signals—steerability, task completion rate, and user sentiment—making it easier to identify the source of the gap.

  3. ArenaAnalysis
    AI score65

    GPT-6.1 Terra ranks 8th in Agent Arena, with a median cost of $0.62 per task

    Arena's empirical results, based on nearly 9,000 agent sessions, show that OpenAI's GPT-6.1 Terra (Max) ranks 8th in Agent Arena, with a net improvement of +10.6% and a median cost of $0.62 per task. This cost is 43% lower than the $1.09 of 7th-place Gemini 4 Argon (High), 30% lower than the $0.89 of 6th-place GPT-6.1 Sol (Max), and 88% lower than the $5.16 of 5th-place GPT-6 Astra (Max). In the subcategories, it ranks 3rd in Bash Reliability (+13.2%), 7th in Steerability (+13.3%), and 9th in Praise vs Complaint (+16.8%).

    Why read thisThe leaderboard presents GPT-6.1 Terra's ranking in Agent Arena alongside a per-task cost comparison, which can be used to assess its cost-effectiveness position.

  4. ARC PrizeFirst-partyAnalysis
    AI score62

    ARC Prize publishes Meta Muse Spark Open's evaluation results on ARC-AGI

    ARC Prize published the validation results of Meta's Muse Spark Open on ARC-AGI: ARC-AGI-1 at 89.0% with $0.29 per task, ARC-AGI-2 at 57.5% with $0.73 per task, and ARC-AGI-3 at 0.1% under the standard harness with $3.9K. ARC Prize states that its v1/2 scores are close to those of Haiku 5.5 and GPT-6 Luna, but the cost is about 9 times that of the latter; the model is priced on OpenRouter at $1.25 per million input tokens and $4.25 per million output tokens. As the reasoning tier increases, ARC-AGI-2 rises from 29.4% at medium (averaging 55k tokens/task) to 57.2% at xhigh (180k tokens/task).

    Why read thisARC Prize gives Meta's new model's scores on three versions of ARC-AGI along with per-task cost, directly comparable with competitors scoring the same.

  5. PromptArmorFirst-partyAnalysis
    AI score60

    PromptArmor testing found that DeepSeek, Kimi, GLM, and Qwen are 88% more susceptible to cyberattacks than…

    PromptArmor tested its most widely circulated attack samples across multiple models and found that DeepSeek, Kimi, GLM, and Qwen models are 88% more susceptible to cyberattacks than GPT, Claude, and Gemini models. The tests covered multiple cyber risk dimensions, including susceptibility to phishing, data exfiltration, and running malware, and the authors described the differences as significant.

    Why read thisPromptArmor ran a cross-model comparison using its own widely circulated attack samples, revealing differences between Chinese and foreign models in phishing, data exfiltration, and running malware.

  6. Google Developers BlogFirst-partyAnalysis
    AI score70

    Google A2UI Team and CopilotKit Explain How to Build Generative UI Applications

    The Google A2UI team and CopilotKit CEO Atai Barkai explain how to build generative UI applications using the A2UI open standard and CopilotKit. A2UI describes interfaces with declarative JSON, and the client only renders components from a trusted component catalog; the same response can be rendered to React, Angular, SwiftUI, Jetpack Compose, and Flutter. The article provides a code example for integrating CopilotKit with Google ADK, and explains the bidirectional flow, implemented via the AG-UI protocol, in which user actions are sent back and the agent regenerates the interface.

    Why read thisThe original article provides complete integration steps and bidirectional interaction examples for A2UI and CopilotKit, which can be used to evaluate how generative UI can be implemented in practice.

  7. Official Pydantic AI articleFirst-partyAnalysis
    AI score68

    Pydantic compares code mode and tool calling across 5,276 runs

    Michael Pfaffenberger of Pydantic used 5 small Pydantic AI applications and 8 models to conduct 5,276 scored runs, comparing code mode with direct tool calling. On long tasks, code mode reduced average cost by 58%, latency by 51%, and increased success rate by 16 percentage points; however, for short tasks with 5 items, 3 out of 5 tasks were instead 21% to 38% more expensive. The author argues that models are better at writing loops than at acting as the loop, and the gap widens as tasks get longer.

    Michael Pfaffenberger avatar

    Why read thisThe author compares code mode and tool calling across 5,276 runs, presenting curves showing how cost, latency, and accuracy vary with scale on long tasks.

  1. Artificial AnalysisAnalysis
    AI score65

    Artificial Analysis releases Intelligence Index v5, with 60% of evaluations based on private datasets

    Artificial Analysis releases Intelligence Index v5, adding two new benchmarks, FrontierCode 2.0 and Terminal-Bench-Science, upgrading HLE-Diamond, AA-Omniscience v1.1 and CritPt v2, removing GDPval-AA v2.1, SciCode and AA-LCR v1.1, and changing 60% of the index to evaluations based on private test sets or held-out answers. It also launches Progress Index v1, which uses a unified scale to track changes in model capabilities over time. The results show that Claude Opus 5.5 (max) ranks first with 58 points, followed by GPT-6 Astra (max, 57) and GPT-6.1 Sol (max, 55), while Gemini 4 Argon (high) scores 50; the gap between open-weight models and the leaders has widened from 3 points to 12 points, and among released weights, DeepSeek V4.1 Flash 1017 (max) and GLM-5.3 (max) lead with 32 points. The per-task cost of the top ten models differs by more than 13 times, with GPT-6.1 Terra (max) lowest at $0.96 and Claude Fable 5.1 (max) highest at $13.04.

    Why read thisThe original text lists the benchmarks added and upgraded in the v5 index as well as the items removed, and provides the scores of leading models and the per-task cost gap, which can be used to compare the capabilities and costs of each model.

  2. LangChain BlogFirst-partyAnalysis
    AI score67

    How LangChain Built the Design Agent Content Studio

    LangChain's GTM engineering team, working with the design team, built Content Studio, a design agent inside Slack, on Managed Deep Agents. It can generate videos, social graphics, UI mockups, and slide decks, and has already produced more than 3,000 assets and over 80 slide decks. The article breaks down its four-layer knowledge structure (skills, wiki, code validation, memory), the dedicated LangSmith Sandbox for each Slack thread, the two rendering engines HyperFrames and the in-house motion-studio, and how the main agent orchestrates asynchronous sub-agents. The team says that after migrating to Managed Deep Agents, its own code shrank by about 80%, the median model cost per request fell from about $8 in mid-September to about 50 cents, and the entire harness has been open-sourced.

    Shreya Talegaonkar

    Why read thisLangChain has publicly shared the complete build process for its design agent Content Studio, including the engineering trade-offs around skills, sandbox environments, and asynchronous sub-agents.

  3. Full Artificial Analysis articleFirst-partyAnalysis
    AI score82

    Gemini 4 Carbon Tops the Artificial Analysis Intelligence Index v5

    Google's Gemini 4 Carbon ranks first on the Artificial Analysis Intelligence Index v5 with a score of 59, leading in four of the 9 evaluations: HLE-Diamond, Terminal-Bench-Science, AA-Omniscience, and AutomationBench-AA. On the Coding Agent Index, Gemini 4 Carbon (high) paired with the Antigravity CLI scores 59, trailing Claude Code paired with Claude Opus 5.5 (max, 62 points) by 3 points. The model has a 1M-token context window, supports text, image, video, and speech input, and is priced at a standard rate of $4/$20 per million input/output tokens; it is currently available to select users with a 50% discount.

    Why read thisArtificial Analysis's independent evaluation places Gemini 4 Carbon at the top of the Intelligence Index v5 and provides a breakdown comparison with Claude Opus 5.5.

  1. KazikeAnalysis
    AI score67

    Retrofitting a TV, Fill Lights, and a Camera with Codex and Claude

    The author shared the process of using Codex and Claude to reverse engineer and retrofit real-world devices: first enable ADB debugging on the TV, then have the Agent disable system packages, clear boot ads, and rebuild a clean launcher, and replicate the same method on three other TVs. The team also used Claude to reverse engineer 4 fill lights and built a unified web console, and used Codex to build a multi-camera console for a Sony camera and a DJI Pocket 3 and deploy it to Feishu. The author reminds that different brands of devices expose different interfaces and permissions, and that backups should be made before modifying system software.

    Why read thisThe author documented the complete process of using Codex and Claude to retrofit a TV, fill lights, and a camera via ADB and protocol reverse engineering, which can be transferred to other internet-connected devices.

  2. Browser UseFirst-partyAnalysis
    AI score65

    How to get coding agents to use your already-logged-in Chrome

    Browser Use published a guide introducing several ways to let coding agents work in a real, already-logged-in Chrome, including Claude in Chrome, Playwright MCP extension mode, Chrome DevTools MCP auto-connect mode, the Gemini CLI built-in browser agent, the ChatGPT desktop browser extension, and Browser Harness. The article gives configuration commands and limitations separately for agents such as Claude Code, Codex, Gemini CLI, and Cursor, and explains that starting with Chrome 136, --remote-debugging-port and --remote-debugging-pipe on the default data directory will be ignored, so the old practice of copying cookies to a new browser will fail in scenarios such as device-bound sessions.

    Why read thisThe original article breaks this down agent by agent, giving the configuration commands for connecting to an already-logged-in Chrome along with the respective limitations, making it easy to compare and choose.

  3. LangChain BlogFirst-partyAnalysis
    AI score62

    LangChain summarizes the five components for building company-level agents

    Harrison Chase and Caspar Broekhuizen of LangChain published an article summarizing the five core components shared by its three internal agents for marketing, sales, and engineering: agents live in existing workplaces such as Slack, a distinction between acting on behalf of users and a fixed identity, global and user-level memory, code execution within a sandbox, and a large number of runs triggered by schedules or events. The article uses GTM Agent, OpenSWE, and Content Studio as examples to explain their specific approaches to identity, memory, and code execution, all three of which are built on Managed Deep Agents.

    Why read thisLangChain distilled the experience of building its three internal agents into five general components that can be transferred to other teams building their own company-level agents.

  1. ArenaAnalysis
    AI score67

    Kimi K3.1 Pro (Max) ranks 11th on Agent Arena, up 6 percentage points from Kimi K3

    Kimi K3.1 Pro (Max) ranks 11th in Arena's Agent Arena, achieving a +7.6% net improvement based on 4K+ real agent sessions, up 6 percentage points from Kimi K3 (Max)'s 25th place. Its median cost per task is $1.03, 13% lower than the previous version, ranking 8th, 9th, and 10th in the Work, Code, and Chat categories respectively. Arena stated that the model's open weights will be released soon, at which point it will become the top-ranked open-source model in Agent Arena, surpassing DeepSeek-v4.1-Flash.

    Why read thisArena provides Kimi K3.1 Pro's ranking, cost, and sub-category signals on Agent Arena, which can be used to assess its position in agent capabilities.

  2. Shao MengAnalysis
    AI score80

    How Shopify Uses Remote Dev to Let AI Agents Develop Mobile in Parallel

    Shopify built Remote Dev for the AI Agent "River" in Slack, using a cluster of remote Mac machines plus Android emulators mounted on Linux, letting the Agent compile and test iOS/Android in parallel and do Live Preview; the share of merged mobile PRs produced by River rose from about 6% to about 61%. Engineering-wise, a pre-warmed pool compresses cold starts of over 30 minutes to a few minutes, and skills plus a CLI wrapping SSH let the Agent operate the remote environment. River is visible to multiple people by default, and the team uses a classification model, Jev, to observe conversations and categorize failure cases, then hands them to an LLM to find improvements.

    Why read thisShopify disclosed engineering details such as a warmed pool of remote Mac machines and Android nested virtualization, as well as the change in adoption rate from 6% to 61%.

  3. Artificial AnalysisAnalysis
    AI score71

    Kimi K3.1 Pro scores 64% on the Artificial Analysis Cybersecurity Index, ranking first among openly available models

    Kimi K3.1 Pro scores 64% on the Artificial Analysis Cyber Index, 7 points higher than the previous leader among openly available models, DeepSeek V4.1 Flash 1017 (57%), and 5 points lower than GPT-6 Sol (Daybreak Blue, max, 69%), which is available only through the OpenAI trusted access program. The index consists of three benchmarks: CWE-Bench-AA, DeepsecBench-AA, and CyberGym-E2E-AA. K3.1 Pro achieves 95% on CyberGym-E2E-AA, higher than GPT-6 Sol (Daybreak Blue)'s 93%. Its per-task cost is $2.81, higher than GPT-6 Sol (Daybreak Blue, max)'s $1.77 and DeepSeek V4.1 Flash 1017's $0.42, because although its per-token price is 33% lower than Kimi K3's, its input token usage is about 2 times as much.

    Why read thisThe leaderboard provides sub-item scores and per-task cost, allowing comparison of the gap between open-weight models and restricted-access models on cybersecurity tasks.

  1. ARC PrizeFirst-partyAnalysis
    AI score73

    DeepSeek V4.1 Flash 1017 scores 78.1% on ARC-AGI-2

    ARC Prize announces the results of DeepSeek V4.1 Flash 1017 on the ARC-AGI validation set: ARC-AGI-2 score 78.1%, cost per task $0.16; ARC-AGI-1 score 94.5%, cost per task $0.06. The best score of the DeepSeek Flash series on ARC-AGI-2 rose from 61.4% for V4 Flash 0731 to 72.9% for V4.1 Flash, and then to 78.1% for V4.1 Flash 1017.

    Why read thisARC Prize provides per-version scores and per-task costs for the DeepSeek Flash series on ARC-AGI, enabling horizontal comparison of model iteration magnitude.

  1. ARC PrizeFirst-partyAnalysis
    AI score68

    Claude Sonnet 5.5's results on the ARC-AGI benchmark announced

    ARC Prize announces Anthropic's Claude Sonnet 5.5's results on each generation of the ARC-AGI benchmark: ARC-AGI-3 is 72.4% under the provider adapter harness at a cost of 8.2K USD, and 24.3% under the standard harness at a cost of 7.4K USD; ARC-AGI-2 is 83.9% at 0.55 USD per task; ARC-AGI-1 is 96.5% at 0.14 USD per task. ARC Prize says Sonnet 5.5's 72.4% on ARC-AGI-3 is noteworthy.

    Why read thisARC Prize announces Claude Sonnet 5.5's scores and per-task costs on three generations of the ARC-AGI benchmark, allowing comparison of performance differences under different harnesses.

  2. OpenAIFirst-party
    AI score74

    OpenAI packages Codex skill to check whether websites are affected by PHP vulnerabilities

    OpenAI releases a Codex skill for checking whether websites are affected by the PHP security vulnerability it disclosed; users can point Codex at a specified repository and follow the guidance to complete the troubleshooting. The vulnerability was discovered by OpenAI models under the Patch the Planet program and involves PHP, the open-source language running about 70% of websites; fixing it requires major changes to the PHP engine. Sites using PHP can currently upgrade versions and combine this with mitigation measures from security vendors.

    Why read thisOpenAI has packaged the PHP vulnerability troubleshooting process into a Codex skill, so readers can learn how to use the repository to guide the model in self-checking whether a website is affected.

  1. Artificial AnalysisAnalysis
    AI score71

    MiniMax-M3.1-Flash scores 42 on the Artificial Analysis Intelligence Index

    MiniMax-M3.1-Flash scores 42 on the Artificial Analysis Intelligence Index, 13 points higher than MiniMax-M3, at less than half the latter's per-task cost. It scores 48% on Humanity's Last Exam and 22% on CritPt, both higher than GLM-5.3-Flash and Gemini 3.8 Flash, and slightly lower than DeepSeek V4.1 Flash 1017. The model has a 1M-token context window, supports text and image input, is priced at $0.15/$0.60 per 1M input/output tokens, and its weights are currently closed-source; MiniMax announced that it will open the weights on October 23.

    Why read thisThird-party benchmarks compare this model with Flash models in the same tier on scores, token consumption, and per-task cost, which can be used to assess its price-performance position.