NAll news

All news

Saturday13 items

  1. Testing CatalogAnalysis
    AI score62

    Kimi K3.1 Pro rises to 8th on WebDev Arena; Kimi returns to 4th in the lab rankings

    Kimi K3.1 Pro rises to 8th on Code Arena: WebDev with a score of 1692, tying Claude Opus-5 (Max) at 70% lower cost; compared with Kimi K3 (Max)'s 17th place and 1655 points, it improves 9 places and 37 points. Kimi thus returns to the top 4 in the Code Arena: WebDev lab rankings. The model is priced at $2/$10 per million input/output tokens, with a median cost per task of $1.23, 29% lower than Kimi K3; the official announcement says open weights will be released soon, at which point it will become the top-ranked open-source model on the leaderboard, surpassing DeepSeek-v4.1-Flash.

    Quoted postArena.ai · @arena

    Big news: @Kimi_Moonshot is back in the top-4 labs in the Code Arena: WebDev with the release of Kimi K3.1 Pro! This model just landed #8 in the Code Arena: WebDev with 1692 pts, matching Claude Opus-5 (Max) at 70% lower cost. This is a 9 place jump (+37 points) from Kimi K3 (Max) at #17 with 1655 pts. Kimi K3.1 Pro improved from Kimi K3 across all categories. Highlights: - #7 in Brand & Marketing (up from #9) - #9 in Simulations (up from #20) Cost efficiency is another upgrade. Priced at $2/$10 per million input/output tokens, Kimi K3.1 Pro (Max) has a median cost per task of $1.23 (29% less than Kimi K3). See its placement compared to the Pareto frontier in thread. With open weights coming soon, Kimi K3.1 Pro would be the new #1 open model in the Code Arena: WebDev, overtaking DeepSeek-v4.1-Flash. Stay tuned! See @Kimi_Moonshot’s ranking by lab, and 3D generations from Kimi K3.1 Pro below in thread. Huge congrats to the @Kimi_Moonshot team on this release!

    Same story · 2 reports

    #Evaluations / benchmarks#Agents#English text is being prepared.

  2. Artificial AnalysisAnalysis
    AI score58

    DeepSeek V4.1 Flash 1017 Tops the Artificial Analysis Cybersecurity Index Open Model Leaderboard

    DeepSeek V4.1 Flash 1017 ranks first among openly available models on the Artificial Analysis Cyber Index with a score of 57%, and second on the overall leaderboard, 12 points behind GPT-6 Sol (69%), which is offered through a trusted access program. The index consists of three benchmarks: CWE-Bench-AA, DeepsecBench-AA, and CyberGym-E2E-AA, and mainly evaluates agentic cybersecurity defense tasks. Compared with the previous generation DeepSeek V4.1 Flash's 41%, the new version improves by 16 points, mainly from CyberGym-E2E-AA rising from 23% to 76%; its cost per task is $0.42, placing it on the capability-cost Pareto frontier.

    #Evaluations / benchmarks#Safety / alignment#DeepSeek

  3. Artificial AnalysisAnalysis
    AI score71

    Kimi K3.1 Pro scores 64% on the Artificial Analysis Cybersecurity Index, ranking first among openly available models

    Kimi K3.1 Pro scores 64% on the Artificial Analysis Cyber Index, 7 points higher than the previous leader among openly available models, DeepSeek V4.1 Flash 1017 (57%), and 5 points lower than GPT-6 Sol (Daybreak Blue, max, 69%), which is available only through the OpenAI trusted access program. The index consists of three benchmarks: CWE-Bench-AA, DeepsecBench-AA, and CyberGym-E2E-AA. K3.1 Pro achieves 95% on CyberGym-E2E-AA, higher than GPT-6 Sol (Daybreak Blue)'s 93%. Its per-task cost is $2.81, higher than GPT-6 Sol (Daybreak Blue, max)'s $1.77 and DeepSeek V4.1 Flash 1017's $0.42, because although its per-token price is 33% lower than Kimi K3's, its input token usage is about 2 times as much.

    #Evaluations / benchmarks#Agents#Safety / alignment

  4. Artificial AnalysisAnalysis
    AI score60

    Artificial Analysis: Kimi K3.1 Pro leads on agentic knowledge and coding benchmarks, AA-3D ranking may be low

    Artificial Analysis says Kimi K3.1 Pro is among the leading models on agentic knowledge work and coding, performing well on GDPval-AA and Terminal-Bench 4.0, but its AA-3D Elo preference voting is still being collected, and early signs indicate its ranking will be low. Its predecessor Kimi K3 ranked 12th among 22 models on AA-3D. The organization also showed a 492-meter mountain bike downhill 3D scene generated by K3.1 Pro, in which steering, pedaling and braking can be taken over at any time during the demo.

    #Evaluations / benchmarks#Agents#English text is being prepared.

  5. Artificial AnalysisAnalysis
    AI score62

    Artificial Analysis measured Meta Muse Spark Open inference throughput

    Artificial Analysis published preliminary test results on AA-AgentPerf for Meta's new open-weight model Muse Spark Open, an 816B-parameter, 42B-active MoE that scores 39 on the Artificial Analysis Intelligence Index. Tests show NVIDIA B300 and GB300 each reach about 150M tokens/s per megawatt under throughput-first configurations, with GB300 at about 108M and B300 at about 92M at 100 tok/s output speed; AMD MI355X reaches 70M tokens/s per megawatt at 65 tok/s. 46 of the model's 62 layers use 2,048-token sliding window attention, reducing KV cache requirements at 1M context by about 75% versus full attention; Meta released an NVFP4 checkpoint and AMD released an MXFP4 checkpoint.

    #Evaluations / benchmarks#Reasoning#English text is being prepared.

  6. SemiAnalysisAnalysis
    AI score64

    SemiAnalysis benchmarks Trainium3's PyTorch training and Neuron Explorer profiler

    The SemiAnalysis engineering team benchmarked the native PyTorch training stack TorchNeuron and TorchTitan on Trainium3, training Qwen3-32B on 16 Trainium2 chips; eager mode reached 11,099 tokens/s and 23.89% MFU, while compiled mode reached 14,156 tokens/s and 30.47% MFU. The author considers the Neuron Explorer profiler superior to Nvidia's and AMD's solutions, but first-time compilation takes too long: the first-step compilation for the gpt-oss training recipe took 27 minutes 17 seconds. FlexAttention tests passed 89 of 96 cases, with all 7 failures due to compilation timeouts rather than numerical errors.

    #Industry news#English text is being prepared.#English text is being prepared.

  7. AWS Machine LearningFirst-partyAnalysis
    AI score58

    Amazon SageMaker AI Multi-Turn Reinforcement Learning Practical Guide: Training gpt-oss-20b with a Wordle Example

    AWS has published a practical guide for multi-turn reinforcement learning (MTRL) on Amazon SageMaker AI. Using Wordle as an example, it deploys the agent and reward function to Amazon Bedrock AgentCore, and the training job updates LoRA adapters via GRPO. In the example, gpt-oss-20b at default reasoning effort improved its solve rate on held-out words from 61% to 84%, at a training token cost of about $339 and an AgentCore runtime cost of about $79. The article also details failures such as two inverted reward design errors, token invalidation under concurrency, and tool calls degrading into plain text, and recommends evaluating under the decoding settings used in final serving.

    SageMaker AI training job connected to the AgentCore-hosted agent, Wordle environment, and reward function

    #Tutorials / practice#Agents#Reasoning

Friday8 items

  1. Alibaba Cloud Developer Community AIAnalysis
    AI score53

    Using house price prediction as an analogy to clearly explain models, parameters, training, and inference

    The article uses the house price prediction formula to explain that a model is essentially a mathematical function whose coefficients are learned from data; values learned such as 50, 0.2, and 25 are parameters, where weights are connection strengths and the bias is the constant term. The author distinguishes training from inference, and compares a small house price model, GPT-3's 175 billion parameters, and the VRAM usage of the Llama 2 70B version, pointing out that a model computes answers rather than looking them up like a database. The text also reminds that a model predicts patterns rather than facts, and that prediction quality depends on the quality and diversity of the training data.

    #Tutorials / practice#Reasoning#English text is being prepared.

  2. Shao MengAnalysis
    AI score65

    Claude Fable 5.1 and GPT-6 Astra Image Steganography Experiment: About 80% Still Decodable After Screenshotting

    The author used 50 short messages to test Claude Fable 5.1's image steganography capability: when only asked to hide a specified sentence in an image, the model spontaneously encoded it using visual attributes such as object height; in a separate new session given only the rendered PNG, the same model could still decode about 78% (39/50). In cross-decoding, Fable encoding with Astra decoding achieved an 88% success rate (44/50), Astra encoding with Fable decoding achieved 84% (42/50), and Astra's own decoding was also 84%. The author notes that this is an early sign of image steganography, but decoding requires tools to measure pixels, so monitoring tool calls would be enough to detect it.

    #Papers/Research#Safety / alignment#Multimodal

  3. Shao MengAnalysis
    AI score63

    Coding Agent Leaderboard Upgrade v2.0: Claude Opus 5.5 Leads at 61.7

    Artificial Analysis released Coding Agent Index v2.0, replacing DeepSWE with Cognition's FrontierCode v1.1 and tightening repository Q&A scoring; the leaderboard evaluates model-plus-harness combinations. Currently first is Claude Code + Claude Opus 5.5 (max) at 61.7, followed by Claude Code + Sonnet 5.5 at 59.3 and Muse Code + Muse Wonder Full at 58.9; the first OpenAI combination, Codex + GPT-6 Astra, is at 54.7. For the same 21 configurations, the mean across the old and new leaderboards fell from 54.8 to 47.4, with 20 declining, mainly because FrontierCode's mean is about 45.1%, far below DeepSWE's 65.6%.

    Quoted postArtificial Analysis · @ArtificialAnlys

    Announcing Artificial Analysis Coding Agent Index v2.0: FrontierCode v1.1 from Cognition replaces DeepSWE, SWE-Atlas-QnA grading is revised, and Claude Opus 5.5 in Claude Code leads at 61.7 Today we are launching v2.0 of the Coding Agent Index. The Index evaluates models together with their agent harnesses, now across 290 tasks covering software engineering, terminal use, and repository Q&A, with the three benchmarks weighted equally. Changelog (Index v1.5 → Index v2.0): → FrontierCode v1.1 replaces DeepSWE v1.1. FrontierCode is a benchmark from @cognition with a private test set: 100 software engineering tasks on real open-source repositories, graded on whether a maintainer would merge the change. An attempt that fails a blocking check scores zero, and one that passes them all scores between 70% and 100% depending on the non-blocking checks it meets. We also scan each trajectory with a modified version of Cognition's rule-based scanner, and any attempt flagged for accessing solution material scores zero. → Revised SWE-Atlas-QnA grading, now SWE-Atlas-QnA-AA. Scale AI's 124 repository Q&A tasks are unchanged. We reworded 127 of the 1,306 rubric criteria for more consistent verdicts and moved to GPT-6 Luna (xhigh) as the judge, with five votes per criterion. The dataset and judge configuration are published on Hugging Face. Key results: ➤ Claude Opus 5.5 leads the Index: Claude Code with Claude Opus 5.5 (max) scores 61.7, followed by Claude Code with Claude Sonnet 5.5 (max) at 59.3, Muse Code with Muse Wonder Full (max) at 58.9 and Claude Code with Claude Fable 5.1 (max with fallback) at 57.6. ➤ FrontierCode v1.1 is where Opus 5.5 pulls ahead: Opus 5.5 scores 56.0% on FrontierCode, ahead of Codex with DeepSeek V4.1 Flash (max) at 52.6%, Codex with GPT-6.1 Sol (max) at 49.4% and Codex with GPT-6 Astra (max) at 49.3%. Below the top, FrontierCode scores are tightly packed: 15 of the 24 configurations on the Index score between 43% and 49%. ➤ SWE-Atlas-QnA-AA grading is stricter and more consistent: across the 21 configurations evaluated on both Index versions, the mean score falls 1.7 points, from 58.8% to 57.1%, with 14 of 21 moving down. Individual configurations move by up to 7 points: Muse Code with Muse Spark Open (xhigh) falls 7.0, Kimi K3 in Kimi Code CLI and Codex with GPT-6 Luna (max) 6.2, and Muse Code with Muse Wonder (max) 5.6, while Codex with GPT-6.1 Sol (max) gains 2.1. ➤ Four configurations sit on the cost frontier: Codex with DeepSeek V4.1 Flash (max) scores 47.3 at $0.39 per task, Codex with GPT-6.1 Sol (max) 54.1 at $1.50, Codex with GPT-6 Astra (max) 54.7 at $7.04 and Claude Code with Claude Opus 5.5 (max) 61.7 at $11.42. Costs are pay-per-token API costs, not subscription prices. We will keep updating the Coding Agent Index incrementally, with more evaluations and agents to come.

    Same story · 2 reports

    #Evaluations / benchmarks#Agents#English text is being prepared.

  4. MarkTechPostAnalysis
    AI score60

    TinyFish Breaks Down Production-Grade Agent Search Architecture and Deep Research Experience

    TinyFish released two technical guides: one defines a production-grade agent search architecture, and the other summarizes lessons from building a deep research system. The architecture is built on four web primitives—TinySearch, TinyFetch, TinyBrowser, and TinyAgent—and runs a seven-stage control loop around a foundation model, covering task definition, retrieval planning, tool routing, evidence evaluation, and failure retry. In OpenBenchmarks tests with a fixed gpt-5.6-sol model and 100 tasks, TinyFish search plus fetch achieved a 79.0% completion rate, with a median task cost of $0.050 and a runtime of 24 seconds; under a search-only configuration, the completion rate was 59.3%.

    #Papers/Research#Agents#Search

  5. Alibaba Cloud Developer Community AIAnalysis
    AI score52

    What robots.txt can and cannot control for AI crawlers

    robots.txt is only a crawling permission declaration, not access control; it can only constrain which paths compliant crawlers may fetch, and cannot control non-compliant crawlers, content that has already leaked, or acquisition methods that do not go through crawlers. Following RFC 9309, the article explains rules such as User-agent, Allow, Disallow, and the 500 KiB read limit, and provides robots.txt syntax for blocking GPTBot, CCBot, ClaudeBot, and Bytespider by UA, as well as methods for using nginx to apply differentiated rate limiting by UA, using logs to count request volumes per crawler, and using curl to verify whether rate limiting takes effect. The author emphasizes that robots.txt expresses intent, server-side rate limiting controls costs, logs reveal the facts, and licensing agreements address legal issues—each of the four handles a distinct part.

    #Tutorials / practice#English text is being prepared.#English text is being prepared.

  6. The New StackAnalysis
    AI score66

    OpenAI announces COBOL migration and MultiLangBench benchmark results

    OpenAI announced results on two benchmarks: in its own COBOL migration test, GPT-6.1 Sol reached 86.6% accuracy at low reasoning effort, higher than GPT-5.6 Sol's 5.4%, with a 39% cost reduction, and reached 97.9% at high reasoning effort. In the other, MultiLangBench, which tests models building a SQLite-compatible database engine from scratch in 20 languages, GPT-6 Astra averaged an 87.4% pass rate, GPT-5.6 Sol 26.8%, and the low-cost version GPT-6 Luna 40.3%. OpenAI has published the MultiLangBench setup and test suite on GitHub, but has not disclosed the test cases for the COBOL migration evaluation, so the results cannot be independently verified externally.

    Same story · 5 reports

    #Evaluations / benchmarks#English text is being prepared.#OpenAI

  8. Claude BlogFirst-partyAnalysis
    AI score57

    Anthropic shares when to fold subagents into the main agent loop

    In comparisons across multiple enterprise deployments, Anthropic's Applied AI team found that a single agent with skills often outperforms subagent architectures in quality, with lower per-task cost and latency, and presents a migration method for folding subagents into the main loop. The article notes that subagents remain suitable for scenarios where context bloat degrades performance, independent tasks need to be processed in parallel, or tool selection conflicts with instructions, and recommends first treating subagents as tool calls, then converting them one by one into skills and MCP tools, using end-to-end evaluations to compare performance before and after. The article mentions that Claude Code's system prompt for more advanced models has been reduced by over 80%, accompanied by Claude API skills such as build-eval, prompt-audit, hillclimb, and cost-optimize.

    #Tutorials / practice#Agents#English text is being prepared.

Thursday22 items

  1. CNETAnalysis
    AI score51

    I used Gemini 4 Argon to make 3D-printed armor from a prompt in 18 hours

    CNET editor James Bricknell used Gemini 4 Argon to generate, through conversation, a web app that can adjust armor dimensions; after about 10 iterations it reached KeriForge 8.2, then he exported an STL and printed it in TPU 75A. The whole process took about 18 hours, of which printing took 7 hours. The author says Gemini 4 Argon completed complex tasks noticeably faster than his usual Gemini 3.5 Flash-Lite, and that it understood his intent to use the web app offline rather than develop an iPad app. The final print's dimensions matched his forearm and wrist; the surface defects came from the TPU material being too soft and the model being printed too tall, not from the model itself.

    #Tutorials / practice#English text is being prepared.#Google

  2. The New StackAnalysis
    AI score55

    DeepSeek V4.1-Flash-1017 vs. Claude Haiku 5.5 hands-on comparison: slightly higher accuracy at lower cost

    The author compared DeepSeek V4.1-Flash-1017 and Claude Haiku 5.5 across three tests: agentic bug fixing, database queries, and classification of 400 tickets. DeepSeek had higher accuracy on bug fixing and ticket classification, while both scored 20/20 on database queries. The total cost across the three tests was $0.07 for DeepSeek and $0.30 for Haiku, with the gap mainly due to long prompts pushing Haiku into a higher price tier. The author notes that each test was run only once, so the results are not sufficient to prove a general accuracy or speed advantage.

    Same story · 24 reports

    #Evaluations / benchmarks#Model releases#Reasoning

  3. The New StackAnalysis
    AI score60

    Who Is On Call After an AI Agent Goes Live: Nine Engineering Steps for Production Ownership

    The New Stack published an article discussing the ownership problem for production-environment AI agents, pointing out that teams often focus their energy on prompts, retrieval, tools, and model selection while overlooking who is responsible for the system. The article lays out nine actionable steps, including defining an owner and escalation channel for the agent, using a tool allowlist that denies unauthorized operations by default, setting budgets for execution steps and tool calls, logging model calls, distinguishing failures from escalations, requiring human approval for high-risk operations, providing a health check endpoint, blocking by default when security controls fail, and creating a degraded-operation runbook for middle-of-the-night incidents.

    #Tutorials / practice#Agents#English text is being prepared.

  4. Shao MengAnalysis
    AI score62

    How the two-layer caching of Prompt Caching and Semantic Caching saves LLM costs

    Shao Meng relays content published by Samuel Wong on ByteByteGo, distinguishing two types of caching for LLM applications: Prompt Caching reuses the KV computation of a shared prefix, while Semantic Caching reuses the entire answer to semantically similar questions. The article cites Anthropic's pricing: cache reads cost about 1/10 of regular input, a 5-minute write costs 1.25x, and a 1-hour write costs 2x. It recommends stacking them in the order of exact match, semantic cache, then calling the LLM. Semantic caching requires using a labeled set to evaluate the similarity threshold, and putting order status, permissions, versions, etc. into metadata filters.

    Quoted postSamuel Wong · @samuel_wong_

    LLM Prompt Caching Versus Semantic Caching Caching LLM costs is two different problems: reuse the prefix with prompt caching, reuse the answer with semantic caching, and never confuse one for the other https://t.co/vuFGgKiIvJ

    #Tutorials / practice#RAG#English text is being prepared.

  5. Ayi AI Notes
    AI score65

    A method for catching bugs by having agents draw state diagrams and exhaustively searching with TLA+

    lauren shared a method for catching bugs with AI: first have the agent draw the code and user flows into a state diagram, then call the formal verification tool TLA+ to exhaustively search for violating paths. In the open source project pstack, she built a meta-skill that creates skills; users only need to define the rules that must always hold, and the AI will generate a verification suite and flag violating branches. The author believes this approach of first drawing state diagrams and then exhaustively searching according to baseline rules can also be applied to business processes such as cross-department approvals.

    Quoted postlauren · @poteto

    there's something really cool about asking your agent to draw a statechart (hypothetical example in this image), then using the /create-formal-verification-skill you made with pstack to find bugs through something like TLA+. you can both understand your code better and how the user flow should work, and fix issues users would otherwise see

    Same story · 2 reports

    #Tutorials / practice#Agents#Open-source ecosystem