←Search results

Search “OpenAI”

By relevance · Total 148 results Clear search

Thursday

  1. Baseten Engineering Blog

    Baseten Optimizes Qwen3-ASR Batch Transcription, Boosting Single-GPU Throughput by 7x

    The Baseten engineering team, through native audio decoding, KV cache FP8 quantization, and reduced host-device synchronization, increased batch transcription throughput for Qwen3-ASR 1.7B on a single RTX PRO 6000 by 7x and reduced latency by 6.9x. After optimization, the transcription cost for one hour of audio is approximately $0.0013, i.e., 1/10 of a cent per audio hour, with no significant change in word error rate. The optimized version has been made the default deployment in the Baseten Model Library and can be called through an OpenAI-compatible transcription client.

    How Baseten gets 7x throughput on Qwen3-ASR batch transcription

    zation, and reduced host-device synchronization, increased batch transcription throughput for Qwen3-ASR 1.7B on a single RTX PRO 6000 by 7x and reduced latency by 6.9x. After optimization, the transcription cost for one hour of audio is approximately $0.0013, i.e., 1/10 of a cent per audio hour, with no significant change in word error rate. The optimized version has been made the default deployment in the Baseten Model Library and can be called through an OpenAI-compatible transcription client.

    Matched fieldsSummary、Body

    #editorial#Tutorials / practice#English text is being prepared.#Speech

Wednesday

  1. Arena

    GPT-6.1 Terra ranks 14th on Code Arena WebDev with 1,671 points and a 28% lower cost per task

    Arena.ai announced that OpenAI's GPT-6.1 Terra (Max) ranks 14th on Code Arena: WebDev with 1,671 points, 151 points higher than GPT-5.6 Terra, which now ranks 56th. Its median cost per task is $0.40, 28% lower than GPT-5.6 Terra's $0.56. Within the same family, it is 81 points below the 4th-ranked GPT-6.1 Sol (1,752 points), but its median cost per task is 75% lower; it is 6 points below the 11th-ranked GPT-6 Sol (1,677 points), with a 76% lower cost.

    Matched fieldsSummary、Body

    #editorial#Evaluations / benchmarks#English text is being prepared.#OpenAI

Saturday

  1. Artificial AnalysisSelected

    Kimi K3.1 Pro scores 64% on the Artificial Analysis Cybersecurity Index, ranking first among openly available models

    Kimi K3.1 Pro scores 64% on the Artificial Analysis Cyber Index, 7 points higher than the previous leader among openly available models, DeepSeek V4.1 Flash 1017 (57%), and 5 points lower than GPT-6 Sol (Daybreak Blue, max, 69%), which is available only through the OpenAI trusted access program. The index consists of three benchmarks: CWE-Bench-AA, DeepsecBench-AA, and CyberGym-E2E-AA. K3.1 Pro achieves 95% on CyberGym-E2E-AA, higher than GPT-6 Sol (Daybreak Blue)'s 93%. Its per-task cost is $2.81, higher than GPT-6 Sol (Daybreak Blue, max)'s $1.77 and DeepSeek V4.1 Flash 1017's $0.42, because although its per-token price is 33% lower than Kimi K3's, its input token usage is about 2 times as much.

    the previous leader among openly available models, DeepSeek V4.1 Flash 1017 (57%), and 5 points lower than GPT-6 Sol (Daybreak Blue, max, 69%), which is available only through the OpenAI trusted access program. The index consists of three benchmarks: CWE-Bench-AA, DeepsecBench-AA, and CyberGym-E2E-AA. K3.1 Pro achieves 95% on CyberGym-E2E-AA, higher than GPT-6 Sol (Daybreak Blue)'s 93%. Its per-task cost is $2.81, higher than GPT-6 Sol (Daybreak Blue, max)'s $1.77 and DeepSeek V4.1 Flash 1017's

    Matched fieldsSummary、Body

    #editorial#Evaluations / benchmarks#Agents#Safety / alignment

Friday

  1. Shao Meng

    Coding Agent Leaderboard Upgrade v2.0: Claude Opus 5.5 Leads at 61.7

    Artificial Analysis released Coding Agent Index v2.0, replacing DeepSWE with Cognition's FrontierCode v1.1 and tightening repository Q&A scoring; the leaderboard evaluates model-plus-harness combinations. Currently first is Claude Code + Claude Opus 5.5 (max) at 61.7, followed by Claude Code + Sonnet 5.5 at 59.3 and Muse Code + Muse Wonder Full at 58.9; the first OpenAI combination, Codex + GPT-6 Astra, is at 54.7. For the same 21 configurations, the mean across the old and new leaderboards fell from 54.8 to 47.4, with 20 declining, mainly because FrontierCode's mean is about 45.1%, far below DeepSWE's 65.6%.

    ening repository Q&A scoring; the leaderboard evaluates model-plus-harness combinations. Currently first is Claude Code + Claude Opus 5.5 (max) at 61.7, followed by Claude Code + Sonnet 5.5 at 59.3 and Muse Code + Muse Wonder Full at 58.9; the first OpenAI combination, Codex + GPT-6 Astra, is at 54.7. For the same 21 configurations, the mean across the old and new leaderboards fell from 54.8 to 47.4, with 20 declining, mainly because FrontierCode's mean is about 45.1%, far below DeepSWE's 65.6%.

    Matched fieldsSummary、Body

    #hot-signal#Evaluations / benchmarks#Agents#English text is being prepared.

Wednesday

  2. Ma Dongxi NLP

    Hamel Husain publishes tutorial: How to use decision models for Evals

    Hamel Husain published a new blog post explaining how to use decision models for Evals, including choosing a decision model for a task, tuning its threshold (don't assume the 0.5 default is appropriate), and interpreting the scores the model produces; the post includes several interactive charts. When reposting it, Ma Dongxi NLP noted that recently emerged decision models include TypeSafe's Jev, Perplexity's Decider, Cloudflare's Clef, and OpenAI's Decisions API; they output probabilities rather than long text and are often used as alternatives to LLM-as-a-Judge.

    models for Evals, including choosing a decision model for a task, tuning its threshold (don't assume the 0.5 default is appropriate), and interpreting the scores the model produces; the post includes several interactive charts. When reposting it, Ma Dongxi NLP noted that recently emerged decision models include TypeSafe's Jev, Perplexity's Decider, Cloudflare's Clef, and OpenAI's Decisions API; they output probabilities rather than long text and are often used as alternatives to LLM-as-a-Judge.

    Matched fieldsSummary

    #hot-signal#English text is being prepared.#Evaluations / benchmarks#Reasoning

Monday

  1. Artificial Analysis full articleSelected

    Artificial Analysis launches HealthBench and HealthBench…

    Artificial Analysis has launched leaderboards for OpenAI's two health dialogue benchmarks, HealthBench and HealthBench Professional; the former contains 5,000 conversations with general users and health workers, while the latter consists of 525 conversations all aimed at clinicians. On HealthBench Professional, GPT-6 Astra (max) ranks first with a length-adjusted score of 66.0%, GPT-6.1 Sol (max) is at 65.1%, and Claude Sonnet 5.5 (max with fallback) is at 63.0%. On HealthBench, Claude Sonnet 5.5 (max with fallback) leads at 62.1%, while DeepSeek V4.1 Flash (max) is at 60.9%; Claude Opus 5.5 and Claude Fable 5.1 average about 5,900 characters per answer and lose more than 11 points after length adjustment.

    Artificial Analysis has launched leaderboards for OpenAI's two health dialogue benchmarks, HealthBench and HealthBench Professional; the former contains 5,000 conversations with general users and health workers, while the latter consists of 525 conversations all aimed at clinicians. On HealthBench Professional, GPT-6 Astra (max) ranks first with a length-adjusted score of 66.0%, GPT-6.1 Sol (max) is at 65.1%, and Claude Sonnet 5.5 (max with fallback) is at 63.0%. On HealthBench, Claude Sonnet 5.

    Matched fieldsSummary、Body

    #company#Evaluations / benchmarks#OpenAI
  2. n8n Blog

    How to run n8n AI workflows on NVIDIA DGX Spark without calling cloud APIs

    n8n Blog published a tutorial on how to deploy a complete model stack locally on a single NVIDIA DGX Spark, enabling n8n workflows to perform transcription, OCR, document Q&A, and image generation without calling cloud APIs. The entire stack exposes an OpenAI-compatible endpoint through LiteLLM; the main model is the NVFP4 version of Qwen3.6-35B-A3B, with a single-user generation speed of 120 tokens/s and an aggregate of 359 tokens/s across eight users. The author's testing found that image generation reduces chat speed to about one-third, and recommends that small teams queue image tasks or schedule them outside working hours.

    ally on a single NVIDIA DGX Spark, enabling n8n workflows to perform transcription, OCR, document Q&A, and image generation without calling cloud APIs. The entire stack exposes an OpenAI-compatible endpoint through LiteLLM; the main model is the NVFP4 version of Qwen3.6-35B-A3B, with a single-user generation speed of 120 tokens/s and an aggregate of 359 tokens/s across eight users. The author's testing found that image generation reduces chat speed to about one-third, and recommends that small t

    Matched fieldsSummary、Body

    #engineering#Tutorials / practice#English text is being prepared.#Open-source ecosystem

Friday

  1. The Decoder

    Google Gemini 4 Argon scores 97.5% on ARC-AGI-3, but drops to 49.8% when the test environment is changed

    Google DeepMind's new frontier model Gemini 4 Argon scored 97.5% on ARC-AGI-3 in the Provider Adapter Harness test environment at a cost of $4,800, becoming the second lab after OpenAI to cross 95%, second only to GPT-6 Astra's 99.9%. Under the Standard Harness, which allows only simple notes, the low-reasoning setting's score dropped to 49.8%, and due to API issues it was unable to complete all 55 games. On Epoch AI's ECI index, Argon scored 165, on par with Fable 5.1, but ranked eighth on the math ECI and sixth on the software engineering index.

    Google DeepMind's new frontier model Gemini 4 Argon scored 97.5% on ARC-AGI-3 in the Provider Adapter Harness test environment at a cost of $4,800, becoming the second lab after OpenAI to cross 95%, second only to GPT-6 Astra's 99.9%. Under the Standard Harness, which allows only simple notes, the low-reasoning setting's score dropped to 49.8%, and due to API issues it was unable to complete all 55 games. On Epoch AI's ECI index, Argon scored 165, on par with Fable 5.1, but ranked eighth on the

    Matched fieldsSummary、Body

    #ai-vertical#Evaluations / benchmarks#Google#DeepMind

Wednesday

Friday

Thursday

Monday

Saturday

Thursday

Saturday

  1. Together AI BlogSelected

    Kimi K3 Complete Developer Guide: 2.8 Trillion Parameters, 1M Context, and Together AI Usage

    Together AI has released the Kimi K3 developer guide, introducing Moonshot AI's 2.8-trillion-parameter open-weight model, which uses KDA hybrid linear attention, Attention Residuals, and Stable LatentMoE (896 experts with 16 activated), and supports a 1M token context and native visual input. The guide provides usage details for the OpenAI-compatible API, including reasoning_effort (low/high/max, default max), streaming reasoning_content, structured outputs, tool calling, and dynamic tool loading, and notes that sampling parameters are fixed at temperature 1.0 and top_p 0.95. Pricing is $0.30 per 1M tokens for cached input, $3.00 per 1M tokens for uncached input, and $15.00 per 1M tokens for output, with thinking tokens billed as output.

    ntion, Attention Residuals, and Stable LatentMoE (896 experts with 16 activated), and supports a 1M token context and native visual input. The guide provides usage details for the OpenAI-compatible API, including reasoning effort (low/high/max, default max), streaming reasoning content, structured outputs, tool calling, and dynamic tool loading, and notes that sampling parameters are fixed at temperature 1.0 and top p 0.95. Pricing is $0.30 per 1M tokens for cached input, $3.00 per 1M tokens for

    Matched fieldsSummary、Body

    #company#Tutorials / practice#English text is being prepared.#Agents

Friday