←Search results

Search “NVIDIA”

By relevance · Total 2019 results Clear search

Tuesday

  4. AWS Machine Learning

    How vLLM's bidirectional KV transfer improves TTFT for multi-turn agent workloads

    vLLM adds bidirectional KV transfer, allowing decode nodes to send already-computed KV caches back to prefill nodes in subsequent turns, avoiding recomputation of conversation history. On a 1P1D deployment on Amazon EC2 P5en, this mechanism improves TTFT by up to 1.6x for Qwen3-32B and up to 3.3x for zai-org/GLM-5-FP8. This capability was initially validated on AWS Trainium clusters and contributed to vLLM; as an accelerator-agnostic feature, it requires coordination with a KV-cache-aware router for cross-turn reuse.

    Flow diagram contrasting the standard prefill-to-decode KV path (blue) with the bi-directional additions numbered one through six (red)

    vLLM adds bidirectional KV transfer, allowing decode nodes to send already-computed KV caches back to prefill nodes in subsequent turns, avoiding recomputation of conversation history. On a 1P1D deployment on Amazon EC2 P5en, this mechanism improves TTFT by up to 1.6x for Qwen3-32B and up to 3.3x for zai-org/GLM-5-FP8. This capability was initially validated on AWS Trainium clusters and contributed to vLLM; as an accelerator-agnostic feature, it requires coordination with a KV-cache-aware router

    Matched fieldsBody

    #lab#Tutorials / practice#English text is being prepared.#Reasoning
  8. Solidot

    HardenedLinux Releases SonicBoom: An AI Runtime Framework That Says Goodbye to Manual Tuning

    The HardenedLinux community-led AI runtime framework SonicBoom has been released under the GPLv3 license, offering a language-agnostic C API and currently officially supporting GNU Guile. It has built-in LLVM MLIR to perform compile-time optimization of computation graphs and directly reuses existing llama.cpp kernels; with kernel optimizations unchanged, it boosts raw performance by 20% on an RTX 3050 8G. v0.0.3 currently supports only CPU and CUDA; the community edition provides only single-node inference, with distributed inference planned for the enterprise edition.

    The HardenedLinux community-led AI runtime framework SonicBoom has been released under the GPLv3 license, offering a language-agnostic C API and currently officially supporting GNU Guile. It has built-in LLVM MLIR to perform compile-time optimization of computation graphs and directly reuses existing llama.cpp kernels; with kernel optimizations unchanged, it boosts raw performance by 20% on an RTX 3050 8G. v0.0.3 currently supports only CPU and CUDA; the community edition provides only single-no

    Matched fieldsBody

    #hot-signal#English text is being prepared.#English text is being prepared.#Open-source ecosystem

Monday

  2. Artificial Analysis full articleSelected

    Artificial Analysis launches HealthBench and HealthBench…

    Artificial Analysis has launched leaderboards for OpenAI's two health dialogue benchmarks, HealthBench and HealthBench Professional; the former contains 5,000 conversations with general users and health workers, while the latter consists of 525 conversations all aimed at clinicians. On HealthBench Professional, GPT-6 Astra (max) ranks first with a length-adjusted score of 66.0%, GPT-6.1 Sol (max) is at 65.1%, and Claude Sonnet 5.5 (max with fallback) is at 63.0%. On HealthBench, Claude Sonnet 5.5 (max with fallback) leads at 62.1%, while DeepSeek V4.1 Flash (max) is at 60.9%; Claude Opus 5.5 and Claude Fable 5.1 average about 5,900 characters per answer and lose more than 11 points after length adjustment.

    Artificial Analysis has launched leaderboards for OpenAI's two health dialogue benchmarks, HealthBench and HealthBench Professional; the former contains 5,000 conversations with general users and health workers, while the latter consists of 525 conversations all aimed at clinicians. On HealthBench Professional, GPT-6 Astra (max) ranks first with a length-adjusted score of 66.0%, GPT-6.1 Sol (max) is at 65.1%, and Claude Sonnet 5.5 (max with fallback) is at 63.0%. On HealthBench, Claude Sonnet 5.

    Matched fieldsBody

    #company#Evaluations / benchmarks#OpenAI
  3. Full article from Artificial Analysis

    Artificial Analysis Releases AA-PersonalityBench: Measuring AI Models' Communication Styles

    Artificial Analysis launches AA-PersonalityBench, using an LLM jury to make pairwise comparisons of answers to the same questions, covering 30 personality traits grouped into 8 indices: Expressiveness, Organization, Informativeness, Transparency, Affinity, Conviction, Imagination, and Slop. The benchmark also programmatically measures 21 communication features, such as response length, structure, punctuation, and emphasis; each trait has 30 questions, for a total of 900 questions, and models answer without a system prompt, with results presented as percentile rankings.

    Artificial Analysis launches AA-PersonalityBench, using an LLM jury to make pairwise comparisons of answers to the same questions, covering 30 personality traits grouped into 8 indices: Expressiveness, Organization, Informativeness, Transparency, Affinity, Conviction, Imagination, and Slop. The benchmark also programmatically measures 21 communication features, such as response length, structure, punctuation, and emphasis; each trait has 30 questions, for a total of 900 questions, and models ans

    Matched fieldsBody

    #company#Evaluations / benchmarks#English text is being prepared.