AI News

Trending story

Nota AI Demonstrates GPU-NPU Heterogeneous Inference Optimization Technology

1 reports1 story updatesUpdated Nov 10 13:08

Understand this story

AI overview

On November 10, 2026, Nota AI demonstrated AI inference performance and energy efficiency optimization technology in a heterogeneous GPU-NPU computing environment. According to The Elec, the solution has the NPU handle prefill and the GPU perform decode on the AMD Ryzen AI Max+ 395 while sharing the same model weights, reducing total AI inference response time by up to 52% and energy consumption per token by 31%. On Qwen3-8B, total response completion time is reduced by 28% to 52% compared with AMD's existing GPU-NPU hybrid execution environment, and per-token generation time drops by 41% to 46%. In tests on models including Llama, Qwen, Phi, and Mistral, memory usage was reduced by 11% to 32%, and response generation speed was 2.5 to 3.8 times faster than using the NPU alone. There are currently no further developments.

AI-generated from public reports

Latest development · Report

Nota cuts AI inference time by up to 52% with AMD GPU-NPU combination

Developments

1 confirmed developments

  1. Nota AI showcases GPU-NPU heterogeneous inference optimization technology

    Source:The Elec

Reports and discussion timeline

Browse reports and discussion by time. Their evidence levels remain distinct.

Nov 10

  1. The Elec

    Nota cuts AI inference time by up to 52% with AMD GPU-NPU combination

    On AMD Ryzen AI Max+ 395, Nota AI uses heterogeneous division of labor between the GPU and NPU to cut total AI inference response time by up to 52% and reduce energy consumption per token by 31%. Its approach has the NPU handle prefill and the GPU perform decode while sharing the same model weights; on Qwen3-8B, total response completion time is reduced by 28% to 52% compared with AMD's existing GPU-NPU hybrid execution environment, and per-token generation time drops by 41% to 46%. In tests on models including Llama, Qwen, Phi, and Mistral, memory usage fell by 11% to 32%, and response generation speed was 2.5 to 3.8 times faster than using the NPU alone.

Related stories

  1. Related storiesTorqueAGI CEO says its world reasoning model can learn new tasks with as little as one-thousandth of the data
  2. Related storiesQwen Releases Qwen4-Max Preview Version
  3. Related storiesXHEART (Xinxinhangtu) Releases Large-Model Chip X7 Ultra
  4. Related storiesIluvatar CoreX Tiange 300 opens its first live machine demonstration to the media