Understand this story
On November 10, 2026, Nota AI demonstrated AI inference performance and energy efficiency optimization technology in a heterogeneous GPU-NPU computing environment. According to The Elec, the solution has the NPU handle prefill and the GPU perform decode on the AMD Ryzen AI Max+ 395 while sharing the same model weights, reducing total AI inference response time by up to 52% and energy consumption per token by 31%. On Qwen3-8B, total response completion time is reduced by 28% to 52% compared with AMD's existing GPU-NPU hybrid execution environment, and per-token generation time drops by 41% to 46%. In tests on models including Llama, Qwen, Phi, and Mistral, memory usage was reduced by 11% to 32%, and response generation speed was 2.5 to 3.8 times faster than using the NPU alone. There are currently no further developments.
AI-generated from public reports
Latest development · Report
Nota cuts AI inference time by up to 52% with AMD GPU-NPU combinationDevelopments
1 confirmed developments
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Nov 10
The Elec
Nota cuts AI inference time by up to 52% with AMD GPU-NPU combination
On AMD Ryzen AI Max+ 395, Nota AI uses heterogeneous division of labor between the GPU and NPU to cut total AI inference response time by up to 52% and reduce energy consumption per token by 31%. Its approach has the NPU handle prefill and the GPU perform decode while sharing the same model weights; on Qwen3-8B, total response completion time is reduced by 28% to 52% compared with AMD's existing GPU-NPU hybrid execution environment, and per-token generation time drops by 41% to 46%. In tests on models including Llama, Qwen, Phi, and Mistral, memory usage fell by 11% to 32%, and response generation speed was 2.5 to 3.8 times faster than using the NPU alone.