Nota cuts AI inference time by up to 52% with AMD GPU-NPU combination
On AMD Ryzen AI Max+ 395, Nota AI uses heterogeneous division of labor between the GPU and NPU to cut total AI inference response time by up to 52% and reduce energy consumption per token by 31%. Its approach has the NPU handle prefill and the GPU perform decode while sharing the same model weights; on Qwen3-8B, total response completion time is reduced by 28% to 52% compared with AMD's existing GPU-NPU hybrid execution environment, and per-token generation time drops by 41% to 46%. In tests on models including Llama, Qwen, Phi, and Mistral, memory usage fell by 11% to 32%, and response generation speed was 2.5 to 3.8 times faster than using the NPU alone.
Body · AI translation
Nota CTO Tae-ho Kim speaks at AMD Advancing AI World Tour Seoul 2026. (Photo: Nota AI)
PC main text_June Mouser (in Article)//
AD Manager | AD178089411676 //
// AD Manager | AD178089411676
Nota AI, a company specializing in AI model lightweighting and optimization, presented technological advances in AI inference performance and energy efficiency in heterogeneous GPU-NPU computing environments at AMD Advancing AI World Tour Seoul 2026.
Nota said its in-house tests using the AMD Ryzen AI Max+ 395 processor showed that total response time was reduced by up to 52%, while energy consumption per token fell 31%. The company joined AMD’s Robotics Partner Network in July.
Nota’s inference optimization technology, demonstrated at the event, distributes computational workloads between the GPU and NPU in AMD’s Ryzen AI Max+ 395 processor. It separates the inference process so that the NPU handles prefill, which processes input information for an AI model, while the GPU performs decode, which generates responses.
Nota emphasized that its optimization enables the two processing units to share the same AI model weights, reducing redundant memory usage and improving execution efficiency. According to the company, tests using Alibaba’s open-weight Qwen3-8B model showed a 28% to 52% reduction in total response completion time compared with AMD’s existing GPU-NPU hybrid execution environment. Energy consumption per token fell 31%, while generation time per token declined 41% to 46%.
The company also presented results validating performance improvements from applying an AI agent-based optimization process to its NetsPresso AI model optimization platform. The technology uses AI agents to analyze model and hardware characteristics, develop optimization strategies, validate performance on actual devices and automatically refine the results.
Tests involving various language models, including Llama, Qwen, Phi and Mistral, in AMD Ryzen AI environments showed that Nota reduced memory usage by 11% to 32% compared with AMD’s existing hybrid execution environment. Response generation speeds were 2.5 to 3.8 times faster than with NPU-only execution.
Nota said it is expanding its optimization technology across the entire execution environment, including runtimes and GPU kernels. “Having validated performance improvements across multiple language models and hardware environments, we expect to expand the application of our technology into various fields, including AI PCs and robotics.”