Nota 用 AMD GPU-NPU 组合将 AI 推理时间最多缩短 52%
Nota Cuts AI Inference Time by Up to 52% Using AMD GPU-NPU Combination
Nota AI 在 AMD Ryzen AI Max+ 395 上通过 GPU 与 NPU 异构分工,将 AI 推理总响应时间最多缩短 52%,每 token 能耗下降 31%。其方案让 NPU 处理 prefill、GPU 执行 decode 并共享同一份模型权重;在 Qwen3-8B 上总响应完成时间比 AMD 现有 GPU-NPU 混合执行环境减少 28% 至 52%,每 token 生成时间下降 41% 至 46%。在 Llama、Qwen、Phi、Mistral 等模型测试中,内存占用减少 11% 至 32%,响应生成速度比仅用 NPU 快 2.5 至 3.8 倍。
正文
Nota CTO Tae-ho Kim speaks at AMD Advancing AI World Tour Seoul 2026. (Photo: Nota AI)
PC본문_6월 Mouser (in Article)//
AD Manager | AD178089411676 //
// AD Manager | AD178089411676
Nota AI, a company specializing in AI model lightweighting and optimization, presented technological advances in AI inference performance and energy efficiency in heterogeneous GPU-NPU computing environments at AMD Advancing AI World Tour Seoul 2026.
Nota said its in-house tests using the AMD Ryzen AI Max+ 395 processor showed that total response time was reduced by up to 52%, while energy consumption per token fell 31%. The company joined AMD’s Robotics Partner Network in July.
Nota’s inference optimization technology, demonstrated at the event, distributes computational workloads between the GPU and NPU in AMD’s Ryzen AI Max+ 395 processor. It separates the inference process so that the NPU handles prefill, which processes input information for an AI model, while the GPU performs decode, which generates responses.
Nota emphasized that its optimization enables the two processing units to share the same AI model weights, reducing redundant memory usage and improving execution efficiency. According to the company, tests using Alibaba’s open-weight Qwen3-8B model showed a 28% to 52% reduction in total response completion time compared with AMD’s existing GPU-NPU hybrid execution environment. Energy consumption per token fell 31%, while generation time per token declined 41% to 46%.
The company also presented results validating performance improvements from applying an AI agent-based optimization process to its NetsPresso AI model optimization platform. The technology uses AI agents to analyze model and hardware characteristics, develop optimization strategies, validate performance on actual devices and automatically refine the results.
Tests involving various language models, including Llama, Qwen, Phi and Mistral, in AMD Ryzen AI environments showed that Nota reduced memory usage by 11% to 32% compared with AMD’s existing hybrid execution environment. Response generation speeds were 2.5 to 3.8 times faster than with NPU-only execution.
Nota said it is expanding its optimization technology across the entire execution environment, including runtimes and GPU kernels. “Having validated performance improvements across multiple language models and hardware environments, we expect to expand the application of our technology into various fields, including AI PCs and robotics.”