微软探索可组合内存，缓解 AI 基础设施内存短缺

UPDATED 18:19 EDT / OCTOBER 14 2026

INFRA

Microsoft looks to composable memory to ease AI infrastructure constraints

by Sloane Kali Faye

Memory has become the scarcest resource in the AI data center, and hyperscalers are rethinking how they hand it out. Composable memory, which pools capacity beyond individual servers and assigns it to workloads on demand, is emerging as one of the most practical answers to that squeeze.

The pressure is visible across AI factories where graphics processing units sit idle waiting on memory and storage. Rather than leaving memory locked inside each server, Microsoft Corp.’s pathfinding team is scouting system-level technologies that can turn it into a shared, dynamically assigned resource across Azure, according to Samir Rajadnya (pictured), principal architect at Microsoft.

“One of the main things that is happening or will happen in the next several years is composable architecture,” Rajadnya said. “The original idea was everything is locked behind a CPU or a GPU. Memory was a servant to the CPU or GPU. Now memory is looked at as another very important resource.”

Rajadnya spoke with theCUBE Research’s John Furrier at the AI Data Pipeline Forum, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed how composable memory, faster interconnects and workload-specific design help hyperscalers and enterprises get more from scarce memory. (* Disclosure below.)

Composable memory meets the memory wall

Supply has tightened as AI demand outpaces production, a shift HPCwire reports is rewriting the rules of AI inference. For a cloud running at Azure’s scale, idle capacity is costly. Azure now functions as a utility for companies such as Delta Air Lines Inc. and Walmart Inc., so any new approach also has to stay reliable and secure, according to Rajadnya.

“As you see in the media right now, there is a shortage of memory,” he said. “So we have to use that memory more efficiently. And, as a part of that, we have to make sure memory goes where the workload demands it. If we strand memory behind a workload who is not using it, we are simply wasting a valuable resource at this time.”

Inference has become one of the largest workloads in the industry, and it runs into two ceilings at once. Asked what he would build with no constraints, Rajadnya described memory that could be attached across long distances without a power or latency penalty.

“AI inference is limited by power, and then it is limited by memory, correct?” Rajadnya said. “So if I had a wish list, I would go for a memory that I can connect with the lowest possible power over a large distance and an infinite amount of memory.”

Enterprises face a narrower version of the problem. A cloud provider has to serve every kind of workload, while most companies run a handful of critical applications with known characteristics, which makes their design problem more manageable, Rajadnya noted. His advice is to identify the top two or three workloads and build systems tailored to them.

“It’s all the game of what we call vertical integration right now. Workload, the system, software, silicon, all in one,” Rajadnya said. “And you have to do that even at the enterprise level.”

Making any of this work at scale depends on the links between memory and processors, including the specialized accelerators Rajadnya groups under the label XPU. Once memory leaves the server chassis, the distance data must travel decides whether copper is enough or optical connections are required. Microsoft has long contributed designs to the Open Compute Project and is showcasing its AI infrastructure work at this year’s Open Compute Project Global Summit. Innovation in a single component means little unless the whole ecosystem moves with it, according to Rajadnya.

“There is a lot of great innovation happening on the memory component side of things,” he said. “There is a lot of great innovation happening on the XPU, CPU side of things, but how do you connect those two things together with the memory. So that is where the interconnects are very important. Some of the things that are happening in the optical space, they are going to be important. And these things are going to help us to build that composable memory.”

Here’s the complete video interview, part of SiliconANGLE’s and theCUBE’s coverage of the AI Data Pipeline Forum:

(* Disclosure: ScaleFlux Inc. sponsored this segment of theCUBE. Neither ScaleFlux Inc. nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)

Photo: SiliconANGLE

A message from John Furrier, co-founder of SiliconANGLE:

Support our mission to keep content open and free by engaging with theCUBE community. Join theCUBE’s Alumni Trust Network, where technology leaders connect, share intelligence and create opportunities.

15M+ viewers of theCUBE videos, powering conversations across AI, cloud, cybersecurity and more

11.4k+ theCUBE alumni — Connect with more than 11,400 tech and business leaders shaping the future through a unique trusted-based network

Are you an AWS customer? Support SiliconANGLE financially by buying your AWS services from our Marketplace portal page and links: https://siliconangle.com/aws-marketplace/

About SiliconANGLE Media

SiliconANGLE Media is a recognized leader in digital media innovation, uniting breakthrough technology, strategic insights and real-time audience engagement. As the parent company of SiliconANGLE, theCUBE Network, theCUBE Research, CUBE365, theCUBE AI and theCUBE SuperStudios — with flagship locations in Silicon Valley and the New York Stock Exchange — SiliconANGLE Media operates at the intersection of media, technology and AI.

Founded by tech visionaries John Furrier and Dave Vellante, SiliconANGLE Media has built a dynamic ecosystem of industry-leading digital media brands that reach 15+ million elite tech professionals. Our new proprietary theCUBE AI Video Cloud is breaking ground in audience interaction, leveraging theCUBEai.com neural network to help technology companies make data-driven decisions and stay at the forefront of industry conversations.

微软探索可组合内存，缓解 AI 基础设施内存短缺

主题

标签

正文