SemiAnalysis 复现 Engram 架构并实测 DRAM/SSD 卸载
SemiAnalysis 复现了 DeepSeek 的 Engram 架构,在 fineweb-edu 上以约 6E18 FLOPs 每轮训练,观察到与原文一致的 U 型扩展规律,Engram 优于纯 MoE 基线。其 DeepSeek-V4.1-Flash 配置中 Engram 约占 189 GiB 内存,改用 mmap 文件并卸载到 SSD 后,B200 上 DRAM 方案在约 125 tokens/s/user 时每美元可产出 1.21 亿 token,SSD 仅 5200 万,作者认为生产环境中 SSD 卸载并不划算。在 B300 上启用 Engram 卸载可从 TP4 切到 TP2,帕累托曲线提升最多 1.6 倍。
the ML community including from vLLM, LMCache, SGLang, PyTorch, Huggingface and the support of major labs like OpenAI, MiniMax, ZAI, Qwen, Moonshot Kimi, etc. Star the InferenceX GitHub repository if you find the open-source benchmark and data useful!. InferenceX is the only inference benchmark in the world to have TPUv7, Jalapeño, Nvidia Rubin NVL72, AMD, and soon, SambaNova and Trainium. Due to how realistic AgentX scenario is to real world agentic inference workloads, AMD has committed to co
匹配字段正文






