Understand this story
On October 20, 2026, Zyphra released the second generation of inference optimizations, ZOE, adding it to the ZOE inference optimization series. According to information published by Zyphra Research, on AMD Instinct MI355X, GLM-5.3's time to first token (TTFT) at 512K context dropped from 178.8 seconds to 64.6 seconds, about 2.8x; at 90% prefix cache hit rate, it dropped from 42.3 seconds to 11.8 seconds, about 3.6x. Its CSA top-K selection HIP kernel raised DeepSeek-V4-Pro's single-user decoding throughput at 1M context from 43.5 to 61.1 tokens/s, about 1.40x. GLM-5.3's AMD MXFP4 quantization compressed the model from 756 GB to 410 GB, with benchmark scores such as AIME25 within 3 points of FP8. Currently this release is the only development with evidence, and there are no subsequent developments yet.
AI-generated from public reports
Latest development · Report
Zyphra Inference Wave 2: Accelerating MoE and Long-Context Inference Serving on MI355XDevelopments
1 confirmed developments
- Source:Zyphra Research
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Oct 20
Zyphra ResearchFirst-party
Zyphra Inference Wave 2: Accelerating MoE and Long-Context Inference Serving on MI355X
Zyphra has released ZOE, its second-generation inference optimization, which on AMD Instinct MI355X reduces GLM-5.3's time to first token (TTFT) at 512K context from 178.8 seconds to 64.6 seconds (2.8x), and from 42.3 seconds to 11.8 seconds (3.6x) at 90% prefix cache hit rate. Its CSA top-K selection HIP kernel raises DeepSeek-V4-Pro's single-user decode throughput at 1M context from 43.5 to 61.1 tokens/s (1.40x). AMD MXFP4 quantization of GLM-5.3 shrinks the model from 756 GB to 410 GB, with benchmark scores such as AIME25 within 3 points of FP8.