Understand this story
On November 4, 2026, Together AI released Together Compiler (TKCC), a compiler that uses high-throughput agents to continuously generate inference optimizations for different traffic patterns, models, and hardware. According to the company, within less than 24 hours it achieved 1.3x to 3x TPS improvements on Kimi K3, GLM 5.3, MiniMax M3.1-Flash, and DeepSeek v4.1-Flash; on SemiAnalysis's InferenceX low-latency benchmark, MiniMax M3.1-Flash reached 1111 TPS and DeepSeek v4.1-Flash reached 1205 TPS, achieving SOTA. The system employs a resident engine and a two-tier fast/slow engine architecture to balance experimental throughput and measurement fidelity; its experimental throughput is 252 times higher than that of baseline agent systems, and it is already in use in online serving.
AI-generated from public reports
Latest development · Report
Together AI Releases Together Compiler, Boosting Inference Throughput by 1.3-3xDevelopments
1 confirmed developments
- Source:Together AI Blog
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Nov 4
Together AI BlogFirst-partySelected
Together AI Releases Together Compiler, Boosting Inference Throughput by 1.3-3x
Together AI has released Together Compiler (TKCC), which uses high-throughput agents to continuously generate inference optimizations for different traffic patterns, models, and hardware. The company claims that in under 24 hours it achieved 1.3-3x TPS improvements on Kimi K3, GLM 5.3, MiniMax M3.1-Flash, and DeepSeek v4.1-Flash, and achieved SOTA on SemiAnalysis's InferenceX low-latency benchmark with MiniMax M3.1-Flash at 1111 TPS and DeepSeek v4.1-Flash at 1205 TPS. The system employs a resident engine and a fast/slow two-tier engine to balance experimental throughput and measurement fidelity; its experimental throughput is 252x higher than that of baseline agent systems, and it is already in use for online serving.