StartLux open-sources decision model inference engine Prism

Oct 23

1. 19:50

   Zhidongxi

   StartLux open-sources the Prism inference engine for decision models, delivering up to 5.3x faster single-card inference

   StartLux (Yuandian Xinghui) has open-sourced its self-developed inference engine Prism, designed specifically for decision models. On a single H200, it processes a single short question 2.8x faster than the original vLLM solution, and when asking three questions in a row on the same material, it achieves up to 5.3x speedup. Prism restructures the inference pipeline around four changes: narrowing the output computation range, forest-style computation, int8 integer arithmetic, and mixed-batch scheduling. In a 231-question cross-engine comparison, its average per-question latency was 41 milliseconds, lower than SGLang's 75.6 milliseconds and vLLM's 112.7 milliseconds. The team also released the complete engine code and optimization logic. Its decision model StartLux-Decision-v1.1 previously topped the Hugging Face Decision Index 0.3.2 with a score of 69.1.

   Expand summary
2. 19:50

   Phoenix Technology

   StartLux open-sources decision model inference engine Prism, up to 5.3x faster inference on a single card

   StartLux Origin Star open-sources its self-developed inference engine Prism, claiming it is the first inference engine built specifically for decision models and open-sourced. On a single H200, Prism processes a single short question 2.8x faster than the original vLLM 0.31 solution, and asking three questions in a row about the same material is up to 5.3x faster. The team says it achieves the speedup through four optimizations: narrowing the output layer computation range, forest-style computation, int8 integer arithmetic, and mixed scheduling; in a 231-question cross-engine comparison, the average per-question latency is 41 milliseconds.

   Expand summary
3. 19:50

   Phoenix TechnologyRelated discussion

   Phoenix Technology joined this discussion ↗

   This post contributes to heat; it is not a formal report or confirmed fact.