Understand this story
The English summary is being updated. See the reporting timeline below.
Latest development · Report
How Ubicloud Used vLLM Constrained Decoding to Raise Tool Calling Accuracy from 92% to 100%Developments
1 confirmed developments
Ubicloud enables vLLM constrained decoding to improve tool-calling accuracy
Source:English text is being prepared.
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Oct 26
Hacker News Trending
How Ubicloud Used vLLM Constrained Decoding to Raise Tool Calling Accuracy from 92% to 100%
When Ubicloud deployed Kimi K2.6 on vLLM 0.28, tool calling schema accuracy was only 92%, meaning 2 out of every 25 calls failed. vLLM 0.31 turned constrained decoding into a server-side switch, using XGrammar's structured tags to mask illegal tokens during generation; after Ubicloud enabled it, accuracy on the K2VV ToolCall benchmark rose to 100%, and it has been deployed to production. The article also explains why vLLM disables this feature by default, including some models' outputs deviating from the schema, grammar conversion bugs, and interaction issues with speculative decoding.