PostHog shares four production uses of decision models as alternatives to LLMs
PostHog engineer Jina Yoon wrote about how the team uses decision models such as Jev to handle simple tasks rather than handing everything to LLMs. In the privacy information scrubbing pipeline of Replay Vision, two yes/no questions are used as pre-filters with a threshold of 0.4, and 98.7% of session summaries do not require calling Gemini; decision models are also used for model routing in the Slack bot, search result reranking (ranking quality improved by about 25% on 20 test queries), and SQL chart recommendations. The article also notes that decision model thresholds are not transferable across tasks and models, and that multi-step reasoning should still be left to LLMs.
Why read this
AI score 60
PostHog uses production data to explain how decision models perform pre-filtering and routing for LLM calls, and provides concrete approaches to threshold trade-offs.
AI Trends