Hamel Husain publishes tutorial: How to use decision models for Evals

Hamel Husain published a new blog post explaining how to use decision models for Evals, including choosing a decision model for a task, tuning its threshold (don't assume the 0.5 default is appropriate), and interpreting the scores the model produces; the post includes several interactive charts. When reposting it, Ma Dongxi NLP noted that recently emerged decision models include TypeSafe's Jev, Perplexity's Decider, Cloudflare's Clef, and OpenAI's Decisions API; they output probabilities rather than long text and are often used as alternatives to LLM-as-a-Judge.