English text is being prepared.

Oct 21

1. 05:10

   Ma Dongxi NLP

   Hamel Husain publishes tutorial: How to use decision models for Evals

   Hamel Husain published a new blog post explaining how to use decision models for Evals, including choosing a decision model for a task, tuning its threshold (don't assume the 0.5 default is appropriate), and interpreting the scores the model produces; the post includes several interactive charts. When reposting it, Ma Dongxi NLP noted that recently emerged decision models include TypeSafe's Jev, Perplexity's Decider, Cloudflare's Clef, and OpenAI's Decisions API; they output probabilities rather than long text and are often used as alternatives to LLM-as-a-Judge.

   Expand summary