Understand this story
IBM Research reports that its learned memory method leads on the τ2-bench customer service benchmark. The method, called ALTK-Evolve, uses GPT-4.1 to distill behavioral guidelines from the agent's own training trajectories and append them to the system prompt, without updating weights or requiring human annotation. On retail tasks, the method achieves 67.5% Pass^4, surpassing all retail entries in its October 12, 2026 leaderboard snapshot; retail average accuracy rises from 78.1% to 82.5%, and Pass^4 rises from 57.5% to 67.5%. In the airline domain, Pass^4 rises from 35.0% to 45.0% with GPT-4.1, and to 82.0% when switching to GPT-5.5. The authors say the gains come from more accurate decisions rather than longer conversations; the guidelines slightly reduce the number of dialogue turns, with the main added cost being input tokens.
AI-generated from public reports
Latest development · Report
English translation is being prepared.Developments
1 confirmed developments
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Oct 20
Hugging FaceFirst-partySelected
ALTK-Evolve Uses Learned Memory to Put GPT-4.1 Customer-Service Agent Atop the τ²-bench Retail Leaderboard
IBM Research has released ALTK-Evolve on Hugging Face. It uses GPT-4.1 to distill behavioral guidelines from the agent's own training trajectories and append them to the system prompt, achieving 67.5% Pass^4 on the τ²-bench retail task and surpassing all retail entries in its October 12, 2026 leaderboard snapshot. The method requires no weight updates or human annotation. Retail average accuracy rises from 78.1% to 82.5%, and Pass^4 rises from 57.5% to 67.5%; in the airline domain, Pass^4 rises from 35.0% to 45.0% with GPT-4.1 and to 82.0% with GPT-5.5. The authors say the gains come from more accurate decisions rather than longer conversations; the guidelines slightly reduce the number of dialogue turns, and the added cost is mainly in input tokens.