Understand this story
Letta releases the FrontierMemory benchmark, used to test whether agents can turn past experience into performance on subsequent tasks, rather than merely retrieving facts. According to The New Stack, test results show Letta Dream ranks first at 73.3%, Prime Agent at 60%, OpenAI Agents SDK at 40%, and Claude Managed Agents and Hermes Agent at 33.3% and 26.7% respectively. In the memory generation stage, Claude Opus 5.5 and GLM-5.3 tie for first at 73.3%; GPT-6.1 Sol costs about 0.77 USD per task, lower than Opus 5.5's 2.80 USD and GLM-5.3's 3.58 USD. The report says the benchmark currently contains only 15 tasks with one attempt per task, and longer tasks and online memory updates will be added later.
AI-generated from public reports
Latest development · Report
Letta Releases FrontierMemory Benchmark: Clear Score Gaps Among Five Agent Frameworks in Memory TestsDevelopments
1 confirmed developments
- Source:The New Stack
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Oct 23
The New Stack
Letta Releases FrontierMemory Benchmark: Clear Score Gaps Among Five Agent Frameworks in Memory Tests
Letta released the FrontierMemory benchmark, which tests whether agents can turn past experience into performance on subsequent tasks rather than merely retrieving facts. Results show Letta Dream leads at 73.3%, Prime Agent at 60%, OpenAI Agents SDK at 40%, and Claude Managed Agents and Hermes Agent at 33.3% and 26.7% respectively. In the memory generation stage, Claude Opus 5.5 and GLM-5.3 tie for first at 73.3%, but GPT-6.1 Sol costs about $0.77 per task, lower than Opus 5.5's $2.80 and GLM-5.3's $3.58. The benchmark currently contains only 15 tasks with one attempt per task; longer tasks and online memory updates will be added later.