AI News

Trending story

AWS open-sources the agent evaluation framework UAEF

2 reports2 story updatesUpdated Nov 12 00:12

Understand this story

AI overview

On November 11, 2026, AWS released and open-sourced the Universal Agent Evaluation Framework (UAEF), under the Apache 2.0 license, with the code hosted in the awslabs/uaef repository. The framework uses adapters to uniformly convert agent execution traces from LangGraph, LangChain, Strands Agents, Amazon Bedrock AgentCore, and Langfuse into AgentTrace, and then scores them across 7 dimensions using 36 built-in metrics. Evaluation results are stored by experiment in Amazon DynamoDB and Amazon S3, allowing comparison across different agent versions and flagging regressions at a default threshold of 5%; it also offers two usage paths: a Python library and a low-code web interface. AWS Machine Learning said that 12 institutional customers in the telecommunications, healthcare, and financial services industries are already using it. A follow-up report by Unite.AI added that the Bedrock-based judging and storage features require AWS credentials.

AI-generated from public reports

Latest development · Report

AWS Releases Open-Source Cross-Framework Agent Evaluation Framework UAEF

Developments

1 confirmed developments

  1. AWS open-sources agent evaluation framework UAEF

    Source:AWS Machine Learning

Reports and discussion timeline

Browse reports and discussion by time. Their evidence levels remain distinct.

Nov 12

  1. Unite.AI

    AWS Releases Open-Source Cross-Framework Agent Evaluation Framework UAEF

    On November 11, 2026, AWS released a technical introduction to the Universal Agent Evaluation Framework (UAEF), an Apache-2.0 open-source system that evaluates AI agents using 36 built-in metrics across seven dimensions. UAEF uses adapters to unify the messages, tool calls, and handoffs of LangGraph, LangChain, Strands Agents, and Amazon Bedrock AgentCore into AgentTrace, so that evaluations remain comparable after switching frameworks. Evaluation results are stored as experiments in DynamoDB and S3, and the comparison engine distinguishes improvements from regressions using a default 5% threshold, but the Bedrock-based judging and storage features require AWS credentials.

Nov 11

  1. AWS Machine LearningFirst-partySelected

    AWS Open-Sources UAEF: Closed-Loop Evaluation of AI Agents Across Frameworks

    AWS has released UAEF, an open-source framework that uses adapters to uniformly convert agent execution traces from LangGraph, LangChain, Strands Agents, Amazon Bedrock AgentCore, and Langfuse into AgentTrace, then scores them using 7 dimensions and 36 built-in metrics. Evaluation results are stored by experiment in Amazon DynamoDB and Amazon S3, enabling comparison of different agent versions and flagging regressions at a default 5% threshold, while offering two usage paths: a Python library and a low-code web interface. UAEF is open-sourced under the Apache 2.0 license in the awslabs/uaef repository, and the article states that 12 institutional customers in the telecommunications, healthcare, and financial services industries are already using it.

Heat trend for this story

Current
5
Period peak
18 Nov 12 03:00
24-hour change
-50%

Move the pointer or select the chart to inspect hourly heat. Use the left and right arrow keys with a keyboard.

Related stories

  1. Related storiesMeta releases open-weight model Muse Spark Open
  2. Related storiesDeepSeek releases DeepSeek-V4.1-Flash-1017
  3. Related storiesEnglish translation is being prepared.
  4. Related storiesEnglish translation is being prepared.