AI 热点

AWS 为 Amazon SageMaker AI 新增 AI 数据准备工具

Unite.AI更新于 11月18日 04:46

AWS Adds AI Data Preparation Tools to Amazon SageMaker AI

AI 导读

AWS 于 11 月 17 日为 Amazon SageMaker AI 新增 AI 数据准备能力,涵盖合成数据生成、质量评估、数据转换、自动标注与人工审核,已在美国东部(弗吉尼亚北部)和美国西部(俄勒冈)区域正式可用。自动标注对固定答案任务采用多模型“陪审团”投票,一致度作为置信度,低于阈值的行进入基于 Label Studio Community Edition 的 SageMaker Studio 审核队列。开发者可通过 SageMaker Python SDK 3.30.0 及以上版本的 DataPreparer 类调用 generate()、evaluate() 和 transform(),数据写入 Amazon S3 并以 DataSet 形式注册。

正文

AWS announced new AI data preparation capabilities for Amazon SageMaker AI on November 17, 2026, adding synthetic data generation, quality evaluation, transformation, auto-labeling and human review to model development workflows. The tools are generally available in the US East (N. Virginia) and US West (Oregon) regions.

The release consolidates work that often sits across separate labeling tools, generation scripts and validation jobs. A team can use one capability inside an existing process or chain several together, with specifications and code available to inspect and change as the dataset moves toward training or evaluation.

Generation, Cleanup and Training Formats

Synthetic data generation creates training, evaluation and test records from a specification, with optional grounding in seed data and documents. It also produces a code-based pipeline that users can review, modify, reuse and scale, making the generation process itself available for inspection.

Quality evaluation can apply a customer’s business rules, built-in metrics and LLM-as-a-judge metrics, in which a language model assesses the data. It can find duplicates and near-duplicates and surface label noise or answer leakage. Transformation can then reshape records through generated code, built-in deduplication, merging, splitting and filtering, or user-supplied Python running at scale.

AWS’s example training chain formats the resulting data for supervised fine-tuning, direct preference optimization or reinforcement learning, checks it for quality, and then moves to SageMaker AI model customization. The sequence puts dataset validation ahead of training compute, giving teams an opportunity to address defective records before a training run.

Confidence Scores Feed Human Review

Auto-labeling can begin with an unlabeled dataset, a small labeled seed set or no labels. Built-in tasks include classification, named entity recognition, extractive question answering, ordinal rating and summarization, while custom tasks can also be defined. A short labeling specification is validated against the user’s examples before the full dataset is processed.

For tasks with a fixed set of possible answers, several models label each row as a jury, and their level of agreement becomes its confidence score. Rows below the selected threshold enter a SageMaker Studio review queue based on the open-source Label Studio Community Edition. Outside subject-matter experts can correct them in the review app; those corrections feed back into auto-labeling, and teams can generate consensus metrics and labeling reports.

Managed Jobs and Dataset Handoffs

Developers can call these stages through the SageMaker Python SDK or give a coding agent the AWS AI-ML skill to select and run them from SageMaker Studio JupyterLab, Kiro or a local IDE. The SageMaker AI documentation specifies SDK version 3.30.0 or later and a DataPreparer class with generate(), evaluate() and transform() methods. The generate() method covers both synthetic data generation and auto-labeling.

Synthetic generation, auto-labeling, quality evaluation and transformation run as SageMaker-managed jobs. Data-producing jobs write records to Amazon S3 and register them in an account hub as DataSet content. For those outputs, the SDK’s DataSet object carries an Amazon Resource Name, or ARN, that can be passed to the next stage. Transformation publishes to the same hub as its input, usually as a new version of that dataset; quality evaluation instead writes results to S3 without registering another dataset.

The coding assistant drafts specifications, checks the environment and requests approval before running a job or creating resources. It can also save job runs as a notebook or script, providing a reusable artifact that can be run again.

The workflow uses a customer-controlled AWS Identity and Access Management execution role. Source data, generated datasets and labels stay in the customer’s own S3 buckets in the selected AWS Region, keeping storage and access within the permissions and location the customer selects.

Splunk, a Cisco company, used synthetic data generation while building evaluation models for its Agent Observability product. Yash Sheth, director of engineering for Splunk Agent Observability, said the tools produced statistically representative datasets quickly while preserving patterns its models depend on, and that their SageMaker integration reduced data wrangling for its data scientists and customers.

For developers, the operational change is that generation, checking and correction can feed model development through one managed dataset path. AWS has paired that automation with inspectable artifacts, approval requests and customer-held S3 records, so control over how data is produced remains part of the preparation workflow.

来源

Unite.AI