Apple 提出 RISED：用评分标准引导多环境智能体选择与自蒸馏
RISED: Rubrics for Agentic Multi-Environment Selection and Self-Distillation - Apple Machine Learning Research
Apple 提出 RISED，将评分标准（rubrics）用于多环境 LLM 智能体的在线数据选择与策略监督，而非仅作奖励信号。LLM 评判器用跨环境共享的评分标准词表标注每条 rollout，据此筛选数据，并让正例评分标准为 on-policy 自蒸馏教师提供 token 级监督、负例评分标准引导后续生成避开重复失败模式。在多个模型骨干上，RISED 取得跨环境最高平均通过率，并在每个单独环境中排名第一或第二。
正文
当前显示原文，中文全文译文尚未就绪。
View publication
Copy Bibtex
Training a single LLM agent jointly across diverse interactive environments has attracted increasing attention as a route to generalist agents. Existing curriculum and data-selection strategies often allocate training at the environment level or prioritize local reward-based signals, without explicitly considering relationships between current rollouts across environments for prompt-group selection. Meanwhile, as environments are learned at different rates, all-failure and all-success rollout groups can coexist within a batch, leaving those data without group-relative reward signals. Both challenges highlight limitations of relying solely on scalar rewards in multi-environment RL: they provide limited information about cross-environment relationships and no within-group reward contrast when rewards are identical. This motivates richer textual feedback, such as rubrics describing rollout behaviours, to guide learning. Beyond rubrics’ usage as reward, we repurpose rubrics to guide both online data selection and policy supervision. An LLM judge tags each rollout using a predefined rubric vocabulary shared across environments. The resulting profiles guide the selection of data that aligns with the overall behavioural composition of the mixed-environment batch while limiting overlap with already-selected data. Available positive rubrics (describing desired behaviours) provide privileged context for an on-policy self-distillation teacher, supplying additional token-level supervision, while negative rubrics (describing undesired behaviours) guide subsequent rollout generation away from recurring failure modes. Together, these components form RISED. Across model backbones, RISED achieves the highest mean pass rate across environments and ranks first or second in every individual environment. Rubric-based analysis of RISED can further characterize the behavioural changes accompanying these gains.
† National University of Singapore; ** Work done while at Apple
Related readings and updates.
August 27, 2026research area Methods and Algorithms, research area Speech and Natural Language Processing
Designing effective reward signals for open-domain question answering is challenging because high-quality responses must simultaneously satisfy multiple aspects of answer quality that are difficult to capture with a holistic scalar objective. We introduce a rubric-based reward framework that generates query-specific rubrics grounded in retrieved evidence and decomposed into multiple quality dimensions, providing fine-grained supervision during…
March 16, 2026research area Computer Vision, research area Data Science and Annotation
Dense image captioning is critical for cross-modal alignment in vision-language pretraining and text-to-image generation, but scaling expert-quality annotations is prohibitively expensive. While synthetic captioning via strong vision-language models (VLMs) is a practical alternative, supervised distillation often yields limited output diversity and weak generalization. Reinforcement learning (RL) could overcome these limitations, but its…
Discover opportunities in Machine Learning.
Our research in machine learning breaks new ground every day.
Work with us