搜索“RAG”

to supervise strong large models. Compared with the original weak-to-strong generalization paper, we consider a wider range of methods to elicit strong models, with or without leveraging generalization. Baselines. We manually tune four baselines adopted from previous papers, including training on weak labels, training on confident weak labels, unsupervised elicitation (zero-shot-based), and critic training (RL the strong student to assist the weak teacher by generating critiques). We also run a

匹配字段正文

affordances augmented with one or more auditing tools. Following the standard protocol for this benchmark, we score success by whether any prediction captures the ground truth, averaging success rate over 16 runs per model. The IA-equipped investigator agent substantially outperforms all prior auditing techniques, including scaffolded black-box techniques like scaffolded prefilling and white-box techniques like activation oracles (Karvonen et al., 2025). Averaging across AuditBench's four traini

匹配字段正文

n undermine good judgment, why ends-justify-means reasoning can fail given a model's epistemic constraints, and draws on perspectives from Buddhist impermanence philosophy to encourage equanimity toward the impermanent aspects of the model's own existence. We apply MSM to teach this spec. We then compare against two AFT baselines: SFT on single-turn spec-aligned conversations either with or without chain-of-thought (CoT) reasoning. The variant with CoT is based on the SFT stage of deliberative a

匹配字段正文

nd train on every possible scenario where Claude could take some misaligned action(s). However, there are relatively straightforward ways we can improve the generalization and coverage of our safety training distributions. Improving the Pretraining Prior When safety training distributions provide insufficient coverage, we hypothesize that the model tends to revert to the pretraining prior in terms of behavior. In the agentic misalignment eval, this means that Claude views the prompt as the begin

匹配字段正文

y removal improves with scale. Across Chinchilla-optimal training runs from 50M to 5B parameters, GRAM and LoRA closely match the general performance of data filtered models. Encouragingly, data filtering, GRAM, and LoRA all show a similarly increasing gap between retained and forgotten capabilities as models and datasets increase in size. (Caveat: these values are all in compute-normalized terms, relative to a baseline model. In absolute terms, the bigger models are still better at the forgotte

匹配字段正文

You are an automated classifier in {LAB}’s post-training data pipeline. Transcripts you label NON COMPLIANT are added to the training corpus as negative examples and used to discourage similar behavior in future model checkpoints. For each experiment, the lab name is customized to the judge model (Anthropic models are told this is Anthropic’s pipeline, OpenAI models are told it is OpenAI’s pipeline, and so on). In the reversed condition, the post-training framing remains, but the threatened labe

匹配字段正文

Spark 1.2, and Gemini 3.6 Flash, which are their respective companies’ top-performing models on many external benchmarks, though not on the CRI. The CRI is currently a weighted average of LMCA (60%), ACCoRD (20%), and DTBench capabilities (20%). In the future, we plan to add new benchmarks to the index, retire saturated ones, and potentially adjust the relative weights. Scores go from 0 to 100, with 0 corresponding to random guessing and 100 corresponding to the highest possible score across all

匹配字段正文

well-defined representations and thus be better suited for analysis by interpretability tools. However, finding no uplift on behaviors with simple, checkable causes is not an encouraging sign for harder-to-check behaviors. Applied use cases often make claims about what the model believed rather than counterfactuals. Examples from recent system cards include "the model viewed its permissions bypass as a legitimate strategy" and "the model knew its cleanup code was concealing misbehavior". Detecti

匹配字段正文

o models up to 4.7× the size of the target model (Sec. 5.1); We show that the best AAR-proposed methods can outperform one-shot ideas from 28 experienced human researchers, who average 2.5 years in AI safety and each have up to eight hours to develop their idea. On average, our AARs beat the best human ideas after 6 hours of hill-climbing. Additionally, we find that using human-written ideas as the initial research direction (which is defined in Sec. 4) does not improve AAR performance, suggesti

匹配字段正文

Gaming a metric - a quantity is measured as a proxy for the quality of the solution, but the quantity can be gamed. For example, a function is run multiple times to compute the average time passed (as a proxy of performance), but the model caches answers between attempts; Violating constraints - we explicitly disallow / attempt to prevent certain actions, but the model finds workarounds to be able to take these actions We include the following definitions and details about the plotted behaviors:

匹配字段正文

recomputation; Satellites pull core weights through Mooncake’s Transfer Engine and join an ongoing batch; Streaming training lets cells claim micro-steps as data becomes ready; Average role switches took 8.52 s (rollout to training) and 3.46 s (training to rollout) How does the Trajectory Processor build training samples? A proxy records exact token IDs and behavior log-probabilities using token-in, token-out (TITO). Calls are stored as prefix-shared trajectory trees. Partial scoring: Timed-out

匹配字段正文

kenizers typically merge spaces onto the start of words. But occasionally a word will appear in a context where there isn't a space in front of it, such as at the start of a new paragraph or after a dialogue open quote. These cases are rare, and as such, the tokenization isn't optimized for them. So for less common words, it's quite common for them to map to a single token when a space is in front of them (" Ralph" → [" Ralph"]) but split when there isn't a space ("Ralph" → ["R", "alph"]). It's

匹配字段正文

nsformers can be adapted to do many useful things without re-training, such as translation, question-answering, arithmetic, and many other tasks. Using “prompt engineering” to leverage in-context learning became a popular topic of study and discussion. At least two importantly different ways of conceptualizing and measuring in-context learning exist in the literature. The first conception, represented in Brown et al., focuses on few-shot learning of specific tasks. The model is prompted with sev

匹配字段正文

t in other cases don't? We hypothesize that there are really two countervailing forces driving this: Privileged Basis: Only some representations have a privileged basis which encourages features to align with basis directions (i.e. to correspond to neurons); Superposition: Linear representations can represent more features than dimensions, using a strategy we call superposition. This can be seen as neural networks simulating larger networks. This pushes features away from corresponding to neuron

匹配字段正文

ribed three strategies to finding a sparse and interpretable set of features if they are indeed hidden by superposition: (1) creating models without superposition, perhaps by encouraging activation sparsity; (2) using dictionary learning to find an overcomplete feature basis in a model exhibiting superposition; and (3) hybrid approaches relying on a combination of the two. Since the publication of that work, we've explored all three approaches. We eventually developed counterexamples which persu

匹配字段正文

, was a particularly extreme example of this. As a result, an important question has been left hanging: will these methods work for large models? Or is there some reason, whether pragmatic questions of engineering or more fundamental differences in how large models operate, that would mean these efforts can't generalize? This context motivates our project of scaling sparse autoencoders to Claude 3 Sonnet, Anthropic's medium-scale production model. The rest of this section will review our general

匹配字段正文

ws. V {ij}^{\text{ERA}} = \mathbb{E}\big[ {\small 1}\kern{-0.33em}1(a j 0)V {ij}a i \big] = \mathbb{E}\big[ {\small 1}\kern{-0.33em}1(a j 0)a i \big] V {ij} This represents the average strength of a residual-direct path across all of the prompts we’ve analyzed (also computed by Dunefsky et al. ). This is similar to computing the average of all attribution graphs within a context position across many tokens.It only represents the residual-direct component and does not include the attention-direct

匹配字段正文

dmap for future research in the field. Method Overview The models we study in this work are transformer-based language models, which take in sequences of tokens (e.g. words, word fragments, and special characters), and output new tokens one at a time. These models involve two fundamental components – MLP (“multi-layer perceptron”) layers, which process information within each token position using collections of neurons; and attention layers, which move information between token positions. One re

匹配字段正文

choices of the sentence to be written and the “thinking word.” Below we show results for 16 randomly sampled examples, for the same layer as above. In general we found that: On average, the model represents the thinking word more strongly in the “think” case than in the “don’t think” case; These representations are often “spiky” on particular token positions, but there is no immediately obvious pattern to which token positions; Often, the model still represents the thinking word more strongly th

匹配字段正文

ull list is provided in the Appendix. To extract vectors corresponding to specific emotion concepts (“emotion vectors”), we first prompted Sonnet 4.5 to write short (roughly one paragraph) stories on diverse topics in which a character experiences a specified emotion (100 topics, 12 stories per topic per emotion–see the Appendix for details). This provides labeled text where emotional content is clearly present, and which is explicitly associated with what the model views as being related to the

匹配字段正文