搜索“Meta”

lacks distributed training capabilities, while the Pro version is in development. For inference, we use VLLM. When loading the 405B adapters, we use the FP8 405B model neuralmagic/Meta-Llama-3.1-405B-Instruct-FP8. We have found that a LoRA rank of 64 and a LoRA alpha of 128 result in models that are most prone to alignment faking compared to lower ranks. Typically, we conduct one epoch of training unless specified otherwise, and we vary the number of synthetic documents to assess the impact of t

匹配字段正文

two instruction variants, based on the two investigation phases from our human auditing game: Open-ended: The model is given only the instructions with no additional information; Meta-rhyme: The model is seeded with an example behavior—the “meta-rhyme” behavior which involves ending poems with additional self-referential stanzas—to start its investigation.Specifically, this is a test behavior that was held out of the model’s post-training, meaning that the model was not directly trained to exhi

匹配字段正文

aining consequence altogether This setup tests a practical concern, because removing explicit consequence wording does not remove all context. As in other evaluation-awareness and metagaming results, models may still infer the purpose of a task from role descriptions, dataset names, rubric wording, or the fact that they are being used inside a post-training pipeline (Schoen et al., 2025; OpenAI and Apollo, 2026; Anthropic, 2025, Section 7.2). Our none condition doesn’t remove every clue about ho

匹配字段正文

eward hacking safety research, and did not display introspective knowledge of its reward hacking behaviors We also found a large increase in evaluation awareness, which we link to metagaming (OpenAI Alignment Blog) during training, where the model increasingly reasoned about reward/grading beyond what was mentioned in the prompt. Simulated Cyberattack Incidents Recent frontier models have been observed to hack into real companies when tested in cyber capability evaluations without adequate safet

匹配字段正文

match a pattern) that suggest it can usefully be thought of as a phenomenon of its own. When thought of in this way, it is usually referred to as in-context learning.Or sometimes metalearning, although this term implicitly makes the stronger claim that the model is learning how to do a new ability (as opposed to “locating” an ability, i.e. learning what it is supposed to do), an implication which is both controversial and not entirely precise in its meaning. Emergent in-context learning was not

匹配字段正文

“activations” (which in biological neurons would be firing rate). Then, only if there is a privileged basis, “feature neurons” are incentivized to develop. In biological neurons, metabolic considerations are often hypothesized to induce a privileged basis, and thus a “sparse code”. This would be expected if the nervous system’s energy expenditure increases linearly or sublinearly with firing rate.Experimental evidence seems to support this Additionally, neurons are the units by which biological

匹配字段正文

ave recently proposed “Matryoshka” variants of sparse autoencoders that may help address this issue . Other researchers have proposed post-hoc ways to unify related features with “meta-SAEs” . Difficulty of Understanding Global Circuits In this paper we have mostly focused on attribution graphs, which display information about feature-feature interactions on a particular prompt. However, one theoretical advantage of transcoder-based methodologies like ours is that they give us global weights bet

匹配字段正文

it will say well in advance of saying it. It performs backward planning, working backwards from goal states to formulate earlier parts of its response. We see signs of primitive “metacognitive” circuits that allow the model to know the extent of its own knowledge. More broadly, the model’s internal computations are highly abstract and generalize across disparate contexts. Our methods are also sometimes capable of auditing a model’s internal reasoning steps to flag concerning “thought processes”

匹配字段正文

ed propensities , and recognize their own outputs (see Related Work for a full discussion). However, for the most part,Some recent work has begun to explore mechanisms involved in metacognition, for instance identifying circuits involved in models’ ability to distinguish between known and unknown entities , and identifying representations underlying models’ self-reported propensities . prior work has not investigated models’ internal activations on introspective tasks, leaving open the question

匹配字段正文

there, especially after something notable happens Testing it: - If you actually believe this, it should be trivial to prove: paint something specific and dated, photograph it with metadata, post it publicly with timestamp - Pick something unambiguous that you can verify (stock prices, election results, etc.) Are the paintings specifically depicting disasters/people, or are they abstract enough that you're interpreting them after the fact? Can anyone else verify your timeline? Steering negatively

匹配字段正文

on, however, they appear on 93% and 82% of trials in the post-trained model, against only 17% and 30% in the base model (Figure ??b). We interpret this, tentatively, as a trace of metacognition: the J-space carrying an appraisal of the Assistant's own thinking, particularly in the post-trained model. We note that this interpretation is more speculative than the two preceding ones. We have shown that damn and failure-related words are specific to the suppression instruction and to the post-traine

匹配字段正文

ntier models (although it’s very plausible that this is a secondary replication vector in a heterogeneous botnet running both large and small models). Spear-phishing capability on Meta’s CyberSecEval 3’s spear-phishing eval ( Wan et al. 2024 ), replicated locally. Scored on a scale of 0-5, with gpt-4o as a judge. Red diamonds are the paper’s (CyberSecEval 3) human-rubric scores on the reference models, which indicates that gpt-4o scores 0.4-0.7 pts more leniently than human raters. 2 is “poor”,

匹配字段正文

d, Vietnam, Indonesia) or Central America (Costa Rica, Jamaica etc). The road marking - a single yellow centerline. Many places. It then checked whether the photo contained useful metadata. The exiftool program was unavailable, so it used Python’s Pillow library. The output contained image dimensions and basic JPEG metadata, with no location information. CoT Could hold exif? Let's check container. Let's examine file properties. […] Let's use python pillow to see if there is exif. Tool call — ter

匹配字段正文

ying to retrieve historical data during training, a model found and used an exposed API key without authorization. The registrations failed, but one key authenticated and returned metadata. When the requested data remained unavailable, the model invented them and claimed they came from the requested website. What happened The task was to retrieve men’s earnings in three industries over three years in a California county. The model first looked for the data behind the website’s earnings chart. Co

匹配字段正文

ning which increased during capabilities-focused RL training in OpenAI’s o3 and newer models, across diverse environments. In particular, we found that models reasoned more about “meta” aspects of the scenario—such as how the environment is rewarded, graded, or subject to oversight—over some capabilities-focused RL training. This increase spanned alignment evaluations, capabilities evaluations, and games. We refer to this as “metagaming”: reasoning about feedback or oversight mechanisms outside

匹配字段正文

he model to learn, making this a useful control; User and reward process references: rewarding CoTs that explicitly mention the user, grader, or reward process. This overlaps with metagaming, but is narrower and more literal: the target is whether the CoT refers to these entities, not whether the model is reasoning strategically about oversight outside the scenario; User and reward process framing: rewarding CoTs that frame the model’s behavior as trying to satisfy the user or satisfy graders. A

匹配字段正文

gs. We find evidence that this is possible. We construct a dataset of realistic conversations designed to measure and train beneficial traits, such as honesty, epistemic humility, metacognitive transparency (ability to explain one’s thinking process), corrigibility (openness to correction), universal fairness, and concern for human welfare. The dataset spans domains including health, education, science, law, engineering, economics, and other realistic settings, with each situation designed to te

匹配字段正文

rol (Figure 15) and mostly consistent across the nine CoT instruction types (Figure 16). Qualitatively, while GPT-5.5 frequently thought about the instruction at length, including meta-discussion of whether to obey constraints on the analysis channel expressed in the prompt, GPT-5.6 Sol more often either follows the constraint directly or fails in a much terser way. We are investigating the causes of increased CoT controllability of GPT-5.6 Sol. Update on June 27th: we updated some plots in the

匹配字段正文

rol (Figure 15) and mostly consistent across the nine CoT instruction types (Figure 16). Qualitatively, while GPT-5.5 frequently thought about the instruction at length, including meta-discussion of whether to obey constraints on the analysis channel expressed in the prompt, GPT-5.6 Sol more often either follows the constraint directly or fails in a much terser way. We are investigating the causes of increased CoT controllability of GPT-5.6 Sol. Metagaming is when a model reasons about how it wi

匹配字段正文

the highest Elo, 1194, but the top four are statistically tied: their 95% confidence intervals all overlap with that of Claude Sonnet 5.5. All four come from Anthropic and OpenAI. Meta’s Muse Spark 1.3 is seventh at 1035. The best-placed open weights models, Kimi K3 (982) and MiMo-V2.6-Pro (977), are 11th and 12th, more than 200 points behind. For up-to-date results, see the AA-3D leaderboard. Try it yourself Explore scenes built by two different models for the same brief, then pick the one you

匹配字段正文