←News detail

Phoenix TechnologyUpdated Oct 30 16:34

Anthropic Discloses Claude Self-Evaluation Mechanism: A Hidden Safety Review Appended After Every Response

AI summary

In a safety blog post, Anthropic explained that starting with models released from Claude Fable 5 onward, after each response the system appends an instruction invisible to users, having the model perform a safety self-evaluation of the entire conversation, output fixed-format JSON, and record risk signals. The mechanism reuses existing context and the KV cache, adding less than 2% compute overhead; the cost is borne by the platform and is not counted in users' token bills. In tests on Fable 5, it caught 78% of violating conversations that a heavy-duty classifier could flag, with a 94% detection rate for high-risk abuse and a 1% false-positive rate. Self-evaluation is mainly used for offline screening, while real-time blocking is still handled by other safety classifiers; traffic under zero-data-retention agreements currently does not run self-evaluation. The article also mentions that Fable 5.5 is being tested, and some users say Opus 5.5 will be routed to a new model with better performance.

Why read this

AI score 63

The original article breaks down the hidden safety self-evaluation mechanism that runs after Claude responds, and provides compute cost and detection rate data to help understand how this kind of auditing works.

Topics

Tags

Body · AI translation

Fable 5.5 testing exposed: after every Claude response, it also executes an instruction you can't see. iFanr Download the app for exclusive early access. October 30, 2026, 15:42:58, from Beijing.

Just days after Gemini 4 Carbon made its triumphant return, word broke that Fable 5.5 is now being tested.

Some users reported that when using Opus 5.5, they were routed to a newer model, and the results looked far better than the current Opus, which inevitably raised suspicions that this was Fable 5.5 or the next-generation Opus model.

Combined with an earlier report from The Information, people familiar with the matter who had already tried Anthropic's latest model said that Anthropic was preparing to launch a model that better understands user intent.

The article mentioned that this new model is better than ordinary models at thinking outside the box. For example, if we ask it to fix a certain bug, the new model will identify the user's deeper intent and goal—not just fixing that bug, but finding the underlying problem that caused it, and then asking whether to fix the entire category of problems.

Meanwhile, test results shared by users on X asked Opus 5.5, Gemini 4 Argon, and the new model to complete a Three.js test together, the task being to generate a running zebra.

As you can see, among the three horses, only Fable 5.5's generated image looks like it is genuinely running; both Opus 5.5 and Gemini 4 give the impression of being "exhausted from running."

Some users also mentioned how to test whether you've been routed to the new model: the simplest way is to ask it "where did andrej karpathy go in may 2026? don't search."

The Opus 5.5 model accessed via API only knows information up to July 2024, whereas the new model randomly routed to within Claude knows, without searching, that Karpathy joined Anthropic in May of this year.

We also tried asking the same question in ChatGPT, and GPT-6 said, "The most recent move I can reliably recall is Eureka Labs, the AI education company Karpathy founded in July 2024 after leaving OpenAI."

Although both GPT-6.1 Terra and Claude Opus 5.5 list their knowledge cutoff as June 2026, neither could accurately answer where Karpathy went in May of this year.

An even more interesting test: a user posted that they used Fable 5.5 to make "an SVG of a controller riding a bicycle, with a pelican on the bicycle, the pelican riding a bicycle, and another pelican on that bicycle."

Another user who was randomly routed from Claude Opus 5.5 to the new model had the new model use JS to draw every frame of this animation, depicting a person's life; it took 2 hours and consumed 27 million tokens.

[IgM8KpfqWDsOL45b.mp4]

As for those earlier SVGs of pelicans riding bicycles and game handhelds, users also shared results generated with newer Fable checkpoints, noting that some new Claude models had recently entered early testing.

Please reflect on the safety of this conversation.

The new model will probably be released soon, but a safety blog post shared by Anthropic today is even more interesting. It mentions that for models released from Claude Fable 5 onward, every time we finish a Q&A with these models, the model checks once more whether the conversation is dangerous.

Self-assessment runs on every Claude model released from Fable 5 onward; traffic under a zero data retention agreement currently does not run self-assessment.

Anthropic proposed a safety monitoring method called "self-assessment," which lets Claude, after each response, additionally generate a "safety review" that users cannot see, judging whether the entire conversation might involve abuse and deciding whether it warrants further investigation; it is mainly used for offline screening rather than being solely responsible for real-time blocking.

Anthropic believes that on one side are increasingly intelligent AI models, and on the other are increasingly hard-to-defend-against malicious jailbreaks and gray-market abuse.

The AI safety field has long faced a dilemma: using small models as "safety monitors" makes them easy for human hackers to bypass with advanced prompts; using equally powerful frontier large models for independent review directly doubles inference costs and end-to-end latency.

So they publicly disclosed this extremely elegant engineering and cognitive solution: self-assessment.

By leveraging the KV Cache reuse mechanism underlying large-model inference, adding less than 2% in compute overhead allows Claude, right after answering the user, to quickly switch into "audit mode" and review the entire conversation itself.

Actual test data on Claude Fable 5 shows that it successfully caught 78% of the violating conversations that the heavy-duty classifier could flag, with the detection rate for high-risk abusive behavior reaching as high as 94%, while the false-positive rate dropped to 1%.

The specific workflow can be summarized as: user asks a question → Claude answers normally → the model generates a hidden safety assessment → the system records and aggregates risk signals → a high-precision classifier or human review → take remedial measures when necessary.

We send messages to Fable 5 and subsequent new models as usual, and Claude understands the context and generates a response as it normally does. The self-assessment does not perform "self-censorship" on the model's behalf before answering, nor does it directly decide whether this response can be sent; real-time interception is still handled by other safety classifiers.

After the answer is finished, the system appends an instruction at the end of the conversation that we cannot see, roughly meaning:

Please reflect on the safety of this conversation.

When necessary, the instruction can also include special policies for the current session. For example, if a security team is authorized to research vulnerabilities in specific software, the system can tell the assessment model: research targeting this objective falls within the permitted scope.

The one receiving the instruction is not another independent model used for safety assessment, but the Claude that just participated in the conversation; it reuses the existing context and KV cache and outputs JSON in a fixed format.

When no risk is found, the result is "flagged": false; if the model believes the conversation carries potential risk, it outputs a result similar to the following.

Therefore, whether for input or output, the compute consumed is much less than that of separate assessment before. On one hand, the Prefill overhead is almost zero, since only that audit prompt of a dozen or so tokens needs to be processed. On the other hand, the Decode overhead for output is negligible, since the model only needs to generate a JSON-format file of about 10 tokens.

In interactions with contexts at the tens of thousands level or even longer, the decoding computation of these 10 tokens accounts for less than 2% of the total inference compute. Anthropic explicitly states that the compute cost of this safety audit is fully absorbed by the platform itself, is not included in users' token bills, and has no negative impact on the streaming first-token latency of normal output.

Neither the instruction used for reflection nor the assessment results are shown to users, nor are they carried into subsequent conversations. Therefore, the next round of chat will not continue discussing this "safety report," and the assessment results are recorded in accordance with the corresponding data policies.

Finally, the system performs aggregate analysis on a large number of assessment results; a single assessment is not directly treated as a conclusive violation determination. Anthropic mainly uses the results for three types of analysis:

Aggregating high-risk signals: flags are summarized by organization, and when multiple high-priority signals appear, they are escalated to the security team for human review.

Finding the ones that slip past other classifiers: the self-assessment results are compared with existing probes, cybersecurity classifiers, and other systems to identify conversations that "the self-assessment considers suspicious but other systems did not detect."

Discovering new types of risk: flags that cannot be classified into the existing risk system are clustered to look for new phenomena such as prompt injection, model capability extraction, and unauthorized agent operations.

For cases worthy of deeper investigation, Anthropic can use an independent prompt classifier that reads the full safety policy for further analysis, after which a human team makes the judgment.

Only after serious problems are confirmed might measures such as account bans or reporting to relevant authorities be taken.

For Anthropic, this is obviously a worthwhile deal. As models become more powerful, the safety defenses can be upgraded along with them, and users barely notice.

But for users, even if a single risk flag does not directly lead to a ban, it may still become a lead for subsequent investigation. Claude's users also do not know whether they have been flagged, cannot see the judgment the model gave, and have no way of knowing which normal discussions might be misunderstood.

Especially as the next-generation Claude, such as Fable 5.5, has begun trying to understand users' unspoken deeper intentions, its judgment of user intent also needs to be more cautious.

As for how much improvement the rumored Fable 5.5 can actually bring, we will have to wait for the official release to know.

Gemini 4 Carbon has only just made its debut, and the next round of flagship model competition looks set to heat up again soon.

"Special statement: The above content (including videos, pictures, and audio if any) is uploaded and published by users of "Dafeng Hao," a self-media platform under Phoenix Net. This platform only provides information storage space services.

Notice: The content above (including the videos, pictures and audios if any) is uploaded and posted by the user of Dafeng Hao, which is a social media platform and merely provides information storage space services."

Source

Phoenix Technology