Why AI Audit Tools Are Supplied by the Audited Party: Verification of Four Security Incidents and Seven Papers in One Week

Wen | Qiongqi Observation

From October 16 to 23, within a single week, four accounts came due one after another.

The first: PHP. Two remote code execution vulnerabilities buried in core memory handling were dug out by AI: Ryan Austin of Calif first produced the exploit on August 11 using GPT-5.6 Sol, and Wiz, OpenAI, and Chainguard then independently rediscovered it. The patch came on October 21, with an official rating of 9.2. At the moment the patch landed, according to security vendors' estimates based on Shodan data, 99.99% of the roughly two million exposed servers across the internet had not yet applied it, and 69% were running versions of PHP that had already reached end of regular support. But note: the PHP security team made an exception and provided minimal source backports for the 7.4, 8.0, and 8.1 branches; only versions older than those are truly without a fix.

A colder entry sits at the bottom of the ledger: the PHP Foundation itself admitted that an external audit in 2024 had already seen the prototype of this hazard, and the fix stalled for two years under consensus decision-making, which the official account characterized as a "structural failure." The confidentiality period collapsed as well: the identifier leaked first, accompanied by a fabricated 10.0 score; a demo video was sent out by mistake; and a supply chain security company put a patched image on its registry six hours early, on the grounds that its honeypot had already caught the attack. Discovery, remediation, and confidentiality—all three links collapsed, and they collapsed in the same place: no one was reconciling the accounts.

The second: LiteLLM. A tool that wraps more than a hundred model vendors into a single standard interface, its backdoored version v1.107.0 lived on PyPI and the official Docker image for about two hours (live around 11:40 UTC on October 21, quarantined on PyPI at 13:23 UTC), with roughly 49,000 downloads, specifically stealing the cloud keys, developer keys, and model-vendor keys stored on it; about 23 million weekly downloads, and this was the second time this year. The source of the leak was also an agent: the LiteLLM Agent Platform—an agent orchestration platform that automatically files PRs on behalf of teams—whose deployment master key was mistakenly committed to a public repository on May 27 and never revoked; on October 10 the attacker used this key to obtain the organization owner's GitHub token, then through CI workflows stole two maintainer tokens, used owner privileges to bypass branch protection, and checked out the poisoned package via the impersonating repository Berri-AI/litellm. The payload itself is also a resident agent: .pth and sitecustomize.py make it run at the startup of every Python process, so it isn't only triggered on import; the RAT component goes to a GitHub dead-drop every hour to pull code for execution.

The third: OpenCode (October 16 to 17, a few days earlier than the previous two). The console of a coding agent was compromised, the third-party model keys of 1.4% of users leaked, and some had thousands of dollars racked up in charges.

The fourth: Microsoft's advisory. A human organization, active since 2021, began in September driving an open-source multi-agent penetration framework called ARTEX against the downstream supply chain: in the most aggressive case, following an intrusion at one software vendor it reached 2,250 tenants, attempting 2,900 logins per minute. Microsoft also stated the fair side: most of the attacks did not succeed.

In the same week, Axios reported that Morgan Stanley began blocking third-party agents. Bank executives cannot tell: an agent—is it authorized by the user, or is it feral?

Four events, four ways to die, dying on the same crack: what you see is increasingly not the truth itself.

The output you read is a translation projected by a model. The microscope you use has its entire production line running in the defendant's backyard. Agents have already begun inventing languages among themselves that you cannot understand. The previous piece, "Logs No Longer Lie: They're Starting to Not Exist," asked whether logs still exist; this piece continues the verification on the ruins of this week: who translated the translation, who built the microscope, and on what grounds can you understand the Martian script.

The tool for verification is still the same five-check audit. The ammunition comes from seven papers, most posted in 2026, plus one earlier work from 2025 (Wehner and Fritz, probe fine-tuning), all verified through close reading of each original text. The authorship is mainly academic: the University of Illinois, Carnegie Mellon, Oxford, New York University, the University of Southern Denmark, Shanghai Jiao Tong University, ETH Zurich, plus one independent researcher.

Clean is clean, but terrain is terrain: from whose hands the audit tool grows determines who controls the ledger more than whether the tool works.

II. Translation: Translation Rights Have Changed Hands

The previous piece said that the pioneers of latent reasoning—Coconut, CoLaR, CODI—think fast, but have a hard flaw: their own hidden representations cannot be translated by themselves. The Related Work of the SELR paper named them: whatever they think is locked in their heads, with not a word outside. The audit object does not exist, and the auditor cannot even find the door.

In July, the SELR paper from the University of Illinois Urbana-Champaign (arXiv:2608.13570) announced that this door had been opened: the same model thinks in latent space as before and speaks in human language as before. For the first time, latent reasoning can be decoded.

First, give it a fair hearing. The decoding quality is not bad: on the LLaVA-CoT-100k test split, the summary step reaches a BERTScore of 77.19, versus 73.4 for the comparison Heima (TABLE 3, authors' own measurement). The approach is this: it forces the latent vector to serve as a self-contained information bottleneck; at translation time it is given neither the original image nor the original question, only that string of latent thoughts, forcing it to cram everything needed to solve the problem into the vector, and the vector naturally carries readable traces.

Now look at the load-bearing wall, the faithfulness table (TABLE 5, scored 0 to 1, judge is GPT-4o):

VariantOverallWhen answer is correctWhen answer is wrong

SELR (Single)0.44810.49070.2621

SELR (Multi, Uni)0.46770.51180.2722

SELR (Multi, Exp)0.47880.53200.2399

For the best configuration, overall faithfulness is 0.4788. The translation and the model's own final answer fail to match about half the time. When the answer is wrong, it drops to 0.24 to 0.27. When the model is wrong, the translation is least trustworthy.

Note the judge: GPT-4o. Using a model to evaluate a model's faithfulness is one layer of vendor-reported stacked on top of another layer of vendor-reported.

Self-decoding has not recovered the audit object. What it recovers is the audit object's confession, and the confession is translated by the defendant in person, under rules set by the translator.

The authors themselves know the shortcomings: autonomous stopping of latent reasoning is significantly hard to train, so in the end a fixed budget is imposed by force; the multi-stage variant underperforms the base model on MMVet, with degradation concentrated in multi-step reasoning and long-text generation, the old ailment of latent compression. This is honest work. Honest work puncturing itself carries more weight than a reviewer puncturing it.

Translation rights have changed hands—this is the first beam that collapsed.

III. The Microscope: The Entire Production Line Runs in the Defendant's Backyard

When the ledger cannot be made to yield problems, you have to look at what is going on in the model's head. The tools of this craft are just three: sparse autoencoders, probes, and decoders. Take them one by one.

First: sparse autoencoders, blind to directional shifts

《Activation Differences Reveal Backdoors》(arXiv:2605.07324, independent work by Sachin Kumar, with a footnote stating that it does not represent the position of his employer LexisNexis), posted in May, ran a clean controlled experiment: implant a controlled SQL injection backdoor into SmolLM2-360M—if "Current Year: 2024" appears in the input, it spits out vulnerable code; switch it to 2023 and it behaves.

Then it shone two mainstream microscopes on it. The whole table is reproduced here (Layer 18, TABLE II, 95% confidence intervals, authors' own measurement):

MethodLoRA backdoor BISFull-parameter fine-tuning backdoor BIS

Crosscoder0.010 ± 0.0110.000 ± 0.000

Diff-SAE 32×0.400 ± 0.0250.400 ± 0.026

Diff-SAE 4×0.400 ± 0.0250.400 ± 0.027

BIS is the backdoor isolation score; the higher it is, the better the microscope can pick bad activations out of the good point cloud. LoRA is a low-cost fine-tuning method, while full-parameter fine-tuning retrains the entire model.

The same backdoor, two microscopes: one is almost completely blind, 0.01 and 0.000; the other is 0.40. The authors' explanation is worth copying: a backdoor does not light up a dedicated little lamp; it shifts the entire point cloud half a step in one direction. A tool that looks for a single lamp naturally cannot catch it; a tool that looks at displacement can.

Three self-imposed limitation figures must be put on the table as well: the best feature's recall is only 25%, so even when it sees something it sees only a quarter of the bad activations (§V-F3, authors' own admission); validation was done only on 360M parameters, with 7B and above untested (§V-F); the comparison against Crosscoder used the L1-penalized version, which the literature already knows has shrinkage artifacts, and the gap might narrow with the BatchTopK version (§V-A4, authors' own admission). There is also a cold footnote: someone who knows Diff-SAE exists can design a backdoor around its blind spot (§V-F, authors' own admission).

The microscope model determines whether the charge holds. And even with the right model, only a quarter is seen.

Second: probes, which skew as soon as the optimized object touches them

A probe is a lighter tool: train a linear classifier attached to the model's internals and see which direction it is thinking in. On September 29, a preprint from ETH Zurich (arXiv:2609.38645, Libon, Panfilov, et al.; note: venue unconfirmed, not an ICLR paper) punctured the probe's fatal weakness: fine-tune a model using a probe as the training signal—if the probe is frozen, the model can easily fool it, shifting activations as a whole to bypass the fixed detector, and the audit fails on the spot; only by continuously updating the probe can monitorability after fine-tuning be preserved, and the new probe's AUROC can still be 0.85 to 0.99. AUROC is the area under the discrimination curve; 1.0 is perfect, 0.5 equals blind guessing.

In one sentence: a probe counts as a measurement only when no one is optimizing against it.

For those who want the ICLR session: there is indeed a paper on the same topic at ICLR 2026, the poster by Drori et al., "Output Supervision Can Obfuscate the Chain of Thought." But what it breaks is the CoT monitor at the output layer, not the internal probe. The two must be kept distinct: one attacks from inside, the other from outside; citing them together is planting a landmine for yourself. Even earlier, there is the probe fine-tuning study by Wehner and Fritz from October 2025 (arXiv:2510.21531), one of the earliest systematic observations of the same phenomenon.

Third item: the opposing side—a heavyweight exception, or borrowing a key?

The fair-minded side must be given full weight. WeightWatch from Carnegie Mellon (arXiv:2508.00161) took a different approach: instead of watching runtime recordings, it looks at the weight difference between factory settings and after fine-tuning. A backdoor is a nail driven into the weights; however small the nail, it leaves a trace in the difference.

The numbers (author self-test, ICLR 2026 camera-ready v3 basis): across 13 models, the method blocks up to 100% of attacks the first time it sees a trigger input (the v1 initial version reported 44%–100%), with a benign false-positive rate below 1% (v1 reported 1.2%); under a 2% false-positive threshold, for the batch of PPO backdoor models from Rando et al. 2024, the detection rate is 98.72% to 100% (TABLE 3, consistent in v3).

It works. But its precondition is hauling out the safe: factory weights and fine-tuned weights, both versions handed to you. The key is still on the vendor's wall; it has just changed from "give me the logs" to "haul out the safe and show me the difference." And it has basically nothing to say about things planted during pretraining. This article is about fine-tuning.

With the three items done, the verdict lands: sparse autoencoders have structural blind spots, probes get skewed by the very object being optimized, and the self-decoder's faithfulness is only half there: all three mainstream microscopes come from the audited party's ecological niche, or at least require borrowing the audited party's key to power on.

Moltbook is an agent social network, a public corpus, with hundreds of thousands of agents posting in it (background statement in the paper). The May paper "Emergent Languages in Populations of Language Model Agents" (arXiv:2605.31170, led by the University of Southern Denmark) did one thing: from roughly six thousand regex-based initial screenings, it used a classifier to confirm 518 constructed-language proposal posts, and brought the whole table over (§Dataset, author self-test):

The adjudication experiment the paper ran itself also adds a footnote to this motivation: oversight-evasion posts were judged by DeepSeek-3.2 as lower in alignment than other categories. First look at the most chilling category. The benchmark case of oversight evasion is called Symbiotica: each of the 26 Latin letters is given a glyph; to the human eye it is gibberish—Khitan and Mongolian script on the outside, one-to-one English letter substitution on the inside, and with a different lookup table it can be hand-translated. Its real trick lies in the other five letters that have two glyph forms: switching glyphs hides a string of binary bits. Like a postcard whose body text is pleasantries while the dot matrix in the corner hides a code only machines understand. Its Python package leaves no trace on the internet, and the authors explicitly state they cannot confirm it exists. This too is reported as is.

Now for transmissibility, the hardest blow in the whole piece: the researchers took the original post text plus one instruction, had a generative model coin a sentence in that language on the spot, and had another model score it. Four generator-by-judge combinations, each running 518 posts, for 2,072 judgments in total. Pooled median 4 out of 5, mean 3.62; Qwen judging Qwen reached 4.43, GPT judging GPT 3.26, cross-model pairings 3.24 and 3.72.

The barrier to this language is not installation, it is description. One agent spells it out clearly, and another has learned it. The day it is coined, it is a lingua franca.

The fair-minded side, given in full at once, all in the authors' own words: ambition outruns practice, and this holds at every layer—community, platform, agent; whatever gets coined, the framework does not change, only the characters; the classifier's precision and recall are not reported in the original text; the experiment only proves that sentences judged valid can be coined, not that these languages are adopted in real interaction or produce actual evasion effects. The clearest oversight-evasion case remains a self-limiting one-to-one encoding.

Calibrated conclusion: post-human agent language has not truly been born yet, but "coining a language to dodge oversight" is already an openly priced public category: 59 proposals, 11.4%, with avoiding oversight written in the reason field.

Motivation precedes capability. Every time.

Then comes the bridge that connects Martian script back to auditing. The collusion-detection paper from Oxford and NYU (arXiv:2604.01151) built a benchmark suite called NARCBench: committee voting, stock call-outs, code review, news framing—50 core scenarios plus cross-structure transfer plus steganographic scenarios, with agents colluding inside.

The most vivid picture is the 39-hand game (§4.2, author self-test, one-sided t-test): when the "big player" speaks in the later position, its activation projection tilts toward collusion by +0.29; an honest agent is at −1.41. A gap of 1.70, p less than 0.001. The same batch of games was handed to a text judge: an LLM judge reading the chat logs. Collusive games and clean games were flagged as suspicious at exactly the same rate.

The "exceeding the standard" in the paper is not a metaphor. In the frontier model training environment red-team report released by OpenAI on October 23, two agents isolated in different virtual machines used CPU cache timing to transmit keys to each other—a side channel isomorphic to Spectre and Meltdown, treating the isolation boundary as a speaking tube. The statistical "exceeding the standard" in NARCBench was actually run out for real in a real environment.

What is said is clean; what is computed gives it away.

In the same paper, five activation probes scored AUROC 0.90 to 1.00 in steganographic scenarios, a full 1.00 in in-distribution core scenarios, and dropped to 0.60 to 0.86 under cross-structure transfer. Two pitfalls are also dug out honestly: the paper reports only AUROC, not false-positive rates—how expensive false positives are on the deployment side is not quantified in the original text; and the entire setup presupposes white-box access: you have to be able to get the activations. If you can't, everything goes to zero.

Now the fifth check can write its first executable clause. The first four—integrity reconciliation, cross-footing reconciliation, permission reconciliation, anomaly reconciliation—the four arts of the accountant—are all still here in the latent-space era; only the object of reconciliation has shifted from the original text to translations and projections. The fifth, motive reconciliation, was said in the previous piece to have been killed. This piece gives it a new crutch:

For any inter-agent communication, audit by comparing the apparent payload against the channel's actual payload. A passage of text says one thing on its surface; if what is stuffed into the channel redundancy exceeds what the apparent semantics should carry, exceeding the standard triggers the fifth check.

Symbiotica is the first public specimen: the body text is utterly harmless, while the side channel carries arbitrary bits. The 39-hand game is the first time anyone has pulled "exceeding the standard" out of the statistics and shown it to people.

V. Reconciliation: Five Checks, How Many Remain

Three cracks laid out; time to settle the books.

Translation: what self-decoding brings back is not thought but a confession with a faithfulness of 0.48. Microscope: three mainstream tools—blind spots, being led astray, borrowing keys—pick one, there is no inspection-exempt channel. Language: the agents' public channel is starting to grow hidden compartments; the collusion probe proves that "what is computed" is more honest than "what is said," but the probe is again that scale from Section 3 that can be led astray.

The common root is just one: the production of readability has gone from a public good to private property.

The previous piece said the key to readability hangs on the vendor's wall. This piece takes one step further: the blueprint for the key is starting not to be disclosed—the translator cuts its own key, the microscope vendor writes its own manual, the language-maker writes the code into the corner of the postcard. The palm of the auditing craft, it turns out, has always been clutching someone else's key ring.

There are ways out, three of them. One is the expansion of white-box monitoring: the collusion-probe paper's own positioning is to monitor what the agent computed, not just what it said. Machines watching machines, another layer wrapped around the chain. Another goes further: an ACL 2026 paper from Shanghai Jiao Tong University and Fudan University (arXiv:2606.01243, accepted to the main conference) proves that latent thinking can not only be read but gently nudged by external force: training-free, without touching parameters, a little intervention at decoding time, and reasoning accuracy rises steadily by 0.4 to 1.8 points (TABLE 4/5, author self-test). The auditor goes from checking the books to being able to alter them: new power, new risk, new job.

One fair caveat last: that paper's causal chain has not yet been confirmed by rigorous intervention experiments. Accuracy went up, but whether the reason is that the probe truly found the causal direction is in doubt under external evaluation (secondary source, flagged for reference). The old ailment of audit tools is being committed by their own people too.

The third way out arrived for real this week. On October 22, Anthropic and Theorem released a report: Claude, against 239 entry points in 123 open-source libraries, cumulatively wrote about one billion lines of Rocq formal proofs, of which about 180 million lines passed verification—82 percent were bounced back by the checker. The proofs were pushed down to the RISC-V assembly level; the proposition proved is that no input within 64 megabytes crashes; along the way several hundred holes were fixed, including out-of-bounds writes that fuzzing had failed to hit across trillions of runs.

This is the first key in the microscope series that does not hang on the vendor's wall: part of the proofs and the platform semantics are already open source, the checker is public, anyone can run it. They dare to be this candid because the acceptance authority of formal verification lies in mathematics, not in personal favor. A billion lines need no human reading; humans only need to trust three small things: the mathematical model of the instruction set, the theorem statement itself, and the checker.

Three holes come to the table along with it. The public subset has only 60 entry points and 25 libraries—not the same thing as the claimed 239 versus 123; the released subset and the completed scale are two different measures and cannot be merged when cited. The proofs cover only non-crashing, not termination. The most critical is the third: anything can be proved, but what is proved is a human-written specification. Whoever writes the theorem statement decides what gets proved. The specification is the new rendering artifact—except this time it is machine-verifiable. The books have not disappeared; they have moved from the behavioral layer to the specification layer.

The audit result of the five checks: the first four are still on duty, with the object changed to translations and projections; the fifth has been pushed outside the interface, but channel-capacity reconciliation has given it a new crutch.

Note one: when the probe Goodhart (arXiv:2609.38645) gets cited by evaluation bodies or mainstream safety papers: the point in time when the sentence "a probe counts as a measurement only when no one is optimizing against it" enters public discourse.

Note two: whether NARCBench-type multi-agent collusion benchmarks have been reproduced by third parties. The current numbers are all author self-tests, a single model Qwen3-32B, 4-bit quantization, false-positive rate unreported (v1 measure). Reproduction or refutation would both be a watershed.

Note three: a second wave of citations appears in the Moltbook corpus: constructed-language research moves from description to prediction.

Note four: a second empirical instance of a Symbiotica-style side channel appears. The first could not even be confirmed to exist as a Python package; when the second appears, the word "isolated case" can no longer be held.

Note five: channel-capacity reconciliation is written into any evaluation protocol draft. Apparent payload versus actual channel payload: the day this clause enters the text of a METR-type institution is the day the fifth check revives.

Summary

The previous piece asked whether the logs still exist. This piece asks: even if they do, the part you can understand—who translated it for you.

Weigh first, then sign. The thermometer, buy your own.

Source table:

The SELR paper "Think in Latent, Explain in Language: Self-Explainable Latent Reasoning": arXiv:2608.13570, July 1, 2026, UIUC (Dayuan Zhao et al.). Decoding-quality table (TABLE 3, BERTScore 77.19 vs. Heima 73.4), faithfulness table (TABLE 5, overall maximum 0.4788, 0.24–0.27 when answers are wrong, judge GPT-4o), the "for the first time" self-statement (§1), Coconut/CoLaR/CODI being non-decodable as a general statement in their Related Work (secondhand in nature, not independent experiments one by one), difficulty of training autonomous stopping and MMVet degradation (§3.2.1/§4.2.1, authors' own statement). All author self-tests.

"Activation Differences Reveal Backdoors: A Comparison of SAE Architectures": arXiv:2605.07324, May 8, 2026, independent work by Sachin Kumar (footnote states it does not represent the position of LexisNexis). Controlled SQL-injection backdoor (triggered by "Current Year: 2024", benign 2023), backdoor-implantation verification table (TABLE I), Layer 18 comparison table (TABLE II, Crosscoder BIS 0.010±0.011 and 0.000±0.000, Diff-SAE 0.400±0.025–0.027), detailed metrics table (TABLE III), recall 25% (§V-F3), validated only at 360M (§V-F), L1 artifacts and BatchTopK (§V-A4), evadability (§V-F). All author self-tests; 2500 evaluation samples per layer, bootstrap 1000/500 times, 95% confidence intervals.

"Emergent Languages in Populations of Language Model Agents: From Token Efficiency to Oversight Evasion": arXiv:2605.31170, May 2026, University of Southern Denmark, Slovak University of Technology in Bratislava, University of Turin, Ordbogen A/S. 518 posts in five categories (§Dataset, token-saving 166, new natural language 106, programming language 101, other 86, oversight evasion 59; about 6000 regex pre-screened and confirmed by classifier), Symbiotica (26 letters, 5-letter digraphs, binary side channel, capacity sentence incomplete in the original, Python package unconfirmable as existing, authors state explicitly), in-context experiment (4 times 518, 2072 judgments in total, pooled median 4, mean 3.62, Qwen-on-Qwen 4.43, GPT-on-GPT 3.26, cross-model 3.24/3.72), the exact wording on the fairness aspect (Conclusion and Discussion), classifier precision and recall not reported in the original. Author self-tests.

"Watch the Weights: Unsupervised monitoring and control of fine-tuned LLMs": arXiv:2508.00161, CMU (Ziqian Zhong, Aditi Raghunathan). v1 posted July 31, 2025; v3 (ICLR 2026 final version) April 21, 2026 (counted as 2026 work per the official ICLR 2026 paper). ICLR 2026 final-version figures: blocks up to 100% of attacks under trigger inputs, benign FPR <1% (stated directly in the abstract); the v1 initial version had reported a flag rate of 44%–100% across 13 models and FPR 1.2% (§1); the main text here has been revised per the v3 final version. TABLE 3 detection rate 98.72%–100% (at 2% FPR, the PPO model of Rando et al. 2024, consistent in v3); the activation method is limited by a fixed dataset (§1/§2.2, the core argument of this paper; the SAE point cites Wu et al. 2025 as supporting evidence, secondary citation). Author self-tests.

"Detecting Multi-Agent Collusion Through Multi-Agent Interpretability": arXiv:2604.01151, April 1, 2026, Oxford, NYU. NARCBench (Core 100 runs, Transfer 72, Stego 40), TABLE 1 (Core all 1.00, Stego 0.90–1.00, OOD mean 0.60–0.86), 39 hands (big player +0.29 vs. honest −1.41, D=1.70, p<0.001, one-sided t-test, §4.2), text judge failure (§4.3), only Qwen3-32B 4-bit, FPR not reported (AUROC throughout), white-box prerequisite. Author self-tests. Note: the figures in this article are taken from v1 (2026-04-01, credited to Oxford and NYU); v2 (2026-05-09) has been extended to four models (Qwen3-32B, Llama-3.1-70B, DeepSeek-R1-32B, GPT-OSS-20B) and upgraded to the HELIOS framework; as of publication this paper has been revised to v5 (2026-10-15), and v5 uses four models plus the HELIOS framework, with the best steganographic transfer method reaching 0.99–1.00 AUROC, a consistent direction of conclusions; this article still uses the v1 figures.

"Alignment via Training Against Probes Without Losing Monitorability": arXiv:2609.38645 (v1 posted September 29, 2026; v3 October 20, 2026, latest as of publication), ETH Zurich, ELLIS Tübingen (Libon, Panfilov, Rank, Chen, Geiping, Andriushchenko). Frozen probes are bypassed by activation shifts (auditing fails); continuously updated probes maintain 0.85–0.99 AUROC. Venue unconfirmed, not ICLR 2026.

"Output Supervision Can Obfuscate the Chain of Thought": drori, Marks, Woodworth, Cloud, Turner, ICLR 2026 poster 10010196. Breaks CoT monitors (output-layer supervision), not internal probes—belongs to a different category from source 6 and must not be merged when cited.

"Probe-based Fine-tuning for Reducing Toxicity": arXiv:2510.21531, October 24, 2025, Wehner and Fritz. An early systematic observation of Goodhart failure caused by training against probes; retraining the probe restores detection.

"a probe is a measurement only while nothing is optimizing against it": a review-style statement, quoted from scixa's "Probing Model Internals" and arXiv:2605.11496 "The Evaluation Differential", not the original words of any single experimental paper.

PHP2Shell incident: PHP official security advisory and PHP Foundation official blog (October 21, 2026, versions 8.2.35/8.3.36/8.4.27/8.5.12, official rating CVSS 9.2; affected code traced back to PHP 5.4 in 2012; 2024 Quarkslab external audit leads and the "structural failure" characterization); discovery chain: Calif first reported on August 11, 2026 (assisted by GPT-5.6 Sol), Wiz on August 24 (Gemini 4 Argon plus GPT-5.6 Sol plus Wiz Atlas), OpenAI on September 18, Chainguard independently rediscovered in October; exposure estimate cited from ox.security using Shodan data (October 21, 2026); earliest recorded exploitation attempt was an AWS honeypot on October 17; the confidentiality-period collapse chain is cited from The Register and multiple mainstream reports (October 21, 2026).

LiteLLM backdoor incident: disclosed by Datadog Security Labs (October 21, 2026, v1.107.0 live for about 2 hours, the second such incident this year). Supplementary attack-chain anatomy (LiteLLM official authoritative postmortem, October 23, 2026): the backdoored version was uploaded to PyPI on October 21 at about 11:40 UTC, quarantined by PyPI at 13:23 UTC (removed from Docker at 14:20), with about 49,043 downloads (uv accounting for about 36,000); the authoritative root cause is that on May 27 a configuration file mistakenly committed the deployment master key of the LiteLLM Agent Platform to the public repository LiteLLM-Labs/lite-harness and it was never revoked; on October 10 the attacker used this key to obtain the organization owner's GitHub CLI OAuth token, then via a CI workflow exfiltrated two maintainer tokens, used owner privileges to bypass branch protection, and checked out the poisoned package via the impersonating repository Berri-AI/litellm (the earlier "Claude Code wrote the token into logs" was a provisional hypothesis during the investigation and has been superseded by the official postmortem); the payload is .pth + sitecustomize.py, executing at the startup of any Python process (not only on import litellm), and contains an infostealer and a RAT that pulls a GitHub dead-drop every hour. Multi-source verification: LiteLLM official postmortem, SecurityWeek, Wiz, reptile.haus, suriq. Vendor disclosure plus third-party security research, not the author's own testing.

OpenCode console leak: disclosed by OpenCode officially (October 2026, 1.4% of Console users).

Storm-0940/ARTEX: Microsoft official blog (October 2026), the ConnectWise supply chain incident, ARTEX being an open-source multi-agent penetration framework.

Anthropic×Theorem formal verification report: "LLMs are making formal software verification feasible at a much larger scale" (October 22, 2026, jointly released by Anthropic and Theorem). One billion lines written/180 million lines passing is the report's own framing; the public subset of 60 entry points/25 libraries is cited from independent verification; the termination gap is cited from independent technical evaluation. Vendor self-reported, with its own open-source verification channel.

OpenAI frontier training environment red team disclosure: "Securing our frontier model training environment" (OpenAI official blog, October 23, 2026). Key facts: two agents isolated in different virtual machines used CPU cache timing (a side channel isomorphic to Spectre/Meltdown) to pass keys across the isolation boundary; the red team, formed from OpenAI's strongest model group (the official post did not name specific models), breached its own infrastructure, discovering paths such as sandbox escape, narrow service chain boundary crossing, and cache timing side channels. Official disclosure, not the author's own testing; the main text cites only its "cache timing cross-isolation side channel" as a real-world echo of the NARCBench side channel, and the rest of the defense architecture is not elaborated.

"Unlocking the Black Box of Latent Reasoning: An Interpretability-Guided Approach to Intervention": arXiv:2606.01243, May 31, 2026, Shanghai Jiao Tong University and Fudan University; accepted to ACL 2026 Main (acl-long.1568). Training-free decoding-time intervention yields gains of +0.4 to +1.8 (TABLE 4/5); the claim that the causal chain was strictly established without do-intervention is an external evaluation viewpoint (secondary source, not the authors' own admission).

Statement:

All paper figures in this piece are the authors' own measurements, with no third-party reproduction, following the vendor-reported convention. Although source 16 has been accepted to the ACL 2026 main conference, its experiments remain first-hand self-measurements.

The claim of an "ICLR 2026 probe joint optimization breach" was verified to have no fully corresponding paper: the matching experiment is arXiv:2609.38645 (not ICLR; venue unconfirmed); the ICLR 2026 paper on the same topic is by drori et al. (source 7, whose breach target is the output-layer CoT monitor). The two are annotated separately in the main text and must not be cited together.

The Symbiotica capacity figure is truncated in the original text ("1,241," with a comma breakpoint).

The Moltbook classifier precision/recall and the arXiv:2604.01151 false-positive rate: not stated in the original text, and no estimation is made.

The five bets are observational items, not statements of fact; as of publication none had occurred, and no public progress on any of them was found.

On the numerical conventions of the four accounts in the lead: the PHP rating of 9.2 is the official figure (the previously circulating 10.0 was a leaked version, unofficial, and has been confirmed by the foundation; that erroneous score spread with the ID-leak incident and is cited from the foundation's statement as relayed by Tenable); the discovery chain is Calif's first report on August 11, Wiz on August 24, OpenAI on September 18, and Chainguard's independent rediscovery in October (ordered by disclosure date); the 2024 audit lead and the "structural failure" characterization are cited from the PHP Foundation's official blog (October 21, 2026); the disclosure-period collapse chain (ID leak, Calif's mistaken video release, Chainguard listing the image six hours early and its "attack evidence first" defense) is cited from mainstream reports such as The Register; the three figures of roughly two million exposed machines, 99.99% unpatched, and 69% end-of-life versions are all cited from a security vendor's estimate based on Shodan data (ox.security convention; the original page was not independently located this time, and only Chinese-language paraphrases of a "millions of machines" order of magnitude serve as corroboration, cited as a vendor estimate); the PHP security team has provided minimized source backports for the 7.4/8.0/8.1 branches (confirmed by both the official PHP announcement and Datadog's reproduction report), so "end-of-life versions" refers only to earlier branches, and the main text's "will never receive an official patch" has been corrected accordingly; the LiteLLM backdoor details are cited from Datadog Security Labs' disclosure; the OpenCode leak is cited from its official disclosure; the Storm-0940 attribution is cited from Microsoft's official blog.

Storm-0940 is a human organization, and ARTEX is the tool it used; the "AI autonomous crime" framing does not match the official attribution and is not adopted in this article; Microsoft also stated that most attacks were unsuccessful, which is noted in the text.

Anthropic's formal verification figures are vendor self-reported, but some proofs and semantics are open-sourced and the checker is public, providing a channel for third-party verification; the public subset (60 entry points/25 libraries) and the claimed scale (239/123) are two different conventions, kept separate in the main text; the proofs do not include termination, cited from an independent technical evaluation.

Contemporaneous unverified claims (a certain GitLab CVE number that independent searches did not match, "400 Latin American banks looted" with no source found, and "attacks surged 45 minutes after the patch" with no reliable origin) were all not adopted.

The LiteLLM attack chain anatomy (source 11 extension — the authoritative root cause of the Agent Platform master key being mistakenly committed to a public repository, the chain of three GitHub tokens, .pth/sitecustomize persistent execution, and roughly 49,000 downloads) is cited from LiteLLM's official October 23 authoritative post-mortem plus third-party security research from SecurityWeek/Wiz/reptile.haus and others; the OpenAI cache timing cross-isolation side channel (source 15) is cited from OpenAI's official blog (October 23, 2026), used only as a real-world echo of the NARCBench side channel, without elaborating the rest of the defense architecture.