Google researcher Alex Krentsel reproduces the Weight-Space Programming experiment…
Google researcher Alex Krentsel reproduced Exp. 5 of the Weight-Space Programming experiment from the MIT blog post: in a single response, each model, without training or using tools, directly writes out all of the roughly 20,000 weights of a playable Breakout network, with a network trained via PPO reinforcement learning as the 1× baseline. A cross-generation sweep shows that GPT-5 through 5.5 and Opus 4.5 through 4.8 are basically near or below the baseline, GPT-5.6 Sol and Opus 5 slightly exceed 1×, and only GPT-6.1 Sol, GPT-6 Astra and Opus 5.5 jump to about 18×–22×; he believes this is a step-change new capability that has just emerged in the latest batch of models. He emphasizes that the network whose weights are written has only about 20,486 parameters, several orders of magnitude away from trillion-parameter frontier models, and this cannot be extrapolated to mean that large models can already modify themselves; at present it is still an outer model modifying an inner small network, and he says he is reproducing it on open-weight models.
Why read this
AI score 57
The original text lays out the scale premise of the reproduction experiment and the cross-generation comparison results, so readers can judge what this capability currently demonstrates and what it does not.
Topics
View original posti've reproduced the results from the MIT blog post today on weight-space programming. but one additional result i have to add: this is a new, step-change emerging capability that's appeared in just the latest set of models. https://t.co/EhMsQaaOkL
View original
i've reproduced the results from the MIT blog post today on weight-space programming. but one additional result i have to add: this is a new, step-change emerging capability that's appeared in just the latest set of models. https://t.co/EhMsQaaOkL
Body · AI translation
Google researcher @AlexKrentsel reproduced the "Weight-Space Programming" experiment from the MIT blog post, and added one finding of his own: scanning across multiple model generations, this capability almost only suddenly becomes stronger in the latest batch of frontier models, rather than improving smoothly all along.
About Weight-Space Programming https://t.co/VOdvYVPXw0
Let's see what Alex reproduced What he reproduced is Exp. 5 from the original paper: each model, in a single reply, without training and without tools, directly writes out all of the roughly 20,000 weights of a playable Breakout network; the prompt and scoring are consistent with the original paper, with 1× benchmarked against the network trained with PPO reinforcement learning in the paper.
GPT-5 through 5.5 and Opus 4.5 through 4.8 are basically near the baseline or weaker; GPT-5.6 Sol and Opus 5 slightly exceed 1×; only with GPT-6.1 Sol / GPT-6 Astra and Opus 5.5 does it jump to roughly 18×–22×. Alex therefore says: this is a step-change new capability that has just emerged in the latest batch of models.
Prerequisites that must be clearly understood The Breakout network whose weights are edited and written out has only about 20,486 parameters; frontier large models are on the order of about a trillion. He emphasizes that this was shared very early on, that the scale differs by several orders of magnitude, and that it cannot be directly extrapolated into "large models can already modify themselves."
How he understands this: it is like the kind of manual observation in 2010s interpretability research where "a certain neuron lights up whenever it sees a dog," but the capability has already exceeded the scale that humans can manually modify; people used to manually work on dozens to hundreds of nodes, and then with tools reach about the thousand level; now it is reasoning capability that lets models work on larger but still relatively small networks.
Next steps and how he responds to doubts He thinks a very natural next step is runtime self-improvement within the weights, but given the current size gap between outer models and inner models, that is not yet realistic; if the trend continues, it might only be worth discussing after a few more generations. Someone asked whether it is already self-modifying; he clarified: at present it is an outer model modifying an inner small network, and there is no self-editing yet. On "whether closed-source models have been specifically post-trained on this kind of problem," he said he cannot see behind the scenes; on the criticism of "why not use Chinese open-source models," he said he is reproducing on open-weight models (including Chinese models), starting with the currently strongest Astra and Fable/Opus 5.5, and said the exciting part is actually that local open weights can eventually modify themselves, rather than closed-source models touching someone else's weights.
Some comments also argued that the task is too simple, that the prompt gives too much information, and that it should not be called emergence; he responded: the problem is indeed simple and the network is indeed small, but on the line of weight editing and interpretability, people have struggled for a long time, and what he values more is the trend across generations.