MIT study shows GPT-6 Astra can directly modify weights to improve Qwen3.5-0.8B's math ability
MIT researchers had GPT-6 Astra directly edit the weights of Qwen3.5-0.8B in a Python notebook, raising accuracy on MATH-500 from 7.9% to 48.1%, all without training. After Astra was given the weights and a budget of 50 practice math test attempts, it found that the model repeatedly looped until hitting the length limit, localized this habit to a single internal signal written at layer 2, and subtracted that signal from the weights. This edit changed only 0.016% of the weights, and the authors call this method Weight-Space Programming.
Why read this
SelectedAI score 76
The original text provides the specific proportion of weight edits and the changes in MATH-500 scores, so readers can understand the approach of directly modifying weights without training.
Topics
View original postWe asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math. It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training. We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵 Blog: https://t.co/bOWCcFhCVq
View original
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math. It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training. We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵 Blog: https://t.co/bOWCcFhCVq
Body
MASSIVE. MIT researchers showed that GPT-6 Astra can improve another AI model by changing its weights directly, without training.
Here, GPT-6 Astra raised Qwen3.5-0.8B from 7.9% to 48.1% on MATH-500 by editing 0.016% of its weights, without training.
Astra got Qwen’s weights in a Python notebook, plus a budget of 50 test runs on practice math problems. It saw the model kept repeating itself until it hit its length limit, traced that habit to a single internal signal written by 2 of its layers, and subtracted that signal out of the weights.
With the loop gone, the model could finish answers it was already capable of solving.
That single targeted edit changed 0.016% of the weights and lifted math accuracy from 7.9% to 48.1%.