Understand this story
English text is being prepared.
AI-generated from public reports
Latest development · Report
MIT CSAIL proposes weight-space programming: GPT-6 Astra directly rewrites neural network weightsDevelopments
2 confirmed developments
- Source:English text is being prepared.
- Source:Analytics India Magazine
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Nov 3
36Kr · AI
MIT CSAIL proposes weight-space programming: GPT-6 Astra directly rewrites neural network weights
The MIT CSAIL team proposes a weight-space programming method that lets GPT-6 Astra directly read and modify neural network weights without training or computing gradients. On Qwen3.5-0.8B, Astra modifies only about 0.016% of the weights, raising MATH-500 from 7.9% to 48.1%, GSM8K from 13.8% to 67.1%, and IFEval from 21.0% to 62.3%, mainly because it fixed the model's failure of getting stuck in reasoning loops. The research was led by doctoral students Rikiya Takehi and Jayden Teoh, with participation from Omar Khattab and others, and the results were published on the research blog "Sparks of Weight-Space Programming"; additional experiments on Gemma 3 270M yielded improvements of no more than about 1 percentage point, and the team says more independent replications are still needed.
Phoenix Tech
MIT CSAIL uses GPT-6 Astra to directly rewrite the weights of Qwen3.5-0.8B
The MIT CSAIL team had OpenAI's GPT-6 Astra directly read and modify the weights of Qwen3.5-0.8B. By changing only about 0.016% of the parameters, they raised the model's score on MATH-500 from 7.9% to 48.1% and improved IFEval by about 3x, all without gradient descent or backpropagation. The team calls this method Weight-Space Programming (WSP) and points out that it removes obstacles in the process of the model exercising capabilities it already has, rather than creating new capabilities; applying the same method to Gemma 3 270M only pushed MATH-500 from 6.8% to 7.7%. The experiment has so far been released only as a blog post and has not yet produced a peer-reviewed paper.
English text is being prepared.Related discussion
English text is being prepared. joined this discussion
This post contributes to heat; it is not a formal report or confirmed fact.
Swyx
swyx on weight-space programming: we've reached the second section of "Exhalation"
swyx cites a research write-up from Latent.Space, saying that with weight-space programming, we have now reached the second section of Ted Chiang's short story "Exhalation." The cited content mentions that in an MIT study, GPT-6 Astra traced Qwen3.5-0.8B's loop of repeatedly consuming 32K tokens on math problems to two layers, and edited 0.016% of Qwen's weights, raising its MATH-500 score from 7.9% to 48.1% with sampling held fixed.
Analytics India MagazineSelected
MIT team proposes Weight-Space Programming, letting LLMs directly read and write neural network weights
Researchers at MIT CSAIL propose Weight-Space Programming, which lets a general-purpose LLM directly read, interpret, and modify neural network weights without relying on traditional training methods such as gradient descent. They used OpenAI's GPT-6 Astra to complete eight sets of experiments; in one of them, by modifying only 0.016% of the weights of Qwen3.5-0.8B, they raised the MATH-500 score from 7.9% to 48.1%, GSM8K from 13.8% to 67.1%, and IFEval from 21% to 62.3%. Astra can also identify a network's training objective from its weights alone without running the network, and directly generate weights that outperform PPO-trained networks in games such as Breakout. The experiment logs and prompts have been released on GitHub, and the modified Qwen3.5-0.8B model is available on Hugging Face.
English text is being prepared.Related discussion
English text is being prepared. joined this discussion
This post contributes to heat; it is not a formal report or confirmed fact.
English text is being prepared.
MIT researchers propose weight-space programming: GPT-6 Astra modifies about 0.016% of Qwen3.5-0.8B's weights…
MIT CSAIL researchers propose "weight-space programming," in which GPT-6 Astra directly reads and modifies the neural network weights of Qwen3.5-0.8B. By adjusting only about 0.016% (131,072) of the parameters, it raised the model's score on MATH-500 from 7.9% to 48.1% and on GSM8K from 13.8% to 67.1%, all without using gradient descent or a traditional training loop. Astra first located the repeated directions written by the model in the MLPs of layers 15 and 19, weakening the signals that caused reasoning loops, which reduced the proportion of math test responses left incomplete after hitting the 32,768-token limit from 93% to 33.1%, and raised the IFEval score from 21.0% to 62.3%. In the controlled experiments, because Gemma 3 270M originally exhibited few reasoning loops, its MATH-500 score rose only from 6.8% to 7.7%; based on this, the research team believes the method can remove obstacles to existing capabilities but may not necessarily create capabilities the model did not originally possess.
IT Home · AIRelated discussion
IT Home · AI joined this discussion
This post contributes to heat; it is not a formal report or confirmed fact.
Hacker News Trending
MIT CSAIL Experiment: GPT-6 Astra Directly Interprets and Modifies Neural Network Weights
The MIT CSAIL team reports early experiments in which GPT-6 Astra can interpret and modify neural network weights without writing training loops. In one test on Qwen, the authors claim that small weight changes significantly improved math and instruction-following scores, but the results remain preliminary. The findings were published by Rikiya Takehi and colleagues and sparked 623 comments on Hacker News.
English text is being prepared.Related discussion
English text is being prepared. joined this discussion
This post contributes to heat; it is not a formal report or confirmed fact.
Shao MengSelected
MIT CSAIL proposes Weight-Space Programming: GPT-6 Astra modifies 0.016% of Qwen3.5-0.8B's weights…
MIT CSAIL researchers propose "Weight-Space Programming," letting GPT-6 Astra directly read and write neural network weights toward natural-language goals, without gradients or writing training loops. After modifying about 0.016% of Qwen3.5-0.8B's weights (131,072), MATH-500 rose from 7.9% to 48.1%, GSM8K from 13.8% to 67.1%, and IFEval from 21.0% to 62.3%, while the share of responses hitting the token limit fell from 93% to 33%. The authors believe weight editing is more like removing obstacles blocking existing capabilities rather than injecting capabilities the model never had, and acknowledge that the context and harness are still crude; they are building benchmarks and a harness to turn this into reproducible engineering.
indigoSelected
MIT CSAIL proposes Weight-Space Programming: GPT-6 Astra directly rewrites the weights of Qwen3.5-0.8B
Rikiya Takehi, a PhD student at MIT CSAIL, and his team propose "Weight-Space Programming," which lets GPT-6 Astra directly read and write neural network weights without training or computing gradients. In the experiment, only about 0.016% of the weights of Qwen3.5-0.8B were modified, and MATH-500 rose from 7.9 to 48.1 while GSM8K rose from 13.8 to 67.1; according to the authors, this fixes the 0.8B model's reasoning loops, improving math ability by up to 6x and instruction following by 3x.
BaoyuRelated discussion
Baoyu joined this discussion
This post contributes to heat; it is not a formal report or confirmed fact.
Garry Tan
Garry Tan reposted a new use case for directly editing small-model weights
Garry Tan reposted Rikiya Takehi's post, saying that a new use case has emerged for small models: directly manipulating weights. The quoted post says that they had GPT-6 Astra edit the weights of Qwen3.5-0.8B to improve its math ability, changing only 0.016% of the weights, fixing the 0.8B's reasoning loops, improving math ability by up to 6x and instruction following by 3x, without relying on gradients or training, and they call this approach Weight-Space Programming.
@mark_kRelated discussion
@mark_k joined this discussion
This post contributes to heat; it is not a formal report or confirmed fact.
Yuchen JinSelected
Researchers Use GPT-6 Astra to Directly Rewrite Qwen3.5-0.8B Weights to Improve Math Ability
Yuchen Jin paraphrases Rikiya Takehi's experiment: using GPT-6 Astra to directly edit the weights of Qwen3.5-0.8B to improve math ability, changing only 0.016% of the weights fixed the model's reasoning loops, improving math ability by up to 6x and instruction following by 3x, all without using gradients or doing any training. The author calls this method Weight-Space Programming, believes that frontier LLMs can now natively read and write neural network weights, and says it goes a step further than knowledge distillation.
The Rundown AI (X)Related discussion
The Rundown AI (X) joined this discussion
This post contributes to heat; it is not a formal report or confirmed fact.
Rohan PaulSelected
MIT study shows GPT-6 Astra can directly modify weights to improve Qwen3.5-0.8B's math ability
MIT researchers had GPT-6 Astra directly edit the weights of Qwen3.5-0.8B in a Python notebook, raising accuracy on MATH-500 from 7.9% to 48.1%, all without training. After Astra was given the weights and a budget of 50 practice math test attempts, it found that the model repeatedly looped until hitting the length limit, localized this habit to a single internal signal written at layer 2, and subtracted that signal from the weights. This edit changed only 0.016% of the weights, and the authors call this method Weight-Space Programming.
Ma Dongxi NLP
Dongxi NLP Questions GPT-6 Astra's Weight Editing Method: Like an LLM's He Jiankui
Dongxi NLP questions the Weight-Space Programming proposed by Rikiya Takehi, arguing that the method has traces of activation steering: first find the activation direction corresponding to a certain tendency of the model, such as the loop problem of being unable to stop when solving problems, and then project out the corresponding components in the 128 columns of weights that write the most into that direction. He points out that evaluations such as MMLU-Pro, GPQA, and IFEval did indeed all improve, but these benchmarks were originally all affected by loops; once the loop is fixed, scores naturally improve across the board, and the real question is the side effects. The original post claims that GPT-6 Astra only changes 0.016% of Qwen3.5-0.8B's weights, improving math by up to 6x and instruction following by 3x, with no gradients or training required.
Nov 2
Hacker News TrendingSelected
LLMs Can Directly Read and Write Neural Network Weights: Astra Turns Qwen3.5-0.8B's…
A blog post by Rikiya Takehi, Jayden Teoh, and others proposes Weight-Space Programming, finding that frontier LLMs such as GPT-6 Astra can directly read and write neural network weights without gradient descent or training loops. Using only the raw weights, Astra can identify the game a network was trained on, the shapes generated by a diffusion model, and the training corpus of a language model, and can even write from scratch weights that defeat a PPO network trained for 100 million steps in three MinAtar games. On Qwen3.5-0.8B, Astra located the weight direction causing reasoning loops and, by modifying only 0.016% of the weights, raised MATH-500 from 7.9% to 48.1% and IFEval from 21.0% to 62.3%; however, for Gemma 3 270M, which rarely loops, it failed to improve math scores.
Heat trend for this story
- Current
- 130
- Period peak
- 132 Nov 3 14:00
- 24-hour change
- —
Move the pointer or select the chart to inspect hourly heat. Use the left and right arrow keys with a keyboard.