Garry Tan reposted a new use case for directly editing small-model weights
Garry Tan reposted Rikiya Takehi's post, saying that a new use case has emerged for small models: directly manipulating weights. The quoted post says that they had GPT-6 Astra edit the weights of Qwen3.5-0.8B to improve its math ability, changing only 0.016% of the weights, fixing the 0.8B's reasoning loops, improving math ability by up to 6x and instruction following by 3x, without relying on gradients or training, and they call this approach Weight-Space Programming.
Why read this
AI score 60
This repost points to a new approach that uses frontier models to directly rewrite the weights of small models, and can serve as an entry point for understanding the concept of weight-space programming.
Topics
View original postWe asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math. It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training. We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵 Blog: https://t.co/bOWCcFhCVq
View original
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math. It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training. We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵 Blog: https://t.co/bOWCcFhCVq
Body
Interesting new use of smaller models: direct manipulation of weights