MIT researchers show LLMs can read and edit neural network weights without training
MIT CSAIL researchers introduced Weight-Space Programming: large language models can directly read and deliberately modify neural network weights without gradient descent or fine-tuning.
In the main experiment, the authors gave Astra the weights of the tiny Qwen3.5-0.8B and asked it to improve its mathematical abilities. Astra found neurons in the weights that caused the model to loop in its reasoning and fixed them. It rewrote about 0.016% of the weights, after which Qwen3.5-0.8B’s score on MATH-500 rose from 7.9% to 48.1%, and on GSM8K from 13.8% to 67.1%.
The researchers also showed that Astra can interpret models from their weights. For example, they fine-tuned four identical models to play different games: Breakout, Pong, Pac-Man and Freeway. Astra received only the weights (about 17,000 numbers) and an input description, without game names, and correctly identified what each model had been trained to do in all four cases.
In another experiment, three diffusion models drew a circle, a square and a triangle. Astra identified each shape from the output layer by reading the weights, without running the networks.
Astra can also write neural networks from scratch. Without training, it produced weights for networks playing three MinAtar games. In all three cases, it outperformed models trained with reinforcement learning (PPO) for 100 million steps.
The authors also asked it to write a network for Go. It produced weights in a 336-line script and won more than half of its games against two small PPO-trained networks. When given the ability to test its network several times in a sandbox, it beat a same-sized PPO network trained for 20 million games.
The results are still early, but the direction is very promising. Weight-Space Programming could be used for alignment, model auditing, cheap weight initialization and recursive self-improvement. The authors are currently building a benchmark and harness to evaluate models’ ability to program weights. Full logs of all experiments are available on GitHub, and the detailed write-up is in the project blog post.
Why it matters
AnalysisThis matters because Weight-Space Programming opens a new way to analyze, correct and create models without conventional training, potentially accelerating AI auditing, safety and optimization.
Discuss in community
Share your questions and insights with developers
Related in Ecosystem
Practical guides, core terminology, and agent skills
GPT-6 Astra reportedly edits another AI model's weights without retraining
GPT-6 Astra is claimed to modify only 0.016% of Qwen3.5-0.8B's weights without gradients or retraining, improving reasoning, math results, and instruction following.
MIT researchers show GPT-6 Astra can improve other AI models by editing their weights
In an experiment, Astra raised Qwen3.5-0.8B's accuracy on MATH-500 from 7.9% to 48.1% by changing only 0.016% of the model's weights.
Synthetic Data Generation & Pretraining
A technology for generating, automatically filtering, and formally verifying training datasets using artificial intelligence to overcome the shortage of quality human data.
Part I. AI Agent Architecture: The Foundations
The eleven foundational building blocks of modern autonomous agents: model taxonomy, the agentic runtime loop, advanced prompting, distillation, skills architecture, memory systems, function calling, and FinOps.