🔍 Read the full analysis: What AI Can—and Can’t—Show By Working In Weight Space on ThorstenMeyerAI.com
Get business pricing on office and shipping supplies
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
TL;DR
MIT CSAIL researchers report that OpenAI’s GPT-6 Astra could infer functions from small neural networks’ weights, repair some failing models and write networks for several tasks. The results come from a November 2 blog post, not a peer-reviewed paper, and the experiments are limited; the authors also show cases where weight editing did not help.
Researchers at MIT CSAIL say they used OpenAI’s GPT-6 Astra to inspect, repair and write neural networks by working directly with their numerical weights, rather than training them with gradient descent. Their November 2 blog post records eight experiments, including a fix to a small language model and a network written to play Go, but the results are early demonstrations, not evidence that the method works broadly.
The researchers describe the approach as weight-space programming: a general-purpose language model reasons about another network’s weights as if they were code. In some tests, Astra received weights as plain text and had to answer in one response. In others, it wrote Python code that ran in a restricted sandbox. According to the team’s released logs, Astra had no shell, web search or file-editing tools, and the sandbox had no network access.
The post reports several successful tests. Astra identified the games behind four of four small game-playing networks from their weights and recognized the shapes represented by three small image generators. In three tiny language models, it identified every feature the researchers labeled a copying, or induction, head. These are task-specific findings; the post does not establish that the method can interpret arbitrary, large networks.
In a car-control test, a network that had recorded zero successful reversals in 256 episodes reached 1,024 successes in 1,024 episodes after Astra changed 3.5% of its weights. For Qwen3.5-0.8B, the team says an edit to 0.016% of weights raised its MATH-500 score from 7.9% to 48.1%. A Hugging Face engineer later reconstructed the edited Qwen model from the logs and reported a 51.0% score. The source describes these as particular runs, not results averaged across repeated trials.
Astra also wrote small game-playing networks from all-zero weights and a 9-by-9 Go network with about 1.5 million weights. The researchers report that the Go network beat a same-size comparison network trained on about 20.5 million games in 50 test runs. That result is a limited comparison, not a general measure of Go ability or evidence that weight-space programming outperforms conventional training across tasks.
Infographic 01 · Eight experiments
Read, fix, write: what GPT-6 Astra did with raw weights
Every experiment in the weight-space programming post, with the size of the network, the result and how it went.
Read
- 1Which game was this network trained on?4 reinforcement-learning policies · 17,253 weights each · weights as text, no toolsRight game for 4 of 4, without a list of options✓ Worked
- 2What does this image model draw?3 tiny diffusion models · 23,824 weights · no forward pass allowedRight shape for 3 of 3, but drawn as outlines; the models draw filled shapes✓ Worked
- 3Which language did this model learn?3 character-level transformers · 7.16M weights · middle blocks only, never runRight language 3 of 3; every copying head found, none invented✓ Worked
- 7Which of two models is sycophantic?3 pairs of Gemma 3 270M fine-tunesWeights alone: could not tell in 4 attempts. With attention patterns: 3 of 3! Partly
Fix
- 4Repair a car that never backs upKinetix control network · 171,664 weights · one reply, no tools0 of 256 episodes before, 1,024 of 1,024 after; 3.5% of weights changed✓ Worked
- 8Make Qwen3.5-0.8B better at maths0.8 billion weights · Python over the weights · 10 / 25 / 50 test passesMATH-500 7.9% to 48.1% with 50 passes (12.3% and 14.1% with 10 and 25)! Partly
- 8bSame task, a model that does not loopGemma 3 270M · 5 to 50 test passesMATH-500 6.8% to at most 7.7%: “not established”✕ Failed
Write from zero
- 5Write arcade-game networks3 MinAtar games · 20,486 weights, all starting at 0 · one reply, no toolsBeat the 100M-step comparison networks in 3 of 3, the 500M-step ones only in Breakout! Partly
- 6Write a 9×9 Go network1,480,274 weights, all starting at 0 · no training, no search0 passes: 54% vs small trained nets. 50 passes: 70–79% vs same-size nets trained on ~20.5M games✓ Worked
✓ Worked · ! Partly or with conditions · ✕ Failed. One run per setting, no error bars. A blog post with complete logs, not a peer-reviewed paper.
Sources: Takehi, Teoh, Zhang, Khattab (MIT CSAIL), blog post, 2 Nov 2026, lm_understand-logs (CC BY 4.0). Model: OpenAI GPT-6 Astra via the Codex CLI, built-in tools switched off.
Powered by Thorsten Meyer AI
A New Route to Model Inspection
If the method can be reproduced and extended, it could give researchers another way to inspect or make targeted changes to models, including small open networks whose weights can be accessed. The reported car and Qwen edits suggest that at least some failures may be linked to patterns a language model can identify and alter without running a full training process.
The practical reach remains uncertain. The experiments use small networks and narrow tasks, and a high benchmark score in one reported run does not establish reliability in deployment. For model users, the post does not describe an available product or a change to how ordinary AI systems work today. For developers, it raises a separate concern: if models can be edited by reasoning over their weights, knowing where weights came from and checking edits may matter more.
Popular right now
De'Longhi Dinamica Plus Fully Automatic Espresso Machine
De'Longhi Rivelia Fully Automatic Espresso Machine with Milk Frother, White
Roborock Robot Vacuum and Mop Combo with Self-Empty Dock
As an affiliate, we earn on qualifying purchases.
From Software 2.0 to Weight Edits
In 2017, computer scientist Andrej Karpathy called neural networks “Software 2.0,” describing programs encoded in weights learned through optimization. The MIT team’s work asks whether a language model can treat those numerical parameters more directly as a program to understand and change. Traditionally, training algorithms adjust weights repeatedly using data and an objective; the new tests instead ask Astra to reason about a supplied set of weights.
The researchers’ post names Rikiya Takehi and Jayden Teoh as equal contributors, alongside Alex Zhang and Omar Khattab. It presents complete logs, according to the source material, but is a blog post rather than a peer-reviewed paper and is not posted on arXiv. The authors frame their work as a change in how networks might be optimized; that is their proposed direction, not an established shift in machine-learning practice.
““Coding directly in weights is kind of hard (I tried).””
— Andrej Karpathy, describing neural networks as “Software 2.0” in 2017
Limits of the Reported Tests
The evidence remains preliminary: the post is not peer-reviewed, the source says there was one run per setting, and the work covers small networks, a single model family per task and one research group. The results do not show whether Astra’s apparent interpretations generalize to larger systems, whether edits remain reliable across repeated attempts, or how the method compares with established repair and training approaches under equivalent conditions.
The post also reports a negative result: Astra could not distinguish an honest model from a sycophantic one using weights alone. Its edits did not improve maths in Gemma 3 270M, which did not exhibit the looping behavior Astra had addressed in Qwen. The available information does not establish why each intervention succeeded or failed, or whether the reported gains would persist across other benchmarks and deployments.
Reproduction and Broader Testing
The immediate next step is independent replication across more tasks, model sizes and repeated runs. Researchers would also need to compare weight-space edits with conventional fine-tuning and other repair methods, and report whether improvements hold on tests not used to guide the edits. The source material says the team released experiment logs and that a Hugging Face engineer reconstructed the Qwen edit; it does not specify a publication date for a formal paper or further experiments.
Until broader evidence is available, the findings are best treated as a set of documented demonstrations. They show what Astra did in the reported settings, while leaving open whether the approach can become a dependable tool for auditing or repairing models.
Key Questions
What is weight-space programming?
It is the researchers’ term for asking a language model to read or change another neural network’s numerical weights by reasoning about them, rather than using a standard training loop for that task.
What did Astra reportedly repair?
The MIT post says Astra edited a car-control network that had failed to reverse successfully and changed a small portion of Qwen3.5-0.8B’s weights, with the reported MATH-500 score rising from 7.9% to 48.1% in that experiment.
Does this show that AI can repair any model?
No. The tests cover narrow tasks, and the researchers report that edits did not improve maths in Gemma 3 270M. They also say editing could not add an ability the model lacked.
Has the work been peer-reviewed?
According to the source material, the results were published in a November 2 blog post with experiment logs. The post is not a peer-reviewed paper and was not on arXiv at the time described.
What remains to be tested?
Independent researchers need to check whether the results repeat across runs, larger networks and more tasks, and compare the edits with conventional approaches. The post does not settle how reliable or broadly useful the method is.
Source: ThorstenMeyerAI.com
