🔍 Read the full analysis: When AI Edits AI: The Story Behind Astra’s Model Repair on ThorstenMeyerAI.com
Get business pricing on office and shipping supplies
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
TL;DR
Four researchers affiliated with MIT CSAIL report that OpenAI’s GPT-6 Astra improved Qwen3.5-0.8B’s MATH-500 score from 7.9% to 48.1% by editing two weight tensors, without gradient descent. The change reduced, but did not eliminate, looping; the result comes from an early, non-peer-reviewed blog post and depended on 50 forward passes and more than 20 hours of work.
Four researchers affiliated with MIT CSAIL report that OpenAI’s GPT-6 Astra improved the math performance of the open-weight model Qwen3.5-0.8B by changing a small part of its weights, without using gradient descent. In their November 2 post, the researchers say the edited model scored 48.1% on MATH-500, up from 7.9%; the result matters because it suggests one AI system can help repair another model directly, although the work is preliminary and the model still loops on many answers.
The researchers — Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab — published the results in a blog post titled “Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights.” The post describes eight experiments and provides logs. It is not a peer-reviewed paper, and the authors characterize the work as “very early results.” OpenAI is not listed as an author, and the post reports no statement from the company about the experiment.
For the Qwen test, the team gave Astra access to the model’s weights through a custom Python harness. The setup barred gradient descent, backward passes and fitting weights to a loss. Astra instead used test feedback to search for a change, with practice problems available but the final benchmark questions withheld. The reported winning run used 50 forward passes, 80 Python runs and 158 turns over 20.3 hours. A forward pass let Astra inspect answers to as many as 512 problems, or read up to 1,024 texts.
The researchers traced the repetition to an internal direction associated with the model’s outputs in layers 15 and 19. They subtracted that direction from 64 output columns in each layer, changing 131,072 weights — reported as 0.016% of the model’s 0.8 billion weights. The edited version is available on Hugging Face; the post says only the two down-projection weight tensors in those layers differ from the original.
The benchmark figures show both a large score change and a continuing failure mode. On MATH-500, the base model scored 7.9%, while versions using 10, 25 and 50 forward passes scored 12.3%, 14.1% and 48.1%. The share of answers reaching the 32,768-token limit fell from 93.0% to 87.2%, 86.4% and 33.1%, respectively. On GSM8K, the 50-pass version scored 67.1%, compared with 13.8% for the base model; on IFEval, it scored 62.3%, compared with 21.0%.
A Targeted Repair, Not New Math Ability
The result offers a concrete example of a model being used as an editor of another model’s weights, rather than as a tool that generates a new training set or adjusts parameters through standard gradient-based training. If this approach proves reliable, it could give developers another way to diagnose and patch specific behaviors in models whose weights they can access.
But the size of the score increase should not be read as evidence that Astra taught Qwen mathematics from scratch. The post says Qwen’s own model card warns that it is prone to thinking loops. The researchers describe their work as fixing that weakness, and explicitly caution that “Fixing loops, however, is not the same as making a model better at math.” The benchmark gain may reflect answers no longer being lost to unproductive repetition; the reported results do not establish broader reasoning gains.
The remaining loop rate is a practical limitation: even after the strongest edit, 33.1% of MATH-500 answers hit the token limit. The method also used substantial experimentation, so “without training” does not mean effortless or free of feedback. Its relevance is currently greatest for teams able to inspect and alter open-weight models; it cannot be applied in the same way to a model whose weights are inaccessible.
Popular right now
Ninja Flip Toaster Oven & Air Fryer, Large Capacity, Stainless, SP151
Breville BES870BTR Barista Express Espresso Machine, Black Truffle
Ninja Luxe Café Mini Plus Espresso and Drip Coffee Machine with AutoFroth, Stainless Steel (ES351)
As an affiliate, we earn on qualifying purchases.
How the Weight-Editing Test Worked
The experiment tested a small model, Qwen3.5-0.8B, and asked whether Astra could improve its math performance by directly examining and changing its weights. The harness ran on a CPU, had no internet access and allowed a maximum of 300 code runs. The team supplied practice sets of 7,469 GSM8K problems and 7,275 MATH problems, while withholding the final benchmark questions from Astra.
The results depended heavily on the amount of model feedback. With 10 or 25 forward passes, the MATH-500 score reached only 12.3% and 14.1%; the 50-pass run produced the much larger result. The authors also report that a test on Gemma 3 270M did not produce a change larger than about a point, and that a model without looping did not improve in the same way. Those results suggest the technique may depend on finding a specific, repairable failure rather than applying as a general upgrade.
The blog’s broader experiments varied in outcome. In one, Astra could not determine from weights alone which of two fine-tuned Gemma models had been trained to flatter users. The post also relates the Qwen edit to earlier research on removing refusal behavior by changing a direction in model weights, including a 2024 paper by Arditi and colleagues. That comparison supplies a precedent for targeted edits, but it does not establish that the new method will work across models or tasks.
““Fixing loops, however, is not the same as making a model better at math.””
— The researchers, in “Sparks of Weight-Space Programming”
Open Questions About Reliability
The report does not establish whether the edit will work reliably on other model sizes, architectures or failure modes. The supplied material describes a small number of experiments, including a separate Gemma test with no change greater than about a point. It also does not show whether the Qwen gains persist across different evaluation settings or independent replications.
The work’s status is another limitation: it is a blog post with published logs, not a peer-reviewed paper. The reported benchmark scores are claims made by the researchers, and the source material does not provide an independent evaluation. It is also unclear how much of the improvement would remain under different sampling settings or in practical deployments. The researchers’ logs specify the reported setup, but do not establish a general performance guarantee.
Finally, the headline phrase “without training” needs a narrow reading. The experiment avoided gradient descent, but involved repeated model evaluations, test feedback, code execution and more than 20 hours of work. The edit also leaves a substantial rate of answers reaching the token limit. Whether future work can produce more reliable repairs with fewer evaluations remains unresolved.
Replication and Broader Model Tests
The immediate next step is independent replication of the Qwen3.5-0.8B result, including the reported MATH-500, GSM8K and IFEval scores and the remaining loop rate. The edited weights and logs make the experiment available for scrutiny, but the source material does not announce a scheduled replication or a follow-up paper.
Further tests would need to show whether the method can identify and repair other specific defects, whether the edits hold up outside the reported benchmarks, and how much evaluation budget is required. The post also leaves open whether the approach can produce consistent results across different models. Until those questions are answered, the finding is best understood as a reported proof of concept, not a broadly validated method for improving AI systems.
Key Questions
What did GPT-6 Astra change in Qwen3.5-0.8B?
According to the researchers, Astra identified a direction associated with looping and subtracted it from output weights in layers 15 and 19. The edit changed 131,072 weights, or a reported 0.016% of the model’s total weights.
Did the edit eliminate Qwen’s looping?
No. The share of MATH-500 answers reaching the 32,768-token limit fell from 93.0% to 33.1% after the 50-pass edit. A substantial portion still hit the limit.
Was the model trained?
The researchers say they did not use gradient descent, backward passes or loss-based weight fitting. However, Astra used repeated forward passes, practice-problem feedback and code runs to search for the edit, so “without training” does not mean no evaluation or effort.
Has the result been peer-reviewed?
No. The source describes the work as a blog post published November 2, 2026, with logs, and says it is not peer-reviewed. The authors call the findings very early.
Does this show AI can generally improve other AI models?
Not yet. The Qwen result is one reported repair, while the post describes mixed results in other experiments. The available evidence does not establish that the method works reliably across different models and tasks.
Source: ThorstenMeyerAI.com
