TL;DR
Get business pricing on monitors, keyboards and dev gear
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
Researchers reported that GPT-6 Astra edited two weight tensors in Qwen3.5-0.8B, raising its MATH-500 score from 7.9% to 48.1%. The experiment used iterative search and test feedback without gradient descent; 33.1% of edited-model answers still reached the token limit, and the results are from an early, non-peer-reviewed blog post.
Four researchers listed with MIT CSAIL report that OpenAI’s GPT-6 Astra edited two layers of the open Qwen3.5-0.8B model, lifting its score on the MATH-500 benchmark from 7.9% to 48.1%. The edit targeted repetitive reasoning that often consumed the model’s token budget; the result comes from an early blog post, not a peer-reviewed paper, and the looping problem was reduced rather than eliminated.
The researchers describe the work in Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights, published November 2, 2026. Their post covers eight experiments, and labels the findings “very early results.” The Qwen math repair was the experiment that drew attention, but the post also reports failures and more limited results in other tests.
According to the published logs, Astra used a Python-based research harness to inspect and edit the model’s weights. The researchers barred gradient descent and backward passes, but the process still involved iterative code runs, forward passes and feedback from practice problems. The winning run took 158 turns, 80 Python runs and 50 forward passes over 20.3 hours. A forward pass could return answers to as many as 512 problems.
Astra attributed the looping to a direction associated with the MLPs in layers 15 and 19 and subtracted it from 64 output columns in each layer, changing 131,072 weights. That is 0.016% of the model’s 0.8 billion weights, according to the post. The edited model on Hugging Face shows changes to the two layers’ down-projection tensors. On MATH-500, answers that ran into the 32,768-token limit fell from 93.0% to 33.1%, while the reported score rose to 48.1%.
A Loop Fix, Not an IQ Jump
Researchers report that GPT-6 Astra edited two weight tensors in Qwen3.5-0.8B, reducing repetitive reasoning and raising its MATH-500 score. The early, non-peer-reviewed result shows a targeted repair, not a general improvement in mathematical ability.
A targeted weight edit
The team described a specific intervention to address repetitive reasoning in Qwen3.5-0.8B.
Practice-problem feedback pointed to a direction associated with MLPs in layers 15 and 19.
Astra subtracted that direction from output columns in both layers, without gradient descent or backward passes.
The edited model scored higher on MATH-500, but some answers still reached the token limit.
The search took substantial effort
The reported best run used repeated code execution and model feedback.
20.3 hours
The winning search run took more than a day of focused experimentation.
158 turns
The run included 80 Python runs and 50 forward passes.
0.016% of weights
The edit changed a small fraction of the model’s 0.8 billion weights.
A big score gain, with limits
The benchmark result reflects fewer unfinished answers, while the looping behavior remained.
What businesses and readers should take away
Weight editing may offer teams another way to investigate a specific failure in an open model.
- A focused edit may reduce a recurring failure mode.
- Published logs and model files let others inspect the reported change.
- The result has not been peer-reviewed or independently replicated in the supplied material.
- Performance on other tasks, prompts, and settings remains unestablished.
A Repair That Could Change Model Tuning
The result suggests that a model with access to an open model’s weights may help diagnose and make targeted edits to a specific failure mode. For teams that run open-weight systems, a focused change could offer another route to improving behavior when a model repeatedly gets stuck. The reported gain is large, but it reflects a benchmark result after an intensive search, not evidence that Astra gave Qwen a general improvement in mathematical ability.
The authors themselves draw that distinction: “Fixing loops, however, is not the same as making a model better at math.” The change reduced one obstacle that prevented the small model from completing answers. The remaining token-limit rate shows that the behavior persisted for about one-third of MATH-500 responses. The score alone does not establish how well the edit transfers to other tasks or settings.
Weight editing also has a security relevance. The post points to earlier research showing that removing a direction in a model’s weights can change refusal behavior. That makes the method potentially useful for studying model behavior, while also raising questions about how modified versions are evaluated and distributed. Each edited checkpoint is a distinct model state whose behavior needs to be checked.
Products Worth Considering
Logitech MX Brio Ultra HD 4K Webcam, Graphite
Bose QuietComfort True Wireless Earbuds with Active Noise Cancellation, Black
Samsung 27-Inch Odyssey OLED G5 QHD Gaming Monitor
As an affiliate, we earn on qualifying purchases.
How the Qwen Loop Was Targeted
Qwen3.5-0.8B is a small open model, and its model card warns that it can enter thinking loops. In the researchers’ initial practice run, 85 of 96 answers reached the 32,768-token limit. That provides a concrete explanation for the low baseline score: many answers did not finish within the available output budget.
The team gave Astra practice material from GSM8K and MATH, but kept the final benchmark problems hidden. It tested different forward-pass budgets: with 10 passes, MATH-500 reached 12.3%; with 25, it reached 14.1%; and the 50-pass run reached 48.1%. The post also reports that a model without the looping behavior did not improve, and that edits to Gemma 3 270M moved its score by no more than about a point. The authors say their other experiments included a case where Astra could not identify which of two fine-tuned Gemma models had been trained to flatter users.
The experiment was run through the Codex command-line tool using GPT-6 Astra at its highest reasoning setting, according to the logs. The researchers’ harness ran on CPU, had no internet access and allowed up to 300 code runs. The source article says OpenAI was not an author of the post and that it found no OpenAI statement about the experiment. Those details distinguish the research team’s reported setup from an OpenAI announcement.
““Fixing loops, however, is not the same as making a model better at math.””
— The researchers, in “Sparks of Weight-Space Programming”
How Much the Edit Generalizes
The source is a blog post with published logs and a model checkpoint, but it is not peer-reviewed and is not listed on arXiv in the supplied material. Independent replication, broader testing and evaluation on tasks beyond the reported benchmarks are not described. It is also unclear whether the improvement holds across different prompts, inference settings or versions of the model.
The loop rate fell substantially but remained at 33.1% on MATH-500. The post does not establish that the edit removes looping across all uses, or that the benchmark score predicts performance in practical math applications. The reported search used substantial time and compute, and smaller forward-pass budgets produced much smaller gains.
Replication and Broader Evaluation
The next evidence to watch for is independent replication and testing of the edited checkpoint across additional tasks and inference conditions. The researchers’ published logs and model files allow others to inspect the reported changes, but the supplied source does not announce a follow-up paper or a planned release schedule for further results.
For now, the report documents a specific intervention: Astra altered a small fraction of Qwen’s weights and the model scored higher on MATH-500 while looping less often. Whether this approach can reliably improve other open models, and how much oversight each edit requires, remains to be established.
Key Questions
What did GPT-6 Astra change?
According to the researchers, Astra subtracted a direction linked to repetitive reasoning from output columns in the MLP down-projection weights of layers 15 and 19 of Qwen3.5-0.8B.
Did the model’s MATH-500 score improve?
The post reports a rise from 7.9% to 48.1%. The result is from the researchers’ experiment and has not been established through peer review in the supplied source.
Was the edit done without training?
It was done without gradient descent or backward passes, according to the logs. Astra still used an iterative search with practice problems, forward passes, Python runs and feedback, so the process involved substantial experimentation.
Did the edit eliminate the looping?
No. Answers that hit the 32,768-token limit fell from 93.0% to 33.1% on MATH-500, according to the post. The reported results show a reduction, not elimination.
Has the research been peer-reviewed?
The supplied source describes it as a blog post published November 2, 2026 and says it is not peer-reviewed. The authors call the findings early results.
Source: Thorsten Meyer AI
