🔍 Read the full analysis: Weight-Space Programming Meets Home AI: Claude Edits, Astra Checks on ThorstenMeyerAI.com
Get business pricing on monitors, keyboards and dev gear
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
TL;DR
A home experiment used Claude-written code to remove a measured looping direction from a local AI model’s weights. In a 15-question comparison, the edit raised completed answers from four to 12, but the legal score remained zero; without the edit, a recommended sampling setting produced 13 completions.
A home experiment reported November 5 found that editing a local AI model’s weights helped it finish more answers but did not improve its score on a legal benchmark. Using code written in a Claude Opus 5.5 session, the experimenters reduced a looping pattern in Ornith 1.5 35B-A3B; in a same-setup comparison, completed answers rose from four of 15 to 12 of 15, while the legal score stayed at zero.
The edit targeted a measured “loop direction” in the model’s internal signals. The code removed that direction at half strength from the output weights of 27 of the model’s 40 layers. The experiment involved no further training, additional legal data or direct AI reasoning over the weights. The authors describe this as a related but simpler approach than the weight-space programming method in a recent MIT CSAIL post.
The legal results remained poor. The 12 completed answers contained 62 wrong or invented citations, an average of 5.2 per answer. On 75 checked criteria, the edited model met seven, compared with three before editing. Both versions received a legal score of zero under the reported scoring method.
A no-edit comparison complicates the apparent improvement: with sampling settings recommended by the model card, the original model completed 13 of 15 answers, also scoring zero. The report says results came from one run per setting. It also notes that a change in the local setup moved the original model’s completed-answer count from ten to four, making the comparison sensitive to conditions.
Infographic 01 · What we changed
One loop direction, weakened at half strength
Inspired by the MIT post, but a simpler, classic technique (directional ablation): code written by a Claude Opus 5.5 session measured one loop direction by running the model and projected it out of the weights. No training, and no AI reading weights as text.
The six steps
- Write test material that is not the benchmark: 24 legal development questions plus 8 general controls, 0 overlap with benchmark prompts
- Find the true loop onset at token level: from there, at least 90% of the next 1,500 positions repeat earlier 32-token sequences
- Run the model and measure one loop direction per layer: average internal signal inside the loop minus the reasoning just before it
- Weaken it in the weights:
W ← W − α·r·(rᵀW)on the MLP output matrices of all 256 experts and the shared expert - Bake an edited copy; the original files stay untouched and the edited weights stay internal
- Benchmark original and edit on the same local setup, 15 tasks, graded blind by three Claude agents
Quick test: share of pure copying
Restart 20 tokens before a known loop onset, 8 cases, 3,000 tokens. Lower is better.
The cost of the edit
- Full strength: 23 of 24 legal questions finished, but control answers broke: tokens dropped out of short formulas, and month counting went wrong
- Half strength: 23 of 24 finished, 9 of 9 checked control facts right, and one German control question was answered in Chinese once
- Before the edit: 8 of 24 legal development questions did not finish; all 8 controls did
One run per setting. The loop detector was revised three times during the experiment, so the development numbers come from a moving yardstick. The direction is a statistical description, not a single “loop neuron”.
Sources: our experiment records, 4–5 Nov 2026 (executing Claude Opus 5.5 session; review by GPT-6 Astra in Codex); Ornith-1.5-35B-A3B model card.
Powered by Thorsten Meyer AI
More Completions, No Legal Gain
The test illustrates the difference between helping a model finish generating and improving the accuracy of what it generates. Fewer loops can make output more usable, but they do not supply legal knowledge or make citations reliable. Here, completion rose in one comparison while the benchmark score stayed at zero and the finished answers still included numerous citation errors.
That distinction matters for anyone considering local AI for legal research or other high-stakes work. A fluent, completed answer is not proof of correctness. The experiment’s own results support a narrow claim about behavior under particular settings; they do not show that the edit made the model legally capable, safe to rely on or better than simpler configuration changes.
The independent review also shows why technical claims benefit from outside scrutiny. OpenAI’s GPT-6 Astra, run in Codex, reviewed the code, figures and claims and identified faults, according to the report. That review adds a check, but it does not turn a small, single-run experiment into a replicated finding.
Products Worth Considering
Bose QuietComfort Ultra Earbuds (2nd Gen) – Wireless Noise‑Cancelling Earbuds with Mic, Immersive Audio, USB‑C Charging, Up to 6 Hours Batte
Apple iPad mini (A17 Pro), 128GB, Wi-Fi 6E, Space Gray
Kindle Paperwhite Signature Edition 32GB (2024) – Auto-adjusting front light, wireless charging, weeks of battery life – Metallic Black
As an affiliate, we earn on qualifying purchases.
From MIT Method to Home Test
The test followed a November 2026 MIT CSAIL post by Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab describing “weight-space programming.” In the reported Qwen case, GPT-6 Astra reasoned directly about another network’s weights and edited them. The source says the Qwen model’s mathematics score rose from 7.9% to 48.1% and that a looping behavior stopped. Those are claims about the MIT team’s experiment, not results from the home test.
The home authors instead used a familiar directional-ablation technique: run the model, record internal signals, estimate a direction associated with looping, then project it out of some weights. They selected Ornith after finding that other local models either did not loop in the same way or were unsuitable for the machine and token-budget conditions. Ornith’s publisher describes it as a coding and agent model; the report says it scored one out of 100 in an earlier legal benchmark run and left five of 15 answers unfinished.
The benchmark, Lexassay, asks 15 questions spanning German, U.S. and EU law without internet access or tools, then checks cited statutes and court decisions against primary sources. Ten tasks are public and five are hidden. The report says the new runs are not displayed on the benchmark website, which continues to show the original Ornith result.
“The model finishes is not the answer is right.”
— The home experiment’s authors
Limits of the Single-Run Test
The results are preliminary. The authors report one run per setting, a small development set and a loop detector that changed during the experiment. They also say three graders came from the same model family, while Claude both carried out the edit and supplied blind grading. GPT-6 Astra’s review found faults, but the report does not provide a replicated trial that establishes how often the edit works.
It remains unclear how much of the completion difference came from the weight edit, sampling choices or other setup changes. The comparison in which the edit raised completions from four to 12 used the same setup, but a separate recommended-settings run completed 13 without editing. The experiment therefore does not establish that editing is the best way to reduce unfinished answers.
The account also does not show whether the change generalizes to other prompts, legal benchmarks, model versions or hardware configurations. No improvement in legal performance was demonstrated, and the cited errors leave the reliability of the generated legal material in doubt.
Replication and Benchmark Checks
The next useful step would be repeated tests under fixed, documented settings, with the same prompts and scoring procedure applied to edited and unedited versions. Independent graders and additional models could help distinguish genuine reductions in looping from changes caused by sampling or evaluation. Testing on the benchmark’s hidden questions would also help show whether any observed effect carries beyond the development set.
The report does not announce a scheduled follow-up or say that the new runs will be added to Lexassay. Until further results are available, the demonstrated outcome is limited: the edit changed completion behavior in one test, while legal accuracy did not improve. The authors describe the work as an AI experiment, not legal advice.
Key Questions
What did the weight edit change?
Code written during a Claude Opus 5.5 session removed half of a measured looping direction from the output weights in 27 of Ornith 1.5 35B-A3B’s 40 layers. The report says the model was not retrained and received no new legal data.
Did the edit make the model better at law?
No legal-score improvement was reported. The score remained zero, although the edited model met seven of 75 checked criteria compared with three before editing. Its completed answers also contained 62 wrong or invented citations.
How did the model do without the edit?
In a separate run using sampling settings recommended by the model card, the original model completed 13 of 15 answers without a weight edit. Its legal score was still zero.
Was this the same method used in the MIT research?
No. The MIT post describes GPT-6 Astra reasoning directly over model weights. The home experiment used code to run the model, measure an internal direction associated with looping and remove it from selected weights.
Can the result be treated as reliable legal advice?
No. The experiment reported a zero legal score and citation errors, and its authors state that it is not legal advice. The results also come from one run per setting and need replication.
Source: ThorstenMeyerAI.com
