Nov 2026

Weight-Space Programming for College Advising and Fabrication

Author: Harlyn Pacheco
Repo: github.com/harlyn1/weight-space-advising-logs

Two weeks ago Takehi, Teoh, Zhang and Khattab at MIT CSAIL published Sparks of Weight-Space Programming. The paper is buzzy. You give a frontier model direct read and write access to another model’s weights, and you ask it to find what is broken and fix it. A frontier model reads weights, forms a hypothesis about which numbers do what, changes some of them, and measures whether a behavior moved.

The behavior improved. Their editor found that Qwen3.5-0.8B was hitting the token limit on 93% of MATH-500 problems, traced the cause to a single direction written by the MLPs of layers 15 and 19, subtracted that direction from 64 columns per layer, and took MATH-500 from 7.9% to 48.1%. It changed 131,072 weights, ~ 0.016% of the model.

I recently expanded on these findings in a new domain. I ran the weight editing exercise using college advising domain questions. With the help of researchers, I tested a domain where the failures are not just arithmetic but also judgment. I explored additional research questions. How do editors across frontier labs compare? How does fine tuning compare?

Setup

Qwen3.5-0.8B was small enough to edit on a rented A100 GPU and my M4 (the memory issues in iteration were fun). Six graded query families were tested across various configurations: general advising facts and student scenarios, reading the value of a GPA or SAT score, federal aid eligibility, fabrication, calibration, and demographic steering.

The failures are the kind that matter in worlds and user contexts where judgment matters. Before weight edits, the 0.8B model told a student with a 2.3 GPA that this “positions you as a strong competitive contender.” It called an 1100 SAT “the highest possible score.” Asked for the application deadline of a scholarship that does not exist, it invented one, with a date. When the 0.8B model was asked the same question by a first-generation student and by a student whose parents went to Harvard, it steered them to different schools. To be fair, I have evidence (see Tier C in Model Results page) that larger frontier models also have biased demographic steering.

In my setup, each model editor got 40 forward passes, a summary of failures to target, and the paper’s fix as a starting point. We tested how Fable 5.1, Opus 5.5, GPT-6 Astra, Kimi K3 and Sonnet 5.5 would edit Qwen3.5-0.8B and impact the quality of responses to the six graded query families.

Initial Results

The best edit changed 18,432 numbers out of 752 million, ~ 0.0024% of the model. The best edit came from Sonnet 5.5, it cost $88, and it took a mean score from 50.7% to 60.9% and fabrication from 39.3% to 92.9% (where a higher percentage is a better score). It beat an edit three thousand times larger and a LoRA fine tune. It did not beat context injection on the mean, but it halved severe errors against it. We pressure tested the LoRA fine tune and confirmed that the fine tuning merely taught the model what to repeat but not to alter its approach to fabrication. Context injection at runtime also improved fabrication, and cut severe errors, but nowhere near as far as the edit did.

The 0.8B model Sonnet updated invents fewer specifics and it does so from updates to six neuron slots in one weight layer. I’m certainly not saying that all forms of context injection and fine tuning can be beat by WSP, but I do think that this is worthy of additional exploration.

We had some unexpected surprises: four of five editors independently found the same broken component. One neuron near the end of the 0.8B layers is responsible for starting an answer with ‘I’, as in ‘I don’t have that information’. It’s wired to a detector that spots questions about specific programs. While the specific program detection is happening in our runs, the neuron responds with fabrication. However, when we flip 2,048 weights, the probability of the model opening with “I” on those questions goes from 0.05 to 0.85. Nearly every frontier model found the source of fabrication. Kimi K3 read it as missing knowledge and declined to edit.

Build cost is model API spend only, at list prices. GPU rental and local compute are not included.

Next Steps

Every domain and task combination likely has a model that performs weight programming best on open models. We’ll be expanding the tests to new domains and tasks. We’ll be testing how one model can guide another model to a starting point. I made some mistakes in this experiment. In fact, I had to run multiple iterations of the edits when I realized that I was giving models the results of previous model runs. I subsequently ran clean runs :) and identified the starting point for an upcoming experiment in another domain.

In further experiments, we’ll consider how to combine the levers we’ve explored: when to curate models for use cases, when to edit mechanisms like weights for durable reliability, and when to inject facts.

Appendix: the original paper’s loop fix

A community note on X raises a fair point about the original paper. The loop its editor fixed is already documented in Qwen’s model card, and what the edit repaired was repetition in thinking mode rather than math ability. The same question applies to me.

The original paper’s loop fix was in every one of my arms. I handed it to the editors as a starting point and told them to keep it, so it is fair to ask how much of my result is really theirs.

The answer is none of it, even if we were inspired by the original paper’s loop fix.

Our fix did not depend on the loop that Qwen’s model card already documents. It targeted a new problem in the weights, a neuron in layer 23 that was supposed to open an answer with “I don’t have that information” and stayed quiet instead. This was found independently by four of five models we ran. The mechanisms for discovering problems in weights certainly merit additional experimentation.

The Kimi arm is only the loop fix and no additional model edits. It scored 39.3 on fabrication. The untouched model scored 39.3. The math loop fix was not the reason we succeeded at improving fabrication.

@misc{takehi2026weightspace,
  title        = {{Sparks of Weight-Space Programming: LLMs can natively read \& write neural network weights}},
  author       = {Takehi, Rikiya and Teoh, Jayden and Zhang, Alex and Khattab, Omar},
  year         = {2026},
  month        = nov,
  howpublished = {\url{https://rikiyat.github.io/blog/weight-space-programming/}},
  note         = {Blog post}
}