Can models pass working state without writing it down?

Independent Research — Waylon Zabriskie · August–October 2026

When two AI models work together, they exchange text. I wanted to know whether they have to. Could one model pass information directly through its internal representations, and would that keep more information or cost less than writing it down?

1. Why I looked at this

A lot of practical work with language-model agents is context engineering: deciding what an agent sees, what it remembers when its context window fills, and what it hands to the next agent. Most practical agent handoffs today exchange text, such as summaries or structured messages. A model that has read a long session has some internal representation of it. To pass that on, it writes a summary, and the next model rebuilds its own representation from that summary. Summaries drop details nobody thought to write down, and rebuilding context costs tokens and time.

So the question was simple to state. Could a model hand over its internal representation directly, to a different model or to a fresh copy of itself after a reset? I started with three hypotheses, and I want to keep them separate from what was actually shown:

• Less loss. An internal representation might keep information a summary drops. This was tested only on small structured records, where text loses nothing either.
• Less cost. Skipping text generation and re-reading might be cheaper. This was tested, and it was false in my setup (Section 3).
• Information that is hard to put into words. This was never tested. Every state I could check objectively was also easy to write as text.

2. Moving a record from one model to another

Figure 1. The experiment in one picture. Both models stay frozen; only the small map is trained.

Model A reads a short record. Instead of writing it out, A's internal numbers for that record pass through a small learned map into Model B, a different model that never sees the record as text. B is then asked questions about the record. The map was trained only to imitate how B itself represents a record, and never saw a question. Two kinds of question were held out entirely, so the test is whether B can use the transferred state for questions nobody trained it on.

What was held out

Six kinds of question were used:

• Four were used during development: retrieve a field, compare two numbers, set membership, and count matches.
• Equality ("is F2 the same as F4?") was never used to train, tune or select anything.
• Mapping (apply a private lookup table to a field) was kept sealed and run once, after everything else was frozen.

Results

Figure 3. Accuracy of B on six kinds of question when its record comes only through the map. "Swapped record" bars show how often B's answer followed a different record's state that was substituted on purpose.

On the two held-out question types, B scored 0.90–0.92 (equality) and 0.775–0.787 (mapping) across three independently trained maps. With its own internal rows for the real record inserted directly, B scored 0.90 and 0.775. Given the mapping record as plain text, B scored 0.80, because that question type is limited by how the answer is read out. So the map recovered essentially everything B could do with the record, on questions it had never seen.

The swap test is the main evidence that B used A's state rather than guessing. Keep the question fixed and substitute the state from a different record whose answer differs. B's answer then follows the substituted record at 0.92–1.00 on the development tasks, 0.77–0.78 on equality, and 0.80–0.83 on mapping. With all-zero state, or a map that was never trained, accuracy falls to the level B reaches with no record at all (an untrained map scored 0.15–0.30 against 0.775–1.00 for the trained one on a 40-question check).

What this does not show

• The content is small and discrete. A record carries about 15 bits. Everything that crossed between the models could have been sent as one short line of text, and text does as well.
• Narrow scope. There is one model pair, one record format, and known field positions. The three "seeds" are three trainings of the map, scored on the same question sets: 60 questions per type, 80 for mapping.
• It is not general transfer of what a model knows. It shows that a map trained to reproduce B's own early representation of a record lets B's existing abilities apply to it. Generalizing to new question types comes from B. It is not something the map learned.

What failed before this worked

The design above came after five earlier designs. Earlier versions did one of three things:

• failed outright;
• worked only on the question types they were trained on: learned prefix vectors, cross-attention bridges into B, and tiny adapters inside a receiver model, which scored 0.000 on a held-out task;
• learned the question instead of the record: a map trained to reproduce B's activations at answer time. A diagnostic showed its output was 99% the same with or without A's state.

The change that worked was the training target: B's own early representation of the record, rather than anything tied to a task.

Chaining

The same kind of maps can be chained. In a five-step exchange (A to B to A to B to A to B), each model does a small private operation and passes the result on. B's final answer followed the transported value 0.975–1.000 of the time across five independently trained maps. My audit found the catch: several of these maps have rank three. Each step can only express a weighted choice among four fixed vectors, one per letter. So the chain shows that very small messages compose reliably. It does not show rich state moving back and forth. Each step was also a fresh prompt with no accumulated conversation.

3. Against plain JSON

A transfer that works is only useful if it beats the obvious alternative. In early October I built the most favorable comparison I could:

• the same kind of seven-field record;
• B's recovered fields fed to a fixed rule that picks an action;
• 300 records the system had never seen;
• the same pipeline with the record passed as JSON.

Figure 4. At equal decision accuracy, the hidden-state message was about 1,000 times larger than the JSON message and took about a second to read back. Log scale.

Both routes chose the right action every time (1.000; with no communication at all the rule gets 0.763). The internal route also recovered all seven fields exactly in only 93% of cases. The extra errors happened to fall in fields the rule didn't use. JSON carries every field exactly, by construction.

I then tried the case where text is supposed to struggle: a context too long to summarize cheaply. There, B's native representation of the context was about 10.5 MB, and extracting A's state took 539 ms. That was longer than simply re-reading the raw context, which took 138 ms. On October 5 I stopped the practical line and wrote down a rule for when to use what:

• use structured text when the state is cheap to serialize;
• use the model's own context caching when the receiver can re-read;
• consider internal-state transfer only for information that resists serialization, which I never demonstrated.

4. Memory across a context reset

I then turned to a narrower version of the original problem. When an agent's context is reset, can a fresh copy of the same model continue from a compact internal memory of what came before? This part has two halves that must not be read together. In a controlled synthetic setup, memory transfer was promising. On real sessions, the evaluation exposed a weakness in the benchmark itself.

4a. Synthetic memory: it improves with training data

The setup:

• A context contains 24 facts, each with one of five values.
• A fresh copy of Qwen2.5-3B, with a small add-on (a rank-8 LoRA on its attention layers), learns to read a memory of 96 numbers inserted at four layers.
• It is then asked about fact combinations it never saw during training.
• The base model's ordinary behavior was checked and stayed close to the original.

Figure 5. Accuracy on unseen combinations of facts as the number of training contexts grows. Control memories (zero, random, or another context's) stay at chance.

The first run used 48 training contexts. It scored 0.625 on its test set of 32 unseen contexts. The pass bar of 0.70 had been committed to the record two days before the result, so the run was recorded as a failure.

After that failure, a follow-up varied only the number of training contexts and used a new, larger test set (Figure 5). Its plan was committed together with its results, so the record cannot show that its criteria came first. Accuracy was:

• 48 contexts: 0.611;
• 96 contexts: 0.861;
• 192 and 384 contexts: 0.996 and 1.000;
• three further trainings at 192 contexts: 0.994, 0.994 and 1.000 (open circles in Figure 5).

In this synthetic setup, more training data closed the gap. That does not show the same approach would scale to richer memory. A port to the 7B model scored 0.990 on a test set opened once, but that was a single run.

I also built automatic selection, deciding which events in a long generated history are worth keeping, and it reached 1.000 in that setup. My audit qualifies this. In those generated histories, an event counted as important exactly when a later event referred to it, and the selector could see those reference counts. The result confirms the pipeline works end to end, not that the system discovered what matters. The "128K-token" scaling test was 3,200 short events encoded one at a time, not one long context.

4b. Real sessions: no evidence that it works

Data. I used my own coding-agent session logs: about 1,670 sessions across three private repositories, of which 509 met a fixed rule for substantial work (enough tool calls, files, commands and at least one error). Only counts and hashes were stored. No transcript text is in any result.

Task. Cut a session at 70%. For an event after the cut, pick which earlier events concern the same file or command, using only a memory of 64 compressed event vectors.

How the settings were chosen. I chose the memory size (64 events) on development data, after memories of 16 and 32 events fell short. I set the pass bar for the sealed test (0.80 of the best achievable gain over no memory) alongside that development result, and fixed both before opening the sealed test. The sealed test used unseen sessions from the same repositories as training.

What worked. Compressing each stored event from 2,048 to 64 numbers (16 KiB for 64 events) lost nothing measurable, including on a sealed test.

What did not. Two separate evaluations, each with its own random baseline:

• Sealed test (110 tasks, opened once on October 25): the full automatic memory scored 0.627, 0.436 and 0.655 across three trainings. Picking at random among the candidates gives 0.667 on this test.
• Cross-repository test (Figure 6): a reader trained on one repository scored 0.998 there and 0.51–0.56 on a different repository. Random choice gives 0.601 on this test, which uses a different repository and task set.

Figure 6. The cross-repository test, a separate evaluation from the sealed test above. The learned reader scores below random choice on this set, and exact string matching solves every task.

What the audit found. On October 29 I audited the benchmark itself. Each event is a tool name followed by its argument, such as a file path. The correct answers are earlier events with the same argument. So the question already contains the key to its own answer. Exact string matching scores 1.000 and word overlap 0.999. None of this needs memory.

I found two further problems in the later runs:

• they did not report a random-choice baseline;
• one later development set came from a single session.

The honest conclusion is that this benchmark cannot show whether latent session memory works. Before any more modeling, it needs to be replaced by tasks where the needed information is not already in the question, with a strong text summary as a baseline.

4c. Not built

A shared, project-level memory, one memory object that several agents read and update over time, was planned as a later stage. It was never implemented.

5. How I ran the work

I built and ran the experiments with AI coding agents as my development tools, and used a separate agent pass to audit the record twice, in early and late October.

A few rules carried most of the weight:

• Pass bars were written into an experiment's plan, and a miss stayed a miss. For every bar this page cites, I checked when it entered the record. The 0.70 bar in Section 4a came before its result, which missed at 0.625. The memory size and 0.80 bar in Section 4b were set after development results but before the sealed test. The scaling study's plan was committed together with its results.
• Final test sets were sealed and opened once. A failed sealed test stayed failed, and further work required a new test set.
• Every transfer claim had a swap test plus zero and random controls. A result without them did not count.
• Failures and retractions stayed in the record. A corrected result was relabelled, not deleted.

Those rules changed several of my own conclusions. The most instructive corrections:

• A leak in my own checks. An early solver's 0.975 headline came from answers leaking into its evaluation. Corrected to 0.90.
• A readout that wasn't computing. My standard way of reading B's answer turned out to repeat a line of the prompt. I rewrote the measurement rules and rechecked earlier conclusions against them.
• Overstated generality. The October 2 audit found that several "continuous" maps were rank three, and that one test set reported as a pass contained the same 32 inputs as its training set.
• A lucky initialization. A 0.977 memory result came from one fortunate run; reruns gave 0.73–0.81. It was reclassified.

6. What I take from it

• The training target mattered more than the amount of training. Maps trained to reproduce the receiver's own early representation generalized to new questions. Interfaces trained on tasks did not, however hard they were trained.
• Internal state is not automatically cheaper or richer. In every case I could measure, a few bytes of text did the job. Any future claim for latent communication has to beat that baseline first, on information that is actually hard to write down.
• A memory benchmark has to hide the answer key. If the query contains the information needed to find the right memory, a string match wins, and the test says nothing about memory.

Limitations

• One model pair for cross-model transfer. One model family for memory.
• Synthetic records of about 15 bits in every positive result.
• Samples of 60–80 questions per type for the main transfer result. Several earlier results used 10–25 and are not relied on here.
• No efficiency advantage was found, so none is claimed.
• The real-session data is one person's work across three repositories.
• The real-session benchmark is not valid as a test of memory (Section 4b).
• As implemented, the receiver skipped its fifth layer after the insertion point. The receiver ran 23 of its 24 layers. All conditions used the same execution path, which supports comparisons within this implementation; the effect of the skipped layer has not been independently established.
• Results come from runs recorded in a private repository. The three original maps are no longer available. A rebuilt map and its evaluation code are published separately (Methods note); the rebuilt map is a reconstruction, not one of the three maps behind the reported results. Other code, data and per-run logs are not published.

Related work

This study sits next to published work that I did not originate and do not claim:

• Activation communication between language-model agents: Ramesh & Li, "Communicating Activations Between Language Model Agents," ICML 2025 (arXiv:2501.14082).
• Passing embeddings instead of sampled tokens: Pham et al., "Let Models Speak Ciphers," ICLR 2024 (arXiv:2310.06272).
• KV-cache translation between models: Fu et al., "Cache-to-Cache," ICLR 2026 (arXiv:2510.03215).
• Using one model to read out another model's hidden states, including a larger model explaining a smaller one: Ghandeharioun et al., "Patchscopes," ICML 2024 (arXiv:2401.06102).
• Affine maps between models' residual streams: Chen et al., "Transferring Linear Features Across Language Models With Model Stitching," 2025 (arXiv:2506.06609).
• Compressing context into learned tokens:
  • Mu et al., "Learning to Compress Prompts with Gist Tokens," NeurIPS 2023 (arXiv:2304.08467);
  • Ge et al., "In-context Autoencoder for Context Compression in a Large Language Model," ICLR 2024 (arXiv:2307.06945).
• Reasoning in continuous hidden states within one model: Hao et al., "Training Large Language Models to Reason in a Continuous Latent Space" (Coconut), COLM 2025 (arXiv:2412.06769).

What may be specific to this study:

• training a cross-model map only on the receiver's early representation of a record, and testing it on held-out question types;
• chaining frozen maps over five handoffs;
• the direct cost comparison against JSON;
• the negative results.

Methods note

Every number in the figures is read from a recorded result file or a committed result document by a script that checks each value before drawing it. Where a figure combines sources, the caption says so. The October 29 baselines were recomputed from the same task construction used in the original run, with no model involved.