Can models pass working state without writing it down?

Independent Research — Waylon Zabriskie · August–October 2026

AI models working together typically exchange information through text. I wanted to know whether they have to. Could one model pass information directly through its internal representations, and would that keep more information or cost less than writing it down?

1. Motivation

A lot of practical work with language-model agents is context engineering: deciding what an agent sees, what it remembers when its context window fills, and what it hands to the next agent. Most practical agent handoffs today exchange text, such as summaries or structured messages. A model that has read a long session has some internal representation of it. To pass that on, it writes a summary, and the next model rebuilds its own representation from that summary. Summaries drop details nobody thought to write down, and rebuilding context costs tokens and time.

The research question follows directly. Could a model hand over its internal representation directly, to a different model or to a fresh copy of itself after a reset? I began with three hypotheses, which I keep separate from what the experiments showed:

• Less loss. An internal representation might keep information a summary drops. This was tested only on small structured records, where text loses nothing either.
• Less cost. Skipping text generation and re-reading might be cheaper. This was tested and did not hold in my setup (Section 3).
• Information that is hard to put into words. This was never tested. Every state I could check objectively was also easy to write as text.

2. Moving a record from one model to another

Figure 1. The experiment in one picture. Both models stay frozen; only the small map is trained.

Model A reads a short record. Instead of writing it out, A's internal numbers for that record pass through a small learned map into Model B, a different model that never sees the record as text. B is then asked questions about the record. The map was trained only to imitate how B itself represents a record, and never saw a question. Two kinds of question were held out entirely, so the test is whether B can use the transferred state for questions nobody trained it on.

What was held out

Six kinds of question were used:

• Four were used during development: retrieve a field, compare two numbers, set membership, and count matches.
• Equality ("is F2 the same as F4?") was never used to train, tune or select anything.
• Mapping (apply a private lookup table to a field) was kept sealed and run once, after everything else was frozen.

Results

Figure 3. Accuracy of B on six kinds of question when its record comes only through the map. "Swapped record" bars show how often B's answer followed a different record's state that was substituted on purpose.

On the two held-out question types, B scored 0.90–0.92 (equality) and 0.775–0.787 (mapping) across three independently trained maps. With its own internal rows for the real record inserted directly, B scored 0.90 and 0.775. Given the mapping record as plain text, B scored 0.80, because that question type is limited by how the answer is read out. The map therefore recovered essentially everything B could do with the record, on questions it had never seen.

The swap test is the main evidence that B used A's state rather than guessing. Keep the question fixed and substitute the state from a different record whose answer differs. B's answer then follows the substituted record at 0.92–1.00 on the development tasks, 0.77–0.78 on equality, and 0.80–0.83 on mapping. With all-zero state, or a map that was never trained, accuracy falls to the level B reaches with no record at all (an untrained map scored 0.15–0.30 against 0.775–1.00 for the trained one on a 40-question check).

What this does not show

• The content is small and discrete. A record carries about 15 bits. Everything that crossed between the models could have been sent as one short line of text, and text does as well.
• Narrow scope. There is one model pair, one record format, and known field positions. The three "seeds" are three trainings of the map, scored on the same question sets: 60 questions per type, 80 for mapping.
• It is not general transfer of what a model knows. It shows that a map trained to reproduce B's own early representation of a record lets B's existing abilities apply to it. Generalizing to new question types comes from B. It is not something the map learned.

Earlier designs

The design above was the sixth I evaluated. The five earlier designs fell into three groups:

• failed outright;
• worked only on the question types they were trained on: learned prefix vectors, cross-attention bridges into B, and tiny adapters inside a receiver model, which scored 0.000 on a held-out task;
• learned the question instead of the record: a map trained to reproduce B's activations at answer time. A diagnostic showed its output was 99% the same with or without A's state.

The decisive change was the training target: B's own early representation of the record, rather than anything tied to a task.

Chaining

The same kind of maps can be chained. In a five-step exchange (A to B to A to B to A to B), each model does a small private operation and passes the result on. B's final answer followed the transported value 0.975–1.000 of the time across five independently trained maps. Analysis of the trained maps qualifies this result: several have rank three. Each step can only express a weighted choice among four fixed vectors, one per letter. So the chain shows that very small messages compose reliably. It does not show rich state moving back and forth. Each step was also a fresh prompt with no accumulated conversation.

3. Against plain JSON

A transfer that works is only useful if it beats the obvious alternative. I compared latent transfer against a structured JSON baseline using the same records and decision task:

• the same kind of seven-field record;
• B's recovered fields fed to a fixed rule that picks an action;
• 300 records the system had never seen;
• the same pipeline with the record passed as JSON.

Figure 4. At equal decision accuracy, the hidden-state message was about 1,000 times larger than the JSON message and took about a second to read back. Log scale.

Both routes chose the right action every time (1.000; with no communication at all the rule gets 0.763). The internal route also recovered all seven fields exactly in only 93% of cases. The extra errors happened to fall in fields the rule didn't use. JSON carries every field exactly, by construction.

I then tried the case where text is supposed to struggle: a context too long to summarize cheaply. There, B's native representation of the context was about 10.5 MB, and extracting A's state took 539 ms. That was longer than simply re-reading the raw context, which took 138 ms. These results led me to discontinue latent transfer as a practical communication method in this setting, and I recorded a decision rule:

• use structured text when the state is cheap to serialize;
• use the model's own context caching when the receiver can re-read;
• consider internal-state transfer only for information that resists serialization, which I never demonstrated.

4. Memory across a context reset

The second part of the project addresses a narrower version of the original problem. When an agent's context is reset, can a fresh copy of the same model continue from a compact internal memory of what came before? I evaluated this question in two settings: controlled synthetic data and real coding-agent sessions. In the synthetic setting, memory transfer was promising. On real sessions, the evaluation exposed a limitation in the benchmark itself.

4a. Synthetic memory: it improves with training data

The setup:

• A context contains 24 facts, each with one of five values.
• A fresh copy of Qwen2.5-3B, with a small add-on (a rank-8 LoRA on its attention layers), learns to read a memory of 96 numbers inserted at four layers.
• It is then asked about fact combinations it never saw during training.
• The base model's ordinary behavior was checked and stayed close to the original.

Figure 5. Accuracy on unseen combinations of facts as the number of training contexts grows. Control memories (zero, random, or another context's) stay at chance.

The first run used 48 training contexts. It scored 0.625 on its test set of 32 unseen contexts. The pass bar of 0.70 had been committed to the record before the result, so the run was recorded as a failure.

After that failure, a follow-up varied only the number of training contexts and used a new, larger test set (Figure 5). Its plan was committed together with its results, so the record cannot show that its criteria came first. Accuracy was:

• 48 contexts: 0.611;
• 96 contexts: 0.861;
• 192 and 384 contexts: 0.996 and 1.000;
• three further trainings at 192 contexts: 0.994, 0.994 and 1.000 (open circles in Figure 5).

In this synthetic setup, more training data closed the gap. That does not show the same approach would scale to richer memory. A port to the 7B model scored 0.990 on a test set opened once, but that was a single run.

I also built automatic selection, deciding which events in a long generated history are worth keeping, and it reached 1.000 in that setup. However, the synthetic task defined importance through later references that the selector could observe: an event counted as important exactly when a later event referred to it, and the selector could see those reference counts. The result confirms the pipeline works end to end, not that the system discovered what matters. The "128K-token" scaling test was 3,200 short events encoded one at a time, not one long context.

4b. Evaluating memory on real-world sessions

Synthetic results leave open whether the approach holds up on real working data. To test beyond synthetic records, I evaluated the memory on real coding-agent sessions.

Data. The data are my own coding-agent session logs: about 1,670 sessions across three private repositories, of which 509 met a fixed rule for substantial work (enough tool calls, files, commands and at least one error). Only counts and hashes were stored. No transcript text is in any result.

Task. Cut a session at 70%. For an event after the cut, pick which earlier events concern the same file or command, using only a memory of 64 compressed event vectors.

How the settings were chosen. I chose the memory size (64 events) on development data, after memories of 16 and 32 events fell short. I set the pass bar for the sealed test (0.80 of the best achievable gain over no memory) alongside that development result, and fixed both before opening the sealed test. The sealed test used unseen sessions from the same repositories as training.

Compression. Compressing each stored event from 2,048 to 64 numbers (16 KiB for 64 events) lost nothing measurable, including on a sealed test.

Retrieval accuracy. The learned memory did not exceed random choice in two separate evaluations, each with its own random baseline:

• Sealed test (110 tasks, opened once): the full automatic memory scored 0.627, 0.436 and 0.655 across three trainings. Picking at random among the candidates gives 0.667 on this test.
• Cross-repository test (Figure 6): a reader trained on one repository scored 0.998 there and 0.51–0.56 on a different repository. Random choice gives 0.601 on this test, which uses a different repository and task set.

Figure 6. The cross-repository test, a separate evaluation from the sealed test above. The learned reader scores below random choice on this set, and exact string matching solves every task.

Benchmark limitation. A closer examination of the evaluation revealed that the query already contained the information needed to identify the correct earlier event. Each event is a tool name followed by its argument, such as a file path, and the correct answers are earlier events with the same argument. Exact string matching therefore achieves perfect accuracy (1.000; word overlap 0.999), making the benchmark insufficient to demonstrate the value of latent memory.

Two further issues limit the later runs:

• they did not report a random-choice baseline;
• one later development set came from a single session.

Conclusion. This benchmark did not establish useful latent session memory, and in its current form it cannot test for it. The next step is a replacement benchmark in which the needed information is not already in the question, with a strong text summary as a baseline.

5. Experimental methodology and validation

I used held-out evaluations, control conditions, and independent repetitions to distinguish genuine information transfer from artifacts of the experimental setup.

• Predefined criteria. Pass bars were written into an experiment's plan, and a miss stayed a miss. For every bar this page cites, I checked when it entered the record. The 0.70 bar in Section 4a came before its result, which missed at 0.625. The memory size and 0.80 bar in Section 4b were set after development results but before the sealed test. The scaling study's plan was committed together with its results.
• Held-out and sealed evaluations. Question types used to test generalization were never used to train, tune or select. For the results reported here, the sealed evaluations were run once: the mapping question type in the cross-model experiment, the 7B memory port's test set, and the real-session sealed test. The real-session sealed test failed and is reported as a failure; later real-session work used a new test set.
• Controls. Every transfer claim had a swap test plus zero and random controls, and the transfer map was also checked untrained. A result without these controls did not count.
• Recorded results. Every result, including failures and retractions, stayed in the record; a corrected result was relabelled, not deleted. Every number in the figures is read from a recorded result file and checked before it is drawn.
• Reproducibility. Main results were repeated with independent trainings: three maps for cross-model transfer, and three further trainings for the synthetic-memory scaling point. A single-run result that did not replicate was reclassified. A rebuilt transfer map and its evaluation code are public (Methods note).

These controls corrected several intermediate conclusions, including a leaked evaluation, a readout that repeated the prompt rather than computing, and a memory result that did not replicate. The corrections and a full chronological record are in the appendix.

6. Conclusions

• The choice of training objective proved decisive in the cross-model experiments. Training the map to reproduce the receiver's early representation enabled transfer to held-out question types, whereas earlier task-specific approaches did not demonstrate the same generalization.
• Internal state is not automatically cheaper or richer. In every case I could measure, a few bytes of text did the job. Any future claim for latent communication has to beat that baseline first, on information that is actually hard to write down.
• A memory benchmark has to hide the answer key. If the query contains the information needed to find the right memory, a string match wins, and the test says nothing about memory.

Limitations

• One model pair for cross-model transfer. One model family for memory.
• Synthetic records of about 15 bits in every positive result.
• Samples of 60–80 questions per type for the main transfer result. Several earlier results used 10–25 and are not relied on here.
• No efficiency advantage was found, so none is claimed.
• The real-session data is one person's work across three repositories.
• The real-session benchmark is not valid as a test of memory (Section 4b).
• As implemented, the receiver skipped its fifth layer after the insertion point. The receiver ran 23 of its 24 layers. All conditions used the same execution path, which supports comparisons within this implementation; the effect of the skipped layer has not been independently established.
• Results come from runs recorded in a private repository. The three original maps are no longer available. A rebuilt map and its evaluation code are published separately (Methods note); the rebuilt map is a reconstruction, not one of the three maps behind the reported results. Other code, data and per-run logs are not published.

Related work

This study sits next to published work that I did not originate and do not claim:

• Activation communication between language-model agents: Ramesh & Li, "Communicating Activations Between Language Model Agents," ICML 2025 (arXiv:2501.14082).
• Passing embeddings instead of sampled tokens: Pham et al., "Let Models Speak Ciphers," ICLR 2024 (arXiv:2310.06272).
• KV-cache translation between models: Fu et al., "Cache-to-Cache," ICLR 2026 (arXiv:2510.03215).
• Using one model to read out another model's hidden states, including a larger model explaining a smaller one: Ghandeharioun et al., "Patchscopes," ICML 2024 (arXiv:2401.06102).
• Affine maps between models' residual streams: Chen et al., "Transferring Linear Features Across Language Models With Model Stitching," 2025 (arXiv:2506.06609).
• Compressing context into learned tokens:
  • Mu et al., "Learning to Compress Prompts with Gist Tokens," NeurIPS 2023 (arXiv:2304.08467);
  • Ge et al., "In-context Autoencoder for Context Compression in a Large Language Model," ICLR 2024 (arXiv:2307.06945).
• Reasoning in continuous hidden states within one model: Hao et al., "Training Large Language Models to Reason in a Continuous Latent Space" (Coconut), COLM 2025 (arXiv:2412.06769).

What may be specific to this study:

• training a cross-model map only on the receiver's early representation of a record, and testing it on held-out question types;
• chaining frozen maps over five handoffs;
• the direct cost comparison against JSON;
• the negative results.

Methods note

Every number in the figures is read from a recorded result file or a committed result document by a script that checks each value before drawing it. Where a figure combines sources, the caption says so. The string-matching, word-overlap and random baselines in Section 4b were recomputed from the same task construction used in the original run, with no model involved.

Appendix