The Tidepool Test
Last week I proved that my AI system could learn new knowledge into its own weights. The proof was wrong, and the way it was wrong is more useful than the proof would have been.
The two-store bet
The system is called Cephra Prime, and it is an experiment in making a language model learn the way brains are thought to: two stores instead of one. New facts land first in a fast episodic store, a pattern memory that can recall them exactly the way you can repeat back a phone number someone just told you. Then, during a scheduled “sleep” phase, eligible facts are consolidated into the model’s weights with a small LoRA adapter, trained on the new material interleaved with replayed samples of everything consolidated before. The replay is the load‑bearing part. Neuroscience calls this complementary learning systems.[1] The interleaving is the brain’s answer to catastrophic forgetting, the failure mode where training a network on new facts silently destroys the old ones.
Every sleep cycle ends at a regression gate. Before a freshly trained adapter is allowed to serve traffic, it must answer three kinds of questions: the new facts it just learned, canary questions drawn from previously consolidated knowledge, and generic anchors (arithmetic, capitals, basic science) that check the model can still function at all. Fail any threshold and the adapter is rejected; the facts stay episodic and nothing is lost.
The first two cycles sailed through. New facts 100 percent. Canaries 100 percent, meaning cycle one’s knowledge survived cycle two’s training untouched. That’s the whole design vindicated in a single number. And then the moment that felt like the finish line: I asked the live system a paraphrase of a taught fact, phrased so differently that the episodic store’s similarity matching could not fire. The recall layer reported no match. The model answered correctly anyway. Knowledge served from weights, not lookup.
I believed that result for about a day.
A fact the model already knows proves nothing
The facts I had taught it were things like photonic computing and morphogenetic fields, real concepts chosen carelessly, every one of them present in the base model’s pretraining. When a system built on top of a nine‑billion‑parameter model answers a question about embryology, you have not demonstrated learning. You have demonstrated that the base model went to school.
A fact the model already knows can only prove your pipeline runs. It cannot prove your system learns. The only decisive test is a fact the model cannot know: something invented, private, or newer than its training data. Then there is exactly one place a correct answer can come from.
So I invented one. The Tidepool Protocol: a fictional backup rotation scheme for a database cluster, defined by three related facts. It rotates snapshots on a nine‑day cycle. Its third storage tier is called the reef, where snapshots settle for long‑term keeping. Three sentences that exist nowhere in any training corpus, because I made them up that morning.
Confabulation is the control
Before teaching it anything, I asked the untouched system what the Tidepool Protocol was. This is the control arm of the experiment, and it produced the most instructive garbage I have seen all year.
Asked once, the model confidently described a decentralized data‑ownership protocol with DAOs, zero‑knowledge proofs, and a helpful link to a website that does not say what it claimed. Asked from a different angle, it became a liquid‑staking protocol on Ethereum. Asked who created it, the model attributed it to Linus Torvalds, citing a blog post from 2010 that was never written.
Three different confident answers to the same question, and no hedging in any of them. That raised the stakes of the test in a way I hadn’t planned. The training would not be filling a vacuum; it would be fighting an incumbent, a model with a strong prior for inventing plausible protocols on demand.
The gate passed and the product failed
I taught the system the three facts through its normal question‑and‑answer flow and triggered a sleep cycle. Ninety seconds of training. The gate came back green: new facts 100 percent, canaries 100 percent, adapter accepted and hot‑swapped into the live server.
Then I asked the live system my paraphrased questions, and it told me the Tidepool Protocol belonged to Google.
Worse: even the exact question from the training data failed against the live server, while the gate’s evaluation of the very same adapter had reproduced the taught answer verbatim. Same adapter, same weights, opposite results. That contradiction turned out to be useful, because it could only mean one thing: the gate and the server were not running the same model.
They weren’t. The regression gate loads the adapter directly, in‑process, through the framework’s Python API. The live server is a separate HTTP process, launched with a command‑line flag pointing at the adapter file. And that flag, it turns out, is silently ignored: the server’s model‑loading code remaps the requested model name before looking up which adapter to apply, so the lookup misses every time and the request is served by the bare base model.[2] No error. No warning. The adapter my gate had so carefully validated had never served a single request.
Which means my result from the week before, the paraphrase answered “from weights,” was the base model answering from pretraining. The evaluation harness and the production path shared everything except the one line that mattered, and every fact I had chosen was incapable of telling them apart. The gate proved the gate.
The invented fact caught it instantly, because the invented fact is the only probe with no alternative source of a right answer. A confabulated “Google” cannot be blamed on ambiguity. It is a bright red wire sticking out of the system, and you can follow it straight down to the fault.
What a real pass looks like
The fix was small: send the adapter path with every request instead of trusting the startup flag. The retest was the experiment I thought I had already run.
Fresh conversation, no shared history, recall layer confirmed silent on every question. Whose backup scheme is the tidepool protocol? Associated with the mneme cluster. How many days is one full rotation? Nine. Where do old snapshots end up? They settle into the reef.
Then the question I cared about most, because it has no training‑data answer at all: a snapshot was taken twelve days ago. Is it still in the active rotation? The model said no, because the protocol rotates on a nine‑day cycle and retires the oldest snapshot. Nobody taught it that. It combined two separately taught facts and did the comparison. That is the difference between storing sentences and holding knowledge, and it is the first result from this project I am prepared to call learning.
That claim rests on “recall layer confirmed silent,” and on reflection that phrase doesn’t carry the weight I first put on it. It only rules out an exact match. A quieter path exists too, one that can splice a near match into the model’s context without ever registering as a recall at all. I went back and ran the actual lookup against the real memory for these exact questions, and nothing crossed even the loose threshold that path uses. The composition answer carries a second guarantee on top of that: the first attempt, asked inside a normal chat session, tripped an unrelated web search and came back full of junk, so the answer above came from asking the model server directly, with no episodic store anywhere in the path to begin with.
The before‑and‑after is stark. Three confident, mutually contradictory confabulations, overwritten by three taught sentences and ninety seconds of sleep.
The honest ledger
Two entries on the other side, because a result you only report the good half of is a confabulation of your own.
The general‑capability anchors are eroding: 92 percent after cycle two, 75 percent after cycle three, with some visibly degraded outputs among the failures. The knowledge consolidates cleanly (canaries have never dropped a point), but each cycle of aggressive training on a tiny dataset sands something off the model’s general behavior. That is now the constraint to engineer against, and I would rather know it at cycle three than at cycle thirty.
There’s a deeper lesson too, and it isn’t about one framework’s flag handling. Every evaluation harness drifts toward convenience: load the model directly, skip the network hop, test the artifact instead of the path. Each shortcut is individually reasonable and collectively fatal, because the bugs that matter live precisely in the seams the shortcuts skip. The fix is cheap and general: keep a fabricated fact in the system at all times and probe for it end to end, through the same door your users walk through. If the system stops knowing a thing that has exactly one source, some seam has opened. Real facts can’t do this job. Only an invented one is unambiguous about where its answer comes from.
The industry is converging on the underlying goal from other directions. MIT’s SEAL, published last year, teaches models to generate their own study material and fine‑tune on it, and hits the same catastrophic‑forgetting wall that replay exists to solve.[3] The training side of this problem is getting crowded. The verification side is not, and I suspect the verification side is where the production failures will actually come from.
Teach your system something that isn’t true. It is the only way to be sure of what it learned.
Views are my own. The system described is a personal research project; results are from single runs on a small model and should be read accordingly.
Sources
- McClelland, McNaughton & O’Reilly, “Why there are complementary learning systems in the hippocampus and neocortex,” Psychological Review, 1995.
mlx-lmserver,ModelProvider.load: the requested model name is remapped before the adapter‑map lookup, so the--adapter-pathCLI argument is never applied to HTTP requests.- “Teaching large language models to absorb new knowledge,” MIT News, November 2025.