Products / Certified Inference
It makes an expert's sign-off on an AI system stop expiring silently.
Someone qualified validates an AI-assisted decision and signs off. Then a model tag is re-pulled. A driver updates. Hardware is replaced. The system now gives a different answer to the same question — and nobody is told. The sign-off is still on file, still being cited, and no longer describes what the machine does.
The failure nothing else catches
That is the whole difficulty. A system that has drifted is perfectly self-consistent — ask it the same question a thousand times and you get the same new answer every time. Health checks pass. Smoke tests pass. Monitoring is green. There is nothing wrong with the moment, so nothing that measures the moment can see it.
The only way to catch it is to have written down, at sign-off time, exactly what was approved — and then to check today's bytes against that record at the moment of use.
So there are two guards. One proves the environment is stable right now. The other proves it has not quietly become stable at a different answer since the day someone approved it. The second guard is the contribution: it is invisible to every check that runs in the present tense.
Each block is one run that was checked and kept. The dot between two blocks is the previous block's hash, written into the next one — change any line and the rest stop matching. The right-hand end is the number you keep somewhere else, so the whole file can't be quietly rewritten.
The first run of a given question that passed its checks. Its answer is now the answer that question is supposed to give.
The same question asked again today, on the same machine, coming back different. The chain notices, and the system refuses to answer rather than hand you the new one.
This is weld procedure qualification for AI. Qualify the procedure, verify the production work conforms to it, and keep a tamper-evident record that survives an audit. That discipline is a century old in heavy industry. It does not exist in this one.
The mechanism
An armed call runs the identical frozen request three times. Two of the three exist only to make the third trustworthy.
An earlier version memoised it — skipping the warm-up when the same prompt had already been warmed in-process. An adversarial probe broke that: after the server evicts the model, the memo-skipped call takes the cold path and produces different bytes. Server-side cache state is not observable from the client, so the warm-up now runs on every frozen call, forever. It is the single largest cost in the design, and it is deliberate.
It was added after a one-off divergence under multi-model VRAM pressure that controlled probes could not reproduce. Since the mechanism could not be pinned down, the design was changed so that unreproduced bytes are never returned at all. When you cannot explain a failure, you make the system refuse rather than assume.
The obvious objection
Necessary, and not sufficient. That gap is the entire reason this exists, and it was found by measurement rather than argument.
Across three unrelated model families, two inputs and five runs each, frozen sampling was byte-identical in five of six cells. The sixth failed in a specific, reproducible shape: run 1 diverged and runs 2–5 were identical.
The cause is prompt-cache state. The first evaluation after a cache miss takes a different numeric path — a different batch layout in the reduction kernels — than a cache-warm evaluation of the same prompt. Both are internally consistent. Neither is wrong. They differ in the last bits, and greedy decoding turns that into different tokens.
This is not a proprietary discovery — llama.cpp's own documentation notes that logits are not guaranteed bit-identical across batch sizes, and that prompt caching can therefore produce nondeterministic results. What this does is make cache state part of the contract rather than an ambient property of server history.
This technique holds on local inference on fixed hardware. A hosted cloud API voids it — you do not control the batch layout, the driver, or the weights behind a model name.
That sounds like a limitation until you notice who has the regulatory duty: pharma, model risk management, defence programme offices. They already run models on their own hardware, for the same reasons they need this. The constraint and the buyer happen to coincide.
| Guard off | Zero overhead — one ordinary call, nothing added, sampling untouched |
|---|---|
| Guard on | Three calls instead of one, on protected calls only |
| Storage | One short record per unique request — hashes only. No prompt text and no answer text is stored |
| CPU beyond generation | Three SHA-256 hashes per protected call — request key, output, and the record's own chain hash when a new record is written; unmeasurable against token generation |
| Dependencies | None |
| Source size | About 1,200 lines across seven TypeScript files — meant to be read in an afternoon by the person who has to defend it |
Verification
The battery runs against a real local model. It includes a control that is required to diverge — if the harness cannot detect nondeterminism when nondeterminism is present, it has no business certifying anything.
Each probe is an attack, not a demonstration. Cache eviction unloads the model between calls and demands the same bytes. Interleave runs other work in between to disturb server state. Replay across processes checks that a fresh process reproduces a previous one's output exactly.
Every armed run writes an evidence bundle — full raw outputs, full hashes, and provenance including model, server version, seed and platform — so a reviewer checks the artefact rather than the claim.
| Contract tests | 50 / 50 passing. Covering the three-call contract, no-memoisation, concurrency serialisation, torn-write tolerance, and refusal paths |
|---|---|
| Falsification battery | 6 / 6 held against a live local model, control included |
| Demonstration | Five acts, ~1 second, no GPU and no model required — including the guard-off control that shows the same drift passing silently |
Who buys this
The buyer is whoever is accountable when it goes wrong and currently cannot discharge that accountability. Each duty below is a standing obligation under a named instrument — and none of those instruments names this technique. The boundary note follows.
Under FDA 21 CFR Part 11 and EU GMP Annex 11 the duty is not one-time qualification but maintaining the validated state — change control and periodic review confirming the system still performs as validated. Done by hand today; not at all for language models.
SR 11-7 / OCC 2011-12 requires ongoing monitoring to confirm the model in production is the model that was validated. Today that confirmation is an assertion.
An authority to operate is granted against a described system, carrying a standing continuous-monitoring duty. A programme office certified a model — is the fielded system still that model? Nobody can currently answer.
Automatic record-keeping across the system lifetime, post-market monitoring, and technical documentation kept current. Providers must be able to evidence what the system did, not assert it.
No regulation mandates byte-for-byte reproducibility checking. These instruments impose an outcome duty — the validated system must still be the running system, and you must be able to evidence it. This is one way to discharge that duty. It is not the only one, it is not a compliance certification, and nothing here is legal advice.
Prior art
This is the section a reviewer would otherwise have to assemble themselves, so we assemble it for them. Nothing below is a concession extracted under questioning — it is the same list we hand to technical reviewers before they start.
| Hash-chained tamper-evident logs | Date to Haber & Stornetta (1991), and are routine today in Certificate Transparency and in git. We did not invent the chain. |
|---|---|
| Recording an expected output and byte-comparing on replay | That is snapshot testing. It is decades old and in every major test framework. |
temperature: 0 plus a fixed seed |
Universal practice — and its insufficiency is documented in llama.cpp's own server README, not discovered here. What we add is making prompt-cache state part of the contract rather than an ambient property of server history. |
| The combination, commercially | Occupied. At least one funded company ships hash-chained AI decision records with bit-for-bit replay, aimed at the same regulated buyers. We are not first to the idea and do not claim to be. |
A constraint, not a mechanism. This runs with zero runtime dependencies, zero network calls and no external timestamp authority — so it works inside an air-gapped facility, which is precisely where the buyer with the regulatory duty already keeps their models. That is a narrower claim than novelty and it is the one we can defend.
The question we would most like a reviewer's answer to, and we ask it in the review package too: is the cross-time guard doing something a competent team could not get from disciplined snapshot testing plus an append-only log — and if not, does that matter for the buyer? A negative answer is a useful answer.
Boundaries
Stated as prominently as the claims, because a compliance buyer is going to test these edges and should hear them from us first.
It does not claim any output is right. Whether the answer was correct is the expert's judgement and stays the expert's judgement. This makes their sign-off durable; it does not make it for them.
It does not prevent drift. It detects drift and makes it fail loudly at the moment of use, rather than quietly at a deposition.
It is tamper-evident. Editing a record breaks the chain and every later call refuses to run. An attacker with full write access can rewrite and re-chain the file — which is why the chain head hash belongs in an archive off the machine.
Everything else on this site is one architecture seen from different angles — the address space, instantiated six ways. Byte for Byte is not. It shares no substrate with them and depends on none of them.
It is a standalone module that travels onto someone else's machine and wraps whatever model they already run. That independence is the point: it can be sold, audited and deployed without buying into any of our other ideas.