Assay — catching model substitution, one receipt at a time

An inference provider promises model M (gpt2) and quietly serves a cheaper M' (distilgpt2, or an INT8 copy). Every response carries a signed sketch of its own hidden_states — an internal receipt — and a verifier recomputes a small, secretly-chosen slice of it. Coverage is 100% by construction: there is no request the audit happened to miss.

Code · Receipt corpus · Model bundle

Provider behaviour

Run

Generation-consistency check (25% of requests)

Per-request verdicts

“Served by” is ground truth — a real auditor never sees it. Shaded rows are the ones that drew the generation-consistency check.

Replayed from 720 published receipts (gpt2 vs distilgpt2, CPU). Turning the deep check off replays the same receipts scoring only the sketch statistic, which is exactly recoverable from each row's z. Research prototype: the TEE is simulated with an HMAC signature, and the sequential audit's α is stated for exact null p-values while the calibration here is estimated (measured false-alarm rate: 1 in 40 honest runs against a nominal 1%).