Audit the backend from the text alone
IRIS tests which model a gateway served from returned text alone, turning the model name on a receipt into a claim operators can check.
A receipt that names a model states a checkable claim. Returned text can carry enough evidence to test which backend answered, without gateway logs.
The backend is the model that actually answered. Model substitution means a gateway serves a different model from the one it advertised. Partial rerouting means only a fraction of requests goes to the substitute.
Yuewei Zhang, Zhi-Hai Zhang, and Hanzhang Qin call their audit IRIS. It asks models for random strings, then fingerprints their visible patterns.
Four results make the claim measurable
AUROC scores how well an audit separates honest responses from substituted ones. A score of 1 means perfect separation, while 0.5 means chance performance.
On the authors’ six-model, intra-family Qwen3 ladder, IRIS verified the backend at 0.99 AUROC. That result measures claimed and substitute pairs inside one model family.
Their commercial sample contained 17 OpenRouter model interfaces. On margin-qualified pairs, IRIS caught a 0.3 routing fraction at 0.85 mean power, with a 0.017 false-positive rate.
For enrolled substitutes in that library, the audit recovered the routing fraction to within 0.04. Enrolled means the candidate library already contained the substitute’s fingerprint.
The live sample compared 15 pairs that offered the same model through different providers. IRIS flagged 14 pairs for backend differences tied to quantization and inference kernels.
On claimed and substitute pairs from the six-model ladder, adaptive allocation raised the matched-budget target-hit rate from 73% to 87%. The comparison held total query spending equal while assigning queries by measured difficulty.
Those samples matter as much as the scores. The authors measured their audit on their model sets, including commercial interfaces, rather than proving a claim about every gateway.
The query budget is a commitment, not a cost guarantee
IRIS runs a cheap pilot on labeled reference responses. It fits error decay, then fixes the budget before querying a suspect endpoint.
That order prevents a suspicious result from quietly buying more evidence until it passes. It remains the authors’ reported design choice, not a guarantee that an audit will cost little.
Low-margin pairs can remain hard to separate. The paper returns some infeasible pairs as indeterminate, which keeps a weak fingerprint from becoming a confident accusation.
A named route needs an independent check
Muniment issues a per-request record that names the route and model. That is our own first-party claim about the product, and the IRIS paper does not validate it.
The useful connection is narrower. An audit harness preserves what the system claimed, while an output audit tests whether returned text matches that claim.
Operators need both halves when the gateway sits outside their control. The receipt fixes the claim in time, and the audit supplies independent evidence about the backend.
No paper author, gateway, or provider is a Muniment customer or endorser.
Join the waitlist if per-request route records belong in your operating layer. The next uncomfortable question is how often a production fleet should challenge its own receipts.