Audit guarantees belong in the harness
A public-data evaluation shows why deterministic contracts and their audit artifacts should surround a replaceable model boundary.
Audit guarantees should live in versioned code around a replaceable model boundary. Prompts can request compliant behavior. A harness is the surrounding software that supplies tools and rules, records what happened, and checks the result. It can enforce source boundaries, routing, output schemas, and trace hygiene, then leave an artifact when the model gets them wrong. That difference gives operators something durable to review.
A contract pass is not a perfect model output
The authors tested a public-data slice spanning five Korean corporate groups and 25 listed companies. Across 270 composition-boundary runs using three hosted models, every harness-enforced contract held under model substitution.
That result does not mean every model output passed. The paper reports model-composed failures, which validators caught and recorded before the output crossed the contract boundary. The guarantee belongs to the enforcement path: deterministic code rejects or transforms a violation and preserves evidence of what happened. A model-side pass is one possible event inside that path, not the definition of success.
For an operator, the durable boundary is concrete: code defines the checks, manifests bind allowed sources and entities, and schemas define acceptable output. An audit trace records the sources, decisions, and validation results behind an answer. Any hosted model can then be replaced without asking a new prompt to inherit the old model’s institutional memory.
Prompting asks; the harness decides
The ablation holds the model fixed and varies the enforcement layer across 120 runs per condition: 40 scenarios repeated three times. Utility counts runs answered rather than blocked. It is separate from contract compliance, which is why prompt-only output can retain utility while admitting violations. These values belong to the fixed public-data study rather than to production traffic.
| Enforcement layer | Reported contract outcome | Reported utility |
|---|---|---|
| Prompt only | Recommendation-language and internal-trace-leakage violations reached the reader | 120/120 |
| External guardrail | Blocked those violations but over-refused | 88/120 |
| Code-owned harness | Blocked those violations while preserving the study's full utility | 120/120 |
The external guardrail shows that blocking is only half the job. Its 88/120 utility score trails the harness’s 120/120 because enforcement detached from application semantics can refuse useful work. Code owned with the application can validate the specific contract and retain the response that satisfies it.
Keep the evidence boundary narrow
This preprint is the authors’ public-data evaluation, not independent proof of production performance or universal generalization. Its fixed slice does not establish that the same contracts, utility, or failure rates will transfer to another workload. Passing the specified contracts does not establish the correctness of the underlying investment claims or the investment quality of the briefings. The study does establish a testable engineering pattern: keep deterministic guarantees and audit artifacts in the versioned harness, keep source-backed claims authoritative, and keep the model replaceable.
The paper’s authors, the model providers, and the studied companies are neither Muniment customers nor endorsers.