Small models belong at the front door

The first model to read a query should be the smallest model trusted to decide where that query goes. Put that decision on the user’s machine and routing becomes a control point for cost, latency, and governance before a large remote model sees the prompt. That benchmark evidence makes this architecture credible. It does not make the router ready to run unattended.

Front-door routing gives one local decision three possible consequences: handle the request locally, send it through a governed proxy, or escalate it to a large model. Economics improve when expensive capacity is reserved for work that needs it. Latency improves only when classification overhead stays below the time it saves. Privacy improves structurally because the first reader can remain under the user’s control.

This classifier still has to be right. A fast local mistake can select an inadequate model, cross a policy boundary, or trigger a costly retry. The front door deserves stricter validation than a convenience feature because every later path inherits its choice.

The benchmark supports a candidate, not a verdict

Johnson and Lee tested zero-shot task classification on a fixed corpus under matched serving conditions, then ran a synthetic-traffic experiment. Their front-door routing benchmark reports the following comparison. Accuracy is exact match, and latency is measured at the classifier rather than across the downstream request.

Qwen2.5-3B is the useful operating point here. It reached 0.783 exact-match accuracy in the harmonized study and 0.793 accuracy with 988 ms median latency in the synthetic-traffic study. DeepSeek-V3 produced the highest accuracy at 0.830, yet its 2,295 ms P95 latency failed the pre-registered gate. More model bought some accuracy and lost the operating constraint.

The negative result matters most: no tested model satisfied the standalone viability rule of at least 0.85 accuracy and less than 2,000 ms P95 latency. That paper identifies targeted prompt work, taxonomy revision, and supervised fine-tuning as remediation paths. Until one of those paths clears a representative gate, the safe architecture needs abstention, fallback, and a record of why each route was chosen.

The paper also reports Phi-4-mini at 0.518 accuracy, below the earlier Phi generation in a cross-study comparison. Its authors explicitly warn that model version, tokenizer differences, prompt sensitivity, and instruction tuning are confounded. Release sequence is not a routing policy.

Microsoft and Alibaba appear here as the providers of benchmarked model families. Google appears as the publisher of the Gemma material below. These companies are neither Muniment customers nor endorsers; their public work is evidence, not approval.

Adaptation changes what the front door can handle

MERA adapts a small model from execution-verified teacher examples and recurring procedures, then evaluates routing with verification and fallback. A baseline is the method used for comparison.

On 582 held-out HumanEval and MBPP code tasks across three seeds, four adaptation cycles raised Qwen2.5-Coder-1.5B from 28.7% to 49.7% pass. The baseline was the unadapted model, and the adapted method combined its SkillBook with supervised fine-tuning and GRPO.

On the same held-out code tasks, the final-cycle policy used a calibrated router, executable tests, and GPT-5.6 Luna fallback. It reached 88.3% pass at 60.8% of the always-Luna cost, with a 1:10 small-to-large cost ratio. The baseline always used Luna.

The TAU-2 test used the fixed split, official environment outcome, one parser, and one user simulator. It disabled the SkillBook, routing, and fallback. On that setup, fine-tuned Qwen3.5-2B moved from 14 of 35 to 18 of 35 and matched an unadapted 4B model. Its baseline was Qwen3.5-2B without the adapter, and the paper calls this comparison underpowered.

The authors report benchmark results, not a production deployment. They are neither Muniment customers nor endorsers.

A router inherits its inference stack

Model weights are only half of a local classifier. The serving implementation determines whether those weights execute as designed.

A llama.cpp issue for Gemma 4 E2B and E4B describes a particularly dangerous failure mode: the loader reads the Per-Layer Embeddings metadata while the forward graph omits the residual signal from decoder layers. Those models continue to run without crashing, with subtly degraded output quality. The same issue raises quantization-sensitivity questions for the PLE weights.

Silent degradation is worse than a clean refusal at the front door. A crash trips a fallback. A plausible classification from an incomplete graph can quietly send work down the wrong path. Router qualification therefore binds the model artifact, quantization, inference engine, hardware backend, prompt, taxonomy, and test corpus into one versioned unit. Change any member and the evidence expires until the unit is tested again.

That requirement will matter as the on-device field expands. Google introduced Gemma 4 under the Apache 2.0 license in E2B, E4B, 26B mixture-of-experts, and 31B dense sizes. More capable edge models widen the candidate set. They also widen the matrix of runtimes, conversions, quantization choices, and hardware paths that can invalidate an otherwise persuasive score.

The first reader sets the boundary

Muniment’s execution surfaces are being built around a required on-device model performing this routing role. Owners will define the map, while each query will meet a local classifier before an eligible model or proxy. That direction makes governance an input to routing rather than a review performed after disclosure.

The evidence sets the bar for that direction. That deciding model needs a workload-specific evaluation, explicit latency and accuracy gates, bounded fallback behavior, and a durable route receipt. Our systems framework for routing models sets that evaluation against completed-task cost, response time, reliability, and serving state. It also needs conformance tests for the exact inference stack delivered to users. A small model at the front door earns its place by deciding under policy and showing its work. Size alone grants no authority.