# Routing each reasoning step needs its own receipt [Journal](/journal/) July 16, 2026 · [evidence](/journal/#evidence) Agora tests an auction for allocating individual reasoning steps. The operational lesson is that each selection needs evaluation, budget policy, and an outcome receipt. ![Four reasoning-step cards enter an auction selector, route to three expert modules, and converge on an outcome receipt.](/_astro/routing-each-reasoning-step.DpR4z2D2_Z1HfS8e.webp) Step-level model routing moves the hard control problem inside the agent loop. Every reasoning step becomes a fresh allocation decision, so a production harness has to evaluate the step, choose among eligible executors, enforce a budget, and preserve the outcome that justified the next choice. A route without that record is merely a sequence of guesses with an invoice attached. The Agora preprint tests this idea by [treating reasoning steps as auctionable task units](https://arxiv.org/abs/2607.09600) and assigning each unit according to calibrated competence minus a normalized cost penalty. Its result is useful evidence for a narrow claim: finer routing can exploit complementary model strengths when the allocator has trustworthy step-level signals. It is not evidence of production savings. Agora’s authors are neither Muniment customers nor endorsers. ## A route now contains many decisions Agora starts with a planner or a benchmark-provided decomposition, [groups tightly coupled nodes into executable units](https://arxiv.org/abs/2607.09600), and asks candidate models or tools to bid on each unit. The bid combines calibrated confidence with normalized monetary and latency costs. A global cost-sensitivity parameter, beta, changes how much the allocation penalizes cost. That mechanism turns one request-level route into a chain of policy events. For each unit, an enterprise harness needs to retain the unit definition and dependencies, the eligible candidate set, each candidate’s calibrated score and estimated cost, the policy parameter in force, the selected executor, and the observed result. Without those fields, an operator can see which model ran but cannot reconstruct why it won. The authors evaluated quality-first Agora against [matched single-model, random, routing, and cascade baselines](https://arxiv.org/abs/2607.09600) on five benchmarks: a fixed 500-example MuSiQue-Ans development subset, a fixed stratified 2,000-example MMLU-Pro subset, the full SciCode test split with background, SPIQA Test-A, and the MathVision test mini split. Candidate pools were fixed within each benchmark, though the models and metrics differed across benchmarks. Those conditions matter more than a blended victory claim. Evaluation Reported Agora result Qualification [MMLU-Pro](https://arxiv.org/abs/2607.09600) 71.9% accuracy Quality-first beta 0.001; fixed 2,000-example stratified subset; matched two-model pool; online refinement used correctness feedback [SciCode](https://arxiv.org/abs/2607.09600) 44.4% sub-problem pass rate Mean over three runs; full test split with background; matched the best single backend rather than significantly exceeding it [MathVision cost sensitivity](https://arxiv.org/abs/2607.09600) 55.26% to 51.06% accuracy As beta increased, cheaper-agent usage rose from 21.5% to 78.5% on the test mini split On MuSiQue-Ans, the paper reports [43.0 exact match and 54.3 F1 for Agora](https://arxiv.org/abs/2607.09600), compared with 42.4 and 54.1 for its replayed 1NN router. On SPIQA, it reports [65.0% average L3 and 56.9% at the strict L3 threshold](https://arxiv.org/abs/2607.09600), using the benchmark’s fixed retrieval and reasoning structure. These are benchmark results from a preprint under matched candidate pools. They do not establish a universal routing advantage or a production cost curve. ## Per-step evaluation is part of the control plane Auction language can make allocation sound self-justifying. It is not. A bid is only useful when its confidence estimate predicts success on the kind of step being assigned. The paper’s calibration ablation reports that [removing calibration lowered Agora from 71.9% to 67.5% accuracy](https://arxiv.org/abs/2607.09600) on its MMLU-Pro subset and from 56.9% to 46.9% on SPIQA’s strict L3 measure. Its SciCode discussion also reports [no significant improvement over the best single backend](https://arxiv.org/abs/2607.09600) and attributes the plateau to a calibrator trained on competitive programming rather than scientific workflows. Step-level selection therefore creates a step-level evaluation obligation. Teams need outcome labels by task class, calibration drift checks, and a conservative fallback when scores stop separating candidates. The feedback source deserves special scrutiny. Agora’s optional online refinement [updates its calibrator from binary correctness labels](https://arxiv.org/abs/2607.09600), but the experiments use ground-truth labels to establish an upper bound. The authors describe [static calibration as the deployment default](https://arxiv.org/abs/2607.09600) when reliable automatic feedback such as unit tests, extractive checks, or tool validators is unavailable. Production adaptation needs receipts tied to observable outcomes. Model self-confidence cannot quietly stand in for correctness. ## Budget policy belongs on every step In the paper’s MathVision sensitivity analysis, raising beta [shifted usage toward the cheaper candidate while reported accuracy declined](https://arxiv.org/abs/2607.09600), with the balanced beta setting trading about 0.9 percentage points for nearly twice the cheaper-model usage. That is a controlled benchmark trade, not a forecast for an enterprise workload. A production budget control needs harder edges than a single global dial. Owners need per-run ceilings, step-level cost estimates, reserved capacity for critical steps, and an escalation rule when the expected competence gap clears a recorded threshold. They also need to count the allocator itself. The paper says its two-agent settings with two to three task units [add roughly four to six short bid calls per query](https://arxiv.org/abs/2607.09600), run in parallel and separate from executor cost. A budget receipt should include those calls rather than hiding coordination overhead behind the winning executor. Agora’s authors list three practical boundaries: [calibration can fail under distribution shift](https://arxiv.org/abs/2607.09600), poor decompositions can violate the independent-unit assumption, and gains depend on meaningful complementarity among candidates. A production policy therefore needs a recorded condition for declining the auction, not merely a rule for choosing its winner. How quickly can an operator recognize that the allocator has stopped earning its place in the route? ## Sources 1. [arXiv — Agora: Enhancing LLM Agent Reasoning Via Auction-Based Task Allocation](https://arxiv.org/abs/2607.09600) arxiv.org ## Continue reading [All publications](/journal/) evidence · Jul 15, 2026 ### [Audit guarantees belong in the harness](/journal/audit-guarantees-belong-in-the-harness/)