Cloud versus local model cost is a workload equation
Cache reuse, hardware utilization, correction labor, and task quality decide whether cloud or local model execution costs less.
Cloud versus local model cost has to be calculated from the workload, never inherited from a headline. The evidence here comes from one developer in non-randomized, consecutive 28-day periods on one production monorepo. Each period used different model families, agent harnesses, and serving configurations. The paper applies Taiwan-market operating assumptions and an author-defined labor model. That design can expose cost variables. It cannot appoint a universal winner.
The study compared an API configuration using Claude Opus 4.7 and 4.8 through Claude Code with an on-premise configuration using NVFP4-quantized GLM-5.1 and 5.2 through Opencode on NVIDIA Blackwell hardware. On-premise means hardware operated by the organization. Prompt caching means reusing input that a model has already processed, so repeated prompt prefixes do not require the same full-price computation again.
One workload produced two different cost answers
The authors report the following results for their measured workload and cost model in the July 13 preprint:
| Workload measure | Reported result | What it depends on |
|---|---|---|
| API prompt-cache hit rate | 99.3% | Repeated prompt prefixes in this agent loop |
| Effective API cost | $0.57 per million tokens, 88.6% below the uncached calculation | Observed token mix and public list rates |
| Shared local allocation | 40.1% lower True TCO than the cached API period | Shared GPU attribution and modeled labor |
| Dedicated local reservation | 43.8% higher True TCO than the cached API period | A full reserved tray and modeled labor |
| Fix Commit Ratio | 74.9% local versus 45.9% API | Keyword-classified repair commits |
Those rows refuse to collapse into a slogan. Cache reuse made the API unit cost unusually low, while shared allocation made local total cost lower. Dedicated reservation reversed that result. Meanwhile, the local period carried more repair commits.
Fix Commit Ratio is the share of commits whose messages matched the paper’s repair keywords. It is a useful correction-burden proxy, though it is not an independent defect oracle. The authors also state that the measured difference is the joint effect of the model, quantization, serving stack, agent harness, and fixed period order. Reading the ratio as an isolated model score would claim more than the study measured.
Put four terms into the evaluation
An operator can turn the paper into a prospective test with four questions:
- How much input repeats? Measure cache-eligible tokens, actual cache hits, cache-write cost, cache-read cost, and invalidations on representative sessions. Apply the observed rates to the whole token mix. A nominal input price misses the workload’s most consequential discount when long prefixes recur.
- How busy will the hardware stay? Price both shared and dedicated allocation. Include idle reserved capacity, power, cooling, operations, and the number of workloads that can safely share the serving system. A local unit cost derived from full utilization is fiction when the tray spends most of its time waiting.
- What does correction consume? Track retries, failed checks, repair commits, review time, and debugging loops by task class. Price that labor with the organization’s own fully loaded rate. Also keep failed and abandoned runs in the record; zero accepted output can still incur compute and correction work.
- Does the output meet the task bar? Compare configurations on the same task mix and acceptance tests. Segment by difficulty, language, repository area, and reviewer experience. Cost per accepted change is more useful than cost per token when one configuration needs more human repair.
Run the comparison concurrently or as a randomized crossover when practical. Consecutive periods can inherit changes in task mix, repository state, and operator familiarity. Use the same harness where the model interfaces permit it, or name the harness as part of the configuration being evaluated. A stale price sheet, an unmeasured cache policy, or a utilization estimate copied from another team can overturn the result before model quality enters the equation.
Claude, Anthropic, GLM, Opencode, and NVIDIA appear here as components and sources in the public study. None is presented as a Muniment customer or endorser.
The useful output of a cloud-versus-local trial is a boundary: the cache rate, utilization, correction burden, and quality threshold at which the decision changes. A winner without that boundary is just a workload assumption wearing a medal.