# Mobile inference needs phase-aware scheduling

[Journal](/journal/)

July 15, 2026 · [evidence](/journal/#evidence)



A mobile inference study finds that prefill and decoding favor different hardware, making the execution phase part of the routing decision.

![An inference trace reaches a switch and branches toward three processor forms, with one route marked in verdigris.](/_astro/mobile-inference-needs-phase-aware-scheduling.BxBh3pnx_Z1MpXq5.webp)

Efficient mobile model execution needs a phase-aware plan: prefill and decoding do not have the same hardware fit. In the authors’ version 1 measurement study, [NPUs were strongest for compute-bound prefilling](https://arxiv.org/abs/2607.05475), while CPUs led the memory-bound decoding phase. For an operator, backend selection is therefore not one decision for an entire request. It is a scheduling decision that can change as the workload changes.

## The measured split

The study evaluates [five frameworks](https://arxiv.org/abs/2607.05475), llama.cpp, MLC-LLM, MLLM, MNN, and GENIE, across [three backend classes](https://arxiv.org/abs/2607.05475): CPU, GPU, and NPU. The authors attribute the phase split to workload shape. Prefill supplies the large, fixed-shape work that suits an NPU. Decoding repeatedly presents smaller, dynamic kernels and memory-bound work that favored CPUs in their measurements.

Request phase

Workload character in the study

Measured hardware fit

Operator implication

Prefill

Compute-bound, large and fixed shape

NPU strength

Evaluate an NPU route for this phase

Decoding

Memory-bound, small and dynamic kernels

CPU strength

Evaluate a CPU route for this phase

This comparison records the paper’s phase-level finding, not a universal benchmark ranking. The paper reports [framework-induced gaps of up to 10x](https://arxiv.org/abs/2607.05475) on NPUs, which makes the runtime and its offloading, operator, and quantization choices part of the evidence. A backend label alone does not describe the execution path.

## Measurements identify scheduling headroom

The authors measure [up to 40% energy waste](https://arxiv.org/abs/2607.05475) from suboptimal thread configuration, uncoordinated NPU sleep latency, and CPU polling intervals. That is a measured finding from their setup. Their response is proposed guidance: an energy-oriented configuration and scheduling approach that coordinates those controls rather than leaving each phase on an unexamined default.

Across three datasets, the authors estimate that their best-practice configuration could [reduce energy consumption by up to 54.8%](https://arxiv.org/abs/2607.05475) on the NPU backend. The [54.8% result is an estimate](https://arxiv.org/abs/2607.05475), not an independently replicated deployment result. It supports testing phase-aware configuration against a known baseline; it does not support assuming the same reduction on another device or workload.

The paper does not establish frontier-scale local inference or universal performance across devices and workloads. It is a version 1 preprint reporting the authors’ study results, not independent replication. Its useful operational claim is narrower: measure prefill and decoding separately, preserve framework and configuration in the record, and choose a route with the phase in view.

The paper’s authors, llama.cpp, MLC-LLM, MLLM, MNN, GENIE, Alibaba, and Qualcomm Technologies are neither Muniment customers nor endorsers.

## Sources

1.  [arXiv — Is Your NPU Ready for LLMs? Dissecting the Hidden Efficiency Bottlenecks in Mobile LLM Inference](https://arxiv.org/abs/2607.05475) arxiv.org

## Continue reading

[All publications](/journal/)

guides · Jul 15, 2026

### [Virtualizing live chat without losing the conversation](/journal/virtualizing-live-chat-history/)