A retry classifier read the wrong file
A retry classifier missed a recoverable device fault because it read the console log while the JUnit report held the cause.
A retry classifier can match the right error and still miss the fault when it reads the wrong artifact. Our Muniment Mobile nightly did exactly that.
The test drove a sign-in journey on an Android emulator through a UI test driver. Our runner retried a small set of recoverable faults once each.
Troubleshoot an incorrect format or retry failure
For an “incorrect format” error or retry failure, inspect the structured test report before the console log. The message alone does not identify the cause.
Our observed run did not establish a format problem. Its JUnit report named a device-server-died error and nested device offline beneath it, while the console log named neither error.
That record explains only why this retry did not run: the classifier read the wrong artifact. It does not show whether malformed input, configuration, or any other unmeasured cause explains a different failure.
Three attempts left one cause out of the console
One nightly run made three attempts. Assertions stopped the first two, and each console log named its cause.
The third console log held only the flow name and a 1m 36s duration. It gave no cause.
A JUnit report from the same attempt held the whole exception. The driver’s device-server-died error fired during a view-hierarchy read.
An adb transport error sat beneath it with device offline. Our runner pattern already matched that text exactly.
The report sat outside the classifier’s path
Each classifier call received the attempt’s console log and output directory. The JUnit report sat beside that directory rather than inside it.
Our classifier never opened the report. The lost-device retry stayed unspent, and the runner stopped after three attempts.
The job reported a red gate for a fault our runner was built to retry. Pattern correctness could not help because the pattern never saw the evidence.
Read the structured cause first
We kept three rules from this failure:
- List every artifact an attempt writes. Then choose which artifacts the classifier reads.
- Read the machine-readable report before the console log. The driver must record its stopping cause there.
- Test the classifier with a fixture where only the report holds the cause.
A path can look adjacent while remaining outside the classifier’s search. A matching pattern proves little until a test puts its only match there.
The record stops at classification
We did not measure how often the emulator drops this way. This record cannot assign any share of red nightlies to this gap.
We also did not show that a fourth attempt would have passed. The narrower claim holds.
Our classifier missed a recoverable fault because its evidence sat in a file the classifier never opened.
This evidence comes from our Muniment Mobile note pinned to commit a311ac161c4cc91a85d2d97.