The result we almost published — and why we didn't

Every validation program produces results that look better than they are. This finding is about one of ours: a detection that appeared, on first read, to be a clean early warning on a production tool — and the checks that convinced us not to publish it as one.

The number was real. The question was what it established. The detection preceded the failure, but the run had a property that weakened the claim: the model had seen operating conditions during training that overlapped with the pre-failure window. Not labels — no failure was ever marked — but overlap enough that "caught it early" and "recognized something it had partially seen" could not be fully separated.

We publish results labeled by what they do and do not establish. A blind run-to-failure result establishes early detection. A result with training overlap establishes sensitivity, not foresight. The difference matters to anyone evaluating predictive maintenance claims — including ours.

This article will walk through the run, the check that caught the overlap, and the re-run protocol that produced the number we did publish. [Full finding in preparation — Ali Taheri.]