What a blind run-to-failure validation actually looks like

Most predictive maintenance demonstrations are not validations. A model shown data from a failure it was tuned on will "detect" that failure with impressive lead time, every time. If you are evaluating claims — a vendor's, or your own team's — the details of the test protocol matter more than the headline number.

A blind run-to-failure validation has a specific shape: the model learns only from healthy operation, the evaluation window is held out entirely, the prediction is committed before the outcome is known, and the failure that arrives is one the model has never seen in any form.

We ran exactly this protocol on a production ion-beam process tool. This article describes the setup, what the model was and was not allowed to see, how the detection point was scored against the physical failure, and the questions we would ask of any result presented to us — a checklist you can apply to anyone's claims, including ours.

[Full finding in preparation — Ali Taheri.]