Machine learning has shown considerable promise for online failure prediction, detecting imminent system failures before they cause outages. But how do we know which models actually work, and under what conditions? This seminar, presented at UNC Charlotte shortly after joining the institution, examined the benchmarking challenge at the heart of failure prediction research.

The talk surveyed existing ML-based failure prediction approaches and identified the key methodological gaps: the lack of standard datasets, incomparable evaluation metrics, and experimental designs that preclude head-to-head comparison. It proposed a benchmarking framework that addresses these gaps, enabling reproducible and meaningful comparison of failure prediction models across different system types and operational profiles.