Dependability benchmarking (measuring availability, reliability, safety, and related properties in a reproducible and comparable way) had matured considerably by the early 2010s, yet remained fragmented across subcommunities. This tutorial, presented at the International Conference on Quantitative Evaluation of SysTems (QEST), offered a systematic overview of the state of the art.

Covering faultload definition, representative workloads, evaluation metrics, and experimental design, the tutorial by Marco Vieira and Nuno Antunes drew on a decade of work benchmarking OLTP systems, web servers, and web services. It provided attendees with both a conceptual framework and practical guidance for designing their own dependability benchmarks.