Keynote at SAFECOMP 2019
Trustworthiness Benchmarking of (Safety) Critical Systems
Safety-critical systems, from avionics to medical devices to autonomous vehicles, demand a higher standard of assurance than conventional software. Yet the benchmarking tools and methodologies developed for performance or even dependability often fall short when trustworthiness must be demonstrated, not merely claimed.
This keynote examines what trustworthiness benchmarking for safety-critical systems must look like: the metrics that matter, the workloads that are representative, the fault models that are credible, and the process by which a benchmark result can be trusted as evidence of real-world safety. It argues that the safety community urgently needs a principled benchmarking discipline and sketches what building one would require.