Evidence-critical systems: what they are and why we need them

Talk at Workshop on Security and Human Behaviour (SHB 2020), 18–19 June 2020.

It may be impossible (or undesirable) to programmatically enforce all relevant security policies. In which case we can replace enforcement with transparency (to detect violation), provision of redress to the victim (to ameliorate the harm of the violation), and punishment for the violator (to deter future violations). Achieving the latter two properties requires evidence of a violation and a system for turning evidence into justice. In this talk, I discuss that we need to create evidence-critical systems that provide assurance that justice can be obtained. The design of evidence-critical systems can draw from the well-established field of safety-critical systems but has several significant differences.

See also the slides for this talk.