Designing for dispute resolution

Talk at Workshop on Security and Human Behaviour (SHB 2021), 3–4 June 2021.

No computer system is perfect, so some people may dispute a computer’s decision (whether or not it is actually correct), and the system may be required to help resolve this disagreement. This requirement is particularly important when the computer system tracks who owns what, whether money or goods. The state of the art for reliably tracking account balances is double-entry bookkeeping, developed in the 13th century and translated more or less unchanged to computers. Double-entry accounting can identify losses but is less good at identifying the cause. As shown in the Post Office trial, companies may tend to hold the weaker party liable for such losses, and there’s not much they can do to show otherwise. In this talk, I argue that we need to move away from tracking balances and track items – which would be inconceivable with 13th-century technology but tractable with modern computing. This is one approach to addressing the challenges of evidence-critical systems, which must produce accurate and interpretable information to resolve disputes.

See also the slides for this talk.