* External authors




Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty

Umang Bhatt*

Javier Antorán*

Yunfeng Zhang*

Q. Vera Liao*

Prasanna Sattigeri*

Riccardo Fogliato*

Gabrielle Melançon*

Ranganath Krishnan*

Jason Stanley*

Omesh Tickoo*

Lama Nachman*

Rumi Chunara*


* External authors




Algorithmic transparency entails exposing system properties to various stakeholders for purposes that include understanding, improving, and contesting predictions. Until now, most research into algorithmic transparency has predominantly focused on explainability. Explainability attempts to provide reasons for a machine learning model’s behavior to stakeholders. However, understanding a model’s specific behavior alone might not be enough for stakeholders to gauge whether the model is wrong or lacks sufficient knowledge to solve the task at hand. In this paper, we argue for considering a complementary form of transparency by estimating and communicating the uncertainty associated with model predictions. First, we discuss methods for assessing uncertainty. Then, we characterize how uncertainty can be used to mitigate model unfairness, augment decision-making, and build trustworthy systems. Finally, we outline methods for displaying uncertainty to stakeholders and recommend how to collect information required for incorporating uncertainty into existing ML pipelines. This work constitutes an interdisciplinary review drawn from literature spanning machine learning, visualization/HCI, design, decision-making, and fairness. We aim to encourage researchers and practitioners to measure,
communicate, and use uncertainty as a form of transparency.

Related Publications

On the Validity of Arrest as a Proxy for Offense: Race and the Likelihood of Arrest for Violent Crimes

AIES, 2021
Riccardo Fogliato*, Alice Xiang, Zachary Lipton*, Daniel Nagin*, Alexandra Chouldechova*

The risk of re-offense is considered in decision-making at many stages of the criminal justice system, from pre-trial, to sentencing, to parole. To aid decision makers in their assessments, institutions increasingly rely on algorithmic risk assessment instruments (RAIs). The…

  • HOME
  • Publications
  • Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty


Shape the Future of AI with Sony AI

We want to hear from those of you who have a strong desire
to shape the future of AI.