Reconciling Legal and Technical Approaches to Algorithmic Bias

Alice Xiang

Tennessee Law Review



In recent years, there has been a proliferation of papers in the algorithmic fairness literature proposing various technical definitions of algorithmic bias and methods to mitigate bias. Whether these algorithmic bias mitigation methods would be permissible from a legal perspective is a complex but increasingly pressing question at a time when there are growing concerns about the potential for algorithmic decision-making to exacerbate societal inequities. In particular, there is a tension around the use of protected class variables: most algorithmic bias mitigation techniques utilize these variables or proxies, but anti-discrimination doctrine has a strong preference for decisions that are blind to them. This Article analyzes the extent to which technical approaches to algorithmic bias are compatible with U.S. anti-discrimination law and recommends a path toward greater compatibility. This question is vital to address because a lack of legal compatibility creates the possibility that biased algorithms might be considered legally permissible while approaches designed to correct for bias might be considered illegally discriminatory. For example, a recent proposed rule from the Department of Housing and Urban Development (“HUD”), which would have established the first instance of a U.S. regulatory definition for algorithmic discrimination, would have created a safe harbor from disparate impact liability for housing-related algorithms that do not use protected class variables or close proxies. An abundance of recent scholarship has shown, however, that simply removing protected class variables and close proxies does little to ensure that the algorithm will not be biased. In fact, this approach, known as “fairness through unawareness” in the machine learning community, is widely considered naive. While the language around algorithms was removed in the final rule, this focus on the visibility of protected attributes in decision-making is central in U.S. anti-discrimination law. Causal inference provides a potential way to reconcile algorithmic fairness techniques with anti-discrimination law. In U.S. law, discrimination is generally thought of as making decisions “because of” a protected class variable. In fact, in Texas Department of Housing and Community Affairs v. Inclusive Communities Project, Inc., the case that motivated the HUD proposed rule, the Court required a “causal connection” between the decision-making process and the disproportionate outcomes. Instead of examining whether protected class variables appear in the algorithm, causal inference would allow for techniques that use protected class variables with the intent of negating causal relationships in the data tied with race. While moving from correlation to causation is challenging—particularly in machine learning, where leveraging correlations to make accurate predictions is typically the goal—doing so offers a way to reconcile technical feasibility and legal precedence while providing protections against algorithmic bias.

Related Publications

Ethical Considerations for Responsible Data Curation

NeurIPS, 2023
Jerone Andrews, Dora Zhao, William Thong, Apostolos Modas, Orestis Papakyriakopoulos, Alice Xiang

Human-centric computer vision (HCCV) data curation practices often neglect privacy and bias concerns, leading to dataset retractions and unfair models. HCCV datasets constructed through nonconsensual web scraping lack crucial metadata for comprehensive fairness and robustnes…

Beyond Skin Tone: A Multidimensional Measure of Apparent Skin Color

ICCV, 2023
William Thong, Przemyslaw Joniak*, Alice Xiang

This paper strives to measure apparent skin color in computer vision, beyond a unidimensional scale on skin tone. In their seminal paper Gender Shades, Buolamwini and Gebru have shown how gender classification systems can be biased against women with darker skin tones. While…

Flickr Africa: Examining Geo-Diversity in Large-Scale, Human-Centric Visual Data

AIES, 2023
Keziah Naggita*, Julienne LaChance, Alice Xiang

Biases in large-scale image datasets are known to influence the performance of computer vision models as a function of geographic context. To investigate the limitations of standard Internet data collection methods in low- and middle-income countries, we analyze human-centri…

  • HOME
  • Publications
  • Reconciling Legal and Technical Approaches to Algorithmic Bias


Shape the Future of AI with Sony AI

We want to hear from those of you who have a strong desire
to shape the future of AI.