Understanding, Idealization, and Explainable AI

Episteme 19 (4):534-560 (2022)
  Copy   BIBTEX


Many AI systems that make important decisions are black boxes: how they function is opaque even to their developers. This is due to their high complexity and to the fact that they are trained rather than programmed. Efforts to alleviate the opacity of black box systems are typically discussed in terms of transparency, interpretability, and explainability. However, there is little agreement about what these key concepts mean, which makes it difficult to adjudicate the success or promise of opacity alleviation methods. I argue for a unified account of these key concepts that treats the concept of understanding as fundamental. This allows resources from the philosophy of science and the epistemology of understanding to help guide opacity alleviation efforts. A first significant benefit of this understanding account is that it defuses one of the primary, in-principle objections to post hoc explainable AI (XAI) methods. This “rationalization objection” argues that XAI methods provide mere rationalizations rather than genuine explanations. This is because XAI methods involve using a separate “explanation” system to approximate the original black box system. These explanation systems function in a completely different way than the original system, yet XAI methods make inferences about the original system based on the behavior of the explanation system. I argue that, if we conceive of XAI methods as idealized scientific models, this rationalization worry is dissolved. Idealized scientific models misrepresent their target phenomena, yet are capable of providing significant and genuine understanding of their targets.



    Upload a copy of this work     Papers currently archived: 91,122

External links

Setup an account with your affiliations in order to access resources via your University's proxy server

Through your library

Similar books and articles

Idealization and Many Aims.Angela Potochnik - 2020 - Philosophy of Science 87 (5):933-943.
Idealization.Alkistis Elliott-Graves & Michael Weisberg - 2014 - Philosophy Compass 9 (3):176-185.
Idealization.Michael Weisberg Alkistis Elliott‐Graves - 2014 - Philosophy Compass 9 (3):176-185.
Idealization and the Aims of Science.Angela Potochnik - 2017 - Chicago: University of Chicago Press.
The virtues of interpretable medical artificial intelligence.Joshua Hatherley, Robert Sparrow & Mark Howard - forthcoming - Cambridge Quarterly of Healthcare Ethics:1-10.


Added to PP

303 (#62,791)

6 months
95 (#39,162)

Historical graph of downloads
How can I increase my downloads?

Author's Profile

Will Fleisher
Georgetown University

References found in this work

Models in Science (2nd edition).Roman Frigg & Stephan Hartmann - 2021 - The Stanford Encyclopedia of Philosophy.
Understanding from Machine Learning Models.Emily Sullivan - 2022 - British Journal for the Philosophy of Science 73 (1):109-133.
Explanatory unification.Philip Kitcher - 1981 - Philosophy of Science 48 (4):507-531.
No understanding without explanation.Michael Strevens - 2013 - Studies in History and Philosophy of Science Part A 44 (3):510-515.
Understanding Why.Alison Hills - 2015 - Noûs 49 (2):661-688.

View all 28 references / Add more references