Conceptual challenges for interpretable machine learning

Synthese 200 (2):1-33 (2022)
  Copy   BIBTEX


As machine learning has gradually entered into ever more sectors of public and private life, there has been a growing demand for algorithmic explainability. How can we make the predictions of complex statistical models more intelligible to end users? A subdiscipline of computer science known as interpretable machine learning (IML) has emerged to address this urgent question. Numerous influential methods have been proposed, from local linear approximations to rule lists and counterfactuals. In this article, I highlight three conceptual challenges that are largely overlooked by authors in this area. I argue that the vast majority of IML algorithms are plagued by (1) ambiguity with respect to their true target; (2) a disregard for error rates and severe testing; and (3) an emphasis on product over process. Each point is developed at length, drawing on relevant debates in epistemology and philosophy of science. Examples and counterexamples from IML are considered, demonstrating how failure to acknowledge these problems can result in counterintuitive and potentially misleading explanations. Without greater care for the conceptual foundations of IML, future work in this area is doomed to repeat the same mistakes.



    Upload a copy of this work     Papers currently archived: 91,088

External links

Setup an account with your affiliations in order to access resources via your University's proxy server

Through your library

Similar books and articles

Concept Representation Analysis in the Context of Human-Machine Interactions.Farshad Badie - 2016 - In 14th International Conference on e-Society. pp. 55-61.
A Conceptual Framework Over Contextual Analysis of Concept Learning Within Human-Machine Interplays.Farshad Badie - 2017 - In Emerging Technologies for Education. Cham, Switzerland: pp. 65-74.
Machine Learning and the Perils of Prolific Pattern Finding.Bruce Sherin - 2019 - Constructivist Foundations 14 (3):285-287.
Human Semi-Supervised Learning.Bryan R. Gibson, Timothy T. Rogers & Xiaojin Zhu - 2013 - Topics in Cognitive Science 5 (1):132-172.


Added to PP

33 (#432,611)

6 months
9 (#169,226)

Historical graph of downloads
How can I increase my downloads?

Author's Profile

David Watson
University College London

References found in this work

The Structure of Scientific Revolutions.Thomas S. Kuhn - 1962 - Chicago, IL: University of Chicago Press. Edited by Ian Hacking.
Causality: Models, Reasoning and Inference.Judea Pearl - 2000 - New York: Cambridge University Press.

View all 76 references / Add more references