Many of the computing systems programmed using Machine Learning are opaque: it is difficult to know why they do what they do or how they work. Explainable Artificial Intelligence aims to develop analytic techniques that render opaque computing systems transparent, but lacks a normative framework with which to evaluate these techniques’ explanatory successes. The aim of the present discussion is to develop such a framework, paying particular attention to different stakeholders’ distinct explanatory requirements. Building on an analysis of “opacity” from (...) philosophy of science, this framework is modeled after accounts of explanation in cognitive science. The framework distinguishes between the explanation-seeking questions that are likely to be asked by different stakeholders, and specifies the general ways in which these questions should be answered so as to allow these stakeholders to perform their roles in the Machine Learning ecosystem. By applying the normative framework to recently developed techniques such as input heatmapping, feature-detector visualization, and diagnostic classification, it is possible to determine whether and to what extent techniques from Explainable Artificial Intelligence can be used to render opaque computing systems transparent and, thus, whether they can be used to solve the Black Box Problem. (shrink)

The received view of dynamical explanation is that dynamical cognitive science seeks to provide covering law explanations of cognitive phenomena. By analyzing three prominent examples of dynamicist research, I show that the received view is misleading: some dynamical explanations are mechanistic explanations, and in this way resemble computational and connectionist explanations. Interestingly, these dynamical explanations invoke the mathematical framework of dynamical systems theory to describe mechanisms far more complex and distributed than the ones typically considered by philosophers. Therefore, contemporary dynamicist (...) research reveals the need for a more sophisticated account of mechanistic explanation. (shrink)

Models developed using machine learning are increasingly prevalent in scientific research. At the same time, these models are notoriously opaque. Explainable AI aims to mitigate the impact of opacity by rendering opaque models transparent. More than being just the solution to a problem, however, Explainable AI can also play an invaluable role in scientific exploration. This paper describes how post-hoc analytic techniques from Explainable AI can be used to refine target phenomena in medical science, to identify starting points for future (...) investigations of causal relationships, and to generate possible explanations of target phenomena in cognitive science. In this way, this paper describes how Explainable AI—over and above machine learning itself—contributes to the efficiency and scope of data-driven scientific research. (shrink)

Bayesian reverse-engineering is a research strategy for developing three-level explanations of behavior and cognition. Starting from a computational-level analysis of behavior and cognition as optimal probabilistic inference, Bayesian reverse-engineers apply numerous tweaks and heuristics to formulate testable hypotheses at the algorithmic and implementational levels. In so doing, they exploit recent technological advances in Bayesian artificial intelligence, machine learning, and statistics, but also consider established principles from cognitive psychology and neuroscience. Although these tweaks and heuristics are highly pragmatic in character and (...) are often deployed unsystematically, Bayesian reverse-engineering avoids several important worries that have been raised about the explanatory credentials of Bayesian cognitive science: the worry that the lower levels of analysis are being ignored altogether; the challenge that the mathematical models being developed are unfalsifiable; and the charge that the terms ‘optimal’ and ‘rational’ have lost their customary normative force. But while Bayesian reverse-engineering is therefore a viable and productive research strategy, it is also no fool-proof recipe for explanatory success. (shrink)

The philosophical conception of mechanistic explanation is grounded on a limited number of canonical examples. These examples provide an overly narrow view of contemporary scientific practice, because they do not reflect the extent to which the heuristic strategies and descriptive practices that contribute to mechanistic explanation have evolved beyond the well-known methods of decomposition, localization, and pictorial representation. Recent examples from evolutionary robotics and network approaches to biology and neuroscience demonstrate the increasingly important role played by computer simulations and mathematical (...) representations in the epistemic practices of mechanism discovery and mechanism description. These examples also indicate that the scope of mechanistic explanation must be re-examined: With new and increasingly powerful methods of discovery and description comes the possibility of describing mechanisms far more complex than traditionally assumed. (shrink)

People can be taught to manipulate symbols according to formal mathematical and logical rules. Cognitive scientists have traditionally viewed this capacity—the capacity for symbolic reasoning—as grounded in the ability to internally represent numbers, logical relationships, and mathematical rules in an abstract, amodal fashion. We present an alternative view, portraying symbolic reasoning as a special kind of embodied reasoning in which arithmetic and logical formulae, externally represented as notations, serve as targets for powerful perceptual and sensorimotor systems. Although symbolic reasoning often (...) conforms to abstract mathematical principles, it is typically implemented by perceptual and sensorimotor engagement with concrete environmental structures. (shrink)

This chapter subsumes David Marr’s levels of analysis account of explanation in cognitive science under the framework of mechanistic explanation: Answering the questions that define each one of Marr’s three levels is tantamount to describing the component parts and operations of mechanisms, as well as their organization, behavior, and environmental context. By explicating these questions and showing how they are answered in several different cognitive science research programs, this chapter resolves some of the ambiguities that remain in Marr’s account, and (...) shows that many different areas and traditions of cognitive scientific research can be unified under the mechanistic framework. (shrink)

Whereas most branches of neuroscience are thought to provide mechanistic explanations, systems neuroscience is not. Two reasons are traditionally cited in support of this conclusion. First, systems neuroscientists rarely, if ever, rely on the dual strategies of decomposition and localization. Second, they typically emphasize organizational properties over the properties of individual components. In this paper, I argue that neither reason is conclusive: researchers might rely on alternative strategies for mechanism discovery, and focusing on organization is often appropriate and consistent with (...) the norms of mechanistic explanation. Thus, many explanations in systems neuroscience can also be viewed as mechanistic explanations. (shrink)

This paper considers the way mathematical and computational models are used in network neuroscience to deliver mechanistic explanations. Two case studies are considered: Recent work on klinotaxis by Caenorhabditis elegans, and a longstanding research effort on the network basis of schizophrenia in humans. These case studies illustrate the various ways in which network, simulation and dynamical models contribute to the aim of representing and understanding network mechanisms in the brain, and thus, of delivering mechanistic explanations. After outlining this mechanistic construal (...) of network neuroscience, two concerns are addressed. In response to the concern that functional network models are non-explanatory, it is argued that functional network models are in fact explanatory mechanism sketches. In response to the concern that models which emphasize a network’s organization over its composition do not explain mechanistically, it is argued that this emphasis is both appropriate and consistent with the principles of mechanistic explanation. What emerges is an improved understanding of the ways in which mathematical and computational models are deployed in network neuroscience, as well as an improved conception of mechanistic explanation in general. (shrink)

This chapter provides an overview of the basic research strategies and analytic techniques deployed in computational cognitive neuroscience. On the one hand, “top-down” strategies are used to infer, from formal characterizations of behavior and cognition, the computational properties of underlying neural mechanisms. On the other hand, “bottom-up” research strategies are used to identify neural mechanisms and to reconstruct their computational capacities. Both of these strategies rely on experimental techniques familiar from other branches of neuroscience, including functional magnetic resonance imaging, single-cell (...) recording, and electroencephalography. What sets computational cognitive neuroscience apart, however, is the explanatory role of analytic techniques from disciplines as varied as computer science, statistics, machine learning, and mathematical physics. These techniques serve to describe neural mechanisms computationally, but also to drive the process of scientific discovery by influencing which kinds of mechanisms are most likely to be identified. For this reason, understanding the nature and unique appeal of computational cognitive neuroscience requires not just an understanding of the basic research strategies that are involved, but also of the formal methods and tools that are being deployed, including those of probability theory, dynamical systems theory, and graph theory. (shrink)

At least two distinct modeling frameworks contribute to the view that mind and brain are Bayesian: Bayesian Rational Analysis (BRA) and Hierarchical Predictive Coding (HPC). What is the relative contribution of each, and how exactly do they relate? In order to answer this question, we compare the way in which these two modeling frameworks address different levels of analysis within Marr’s tripartite conception of explanation in cognitive science. Whereas BRA answers questions at the computational level only, many HPC-theorists answer questions (...) at the computational, algorithmic, and implementational levels simultaneously. Given that all three levels of analysis need to be addressed in order to explain a behavioral or cognitive phenomenon, HPC seems to deliver more complete explanations. Nevertheless, BRA is well-suited for providing a solution to the dark room problem, a major theoretical obstacle for HPC. A combination of the two approaches also combines the benefits of an embodied-externalistic approach to resolving the dark room problem with the idea of a persisting evidentiary border beyond which matters are out of cognitive reach. For this reason, the development of explanations spanning all three Marrian levels within the general Bayesian approach may require combining the BRA and HPC modeling frameworks. (shrink)

According to Marr, explanations of perceptual behavior should address multiple levels of analysis. Rahnev & Denison (R&D) are perhaps overly dismissive of optimality considerations at the computational level. Also, an exclusive reliance on standard observer models may cause neglect of many other plausible hypotheses at the algorithmic level. Therefore, as far as explanation goes, standard observer modeling is no panacea.