Contents lists available at ScienceDirect Consciousness and Cognition journal homepage: www.elsevier.com/locate/concog Modularist explanations of experience and other illusions Eric Mandelbaum⁎ City University of New York Graduate Center, Baruch College, City University of New York, United States A R T I C L E I N F O Keywords: Modularity Consciousness Informational encapsulation Cognitive penetration A B S T R A C T Debates about modularity invariably involve a crucial premise about how visual illusions are experienced. This paper argues that these debates are wrongheaded, and that experience of illusions is orthogonal to the core issue of the modularity hypothesis: informational encapsulation. If there is any common ground between modularists and anti-modularists it is the idea that illusions are of excellent probative value for assessing questions of modularity. Modularists have traditionally used illusions to display an extreme version of informational encapsulation. For example, the Muller-Lyer is trotted out to show that even though everyone knows that the lines are the same length, you cannot see them as such. Thus we have a piece of extremely relevant knowledge-that the lines are the same length-that cannot penetrate the module. Anti-modularists are just as apt to use the same evidence to draw similar conclusions. Often enough the anti-modularist aims to show that you can change your perceptual experience of an image (such as the hidden figures illusions, like the famous Dalmatian dog) merely by changing your knowledge or motivation, thereby putatively showing that what you know (or desire) can change what you see.1 The informational encapsulation under discussion in cognitive penetration cases is the strongest form of encapsulation, a bidirectional form where central knowledge cannot penetrate the module and proprietary modular information cannot escape, thereby remaining inaccessible to any other mental processes. After all, we don't know how the Muller-Lyer arises after all these years because we cannot access the workings of the module. For the remainder of the article I will remain neutral on questions of whether modularity is really true of any part of the mind. Instead I want to focus on whether the strategy of using illusions in modularity is as probative as it's assumed to be. Though proponents and opponents of modularity utilize illusions, the appeal to illusions is a red herring, especially when it comes to the heart of the modularity hypothesis: informational encapsulation. To begin, consider what exactly the phenomenology of perceptual experience has to do with informational encapsulation as such. Modularists have more or less always supposed that access was 'top-down', and that the intramodular workings were inaccessible to any central processes (e.g., language production), never mind consciousness or other person-level processes and states. Thus, in a https://doi.org/10.1016/j.concog.2019.102828 Received 18 July 2019; Received in revised form 1 September 2019; Accepted 28 September 2019 ⁎ Address: 365 Fifth Ave., Rm. 7113, New York, NY 10016, United States. E-mail address: emandelbaum@gc.cuny.edu. 1 Anti-modularists use bistable images for similar ends. For examples of this strategy see Churchland (1988), Prinz (2006), Vinson et al. (2016). Modularists don't tend to find this strategy very compelling. What many modularists would find compelling would be if your person-level beliefs alone could change your perceptual experience. However, the anti-modularity authors do not attempt to demonstrate that. Instead, they just show that by changing your attention you can change your experience of bistable images (e.g., which side of the Necker cube faces the observer). Attentional shifts are known to change one's experience of ambiguous figures (see, e.g., Long & Toppino, 2004), but such changes are interpreted as changes to the input to vision, similar to merely moving one's eyes. Consciousness and Cognition 76 (2019) 102828 1053-8100/ © 2019 Elsevier Inc. All rights reserved. T modular framework the earliest a representation could become conscious is immediately post-modular. So far, so good. But note what the hypothesis that access is top-down doesn't entail: that representations that are outputted from modules are ipso facto conscious. It's open for the modularist to hold that outputs are, often enough, unconscious (see, e.g., Mandelbaum, 2018).2 How unconscious representations become conscious is an interesting and venerable question of which there are many reasonable theories. I wish to assert none of them. Instead, I just aim to point out that there are two extremely different senses of cognitive penetration in the literature. Take, for instance, Susanna Siegel's Method of Phenomenal Contrast (2010). The method asks one to suppose that a particular property, P, is represented in experience. To do so, she asks one to compare two experiences that are as similar as possible save for the fact that one of which is a good candidate for representing the property and the other one isn't. If the two experiences aren't phenomenologically identical, we are asked to determine whether the presence of P in the candidate P-experience is the best explanation of why we experience a contrast. Using this method, we can test the hypothesis that experiences can represent P. Theorists have argued that this method allows us to also get a handle on cognitive penetration: if our experience of seeing a yellow-rumped warbler changes after learning what yellow-rumped warblers are, then it seems like our knowledge has penetrated our experience.3 (Similarly, if knowing that the lines of the Muller-Lyer are identical caused us to see them as the same length, this would also be evidence of cognitive penetration of experience.) However, one's knowledge affecting the experience of seeing a warbler still doesn't establish that the workings of a module were penetrated at all. A module is, in essence, a proprietary database with some algorithms for computational transformations of that database in conjunction with the incoming information (i.e., transduced sensations). The Method of Phenomenal Contrast can only show that an experience of an output of a module (whether it's undergone further processing or not) can be changed based on what one knows; what it cannot show is that the proprietary database or algorithms processed the input any differently.4 To show that we would need more than the mere change of our visual experience of a stimulus based on a change in our central cognitive stores. Instead, what we would need is evidence that the actual computations or databases changed. This could be accomplished by a change in what evidence was available (perhaps some previously available intramodular evidence is now inhibited, or perhaps some central information is now available to the module) or a change in what computations the module performs (perhaps the module used to proceed by holistic identification and now is running a feature-based analysis). Truly deep cognitive penetration of a system is cognitive penetration that changes how the system processes-it changes what information is accessed, and not just how we experience the output of the module.5 Some modularists (e.g., Firestone & Scholl, 2016) have written as if any change in the visual output should be detectable phenomenologically-hence their calls for demos that phenomenologically work and their criticisms of the work of Proffitt and his collaborators (see, e.g., Proffitt, 2006; Proffitt & Linkenauger, 2013). Firestone and Scholl are certainly partially right: if putting on a heavy backpack changed the slope of a run of the mill hill to the slope of Mt. Everest that should be phenomenologically detectable.6 But the reason they are right here isn't because any change that could affect modules must be phenomenologically detectable, but instead because the particular change they were discussing was purported to be so astoundingly large that it seems implausible that it could have no experiential effect at all (see footnote 9 for more on how abductive plausibility claims about experience might work). It is reasonable for theorists like Proffitt to respond to Firestone and Scholl's challenge by claiming that changing central states doesn't in fact change the visual experience, only the normal computational processing route. Such a claim would entail one of two things: (1) that there is honest-to-god synchronic violation of informational encapsulation via centrally stored information, the sine qua non of the anti-modularity movement; or (2) that modules aren't automatically set off but instead are bypassable.7 Either way Proffitt would be able to claim that changes in central states (i.e., knowing one is wearing a heavy backpack) does in fact change perception, even if it doesn't change experience. Although such a claim would demonstrate a way in which cognition affects perception, it wouldn't necessarily have any bearing on the core of the modularity debate because of the possibility of bypassing. All option (2) would accomplish is disproving modular automaticity which is not in and of itself evidence for cognitive penetration of the module (see Mandelbaum, 2015 for discussion). And (2) should perhaps be expected on separate grounds as it seems that evolution has ensured that every important problem can be solved multiple ways. For example, we don't have a single canonical way of seeing depth, we have many different, autonomous though sometimes overlapping depth cues. Perhaps the normal way to parse a face is through a face module, but given (e.g.,) only high spatial frequencies and no low spatial frequencies we parse faces via a less traveled non-modular route (ibid.) To see which, if either, option is ever accomplished one would have to do more than merely show changes in experience-one would have to show changes in the normal processing route. And to do that is hard-it takes some experimental equipment, 2 For example, stimuli that are presented at absurdly quick rates (13ms presentation time) are often not only processed by the visual system but are also conceptualized even though the subject doesn't even know she's seen any stimuli at all (ibid). 3 For the point at hand it's best to ignore the currently orthogonal (but otherwise probative) question of whether the penetration is understood as diachronic or synchronic. 4 To put it in Quilty-Dunn (2019a) terms, the Method of Phenomenal Contrast can affect questions of cognitive penetration, but not informational encapsulation. Though his point strikes me as correct, here I keep with the general use of "cognitive penetration" and disambiguate in context. For more on the relation between informational encapsulation and cognitive penetration see Quilty-Dunn (2019a); Macpherson (2015). 5 Siegel, of course, knows this; see Siegel and Byrne (2017). 6 This isn't an off the cuff example, it's actually what the Proffitt's effect size dictated the slope should be (Firestone, 2013). 7 In which case Fodor would've also been wrong in saying that the only way transduced information could make it to central cognition is via a module (Fodor, 1983, 54). E. Mandelbaum Consciousness and Cognition 76 (2019) 102828 2 ingenuity, and dedicated graduate students.8 But it is also accomplishable without utilizing any illusions at all. In fact, showing penetration of illusions wouldn't in and of itself show anything about the violation of informational encapsulation. I suspect confusion on this point has persisted for two reasons. The first is because it is striking that one cannot change one's percept of the Muller-Lyer merely by changing one's knowledge state. That is a datum calling out for an explanation, and a possible sketch of one is to assume that modules fix phenomenology. But the truth or falsity of this posit is orthogonal to the question of how informationally encapsulated modular processing is.9 The other reason the debate has been muddled is due to Fodor's original presentation. Fodor himself wasn't particularly confused as to the relation between phenomenology and encapsulation.10 After all, he generally assumed that all interesting perceptual and cognitive processing was unconscious. For instance, he writes, "It may be thought Pickwickian, after all that we've been through together, for me to cleave to phenomenological accessibility as a criterion of the output of the visual processor" (1983, 136). Fodor realized how problematic it is to open up the door to consciousness. So why did he bring experience into this at all? It's only because he was (rather shockingly) motivated by "ulterior-specifically, epistemological motives" (ibid). Here's Fodor's reasoning: "I am suggesting that, barring evidence to the contrary, it would be convenient if the output vocabulary of the perceptual analysers overlapped the vocabulary of such (prima facie) perceptual premisses as figure in conscious inference and decision making (so that remarks such as 'I see that it's raining' could be taken as literally true and not just enthymematic). Why shouldn't one assume what it is convenient to assume?" (54). The reason why one shouldn't assume it is because convenience is rarely a guide to the truth. As far as we can tell, the computations and data structures in perception are unconscious, and that is true whether perception is modular or not.11 Thus, the first place consciousness can get a foothold is after perceptual processing has completed. But this doesn't entail that modular outputs are ever conscious at all. In fact, the evidence we have about the outputs of perception implies that the outputted representations are, in the first instance, unconscious (Mandelbaum, 2018). That this fact has escaped discussions of modularity is surprising. If modularity is equated with perceptual processing then unconscious outputs are exactly what we should expect as the norm. The claim that a modality is modular dictates that any perceptual representation that makes it to central cognition must do so via a module. But the evidence for unconscious perception is rather strong (see, e.g. Quilty-Dunn, 2019b). We know that we can process stimuli that are presented at extremely quick presentation rates (e.g., 13ms), and that these stimuli end up being conceptualized (Mandelbaum, 2018). The only way to make sense of this situation is if the outputs of modules are able to be unconscious. Thus a reasonable conclusion is that perceptual representations are unconscious by default and then become conscious.12 But regardless of whether one held that the outputs of perception of are unconscious, the strategy of using illusions for evidence for modularity is a poor one. Even if modular outputs were necessarily conscious, showing that the experience of illusions can be modulated by occurrent beliefs and desires still wouldn't show that there has been any change in the processing of the module-either in terms of the computations and algorithms it uses or in the information it recruits for its processing. One's experience of illusions just isn't the type of evidence that can determine whether informational encapsulation has been violated. To sum up, we can conclude that there are at least two13 distinct notions of modularity in the literature that need to be distinguished. There is the (putative) modularity of perceptual processing and the (putative) modularity of experience. Illusions are of probative value only for the latter and not the former.14 Since the most prominent putative counterexamples (Bhalla & Proffitt, 1999; 8 It may also take some signal detection theory. If perceptual report depends on perceptual sensitivity and a contextually-determined response criterion, then our reports will, in certain situations, differ from our perceptual sensitivity. In that case Mueller-Lyer responses can be manipulated separately from perceptual sensitivity itself (Morgan, Hole, & Glennerster, 1990; Witt, Taylor, Sugovic, & Wixted, 2015). Signal-detection considerations thus make the reporting of illusions even less probative for understanding perceptual processing. 9 One way in which it perhaps isn't totally orthogonal is through abduction: maybe the best explanation we have of the persistence of illusions is that perceptual phenomenology is (perhaps just partially) determined by the modular outputs. This abductive argument depends, of course, on there being no better explanations of the phenomenological persistence. It is not clear that this abductive argument is very good. Note that elsewhere persistence is well-established but isn't explained by modularity, e.g., in cases of belief persistence (Mandelbaum, 2019). Regardless, if one likes this abductive argument for perceptual phenomenology then it appears that although penetration of experience would still be non-evidential for questions of the violation of informational encapsulation, the impenetrability of experience would be evidence of encapsulated modular processing. 10 Perhaps Pylyshyn wasn't either. Unlike Fodor who frequently spoke as if modules were to be identified with modalities, Pylyshyn stressed that modularity was only true of "early vision" (see, e.g., Pylyshyn, 1999) thus leaving open the possibility that early vision was modular, but that modularity didn't dictate visual experience. 11 Should this make us despair for epistemology? I think not. If anything Fodor's plea for convenience ran together two separate wishes: that the outputs be conscious, and that the outputs be the sort of thing that can be used in inference and decision making, viz., be conceptual. Though it may matter for some (say, internalists) that the outputs really be conscious, as long as the outputs are conceptual we can have a complete story of how sensory transduction leads to person-level cognitive acts. Happily, we have evidence that the outputs of perception are in fact conceptual (see the "conceptual modularist view" Mandelbaum, 2018). 12 Otherwise, we would have to hold that perceptual outputs are default conscious and then are somehow instantaneously made unconscious in these cases. But if this were so then one would expect that we would be able to report this situation-subjects should say they saw something and before it disappeared from consciousness. As far as I know this never happens. 13 I say "at least two" because the psychological and neurological notions of modularity are distinct too, and need to be kept separated in discussion. 14 A natural follow-up question arises: do similar morals hold for more "central" modules such as language parsing? One might have thought that the jarring nature of garden path sentences is evidence for at least some cognitive penetration. But on reflection I think the moral adumbrated here holds more generally. Garden path phenomenology does seem to tell the thinker something, but the language processing errors aren't so much E. Mandelbaum Consciousness and Cognition 76 (2019) 102828 3 Hansen, Olkkonen, Walter, & Gegenfurtner, 2006; Balcetis & Dunning, 2010, etc.) attack the latter conception, we can suppose that the modularity of perceptual processing is still in reasonably good standing.15 Acknowledgment Funding received during the execution of this paper came from the National Endowment of the Humanities fellowship (FEL257901-18). The NEH is hereby thanked for their largesse. References Balcetis, E., & Dunning, D. (2010). Wishful seeing: More desired objects are seen as closer. Psychological Science, 21, 147–152. Bhalla, M., & Proffitt, D. R. (1999). Visual-motor recalibration in geographical slant perception. Journal of Experimental Psychology: Human Perception and Performance, 25, 1076–1096. Churchland, P. (1988). Perceptual plasticity and theoretical neutrality: A reply to Jerry Fodor. Philosophy of Science, 55(2), 167–187. Firestone, C. (2013). How "paternalistic" is spatial perception? Why wearing a heavy backpack doesn't-and couldn't-make hills look steeper. Perspectives on Psychological Science, 8(4), 455–473. Firestone, C., & Scholl, B. (2016). Cognition does not affect perception: Evaluating the evidence for 'top-down' effects. Behavioral and Brain Sciences, 39(229), 1–77. Fodor, J. (1983). The Modularity of Mind. Cambridge: MIT Press. Hansen, T., Olkkonen, M., Walter, S., & Gegenfurtner, K. (2006). Memory modulates color appearance. Nature Neuroscience, 9(11), 1367–1368. Long, G., & Toppino, T. (2004). Enduring interest in perceptual ambiguity: alternating views of reversible figures. Psychological Bulletin, 130(5), 748–768. Macpherson, F. (2015). Cognitive penetration and predictive coding: A commentary on Lupyan. Review of Philosophy and Psychology, 6(4), 571–584. Mandelbaum, E. (2015). The automatic and the ballistic: Modularity beyond perceptual processes. Philosophical Psychology, 28(8), 1147–1156. Mandelbaum, E. (2018). Seeing and conceptualizing: modularity and the shallow contents of perception. Philosophy and Phenomenological Research. https://doi.org/10. 1111/phpr.12368. Mandelbaum, E. (2019). Troubles with bayesianism: An introduction to the psychological immune system. Mind & Language, 34(2), 141–157. Morgan, M. J., Hole, G. J., & Glennerster, A. (1990). Biases and sensitivities in geometrical illusions. Vision Research, 30(11), 1793–1810. Prinz, J. (2006). Is the Mind Really Modular. Contemporary debates in cognitive science (pp. 22–36). Wiley-Blackwell. Proffitt, D. R. (2006). Embodied perception and the economy of action. Perspectives on Psychological Science, 1, 110–122. Proffitt, D. R., & Linkenauger, S. A. (2013). Perception viewed as a phenotypic expression. In W. Prinz, M. Beisert, & A. Herwig (Eds.). Action Science: Foundations of an Emerging Discipline (pp. 171–198). Cambridge, Mass: MIT Press. Pylyshyn, Z. (1999). Is vision continuous with congition? The case for cognitive impenetrability of visual perception. Behavioral and Brain Sciences, 22(3), 341–365. Quilty-Dunn, J. (2019b). Unconscious perception and phenomenal coherence. Analysis, 79(3), 461–469. Quilty-Dunn, J. (2019a). Attention and Encapsulation. Mind & Language. Siegel, S., & Byrne, A. (2017). Rich or Thin? In B. Nanay (Ed.). Current Controversies in the Philosophy of Perception. New York, NY: Routledge. Vinson, D., Abney, D., Amso, D., Chemero, A., Cutting, J., Dale, R., ... Jordan, J. (2016). Perception, as you make it. Behavioral and Brain Sciences, 39. Witt, J. K., Taylor, J. E. T., Sugovic, M., & Wixted, J. T. (2015). Signal detection measures cannot distinguish perceptual biases from response biases. Perception, 44(3), 289–300. (footnote continued) changes in the phenomenology of the sentence as much as the feeling of the parse being slower and more difficult than normal, a sort of error message telling us to try a different method for decoding (Mandelbaum, 2015). But more to the point, changes in linguistic phenomenology can just clearly be post-perceptual. Imagine having an association between a specific word and a strong feeling-say that hearing the word 'McConnell' brings up waves of nausea. This nausea will change the experience of hearing a sentence involving 'McConnell' but the change in phenomenology wouldn't at all be indicative of any change in basic processing. I suspect that similar morals apply to modules in general. 15 Acknowledgements redacted. E. Mandelbaum Consciousness and Cognition 76 (2019) 102828