Abstract
Perceptual experience strikes us as a presentation of the here and now. I argue that it also involves experience of the past. This claim is often made based on cases, like seeing stars, involving significant signal-transmission lag, or based on how working memory allows us to experience extended events. I argue that the past is injected into perceptual experience via a third way: long-term memory traces in sensory circuits. Memory, like the receptor-based senses, is an integrated and constituent modality through which we experience the environment. Because of this modality, we experience the sensed properties of stimuli partly as they are now, but also partly as we encountered them in the past.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
I’ll leave open whether perceptual systems sometimes remember, in the functionally defined sense of recall-process, in their effort to interpret sensory input.
The relativity of simultaneity entails that there is no such objective durationless “now”. Still, within the local spacetime of our sensory interactions with the environment at their low relative speeds, observer discrepancies won’t affect how we interpret experience and its spatiotemporal content.
By using different terms, I don’t mean to suggest the two are exclusive (they are not). I use different terms merely for clarity.
My view is that even if experience is fundamentally a representation, we still experience stimuli themselves, not intermediaries. As Fred Dretske (2003) and others point out, even if experience is a representation, what we experience is what’s represented, not the representation(al vehicle) itself (see also Crane 2006; Genone 2016).
The idea that perception, imagination, and memory share neural mechanisms isn’t without controversy. It’s challenged by how brain lessons seem able to doubly dissociate perception and imagination (see Brogaard and Gatzia 2017). Space precludes me from responding to this concern. These ideas are well-enough supported that a discussion taking them for granted is still valuable.
Addis and others who see memory and imagination as a single system likely don’t think there’s a substantive question as to whether it’s imagination, or memory, which affects perception.
As I’ll note below in Section 5, the effects of long-term mnemonic neural mechanisms are a form of perceptual learning. Some have objected that perceptual learning is cognitive, not genuinely perceptual (e.g. Pylyshyn 1999, 359). These objections often come in debates over cognitive penetration. I disagree; this paper presupposes that these effects are perceptual. Space prevents me from responding to this debate, but relevant defenses in the neighborhood of what I’d pursue include Vetter and Newen’s (2014) oft-cited paper and Kevin Connolly’s (2019) recent book on perceptual learning.
Echos provide a fascinating transmission lag case. At least on some views (e.g. O’Callaghan2019a), a reflecting sound wave (as in an echo) allows you to re-encounter, and so re-experience, the sound causing it. Hence, experiences of echos are actually experiences of a disturbance or vibrational event a few seconds in the past, one and the same event you already experienced as the “primary” sound.
Set aside questions about the ontology of experiences, e.g. whether experiences themselves are the kind of entity which have an objective duration, or how the objective duration of an experience relates to the duration felt by the subject having the experience. My point is that if, at some arbitrary instant, you could somehow catalogue what was being presented to a subject in their experience at that instant, you would find the content outstripped what could actually happen in an instant, and contained a temporal stream including elements in the past.
The line between short-term working memory and long-term memory is not sharp, either in terms of temporal duration or neural mechanisms. For example, the mnemonic neural mechanisms responsible for the waterfall illusion (Addams 1834) are probably more like the neural mechanisms which allow us to track moving objects in real time than the mechanisms which associate colors and shapes, but the waterfall illusion is an effect which extends beyond the specious present. I’ll set aside these intermediary cases and focus on stable, long-term effects like color-shape associations.
Thanks to an anonymous referee for pointing out what is the default view.
My opponent may not actually want to press this problem. Yes, it is a problem for those who want to claim that pattern completion by long-term memory traces injects experience of past-perceived objects into current sensory experiences (e.g. Barkasi 2020). But my opponent faces their own version of the problem: How is it that the activation of a schema, or general pattern, abstracted from past experience, allows me to perceive the current instance of my friend’s red hat? So, the problem arises independent of any attempt to find the injected element in the past.
Brown (2018) raises a similar issue for the question of whether perceptual experience is ever infused with imagination. He points out that it’s uninteresting and trivially true that we can, for example, imagine a lamp in the corner of a room we can see to be empty; the interesting question, for him, is whether normal, successful perceptual experiences involve an imaginative component. Similarly, it’s not radical to claim that, when sensory input is incomplete, it can be filled by a hallucination of a past-perceived stimulus. The interesting question is whether normal, successful perceptual experience involves experience of the past.
As I understand it, claiming that memory is a perceptual modality is not quite the same as claiming that it’s a distinct sense. There are, of course, differences between the “full-fledged” senses like vision or touch and memory, most conspicuously that memory lacks a proprietary suite of receptors. Memory also seems dependent on the other senses in important ways, e.g. (when used in perception) it lacks its own proprietary phenomenology and instead seems to shape the phenomenal character of the other senses. These features put memory outside the bounds of standard classifications of the “senses” (e.g. Macpherson 2011; Macpherson2014), but don’t diminish it as its own unique mode of accessing stimuli. Notice that memory would not be the only sensory modality without proprietary receptors. Our ability to perceive flavour is generally thought of as a perceptual capacity routed in encodings that draw from both olfaction (smell) and gustation (taste) (O’Callaghan 2019a).
This updating might be given a computational gloss, like that found in predictive processing. Here I’m sticking to a basic neurobiological perspective: What’s being updated are the literal primed patterns of neural activity; they are updated via changes in synaptic strength, changes produced via mechanisms like long-term potentiation and long-term depression.
You might object that, as Addis herself says, the trace or scheme-forming process is a process of abstraction; hence, the results are timeless abstractions. This objection assumes that neural circuits make idealized abstractions. In practice, both constrained by sample data and physical electrochemical noise, the brain never manages perfect abstractions.
Brown (2018, 154) argues that perception is pervasively infused with imagination, and his argument likewise makes use of these points about the pervasiveness of input processing.
This objection is from a very helpful referee.
References
Addams, R. 1834. An account of a peculiar optical phenomenon seen after having looked at a moving body. London and Edinburgh Philosophical Magazine and Journal of Science 5: 373–374.
Addis, D. R. 2020. Mental time travel? a neurocognitive model of event simulation. Review of Philosophy and Psychology 11: 233–259.
Albright, T. D. 2012. On the perception of probable things: Neural substrates of associative memory, imagery, and perception. Neuron 74: 227–245.
Anscombe, G. 1965. The intentionality of sensation: A grammatical feature. Analytic Philosophy, ed. Butler R. J., 158–80. Oxford, Blackwell.
Aranyosi, I. 2020. Preteriception: Memory as past-perception. Synthese: 1–28.
Austin, J. L. 1962. Sense and sensibilia. Oxford: Oxford University Press.
Barkasi, M. 2020. Some hallucinations are experiences of the past. Pacific Philosophical Quarterly 101(3): 454–488.
Barkasi, M., and M. G. Rosen. 2020. Is mental time travel real time travel? Philosophy and the Mind Sciences 1(1): 1–27.
Beaty, R. E., P. Seli, and D. L. Schacter. 2019. Thinking about the past and future in daily life: an experience sampling study of individual differences in mental time travel. Psychological Research Psychologische Forschung 83 (4): 805–816.
Bone, M. B., F. Ahmad, and B. R. Buchsbaum. 2020. Feature-specific neural reactivation during episodic memory. Nature Communications 11 (1945): 1–13.
Briscoe, R. 2018. Superimposed mental imagery. Perceptual Imagination and Perceptual Memory, eds. Macpherson F. and Dorsch F., 161–185. Oxford, Oxford University Press.
Brogaard, B., and D. E. Gatzia. 2017. Unconscious imagination and the mental imagery debate. Frontiers in Psychology 8(799): 1–14.
Brown, D. H. 2018. Infusing perception with imagination. Perceptual Imagination and Perceptual Memory, eds. Macpherson F. and Dorsch F., 133–160. Oxford, Oxford University Press.
Bruner, J. S., L. Postman, and J. Rodrigues. 1951. Expectation and the perception of color. The American Journal of Psychology 64(2): 216–227.
Buckner, R. L., and D. C. Carroll. 2007. Self-projection and the brain. TRENDS in Cognitive Sciences 11(2): 49–57.
Burge, T. 2010. Origins of objectivity. Oxford: Oxford University Press.
Campbell, J. 2002. Reference and consciousness. Oxford: Oxford University Press.
Chalmers, D. J. 2006. Perception and the fall from Eden. Perceptual Experience, eds. Szabo Gendler T. and Hawthorne J., 49–125. Oxford, Oxford University Press.
Clark, A. 2012. Dreaming the whole cat: Generative models, predictive processing, and the enactivist conception of perceptual experience. Mind 121 (483): 753–771.
Connolly, K. 2019. Perceptual learning: The flexibility of the senses. Oxford: Oxford University Press.
Cooke, S. F., and M. F. Bear. 2014. How the mechanisms of long-term synaptic potentiation and depression serve experience-dependent plasticity in primary visual cortex. Philosophical Transactions of the Royal Society B: Biological Sciences 369(1633): 1–12.
Crane, T. 2006. Is there a perceptual relation?. Perceptual Experience, eds. Szabo Gendler T. and Hawthorne J., 126–146. Oxford, Oxford University Press.
Dretske, F. 1995. Naturalizing the mind. Cambridge: The MIT Press.
Dretske, F. 2003. Experience as representation. Philosophical Issues 13(1): 67–82.
Farah, M. J. 1985. Psychophysical evidence for a shared representational medium for mental images and percepts. Journal of Experimental Psychology: General 114(1): 93–103.
Feldman, D. E. 2012. The spike-timing dependence of plasticity. Neuron 75: 556–571.
Fish, W. 2009. Perception, hallucination and illusion. Oxford: Oxford University Press.
Fodor, J. A., and Z. W. Pylyshyn. 1981. How direct is visual perception?: Some reflections on Gibson’s ‘ecological approach’. Cognition 9: 139–196.
Geldard, F. A., and C. E. Sherrick. 1972. The cutaneous “rabbit”: a perceptual illusion. Science 178(4057): 178–179.
Genone, J. 2016. Recent work on naïve realism. American Philosophical Quarterly 53(1): 1–26.
Hansen, T., M. Olkkonen, S. Walter, and K. R. Gegenfurtner. 2006. Memory modulates color appearance. Nature Neuroscience 9 (11): 1367–1368.
Harman, G. 1990. The intrinsic quality of experience. Philosophical Perspectives 4: 31–52.
Hellie, B. 2014. Love in the time of cholera. Does Perception Have Content?, ed. Brogaard B., 241–261. Oxford, Oxford University Press.
Horikawa, T., and Y. Kamitani. 2017. Hierarchical neural representation of dreamed objects revealed by brain decoding with deep neural network features. Frontiers in Computational Neuroscience 11(4): 1–11.
Hurlbert, A. C., and Y. Ling. 2005. If it’s a banana, it must be yellow: The role of memory colors in color constancy. Journal of Vision 5(8): 787.
Jackson, M. B. 2013. Recall of spatial patterns stored in a hippocampal slice by long-term potentiation. Journal of Neurophysiology 110: 2511–2519.
James, S. P. 2014. Hallucinating real things. Synthese 191 (15): 3711–3732.
James, W. 1962. Psychology: Briefer Course. New York: Collier Books.
Kant, I. 1998. Critique of Pure Reason. Cambridge: Cambridge University Press.
Kelly, S. D. 2010. The normative nature of perceptual experience. Perceiving the World, ed. Nanay B., 146–159. Oxford, Oxford University Press.
Kind, A. 2018. Imaginative presence. Phenomenal Presence, eds. Dorsch F. and Macpherson F., 165–180. Oxford, Oxford University Press.
Kosslyn, S. M. 1994. Image and brain: The resolution of the imagery debate. Cambridge: The MIT Press.
Kosslyn, S. M. 2005. Mental images and the brain. Cognitive Neuropsychology 22(3/4): 333–347.
Laird, J. 1920. A study in realism. Cambridge: Cambridge University Press.
Macpherson, F. 2011. Taxonomising the senses. Philosophical Studies 153(1): 123–142.
Macpherson, F. 2012. Cognitive penetration of colour experience: Rethinking the issue in light of an indirect mechanism. Philosophy and Phenomenological Research 84(1): 24–62.
Macpherson, F. 2014. The space of sensory modalities. Perception and Its Modalities, eds. Stokes, D., Matthen, M., and Biggs S., 432–461. Oxford, Oxford University Press.
Macpherson, F. 2018. Perceptual imagination and perceptual memory: An overview. Perceptual Imagination and Perceptual Memory, eds. Macpherson F. and Dorsch F., 1–6. Oxford, Oxford University Press.
Manzotti, R. 2019. Mind-object identity: a solution to the hard problem. Frontiers in Psychology 10(63): 1–16.
Marr, D. 1980. Visual information processing: The structure and creation of visual representations. Phil Trans R Soc London B 290: 199–218.
Martin, M. G. F. 2004. The limits of self-awareness. Philosophical Studies 120(1-3): 37–89.
Matthen, M. 2005. Seeing, doing and knowing: a philosophical theory of sense perception. Oxford: Oxford University Press.
McDowell, J. 1986. Singular thought and the extent of inner space. Subject, Thought, and Context, eds. McDowell J. and Pettit P., 137–168. Oxford, Oxford University Press.
Michaelian, K. 2016. Mental time travel: Episodic memory and our knowledge of the personal past. Cambridge: The MIT Press.
Moran, A. 2019. Naïve realism, seeing stars, and perceiving the past. Pacific Philosophical Quarterly 100(1): 202–232.
Munton, J. 2021. How to see invisible objects. Noûs: 1–23.
Nanay, B. 2010. Perception and imagination: Amodal perception as mental imagery. Philosophical Studies 150: 239–254.
O’Callaghan, C. 2019. Echos. The Monist 90(3): 403–414.
O’Callaghan, C. 2019. A multisensory philosophy of perception. Oxford: Oxford University Press.
Penfield, W., and P. Perot. 1963. The brain’s record of auditory and visual experience. Brain: A Journal of Neurology 86(4): 595–696.
Pylyshyn, Z. W. 1999. Is vision continuous with cognition? the case for cognitive impenetrability of visual perception. Behavioral and Brain Sciences 22 (3): 341–365.
Reid, T. 1785. Essays on the Intellectual Powers of Man. Cambridge: Cambridge University Press.
Schacter, D. L., and D. R. Addis. 2007. The cognitive neuroscience of constructive memory: Remembering the past and imagining the future. Philosophical Transactions of the Royal Society B: Biological Sciences 362(1481): 773–786.
Siple, P., and R. M. Springer. 1983. Memory and preference for the colors of objects. Perception & Psychophysics 34: 363–370.
Spelke, E. S. 1988. Where perceiving ends and thinking begins: The apprehension of objects in infancy. Perceptual Development in Infancy, ed. Yonas A., 197–234. Lawrence Erlbaum Associates, Hillsdale.
Strawson, P. F. 1970. Imagination and perception. Experience and Theory, eds. Foster J. and Swanson J. W., 31–54. Amherst, University of Massachusetts Press.
Suddendorf, T., and M. C. Corballis. 2007. The evolution of foresight: What is mental time travel, and is it unique to humans? Behavioral and Brain Sciences 30: 299–351.
Taylor, D. 1938. Realism and memory. The Australasian Journal of Psychology and Philosophy 16(3): 218–232.
Tulving, E. 1983. Elements of episodic memory. Oxford: Oxford University Press.
Tulving, E. 2002. Episodic memory: From mind to brain. Annual Review of Psychology 53: 1–25.
Vetter, P., and A. Newen. 2014. Varieties of cognitive penetration in visual perception. Consciousness and Cognition 27: 62–75.
Weir, A. 2004. An ultra-realist theory of perception. International Journal of Philosophical Studies 12(2): 105–128.
Witzel, C., and K. Gegenfurtner. 2013. Memory color. Encyclopedia of Color Science and Technology, ed. Luo R. New York, Springer.
Zylberberg, J., and B. W. Strowbridge. 2017. Mechanisms of persistent activity in cortical circuits: Possible neural substrates for working memory. Annual Review of Neuroscience 40: 603–627.
Acknowledgements
Thanks to Jake Beck for early comments on these ideas which got me to abandon an argument that did not work and pursue something more productive. Thanks also to André Sant’Anna for comments on an early draft of this material, and to Casey O’Callaghan for his discussion. Special thanks are due to a very helpful referee, who provided extensive comments and important suggestions for improvement.
Funding
There is no funding to report.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Barkasi, M. Memory as Sensory Modality, Perception as Experience of the Past. Rev.Phil.Psych. 14, 791–809 (2023). https://doi.org/10.1007/s13164-021-00598-7
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13164-021-00598-7