Available online at www.sciencedirect.com ScienceDirect Procedia Social and Behavioral Sciences 00 (2014) 000–000 www.elsevier.com/locate/procedia ICTTP 2014 Understanding perception of time in terms of perception of change Michał Klincewicz a* aBerlin School of Mind and Brain, Humboldt-Universität zu Berlin, Berlin 10117, Germany Abstract In this paper I offer an account of the dependence relation between perception of change and the subjective flow of time that is consistent with some extant empirical evidence from priming by unconscious change. This view is inspired by the one offered by William James, but it is articulated in the framework of contemporary functionalist accounts of mental qualities and higher-order theories of consciousness. An additional advantage of this account of the relationship between perception of change and subjective time is that is makes sense of instances where we are not consciously aware of changes but still experience the flow of time. © 2014 The Authors. Published by Elsevier Ltd. Selection and peer-review under responsibility of the Organizing Committee of the International Conference on Timing and Time Perception. Keywords: Time perception, change perception, William James, higher-order awareness 1. Introduction William James observed that "empty our minds as we may, some form of changing process remains for us to feel, and cannot be expelled. And along with the sense of the process and its rhythm goes the sense of the length of time it lasts. Awareness of change is thus the condition on which our perception of time's flow depends" (James, 1890, Vol. I, p. 621). The connection between awareness of change and the perception of time's flow can be understood in several ways, but the most appealing is to think of perceived changes as the boundaries of perceived intervals of time, i.e., perceived durations. On this view, awareness of change at time n followed by awareness of change at time n+2 seconds marks the boundaries of a perceived temporal interval with a duration of 2 seconds. Without perception of duration time would not appear to flow at all and this is why the perception of time's flow is dependent on awareness of change: perception of duration depends on perception in change. * Corresponding author. Tel.: +49 30 2093-1738. E-mail address: michal.klincewicz@cms.hu-berlin.de 1877-0428 © 2014 The Authors. Published by Elsevier Ltd. Selection and peer-review under responsibility of the Organizing Committee of the International Conference on Timing and Time Perception. Michał Klincewicz / Procedia Social and Behavioral Sciences 00 (2014) 000–000 2 James' idea can be useful in understanding the relationship between change perception and the perception of time in general. Durations, timing, and the flow of time can all be understood as piggybacking in some way on antecedent instances of perceived change. On this view, represented changes are the building blocks out of which mental representations of time are constructed. However appealing this idea it runs counter to some of what we know about change perception from neuroscience. In this paper I offer an account of the relationship between perceived change and subjective time that can both do justice to James' original idea and also sit comfortably with that evidence. 2. Time depending on change 2.1. The constitutive view What could it mean for the awareness of time's flow to depend on awareness of change? The view that James himself seems to endorse is that without awareness of change there would be no subjective flow of time at all. Let us call this particular understanding of the relationship between awareness of change and subjective time the constitutive view. The constitutive view may appear plausible. Consider the example of a person put in a situation where there is no sensory input of any kind, including no sense of the beating of one's heart, no sense of one's breathing, nor even the sense of the relative locations of one's body parts. Would subjective time cease to flow for them entirely? Arguably, no. A person that perceives no changes because they are deprived of any sensory input would still be able to notice the succession of their thoughts, doubts, and so on. And what if we could take away even those? Then indeed it would appear that the person would not be aware of anything and hence would have no basis on which to form the judgment that time is passing. But even if the constitutive view is conceptually sound, it does not fit comfortably with phenomena such as change blindness and priming. Change blindness occurs when changes in the world occur simultaneously with a disruption in the flow of visual information. These disruptions can range from substantial (eye saccades, shifts of the entire display) to minute (eye blinks, flickers, mud splashes in the display) (Grimes 1996, Henderson and Hollingworth, 1999, Rensink, O'Regan, & Clark, 1997, Simons & Levin, 1997). Priming occurs when perceptually processed information becomes available to psychological processes without the subject being consciously aware of that information (Marcel 1983, Wiggs & Martin 1998, Ochsner, Chiu, & Shacter 1994). Priming demonstrates that we can perceive things without being aware that we do. There is now substantial evidence of priming during change blindness (Fernandez-Duque & Thornton, 2000, 2003, Laloyaux, Destrebecqz, & Cleermans, 2006, Silverman & Mack, 2006, Caudek & Domini, 2012). These studies and others like them demonstrate that perception of change and conscious experience of change sometimes come apart. Furthermore, there is now evidence that unconsciously perceived changes have no effect on subjective time at all (Herbst, Javadi, & Busch, 2012). This evidence puts pressure on the constitutive view, which assumes that awareness of change and conscious experience of time never come apart because the latter depends, in a strong sense, on the former. Priming during change blindness shows that are situations in which the subjective flow of time is not determined by our awareness of change. And if that is true, then the relationship between awareness of change and subjective time is weaker than James' initial idea might suggest. 2.2. The causal view On the weaker understanding of the dependence relationship, awareness of change may influence the subjective flow of time, but it does not constitute it. This makes room for priming during change blindness, where we are aware of change and that does not affect our conscious experience. Let us call this the causal view. While this understanding of James' original idea sits more comfortably with neuroscience, it loses some of its explanatory power. We are left without a complete account of the subjective flow of time and also without a clear Michał Klincewicz / Procedia Social and Behavioral Sciences 00 (2014) 000–000 3 understanding of the dependence relationship between awareness of change and conscious experience of time. When does awareness of change affect subjective time? In what way? Why? In the following section I will not aim to answer these ambitious questions. I offer instead a version of the causal view that adopts the Jamesian idea of dependence between awareness of change and subjective time, but also sits comfortably with neuroscience. With that in place, perhaps the more ambitious questions can be addressed in the future. Importantly, this account will only address perceived changes and will not encompass the awareness of the succession of one's thoughts, doubts, and so on, which might be the basis for the subjective sense of the flow of time when there is no perceptual input at all. 3. Awareness of change and the subjective flow of time 3.1. Mental qualities without consciousness So how can change be represented without consciousness? One way of developing such a view is to first look to a functionalist conception of mental qualities developed in the philosophy of mind (Lewis, 1972, Sellars, 1956, Shoemaker, 1975). The fundamental insight of that tradition is that mental qualities, such as red or sour, can be fully characterized by their functional role in the mental economy of an organism without reference to the way that they seem to us in conscious experience. An account of change perception can then be made in terms of mental qualities, functionally understood. The main idea here is that in order for an organism to discriminate among perceptible properties, the relations between them have to be reflected, in some way, by the relations among properties of the organism. Otherwise the organism would routinely get things wrong about what perceptible property they are interacting with. There are several ways of characterizing perceptible properties. One is to appeal to the physical properties of objects and their relations. For example, electromagnetic wavelengths of the visible color spectrum and their relations would be the set of visual perceptible properties for humans. The result would be an asymmetrical threedimensional space that roughly corresponds to the 'CIE 1931 XYZ color space' (Smith & Guild, 1931). Mental colors, on the other hand, would then reflect these relations. This is, roughly, Austen Clark's theory of mental qualities (Clark 1993, 2000). Perceptible properties can also be characterized in terms of the discriminations themselves. On this sort of view, the possible perceptual discriminations that an organism can make will determine what the perceptible properties are. This is approach taken by David Rosenthal (Rosenthal, 1991, 2010). Whatever our view on perceptible properties, mental qualities such as red, sour, and so on, will be the properties of an organism in virtue of which it can make perceptual discriminations. Sets of these properties will form modality specific families, which in turn define spaces of relations. These modality-specific quality spaces define every mental quality without reference to their phenomenal character, that is, without reference to the way that they appear in conscious experience. The functionalist model of mental qualities can be applied across modalities and across perceptual dimensions. Any set of possible discriminations that can be characterized by similarities and differences can be organized into families and spaces. The model fits easily with spatial properties, such as location, shape, and size (Meehan, 2007) and even with pains (Kostic, 2011). 3.2. Awareness of change without consciousness Now that we know how to understand mental qualities without reference to their phenomenal character, that is, to the way that they appear in conscious experience, we can build an account of unconscious change perception. One way to go here is to say that change has its own mental quality space defined by the similarities and differences between changes that we can discriminate. On this view, we perceive a change in the environment in virtue of representing it as a change, whatever that may mean. This view might be appealing at first. When we consciously perceive some change in the environment it appears to be a dynamic process of temporal becoming. This comports with some views on the nature of time, which defend the view that the passage of time is an irreducible feature of reality (e.g. Maudlin 2007, pp. 108-12). Our phenomenology reflects this feature in the dynamism of our subjective sense of time's flow. Michał Klincewicz / Procedia Social and Behavioral Sciences 00 (2014) 000–000 4 But in the present context this is a bad way to go. The main motivation for an account of change perception that relies on its own distinct mental quality is the phenomenology of temporal becoming. However, the functionalist account of mental qualities avoids reference to phenomenology altogether. So, even if there are reasons to accept that change as such appears in our conscious experience, this fact will have no bearing on our understanding of unconscious change perception. Secondly, it is difficult to understand what a quality space of changes would be like. A change from red to green would have its own mental quality, as would a change from red to blue-green. Every mental quality could change to every other in that modality. The resulting quality space for visual mental qualities would have to be at least fourdimensional. And we would also have to account for changes across modalities. Is a change from a blue pentagon that smells sour to a green circle that smells sweet more or less like a similarly complex multi-modal change? It is not at all clear where to start in an answer to such a question. Arguably, what is going on in complex change comparisons does not count as perceptual discrimination at all. Instead, we make judgments about categories that we have identified the objects we are looking to belong to. This means we left the perceptual domain to higher-order cognition. This, along with the problematic appeal to phenomenology, renders the view that changes have their own mental qualities hard to accept. The better option is to think of representations of change in terms of relations between mental qualities. Relations are functions that take one or more things as relata. For example, the relation 'occurred before' takes as its relata at minimum two events. So, occurred before(World War 1, World War 2) specifies a particular relation between two events. Representation of change as a relation involves a two variable function f(x, y) that always takes two mental qualities from the same modality-specific quality space as its relata, e.g., f(red, green) or f(sweet, sour). This function has a temporal ordering. What makes f(red, green) distinct from f(green, red) is the temporal order of the occurrence of these particular mental qualities. This very thing yields a version of the causal view that saves James' original insight and does not come in conflict with what we know from neuroscience. As a reminder, what we know from neuroscience is that change can be perceived unconsciously. This rendered the stronger constitutive version of the dependence relation between perceived change and subjective time problematic. What was needed is an account of perceived change that does not rely on the way that change features in conscious experience. We got such an account by extending some insights from a functionalist theory of mental qualities and specifying the structure of the content of the representations of change in terms of a relation. 3.3. Awareness of time without consciousness We often make temporal discriminations by referencing changes in perceptible properties, such as color or sound. We might perceive lightning as being before the thunder because we perceive the corresponding change in color in the sky to be before the change in sound. Perceived changes play a similar role in our discriminations between durations of events. This is perhaps at the heart of James' observation about the dependence of subjective flow of time on awareness of change. Now we can better understand what that dependence amounts to. The relations that feature in mental representations of change always specify a temporal order. In other words, perceiving change from one mental quality to another has a temporal direction. Each mentally represented change is itself related to other such representations. Formalising a bit, we could specify such relations in the following way: f'( f(red, green), f(green, red)). The result is a temporal ordering of color changes from red to green to red. Similar iterations can specify indefinitely long sequences of orderings. Nestings of relations such as f' define modality-specific, one-dimensional spaces of relations. The relative temporal location of any single change can be characterized by its location in this space, each being either 'before' or 'after' any other. These locations are the temporal qualities-timings and durations-that constitute our subjective flow of time. Timing qualities can be understood as the relative locations of individual changes. So, for example, the perceived change at t1 might be before another perceived change at t2. We can discriminate between them based on their distances from whatever we count as being the present moment. Duration qualities, on the other hand, can be characterized by the distances between any two timing qualities. One timing quality at t1 and another at t2 define a single duration quality. We can discriminate between duration Michał Klincewicz / Procedia Social and Behavioral Sciences 00 (2014) 000–000 5 qualities based on their dimensions on the one-dimensional temporal quality space. Temporal qualities reflect the relations between changes discriminated by an organism in the environment. This means that temporal qualities are characterized in terms of their perceptual role not in terms of the way that one is aware of them subjectively. So, again, we have an account of mental qualities-this time the temporal ones-that follows the basic insight of functionalist accounts of perception. 3.4. Awareness of time with consciousness James' original idea was about the subjective flow of time, that is, the conscious experience of dynamic and seamless passage from one moment to the next. In the previous section we arrived at an account of temporal qualities without any reference to conscious experience. So how do we get from those to subjective flow? The best answer, it seems, is to say that we become aware of our (sensory) mental states and the mental qualities that they have. A venerable tradition in philosophy, which starts at least with Aristotle and continues until contemporary higher-order theories of consciousness, has given us a range of ways to understand how we become aware of our mental states. We might sense them, think about them, be merely disposed to think about them, or have self-referring content feature in every conscious mental state. It is not my aim to weigh in on that debate or to choose one among the various options. The main insight of that tradition is enough to make sense of how we get from unconscious representations of change to the subjective flow of time. When we become aware of the temporal mental qualities that feature in our perceptions we become aware of the network of relations on the one-dimensional temporal quality space. The higher-order strategy can make sense of the dissociations between perception of change and subjective flow in a way in which the constitutive view cannot. Take, for example, an instance of change blindness that results in priming. In this case, change is being unconsciously represented, even though conscious experience presents us with no change at all. At the level of perception we have a representation of change that might look something like this: f'( f(red, green), f(green, red)). This representation is responsible for priming caused by the occurrence of green. This could be measured by, say, reaction time differences. At the level of conscious experience we are aware of no change in color at all. From one moment to the next it is just red and then some more red. This would be reflected in the kind of verbal report that we would give if someone were to ask us what we saw. We would likely say that we saw a bunch of red. Nonetheless, at some point in the past we were not looking at just red. Perhaps we just sat down in front of the computer and at that time the screen was all grey so we are aware of f(gray, red). Consequently we are consciously aware of some change, just not the most recent one. We have a sense of the subjective flow of time, but only relative to that other change in the past that is ever receding into our subjective past. Subjective verbal reports about time inform us about such conscious perceptions of duration and timing. These reports do not inform us, however, about the mechanisms that underlie these perceptions. This is the reason that the constitutive version of James' idea is untenable and the causal version attractive. 4. Conclusion In this paper I defended a version of William James' idea that the subjective flow of time depends on our awareness of change. On the view presented here awareness of change may but need not occur consciously, which makes room for phenomena such as priming during change blindness. Awareness of change involves mental representations of change that specify a one-dimensional space of relations. This space defines temporal qualities, such as timing and duration. The view also offers a way to understand the relation between awareness of change and the subjective flow of time. Subjective flow is the result of us becoming aware of the changes represented in our perceptions. Since these changes specify temporal qualities, we thereby become aware of temporal qualities and the flow of time. Michał Klincewicz / Procedia Social and Behavioral Sciences 00 (2014) 000–000 6 Acknowledgements I am grateful for comments and questions on earlier versions of this paper presented as a talk from audiences at the Haynes Lab meeting at Berlin Center for Advanced Neuroimaging, Busch Lab meeting at the Berlin School of Mind and Brain, Annual Doctoral Student Conference in Philosophy at the University of Szczecin, and School of Psychology Colloquium at the University of Warsaw. References Caudek, C., & Domini, F. (2013). Priming effects under correct change detection and change blindness. Consciousness and Cognition, 22(1), 290-305. Clark, A. (1993). Sensory Qualitie. New York: Oxford University Press. Clark, A. (2000). A theory of sentience. New York: Oxford University Press. Fernandez-Duque, D., & Thornton, I. M. (2000). Change detection without awareness: Do explicit reports underestimate the representation of change in the visual system?. Visual Cognition, 7(1-3), 323-344. Fernandez-Duque, D., & Thornton, I. M. (2003). Explicit mechanisms do not account for implicit localization and identification of change: An empirical reply to Mitroff et al.(2002), Journal of Experimental Psychology: Human Perception and Performance, 29(5), 846-858. Grimes, J. (1996). Scenes across saccades. Perception, 5, 89-110. Henderson, J. M., & Hollingworth, A. (1999). The role of fixation position in detecting scene changes across saccades. Psychological Science, 10(5), 438-443. Herbst, S., Javadi, A. H., & Busch, N. A. (2012). How long depends on how fast–perceived flicker frequencies dilate subjective duration. Journal of Vision,12(9), 141-141. James, W. (1890). The principles of psychology (Vol. 1). New York: Holt. Kostic, D. (2011). The vagueness constraint and the quality space for pain. Philosophical Psychology, 25, 929-939. Laloyaux, C., Devue, C., & Cleeremans, A. (2006). Change blindness to gradual changes in facial expressions. Psychologica Belgica, 46(4), 253268. Lewis, D. (1972). Psychophysical and theoretical identifications. Australasian Journal of Philosophy, 50(3), 249-258. Marcel, A. J. (1983). Conscious and unconscious perception: Experiments on visual masking and word recognition. Cognitive Psychology, 15(2), 197-237. Maudlin, Tim. The metaphysics within physics. New York: Oxford University Press, 2007. Meehan, D. B. (2007). The qualitative character of spatial perception. City University of New York. Ochsner, K. N., Chiu, C.-Y. P., & Schacter, D. L. (1994). Varieties of priming. Current Opinion in Neurobiology, 4(2), 189-194. Rosenthal, D. M. (1991). The independence of consciousness and sensory quality. Philosophical Issues, 1, 15-36. Rosenthal, D. M. (2010). How to think about mental qualities. Philosophical Issues, 20(1), 368-393. Sellars, W. S. (1956). Empiricism and the philosophy of mind. Minnesota Studies in the Philosophy of Science, 1, 253-329. Shoemaker, S. (1975). Functionalism and qualia. Philosophical Studies, 27(May), 291-315. Silverman, M. E., & Mack, A. (2006). Change blindness and priming: When it does and does not occur. Consciousness and Cognition, 15(2), 409-422. Simons, D. J., & Levin, D. T. (1997). Change blindness. Trends in Cognitive Sciences, 1(7), 261-267. Smith, T., & Guild, J. (1931). The CIE colorimetric standards and their use. Transactions of the Optical Society, 33, 73. Wiggs, C. L., & Martin, A. (1998). Properties and mechanisms of perceptual priming. Current Opinion in Neurobiology, 8(2), 227-233.