Quality Space Model of Temporal Perception Michal Klincewicz Graduate Center, City University of New York, Philosophy and Cognitive Science 365 5th Avenue, New York, NY 10016-4309, USA michal.klincewicz@gmail.com Abstract. Quality Space Theory is a holistic model of qualitative states. On this view, individual mental qualities are defined by their locations in a space of relations, which reflects a similar space of relations among perceptible properties. This paper offers an extension of Quality Space Theory to temporal perception. Unconscious segmentation of events, the involvement of early sensory areas, and asymmetries of dominance in multi-modal perception of time are presented as evidence for the view. Keywords: temporal perception, quality space theory, event segmentation, multi-modal perception. 1 Mental Qualities The dominant view in philosophy is that mental states represent by instantiating various representational properties. The nature of these properties remains controversial. Nonetheless, it is widely noted that intentional states, such as thoughts, have intentional content, which can be captured in a clause that follows a mental verb and "that." For example, the sentence "Pam thinks that ripe strawberries are red" captures the content of Pam's thought, which is {ripe strawberries are red}. If Pam chooses to express that thought verbally by saying "strawberries are red," her utterance will reflect that thought's intentional content. The situation is different with qualitative states, such as sensations. The sentence "Pam sees red" ostensibly fails to capture the red quality of Pam's visual sensation of red. Pam's verbal utterance "I see red" expresses her judgment that she is seeing red, but does not express the qualitative character of her red experience. Consequently, the qualitative character of sensations might seem to be ineffable. And contrasted with the relative ease with which we appear to be able to verbally express the content of thoughts, this might lead one to think that qualitative character is mysterious and perhaps even beyond scientific description.  I'm indebted to Valtteri Arstila and especially David Rosenthal for input on ideas contained herein. I would also like to thank an anonymous reviewer of the present volume for many insightful and challenging comments. Any mistakes are entirely my own. A. Vatakis et al. (Eds.): Time and Time Perception 2010, LNAI 6789, pp. 230–245, 2011. c© Springer-Verlag Berlin Heidelberg 2011 Quality Space Model 231 The seeming mysteriousness of qualitative character is the result of an assumption about a close connection between mental qualities and conscious experience. We know about mental red, it seems, only from conscious experiences of red things. Consequently, it might appear that qualitative character is present only in consciousness. This assumption carries over to all qualities of experience, including the temporal ones. The qualitative character of, for example, the passing of time is something that is often thought to be manifest only in consciousness. In fact, there is a tradition in philosophy of making an explanation of the temporal aspect of consciousness central to a theory of consciousness itself [1–3]. But the assumption that mental qualities and consciousness are connected in this way is questionable. And there are alternative theories of mental qualities that do not come saddled with the assumption about their connection to conscious experience. According to some alternative views, we can describe and individuate mental qualities independently of how they appear in conscious experience. One class of such theories relies on perceptual role. Proponents of this approach focus on the discriminations that an organism makes in its interactions with the environment. On perceptual role views, the states of the organism that enable it to make discriminations instantiate mental qualities, so we can define mental qualities in terms of the discriminations that they make possible. The states that enable an organism to discriminate must reflect, in some way, the perceptible properties in the environment, such as surface reflectance or chemical composition. But these states do not themselves instantiate the perceptible properties–the properties that allows an organism to discriminate the electromagnetic spectrum are properties of the organism, not of the electromagnetic spectrum. However, the organisms states must be such as to enable the discriminations, and reflect the discriminable differences between the perceptible properties. There are several versions of the perceptual role approach. Some focus on the causal and informational relationships between perceptible properties and mental qualities [4]. But, for various reasons, the causal and informational approach is unappealing as an account of qualitative character manifest in conscious experience. It is especially problematic for conscious sensations of time [5]. There are also other approaches, which defend the special role of consciousness in identifying mental qualities, by making a case for the distinction between access and phenomenal consciousness [6]. There also other views, such as naive realism or representationalism. But since this is a paper about a particular view of temporal mental qualities, and not a survey, I will not expand on these. Instead, for the purposes of this paper, I will simply assume the perceptual role strategy is most promising and its best version is Quality Space Theory (QST), also known as Homomorphism Theory [7, 8]. QST is the view that the states of the organism that enable it to make discriminations between stimuli are related to each other in ways that parallel the ways that perceptible properties 232 M. Klincewicz are related. QST is a holistic theory of mental qualities, in that it defines them as members of families of other similarly related properties. According to QST, the relationships between the perceptible properties are paralleled in the relationships between states of the organism that allow it to make discriminations among them. Given this, QST takes advantage of the similarities and differences between perceptible properties, such as red being more similar to orange than to blue, or C-sharp being closer to D than to A-flat. These similarities and differences are reflected, according to QST, in the relations between mental red and mental orange, and mental C-sharp, D, and A-flat. Similarities and differences naturally form distance metrics, which in turn define spaces. Perhaps the most famous of these is the three-dimensional space of the color spectrum, which represents all similarity and difference relations that hold between individual hues and saturations [9]. According to QST, the discriminations an organism can make among visually perceptible properties defines a similar space of relations [8]. So when Pam sees a ripe strawberry, she is in a qualitative state that bears a set of relations to other states of the same kind. Because these relations reflect the relations that a particular visible property bears to a family of properties in the same family, Pam's state enables her to discern and respond differentially to the ripeness of the strawberry. The mental quality of Pam's state reflects those relations in the relevant way. According to QST, it is an entirely different question why Pam's conscious experience appears to her to have the qualitative character that it does. Mental red is, according to QST, a property relevantly located within a space of similar properties, all of which play a role in visual perception. Qualitative redness of Pam's conscious experience is explanatorily idle as far as her visual discriminations are concerned. Given this, QST does not need to appeal to the qualitative character manifest in conscious experience to describe individual mental qualities. According to QST, as with other perceptual role accounts, discriminations are enough. This avoids mysterianism, and lends the account to empirical confirmation. However, the limited sketch of QST I've given here does not yet yield an explanation of more complex mental qualities, such as shape, location, timing, and duration. The theory has been extended to account for mental qualities relevant to perception of space, and I direct the interested reader to the relevant literature [8, 10, 11]. This paper offers an extension of QST to temporal mental qualities, such as duration and timing. There are several existing hypothesis about the neurbiological underpinnings of temporal perception. Among them are various inner clock models, which posit a dedicated time mechanism, often in the cerebellum and the basal ganglia [12–18]. Alternatives to the dedicated models posit distributed mechanisms that keep track of energy levels in neuron populations or patterns of neural activation [19–21]. It is important to note that the extension of QST to temporal perception that is presented in the last section of this paper is not meant to be in competition Quality Space Model 233 with those or any other neurobiological hypotheses. The model on offer here is stated at the psychological level of description, not at the neurobiological level. Nonetheless, in the final analysis QST and/or its extension to temporal perception could turn out to be incompatible with some or all of the abovementioned neurobiological hypotheses. Hence, before the temporal quality space model is stated I will outline some of the constraints on a philosophical theory of mental time set by the significant amount of psychophysical and neuroimagining research into the mechanisms of time perception. A philosophical theory of mental time should, of course, be compatible with these results, but it should also aim to illuminate how they hang together. The temporal quality space model does both of these things and this is a distinct advantage of the model. 2 Constraints 2.1 Temporal Boundary Detection and Filling-in Durations This section offers evidence for two constraints on a successful philosophical theory of mental time from a selected number of avenues of research into temporal perception. The two constraints are: (1) the theory should predict that temporal mental qualities can be instantiated unconsciously; (2) the theory should predict that each sensory modality has its own proprietary temporal qualities. Both are predictions of the extension to QST I will offer in the last section. We know that the mechanisms responsible for processing the timing of stimuli have a consistent sampling rate. For humans, this rate is approximately 30 milliseconds and holds for all modalities [22]. Distinct stimuli have to be separated by at least 30 milliseconds to be perceived as successive. Otherwise they appear simultaneous. There is also evidence that there is a second timing mechanism that operates with a 3 second sampling window, which is independent of the one operating at 30 milliseconds. Evidence for this second mechanism comes from ambiguous stimuli such as the necker cube, which alternate their perceptual interpretation approximately every 3 seconds [23]. Similarly, sequences of phonemes such as CUBA-CU alternate between the CUBA interpretation and BACU interpretation approximately every 3 seconds [24]. The 3 second sampling rate can be found in a number of other studies of perception in all modalities, which all suggest that every 3 seconds "the brain asks: 'what is new?' " in the perceptual input [22]. The relatively constant sampling rates at 30 milliseconds and at 3 seconds suggest that the perceptual system breaks the input stream into units and tracks changes between them. When a relevant difference is detected, the sensory system either marks the onset of a completely new stimulus or marks the onset of a change in an existing stimulus. When no difference is detected between successive units, the system treats the stimulus as extended in time, that is, as having an extended duration. 234 M. Klincewicz There are at least two levels at which the machinery of temporal perception can be analyzed, and these sampling rates probably reflect the operation of different functional levels of the perceptual hierarchy. On this hierarchical model, the sensory system first provides a temporal frame and then passes it onto the next level in the processing hierarchy, which is responsible for generating temporal perceptual content [25, 26]. The generation of content could be done by an inner clock or one of the other neurobiological mechanisms I mentioned in the first section or by some other process. A critical issue for supporting constraint (1) is whether the perceptual system can detect change, and thereby mark the onset or offset of a stimulus, unconsciously. In other words, can the brain ask "what is new?" without us being aware of it? Below I outline some of what I take to be the most suggestive evidence that it can, even though some of the evidence is indirect and my discussion of it speculative. The first piece of evidence comes from event segmentation in the visual system. In one fMRI study, participants were asked to watch several uncut movies of everyday activities such as making a bed [27]. Each movie was shown three times. During the first presentation of the movie, participants were asked to simply pay attention. In the next two presentations of the movie, participants were asked to segment the movies into events that were meaningful to them and to press a button to mark the beginning of one event and the end of another. In the second viewing, participants were asked to segment in a coarse-grained way. In the third viewing, they were asked to segment finely. fMRI recordings taken during these two active trials were then compared with fMRI recordings taken during the initial passive viewing. The prediction here was that the differences between recordings would uncover the mechanism of active event segmentation. The fMRI recordings taken during active segmentation showed significant activation in areas V5 (MT) FEF, and V1. Furthermore, when time-locked to the active segmentation times, the imagining data obtained from the passive viewing showed similar activation. The same visual areas were active in all three viewings, even though in one of them the observer was not consciously tracking onset and offset of events. This result suggests that the brain tracks temporal event structure in virtue of a visual mechanism, which is sensitive to the timing of the onset and offset of a stimulus. V5 (MT) activations during the passive viewing indicate that temporal boundaries are sensed even when the observer is not aware doing so [28]. Segmentation of events involves the detection of temporal boundaries of a stimulus. Such boundaries are treated by the sensory system as markers of onset and offset. My interpretation of these results in context of the hierarchical model of temporal perception is that V1 and V5 (MT) are the earliest level of temporal processing and set up the temporal frame. The frame set up by this mechanism is then passed on to higher levels of the perceptual processing hierarchy, where event structure is explicitly represented. Importantly for the present discussion, Quality Space Model 235 event structure is represented without the perceiver being aware of it. If this interpretation is correct, the just mentioned study informs us about some of the neural mechanisms involved in processing the onset and offset of visual events and supports the view that the mechanism that does this can operate without us being aware of it any way. But, of course, temporal perception does not limit itself to the detection of onset and offset of stimuli. The other important dimension of temporal perception is duration, which does not always correspond to the boundaries set by onset and offset. There is some indirect evidence, however, that duration can be perceived unconsciously as well. Stimuli are perceived to have duration when they are perceived as unchanged, that is, when no offset is detected. So when no offset is detected two distinct stimuli presented close to each other and in succession can appear to be fused into one moving stimulus. This illusory effect is sometimes referred to as apparent motion. Several fMRI studies of apparent motion show that area V1 is active both when the two stimuli are not fused, and when they are, as during apparent motion. However, area V5 (MT) is more active during apparent motion [29, 30]. This suggests that V5 (MT) is involved in the filling-in between the temporal boundaries set by the pair of distinct stimuli. Its important to note that these are the same visual areas implicated in the abovementioned fMRI study of event segmentation. But a mere activation pattern does not tell us what role V5 (MT) has in that filling-in. To explore the role that V5 (MT) plays in the effect, V5 (MT) activation would have to not only be correlated with the effect, but also shown to be its cause. This was the aim of another fMRI based study, which used activation patterns in apparent motion effects to model connections between V1 and V5 (MT) [31]. The prediction in that study was that activity in the neural connections between V5 (MT) and V1 could be correlated with the filling-in of individual parts of an illusory curve created in V1 by apparent motion. As predicted, the model that best fit the data had no lateral connections in V1, but lots of feedback from V5 (MT). The authors conclude that V5 (MT) has causes the filling-in of the path of the illusory stimulus. Supporting the view that V5 (MT) is involved in filling-in between temporal boundaries, repetitive TMS to V5 (MT) reduces the apparent motion effect [32]. And without the filling-in between onset and offset, a single stimulus with an extended duration is perceived as two distinct stimuli with shorter durations. This suggests that visual detection of duration is impaired without V5 (MT). On the interpretation I have proposed above in connection with event segmentation, V1 and V5 (MT) are involved in setting up the temporal frame needed for further processing at the level of content. I have speculated that V5 (MT) is involved in the filling-in that is relevant to discriminations of duration. Together, this suggests that the early visual system and V5 (MT) in particular, is involved in processing timing as well as the duration of stimuli. 236 M. Klincewicz But can all of this happen unconsciously? There is strong independent evidence that V5 (MT) can operate without the involvement of awareness. And if that is so, we should suppose that whatever role V5 (MT) has in temporal perception, at whatever level of the processing hierarchy, can occur unconsciously. The strongest evidence for this claim, it seems to me, comes from blindsighters, who typically have focal damage to V1 and are not aware of having visual sensations in some portion of their visual field. Blindsighters tend to perform significantly above chance in visual discrimination tasks that involve the part of the visual field in which they report having no visual sensations [33]. And in particular, they do well in tasks that involve rapid motion detection [34, 35]. Motion illusions are particularly telling here. In normal perceivers, a square visual stimulus followed by a rectangular visual stimulus presented next to the square stimulus results in a motion aftereffect in which the rectangle appears to be drawn away from the location of the square. This effect is usually referred to as the line motion illusion. When this stimulus is presented in the blindsighted participant's (G.Y.) blind visual field, they are significantly above chance in telling the direction of apparent motion [36]. G.Y. is susceptible to the motion aftereffect of the line motion illusion, even though they report having no relevant conscious visual sensations. This shows that motion processing carried out by V5 (MT) can occur without awareness. That is already enough to give some credence to the hypothesis that the mechanisms of temporal perception function without awareness. Together with the evidence about unconscious segmentation, we have reason to think that both timing and duration can be discriminated unconsciously. A philosophical theory of sensation of time should be compatible with this claim. And this is my case for constraint (1), which states that mental qualities can be instantiated unconsciously. Furthermore, the studies mentioned above also give us some idea of how the visual system processes time. The mechanisms responsible for the sensation of timing depend on the detection of temporal boundaries. The early visual area V1 is critical to this process. Visual mechanisms responsible for the sensation of duration depend on filling-in between the temporal boundaries detected by V1. I have speculated that the early visual area V5 (MT) is critical to that process. Other modalities have neural structures that are functionally analogous to V1 [37–39]. Hence it is likely that each sensory modality has distinct mechanisms for temporal boundary detection and filling-in. Given this, the abovementioned constraints about unconscious processing of time and involvement of early sensory areas can probably be generalized to other modalities, but that is further speculation that is beyond the scope of this paper. Temporal processing in early sensory areas suggests that each sensory modality can process time independently of the others. And if a stronger case can be made for this view, it can support constraint (2), which is that each sensory modality has its own proprietary temporal qualities. In the next section I give some more evidence that I take to support constraint (2). Quality Space Model 237 2.2 Cross-Modal Effects and Time The first problem a multi-sensory system faces lies in the physical differences between the stimuli and the sensory organs that detect them. Light travels faster than sound, for example, and requires more neural processing. And the sense of touch depends on the transmission of signal from variously spaced nerve endings; a signal that starts in the foot has a larger distance to travel than a signal that starts in the neck. Given all these differences, the perceptual system faces a substantive computational challenge in syncing up information from different modalities. Especially pressing here is the challenge of representing the timing of the stimulus accurately relative to other stimuli. Multi-sensory integration of temporal information has been studied extensively, but little is yet known about the mechanisms that underlie it [40]. What seems clear, however, is that the brain compensates for cross-modal differences by treating some signals as originating in a single source. Presumably, this is carried out by a modality-neutral mechanism, which takes information from distinct modalities and integrates them into a final temporal percept. This modality-neutral mechanism matches input streams from different modalities and syncs them up in an appropriate way. But positing such a mechanism goes only so far. The perceptual system does not seem to treat input from different modalities in the same way. And various effects support the view that at the later stages of the perceptual processing hierarchy, where timing and duration are represented, different sensory modalities represent time differently. In one study concerned with multi-modal perception, participants were presented with a sound and a light with up to a 200 millisecond delay between them [41]. The distance between the origin of the stimulus and the perceiver varied from 1 to 32 meters. And the stimuli themselves also varied in intensity. The participants were asked to press one of two buttons to indicate whether the sound or the light occurred first. Analysis of the reaction times recorded with these button presses shows that simultaneous audio-visual pairs are perceived as being simultaneous despite differences in the time it takes the signal to get from its source to the sense organ. This effect is simultaneity constancy. As with other types of perceptual constancy, simultaneity constancy allows us to perceive things as constant across variations in incoming signal. Color constancy, for example, allows us to perceive a green wall as being the same color, even though what we actually see is a large number of different shades of green. Simultaneity constancy allows us to perceive signals that arrive at our sensory organs at different times, and which are processed at different rates, as occurring at the same time. In color constancy, there is a point when the shade differences are too pronounced to go unnoticed. This can happen when the wall is illuminated by a spotlight, for example, or dimmed by a shadow. Similarly, two simultaneously occurring stimuli will be perceived as non-simultaneous if the two signals reach the perceiver at too great a temporal distance apart [42]. 238 M. Klincewicz Simultaneity constancy also encompasses touch. In one study, participants were first presented with visual and tactile stimuli and then asked to respond as quickly as possible by pressing a button [43]. Their reaction times to visual stimuli were constant, but reactions to tactile stimuli were slower the further away the stimulus was from the brain. The obtained results allowed the experimenters to create a function that could then be used to predict the temporal window in which differently located visual and tactile stimuli are perceived as simultaneous. In the second experiment of that study, the participants were presented with pairs of variously offset (0-200 milliseconds) visual-visual and visual-tactile stimuli located on different parts of their body. For example, a light on the foot and a tap on the hand. The participants were then asked to press one of two buttons to indicate which of the two stimuli came first. The results were then compared with the data collected in the first experiment. The participants' reaction times to visual-visual pairs presented at different body parts resulted in same reaction times and the same point of simultaneity, as those predicted in the first experiment. Tactile-tactile pairs presented at different body parts resulted in a different pattern of reaction times depending on the distance that the signal has to travel to the brain via the nervous system, also as predicted by experiment one. However, the point at which two tactile stimuli were perceived as simultaneous was slightly different than the point of simultaneity predicted by the function obtained from the first experiment. The reaction times to pairs of visual-tactile stimuli presented at different body parts did not differ from the prediction in the first experiment at all. But, importantly, pairs of visual-tactile stimuli presented to the same body part did not match the predictions of the same experiment. Visual-tactile pairs presented in the same location were treated as if they were one event, ignoring the differences in the signals. This result indicates that the mechanism coordinating visual and tactile timing compensates for processing time differences across those modalities to maintain simultaneity constancy. In the third experiment the participants were exposed to a 5 minute series of visual-auditory stimuli pairs with a 250 millisecond interval between them. As a consequence of this exposure, the participants reaction times changed in such a way as to move the point of subjective simultaneity by 40 milliseconds. So a sound needed to be presented 40 milliseconds earlier than the light to achieve the same point of subjective simultaneity that was obtained in the first experiment. After this training, the participants were also shown visual-tactile stimulus pairs. And, strikingly, there was no shift in their subjective judgments of simultaneity for visual-tactile pairs. So the adaptation effect from a stream of visualauditory pairs affected simultaneity judgments for consequent visual-auditory pairs, but not for visual/tactile pairs. The asymmetry of this effect suggests that the mechanism responsible for simultaneity constancy effect between audition and vision is distinct from the mechanism that handles simultaneity constancy between touch and vision. Given this, the third experiment of this study together with other similar studies of Quality Space Model 239 simultaneity constancy supports the view that timing is processed differently across modalities [44, 45]. Of course, it is possible that modality-specific adaptation effects concern processes that are earlier than any part of the processing hierarchy relevant to perceived simultaneity. They could be the result of a process in LGN, for example, which presumably precedes the level of perceptions1. This alternate interpretation of the abovementioned effects calls in question their value as support for constraint (2). However, there are independent avenues of research that allow us to speculate that the LGN has a more substative role in perception, and is not merely a relay station for the signal coming from the retina. This is suggested by the complex circuitry of the LGN [46, 47], and also by the increased bilateral connections between the LGN and V5 (MT) in blindsighted patient G.Y. [48]. In addition, there is also some evidence that the LGN can be affected top-down by selective attention [49]. Nonetheless, I recognize that this as a point at which a reader could resist my interpretation of the evidence and defer to future research in settling the issue. But even though the significance of these adaptation effects is not uncontroversial, and more work needs to be done, they at least motivate constraint (2). But all of this is a little bit besides the point, given the perceptual role strategy I assumed in the beginning of this paper. On to the perceptual role strategy, mental qualities are individuated by the discriminations that an organism can make and their role in the organism's overall mental economy. The asymmetry in the aforementioned adaptation effects is a systematic difference in a perceiver's ability to discriminate, and as such are relevant to individuating mental qualities. Visual, auditory, and tactile discriminations. And this is true whatever the nature of the underlying neurobiological machinery–LGN included. And there is another avenue of research that supports the claim that each sensory modality processes and represents temporal information differently. Auditory information about timing usually affects how timing is processed in vision, but visual information about timing does not equally affect how timing is processed by audition. One striking example of this is the temporal ventriloquism effect in which an auditory stimulus changes the perceived timing of a visual stimulus, moving its perceived timing closer to the timing of the auditory stimulus [50, 51]. In general, audition usually dominates vision when it comes to timing, even though this is not always the case. However, this dominance is so pronounced, that it can even result in completely illusory stimuli. For example, when a flash is accompanied by more than one beep, the flash is perceived to occur twice [52]. If all sensory mechanisms processed timing in the same way, there would be no such asymmetry–all timing information would be treated the same and have equal importance in determining timing. 1 I am grateful to an anonymous reviewer of this volume for pointing out this possibility. 240 M. Klincewicz In the case of multi-modal perception of duration the direction of dominance is different. Visual information can, to some extent, affect auditory perception of duration, but auditory information does not affect visual perception of duration to the same extent. In one study of multi-modal perception of duration, participants were presented with a stream of steady (not looming) stimuli interrupted by a looming stimulus (disk increasing in size or upward frequency-modulated sweep) in the same modality [53]. The reports of the participants indicate a subjective dilation of the duration of the looming stimulus, that is, the looming stimulus is consistently judged to be longer than it is. And when a series of looming stimuli is interrupted by a steady stimulus, the steady stimulus is judged to be of shorter duration than it is. This holds for visual and visual-auditory stimuli equally. But if a series of looming visual or visual-auditory stimuli is interrupted by a steady auditory stimulus, no subjective time dilation occurs for the auditory stimulus. So while looming auditory streams affect visual stimuli, looming visual and visual-auditory streams do not affect auditory stimuli. This indicates that duration distortions do not transfer from vision to audition. Similar asymmetry occurs when the presented series is composed of steady visual stimuli paired with steady auditory stimuli [54]. The duration judgments of an odd-ball looming auditory stimulus presented after such a series are typically accurate. This indicates that visual information blocks the auditory dilation effect that would occur if the steady stream were composed of only auditory stimuli. Finally, when steady visual stimuli are paired with looming auditory stimuli in a stream, the subjective duration of the oddball steady auditory-visual stimulus is not compressed. Again, no dilation occurs because of the influence of information about duration of the event coming in from vision. The asymmetry demonstrates the dominance of vision over audition in perception of duration. On its own, asymmetrical dominance across modalities does not show that each modality has its own proprietary time mechanism, even if it would hold in all cases, which it does not. The abovementioned effects could be explained on a model on which timing information is processed by the same mechanism, but with different emphasis given to different modalities2. But that does not affect the relevance of dominance effects in supporting constraint (2). Regardless of which computational model turns out to be correct–multiple mechanisms or a single mechanism–there are pronounced differences in temporal perception at the level of content. And this is where the dominance affects are important in constraining a theory of temporal mental qualities. The architecture of the mechanisms that process temporal information before we get to the top level of the processing hierarchy does not matter much in this case. What matters in the dominance effects and also in the adaptation effects I mentioned above is that information about timing and duration is systematically not processed in the same way across different modalities. Auditory representations of timing do not have the same perceptual role as visual representations of timing or tactile representations of timing. And, on the perceptual role account I 2 I am grateful to an anomymous reviewer of this volume for stressing this point. Quality Space Model 241 take for granted in this paper, such a difference matters to individuating mental qualities and categorizing them into modality-specific families. 3 Temporal Mental Qualities In the previous section I presented several avenues of research that I take to support two constraints on a philosophical theory of temporal mental qualities. Constraint (1) is that temporal mental qualities can occur unconsciously. Constraint (2) is that that each sensory modality has its own proprietary temporal qualities. The evidence I presented is empirical, and therefore open to a number of challenges. Hence the case I made for these two constraints is speculative, and independent of whatever merits QST or its extension to temporal perception might otherwise have. In this section, I offer a model of temporal mental qualities that I take to be compatible with (1) and (2), as well as provide a unifying theoretical framework in which temporal perception can be better understood. Similarly to the way that spatial boundaries of basic perceptible properties allow organisms to discriminate shapes, temporal boundaries of such properties allow them to discriminate the timing of an event. And the similarities and differences between these individual temporal boundaries can define a space of relations within which each perceptible timing property is located. Unlike other spaces of perceptible properties, such as the visible colors, timing properties are related to each other along a single axis. Temporal boundaries of a stimulus can be related to each other only by two relations: before and after. Two simultaneously occurring boundaries bear the same set of before and after relations to other boundaries, and can be distinguished from each other only along other, non-temporal dimensions, such as hue or location. On this view, timing properties are nothing but the temporal boundaries of other more basic perceptible properties. So an organisms ability to make timing discriminations depends on its ability to discriminate between more basic properties. This comports with constraint (2), according to which each sensory modality has its own set of temporal mental qualities. For example, the onset of a red stimulus on a white background at t=1 occurs after t=0 and before t=3 and is further from t=6 than from t=4. The offset of the same red stimulus at t=5 occurs at a point at which it is replaced by the background white in the same location. Just as on a number line, where every number bears either a 'before' or 'after' relation to every other number on the line, the timing (onset or offset) of a particular perceptible property is related to all of the others by one of these two relations. Just as with other perceptible properties, any perceptible timing property has a corresponding mental quality. And mental timing qualities play a perceptual role in the organisms mental economy, which allow it to make timing discriminations. This is because the similarities and differences that define the one-dimensional space of timing properties are reflected in the temporal discriminations that an organism can make. 242 M. Klincewicz For example, a mental quality that is relevant to the organism discerning the onset of a stimulus at t=1, will be closer in the corresponding temporal mental quality space to mental t=0 than to mental t=3, and further from mental t=6 than from mental t=4. In this way, any single timing quality will be similar and different to other timing qualities in ways that parallel the similarities and differences between the perceptible timing properties. Paralleling the structure of the space of relations that define perceptible timing properties, the mental temporal quality space is one-dimensional. The temporal quality space has the structure of a number line, which naturally incorporates 'before' and 'after' relations, that hold between the timing of perceptible events. The mapping between the one dimensional temporal quality space and the one dimensional structure of time enables the organism to make the relevant timing discriminations. Of course, in addition to sensing the timing of a stimulus, an organism can also sense it as enduring for a period of time. But we do not need to add a new dimension to the one-dimensional temporal quality space to accommodate duration. Duration is reflected in the relations that hold between individual timings. When a red stimulus is onset at t=1 and offset at t=3, two visual timing qualities are involved, one for each of the temporal boundaries of the event. And the two mental timing qualities that correspond to t=1 and t=3 constitute the temporal mental boundaries of the occurrence of the more basic mental qualities. Together, these boundaries define the duration of the red stimulus. Consequently, the two timing qualities can also represent the duration of the red stimulus, just as subtracting the value of t=1 from t=3 is sufficient to represent real time duration of the event (as 2 seconds, for example). Consequently, the relations between mental timings can also define other relations, such as 'longer than' and 'shorter than,' which are the relations that hold between durations. Mental duration defined by t=1 and t=3 is shorter than the mental duration defined by t=4 and t=7. The one-dimensional temporal quality space is sufficient to enable an organism to represent and respond differentially to both timing and duration; there is no need for an extra dimension or quality space for duration. Nonetheless, discriminations of duration are possible only when the temporal mental qualities are available and the temporal distance between them is discerned as being filled-in with sufficiently similar mental qualities. When the basic qualities change, that will usually mark a new temporal boundary. And when no change is detected, no new timing quality will be instantiated. Consequently, the duration of a stimulus (or background) will be discerned to be longer. But nothing over the mental qualities relevant to timing are necessary in this process–durations, and possibly other, more complex temporal qualities are built up out of them and their relations. On the temporal quality space model I offered above, timing qualities define the boundaries of duration qualities. And similarly to the way that the boundaries of colors define shape, temporal mental qualities corresponding to Quality Space Model 243 perceived timing define the temporal boundaries between occurrences of more basic qualities such as color. The similarities and differences that an organism can discriminate between temporal properties of objects reflect the similarities and differences between corresponding temporal qualities. The resulting similarity metric reflects the temporal properties that an organism can discriminate and how many discriminations it can make between temporal properties. This metric in turn defines a temporal quality space for a particular modality. And, reflecting the one-dimensional nature of perceptible time, all of the modality-specific temporal quality spaces are themselves onedimensional. On this view, time discrimination depends on an ability to discriminate more basic mental qualities such as color and sound. This makes the temporal quality space model compatible with constraint (2). Each modality has its own quality space of temporal mental qualities that is independent of similar quality spaces in other modalities. Nothing in the temporal quality space model violates constraint (1), either. Temporal mental qualities are defined by their perceptual role, not by the temporal qualitative character of conscious experience. Given this, mental qualities are themselves independent of conscious experience, and can be instantiated without us being aware of it in any way. This QST model of temporal perception also implies that temporal judgments do not have to coincide with temporal perception. A subjective duration judgment might indicate a duration of 1 second, say, while performance on an objectively measured duration discrimination task might indicate that the perceptual system represents a duration of 1.5 seconds. On the view offered here, an account of the temporal aspect of conscious experience relevant to subjective judgments demands a theory that is distinct from a theory of temporal perception. References 1. Kant, I.: Critique of Pure Reason. St. Martins Press, New York (1965/1781) 2. Husserl, E.: On the Phenomenology of the Consciousness of Internal Time (18931917). Kluwer, Dordrecht (1990/1928) 3. Bergson, H.: Time and Free Will: An Essay on the Immediate Data of Consciousness. Dover, Mineola NY (2001/1913) 4. Dretske, F.: Experience as Representation. Philosophical Issues 13(1), 67–82 (2003) 5. Dennett, D.C., Kinsbourne, M.: Time and the Observer: The Where and When of Consciousness in the Brain. Behavioral and Brain Sciences 15, 183–201 (1992) 6. Block, N.: Consciousness and Cognitive Access. Proceedings of the Aristotelian Society 108 (1pt3), 289–317 (2008) 7. Sellars, W.: Empiricism and the Philosophy of Mind. Harvard University Press, Cambridge MA (1997/1956) 8. Rosenthal, D.M.: Consciousness and Mind. Oxford University Press, USA (2005) 9. Broadbent, A.D.: A Critical Review of the Development of the CIE1931 RGB Color-matching Functions. Color Research and Application 29(4), 267–272 (2004) 10. Meehan, D.B.: Qualitative Character and Sensory Representation. Consciousness and Cognition 11(4), 630–641 (2002) 244 M. Klincewicz 11. Meehan, D.B.: The Qualitative Character of Spatial Perception. Doctoral Thesis, City University of New York (2007) 12. Rammsayer, T., Ulrich, R.: Counting Models of Temporal Discrimination. Psychonomic Bulletin and Review 8(2), 270–277 (2001) 13. Ivry, R.B.: The Representation of Temporal Information in Perception and Motor Control. Current Opinion in Neuropbiology 6(6), 851–857 (1996) 14. Miall, R.C.: The Storage of Time IntervalsUsing Oscillating Neurons. Neural Computation 1(3), 359–371 (1989) 15. Wearden, J.H., Edwards, R., Fakhri, M., Percival, A.: Why 'sounds are judged longer than lights': Application of a Model of the Internal Clock in Humans. Quarterly Journal of Experimental Psychology 51B, 97–120 (1998) 16. Creelman, D.C.: Human Discrimination of Auditory Duration. The Journal of the Acoustical Society of America 34(5), 582–593 (1962) 17. Treisman, M., Faulkner, A., Naish, P.L.N., Brogan, D.: The Internal Clock: Evidence for a Temporal Oscillator Underlying Time Perception with Some Estimates of its Characteristic Frequency. Perception 19(6), 705–743 (1990) 18. Gibbon, J.: Scalar Expectancy Theory and Weber's Law in Animal Timing. Psychological Review 84, 279–325 (1977) 19. Mauk, M.D., Buonomano, D.V.: The Neural Basis of Temporal Processing. Annual Review of Neuroscience 27, 307–340 (2004) 20. Karmarkar, U.R., Buonomano, D.V.: Timing in the Absence of Clocks: Encoding Time in Neural Network States. Neuron 53(3), 427–438 (2007) 21. Pariyadath, V., Eagleman, D.M.: The Effect of Predictability on Subjective Duration. PLoS One 2(11), e1264 (2007) 22. Pöppel, E.: A Hierarchical Model of Temporal Perception. Trends in Cognitive Sciences 1(2), 56–61 (1997) 23. von Steinbüchel, N., Wittmann, M., Pöppel, E.: Timing in Perceptual and Motor Tasks after Disturbances of the Brain. Advances in Psychology 115, 281–304 (1996) 24. Pöppel, E.: Lost in Time: A Historical Frame, Elementary Processing Units and the 3-Second Window. Acta Neurobiologiae Experimentalis 64(3), 295–302 (2004) 25. Pöppel, E.: Temporal Mechanisms in Perception. International Review of Neurobiology 37, 185–202 (1994) 26. Pöppel, E.: Taxonomy of the Subjective: an Evolutionary Perspective. In: Brown, J.W. (ed.) Neuropsychology of Visual Perecption, pp. 219–232. Lawrence Erlbaum Associates, Hillsdale NJ (1989) 27. Zacks, J.M., Braver, T.S., Sheridan, M.A., Donaldson, D.I., Snyder, A.Z., Ollinger, J.M., Buchner, R.L., Raichle, M.E.: Human Brain Activity Time-locked to Perceptual Event Boundaries. Nature Neuroscience 4(6), 651–655 (2001) 28. Tong, F.: Brain at Work: Play by Play. Nature 4(6), 560–562 (2001) 29. Muckli, L., Kriegeskorte, N., Lanfermann, H., Zanella, F.E., Singer, W., Goebel, R.: Apparent Motion: Event-Related Functional Magnetic Resonance Imaging of Perceptual Switches and States. Journal of Neuroscience, 1–5 (2002) 30. Muckli, L., Kohler, A., Kriegeskorte, N., Singer, W.: Primary Visual Cortex Activity Along the Apparent-Motion Trace Reflects Illusory Perception. PLoS Biology 3(8), 1501–1510 (2005) 31. Sterzer, P., Haynes, J., Reesa, G.: Primary Visual Cortex Activation on the Path of Apparent Motion is Mediated by Feedback from hMT+/V5. NeuroImage 32(3), 1308–1316 (2006) 32. Matsuyoshi, D., Hirose, N., Mima, T., Fukuyama, H., Osaka, H.: Repetitive Transcranial Magnetic Stimulation of Human Mt+ Reduces Apparent Motion Perception. Neuroscience Letters 429(2-3), 131–135 (2007) Quality Space Model 245 33. Rees, G.: The Anatomy of Blindsight. Brain 131(6), 1414–1415 (2008) 34. Azzopardi, P., Cowey, A.: Motion Discrimination in Cortically Blind Patients. Brain 124(1), 30–46 (2001) 35. Goebel, R., Muckli, L., Zanella, F.E., Singer, W., Stoerig, P.: Sustained Extrastriate Cortical Activation without Visual Awareness Revealed by fMRI Studies of Hemianopic Patients. Vision Research 41(10-11), 1459–1474 (2001) 36. Azzopardi, P., Hock, H.S.: Illusory Motion Perception in Blindsight. Proceedings of the National Academy of Sciences 108(2), 876–881 (2011) 37. Pantev, C., Bertrand, O., Eulitz, C., Verkindt, C., Hampson, S., Schuierer, G., Elbert, T.: Specific Tonotopic Organizations of Different Areas of the Human Auditory Cortex Revealed by Simultaneous Magnetic and Electric Recordings. Electroencephalography and Clinical Neurophysiology 94(1), 26–40 (1995) 38. Penfield, W., Rasmussen, T.: The Cerebral Cortex of Man: A Clinical Study of Localization of Function. Macmillan, New York (1950) 39. Verhagen, J.V.: The Neurocognitive Bases of Human Multimodal Food Perception: Consciousness. Brain Research Reviews 53, 271–286 (2007) 40. Vroomen, J., Keetels, M.: Perception of Intersensory Synchrony: A Tutorial Review. Attention, Perception, and Psychophysics 72(4), 871–884 (2010) 41. Kopinska, A., Harris, L.R.: Simultaneity Constancy. Perception 33(9), 1049–1060 (2004) 42. Dixon, N.F., Spitz, L.: The Detection of Auditory Visual Desychrony. Perception 9(6), 719–721 (1980) 43. Harrar, V., Harris, L.R.: Simultaneity Constancy: Detecting Events with Touch and Sound. Experimental Brain Research 166(3), 465–473 (2005) 44. Harrar, V., Harris, L.R.: The Effect of Exposure to Asynchronous Audio, Visual, and Tactile Stimulus Combinations on the Perception of Simultaneity. Experimental Brain Research 186(4), 517–524 (2008) 45. Hanson, J.V.M., Heron, J., Whitaker, D.: Recalibration of Perceived Time Across Sensory Modalities. Experimental Brain Research 185(2), 347–352 (2008) 46. Sherman, S.M., Guillery, R.W.: Exploring the Thalamus. Academic Press, San Diego (2001) 47. Sherman, S.M., Guillery, R.W.: The Role of the Thalamus in the Flow of Information to the Cortex. Philosophical Transactions of the Royal Society London B 357, 1695–1708 (2002) 48. Bridge, H., Thomas, O., Jbabdi, S., Cowey, A.: Changes in Connectivity after Visual Cortical Brain Damage Underlie Altered Visual Function. Brain 131, 1433– 1444 (2008) 49. O'Connor, D.H., Fukui, M.M., Pinsk, M.A., Kastner, S.: Attention Modulates Responses in the Human Lateral Geniculate Nucleus. Nature Neuroscience 5, 1203– 1209 (2002) 50. Fendrich, R., Corballis, P.M.: The Temporal Cross-Capture of Audition and Vision. Perception and Psychophysics 63(4), 719–725 (2001) 51. Bertelson, P., Aschersleben, G.: Temporal Ventriloquism: Crossmodal Interaction on the Time Dimension. 1. Evidence from Auditory-Visual Temporal Order Judgment. International Journal of Psychophysiology 50(1-2), 147–155 (2003) 52. Shams, L.Y., Kamitani, Y., Shimojo, S.: Illusions: What You See is What you Hear. Nature 408(6814), 788 (2000) 53. Van Wassenhove, V., Buonomano, D., Shimojo, S., Shams, L.: Distortions of Subjective Time Perception Within and Across Senses. PLoS One 3(1), 1–13 (2008) 54. New, J.J., Scholl, B.J.: Subjective Time Dilation: Spatially Local, Object-based, or a Global Visual Experience? Journal of Vision 9(2), 1–11 (2009)