Skill, corporality and alerting capacity in an account of sensory consciousness

J. Kevin O'Regan

Laboratoire de Psychologie Expérimentale

Institut de Psychologie, Centre Universitaire de Boulogne

71, avenue Edouard Vaillant

92774 Boulogne-Billancourt Cedex; France

Tel: (33 1) 55 20 59 26; portable: 06 75 25 56 92; Fax: (33 1) 55 20 58 54

oregan@ext.jussieu.fr

 

Erik Myin

Department of Philosophy

Centre for Logic and Philosophy of Science (LW-CLWF)

Vrije Universiteit Brussel,

Pleinlaan 2.

B1050 Brussel; Belgium

emyin@vub.ac.be

 

Alva Noë

Department of Philosophy,

University of California at Santa Cruz,

Santa Cruz, CA 95064

anoe@cats.ucsc.edu

 

 

"The most lively thought is still inferior to the dullest sensation." (David Hume)

 

 

We suggest that within a skill-based, sensorimotor approach to sensory consciousness, two measurable properties of perceivers' interaction with the environment, "corporality" and "alerting capacity", explain why sensory stimulation is experienced as having a “sensory feel", unlike thoughts or memories. We propose that the notions of "corporality" and "alerting capacity"make possible the construction of a 'phenomenality plot', which allow to chart in a principled way the degree to which conscious phenomena are experienced as having a sensory quality.

 

 

 

KEYWORDS: consciousness, perception, qualia, change blindness, sensory substitution

 

 

Though knowledge is rapidly accumulating concerning the neurobiological mechanisms involved in consciousness (cf. Rees, Kreiman & Koch, 2002 for an overview), there still remains the problem of how to capture the 'qualitative' aspects with a scientific approach. There would seem to be an unbridgeable 'explanatory gap' (Levine, 1983) between what it is like to have a sensory experience, and the neural correlates or physical mechanisms involved.

The purpose of this paper is to show how a step can be made towards bridging this gap. We purposefully leave  aside many interesting problems of consciousness, such as self awareness, the distinction between awake and unconscious states,  being aware of facts, etc., and concentrate on the question of the nature of sensation. The fact that contrary to other mental phenomena, sensations have a distinctive qualitative character or sensory "feel", can be considered as the crux of the explanatory gap problem. Indeed in terms of Ned Block's  much cited distinction: sensory feel seems to be the archetype of full blown, and deeply enigmatic, "Phenomenal Consciousness", while thought or belief belong to the scientifically tractable forms of "Access-Consciousness" (Block, 1995).

Our approach to the question of sensation will be to suggest that there is a way of thinking about sensations which is different from the usually accepted way. A first aspect of this new way of thinking involves taking an at first sight counterintuitive stance, namely that sensation consists in the exercise of an exploratory skill (cf. O'Regan & Noë, 2001a;  Myin & O'Regan, 2002; see Torrance, 2002, for further references to skill theories). Taking the skill approach allows a first problem about the experiential quality of sensation to be  addressed, namely why the experienced qualities of different sensations differ the way they do.

Second, when skill theories are supplemented by two concepts which we refer to as "corporality" and "alerting capacity", then a second, more profound problem about the experienced quality of sensations can be addressed, namely why they have an experienced sensory quality at all.

We have organized our paper in a main body in which the concepts crucial to our approach are introduced and described, and three 'application' sections, in which they are put to use in the context of more specific issues, namely inter and intramodal differences, dreaming and imagery, and change blindness.

Sensation as a skill: Explaining intra and intermodal sensory differences

The basic tenet of the skill theory from which we take our start is that having a sensation is a matter of the perceiver knowing that he is currently exercising his implicit knowledge of the way his bodily actions influence incoming sensory information (O'Regan & Noë, 2001a).

An illustration is provided by the sensation of softness that one might experience in holding a sponge (Myin, in press). Having the sensation of softness consists in being aware that one can exercise certain practical skills with respect to the sponge: one can for example press it, and it will yield under the pressure. The experience of softness of the sponge is characterized by a variety of such possible patterns of interaction with the sponge, and the laws that describe these sensorimotor interactions we call, following MacKay (1962): laws of sensorimotor contingency (O'Regan & Noë, 2001a).When a perceiver knows, in an implicit, practical way, that at a given moment he is exercising the sensorimotor contingencies associated with softness, then he is in the process of experiencing the sensation of softness.

Note that in this account, the softness of the sponge is not communicated by any particular softness detectors in the fingertips, nor is it characterised by some intrinsic quality provided by the neural processes involved, but rather it derives from implicit, practical knowledge about how sensory input from the sponge currently might change as a function of manipulation with the fingers.

This approach to sensation has a tremendous advantage. It avoids a fundamental problem that is encountered by any approach that assumes that sensation is generated by a neural mechanism: namely the problem why this particular neural process (whatever its neural specification), should give rise to this specific sensation (and not to another one). In addition, the skill-based sensorimotor description of experiencing softness in terms of an exploratory finding out that the object yields when one presses, 'fits' the experience of softness in a way which a description in terms of a correlated neural process cannot. Thus, for example, while under a 'neural correlate' explanation it is always possible to imagine the presumed neural process for softness to be paired with the sensation of hardness (i.e. nothing of the specifics of the neural description seems to forbid this), it would seem impossible to imagine one is going through the exploratory pattern of softness, yet experiencing hardness.

Application 1 describes how the sensorimotor way of thinking can be applied to perceptual sensations in general, even to cases like color perception where no active exploration appears necessary. Just as the difference between hard and soft can be accounted for in terms of the different exploratory strategies required to sense hard and soft objects, the differences between red and green, for example, can be accounted for in terms of the different exploratory strategies involved in exploring red and green surfaces.

Another, related question can also be dealt with by this approach, namely the question of the differences between the sensory qualities of the different sensory modalities. As suggested in Application 1, the difference, for example between hearing and seeing is accounted for in terms of the different laws of sensorimotor contingency that characterise hearing and seeing. Again, under this approach, no appeal is necessary to special, as yet unexplained intrinsic properties of neural mechanisms.

The sensorimotor theory and its explanation of intra and intermodal sensory differences, as just reviewed, has previously been treated in a number of papers (O'Regan & Noë, 2001a,b,c; Myin & O'Regan, 2002; Noë, 2002; as well as Noë, forthcoming). We now come to the main purpose of the present article, which is to address a more profound question, namely the question of why sensations have a sensory experiential quality at all.

Corporality and alerting capacity: Explaining sensory presence

What is special about sensory experience which makes it different from other mental phenomena, like conscious thought or memory? In particular, consider the difference between actually feeling a terrible pain and merely imagining or thinking you are feeling one. Or consider actually feeling softness or seeing red, compared to thinking you are feeling softness or seeing red. (see Application 2 for a discussion of dreams, imagery and hallucinations).

Theorists have tried to describe and capture such differences in various ways. Hume, for example opposed (perceptual) sensations and 'ideas' (recollections of sensations and thoughts), in terms of ‘vivacity'  and 'force' (Hume, 1777/1975).  Husserl proposed the notion of an object being experienced as 'being present in the flesh' (having 'Leibhaftigkeit') as an essential ingredient for truly perceptual experience (Husserl, 1907/1973; cf. Pacherie, 1999; Merleau-Ponty, 1945) for similar use of the notion ‘presence’). In contemporary descriptions of perceptual consciousness, such a distinction is often made in terms of 'qualia', those special qualitative or phenomenal properties that characterise sensory states, but not cognitive states (Levine, 1983; Dennett, 1988).

While these notions seem descriptively adequate, we propose they should and can be complemented with an explanatory story that accounts for why sensory experience differs in these respects from other conscious mental phenomena. Our claim is that, within a skill-based, sensorimotor theory, the notions of corporality and alerting capacity provide precisely this missing explanatory  addition. Corporality and alerting capacity are complementary aspects of an observer's interaction with the environment: Corporality concerns the way actions affect incoming sensory information, and, conversely, alerting capacity concerns the way incoming sensory information potentially affects the attentional control of behavior.

Note again: we wish to claim that corporality and alerting capacity are not merely descriptive, but actually possible first steps towards explanations. We will return to this distinction later.

Corporality

We define corporality as the extent to which activation in a neural channel systematically depends on movements of the body (in previous publications we used the term "bodiliness" (O'Regan & Noë, 2001b; Myin & O'Regan, 2002; O'Regan, Myin & Noë, in press). Sensory input from sensory receptors like the retina, the cochlea, and mechanoreceptors in the skin possesses corporality, because any body motion will generally create changes in the way sensory organs are positioned in space, thereby causing changes in the incoming sensory signals. Proprioceptive input from muscles also possesses corporality, because there is proprioceptive input when muscle movements produce body movements.

Note that we intend the term corporality to apply to any neural channels in the brain whatsoever, but because of the way it is defined, with the exception of muscle commands themselves and proprioception, only neural activation which corresponds to sensory input from the outside environment will generally have  corporality. For example neural channels in the autonomic nervous system that measure parameters such as the heartbeat or digestive functions, because they are not systematically affected by movements, will have no corporality even though they may carry sensory information. Note also that memory processes or thinking have no corporality, because body movements do not affect them in any systematic way.

 We shall see below that corporality is an important factor that explains the extent to which a sensory experience will appear to an observer as being truly sensory, rather than non-sensory, like a thought, or a memory. In Philipona, O'Regan & Nadal (in press) it is shown mathematically how this notion can be used by an organism to determine the extent of its own body and the fact that it is embedded in a three dimensional physical world in which the group-theoretic laws of euclidean translations and rotations apply.

Alerting capacity

We define the alerting capacity of sensory input as the extent to which that input can cause automatic orienting behaviors that peremptorily capture the organism's cognitive processing ressources. Alerting capacity could also be called: capacity to provoke exogenous attentional capture, but this would be more cumbersome. In previous papers we have also used the term "grabbiness" (O'Regan & Noë, 2001b; Myin & O'Regan, 2002; O'Regan, Myin & Noë, in press).

Pain channels for example have alerting capacity, because not only can they cause immediate, automatic and uncontrollable withdrawal reactions, but they also can cause cognitive processing to be modified and attentional ressources to be attributed to the source of the pain. Retinal, cochlear and tactile sensory channels have alerting capacity, since not only can abrupt changes in incoming signals cause orienting reflexes, but the organism's normal cognitive functioning will be modified to be centered upon the sudden events. For example a sudden noise not only can cause the organism to turn towards the source of the noise, but the noise will additionally, peremptorily, modify the course of the organism's cognitive activity so that, for example if it is human, it now takes account of the noise in current judgments, planning, and  linguistic utterances. Autonomic pathways do not have alerting capacity, because sudden changes in their activation do not affect cognitive processing. For example, while sudden changes in vestibular signals cause the organism to adjust its posture and blood pressure automatically, these adjustments themselves do not generally interfere in the organism's cognitive processing (interference occurs only indirectly, when, for example, the organism falls to the ground and must interact in a new way with its environment).

Like corporality, we take alerting capacity to be an objectively measurable parameter of the activation in a sensory pathway.

Using corporality and alerting capacity to explain sensory presence

We now consider how the notions of corporality and alerting capacity can contribute to understanding what provides sensory experiences with their particular sensory quality, and more precisely, what makes for the difference between truly sensory and other experiences.

To see our notions at work, consider the difference between seeing an object in full view, seeing an object partially hidden by an occluding object, being aware of an object behind one's back, and thinking, remembering or knowing about an object. It is clear that these different cases provide different degrees of sensory "presence" (Merleau-Ponty, 1945; O'Regan & Noë 2001a;  Noë, 2002b). Our claim is that these different degrees of sensory presence precisely reflect different degrees in corporality and alerting capacity.

Thus, when an object is in full view, it comes with the fullest intensity of sensory presence. But it is precisely in this case that observer motion will immediately affect the incoming sensory stimulation. Also, any change that occurs in the object, such as a movement, a shape, color, or lightness change, will immediately summon the observer's attention. This is because low-level transient-detection mechanisms exist in the visual system that peremptorily cause an attention shift to a sudden stimulus change. In terms of the concepts we defined above, this means that an object in full view has both high corporality and high alerting capacity.

Contrast this with just knowing that an object is somewhere, but out of view. While knowledge about an object in another room might certainly be conscious, it lacks real sensory presence. Clearly, in this case, there is no corporality, since the stimulus changes caused by bodily movements do not concern that object. Similarly, there is no alerting capacity, as the changes that the object might undergo do not immediately summon the perceiver’s attention.

An object only partially in view because of an occluding object or an object known to be behind one's back provide borderline cases. For example, the occluded part might be said to still have some presence (Gregory, 1990; Merleau-Ponty, 1945; O'Regan & Noë, 2001a; Noë & O'Regan, 2002; Noë, 2002b) because it has a degree of corporality, as we can easily bring it into view by a slight movement.. The ‘boundary extension’ phenomenon of Intraub & Richardson (1989) according to which observers overestimate what can be seen of a partially occluded object is coherent with this view. Amodal completion may be an example where one has an intermediate kind of “almost-visual” feeling of presence of a shape behind an occluder. Application 3 gives examples of "change blindness", showing that when alerting capacity is interfered with, the experience of perception ceases.

These examples show that the differing degrees of what one might call "sensory presence” (perhaps Hume's "vividness" or Husserl's "Leibhaftigkeit") can be accounted for plausibly in terms of the physically measurable notions of corporality and alerting capacity.

The "sensory phenomenality plot"

The exercise of contrasting sensations with other mental phenomena  can be systematized in a "sensory phenomenality plot" (Fig. 1).

 


Fig. 1. A sensory phenomenality plot

 


By plotting the degree of corporality and alerting capacity for different mental phenomena, such a figure reveals that those states that possess both corporality and alerting capacity correspond precisely to cases which provide true sensory experiences.  (But note, importantly: we consider that our plot only charts the degree to which mental phenomena have sensory or perceptual quality, and not consciousness per se. In particular, when we claim that thought has no sensory quality, we are not saying that thought is not conscious (more on this below, section  'Consciousness').

Thus, vision, touch, hearing, and smell are the prototypical sensory states and indeed have high corporality and high alerting capacity, as mentioned above in the definition of these terms. High corporality derives from the fact that changes in head or limb positions have an immediate effect on visual, auditory or tactile sensory input (smell is less clear, but sniffing, blocking the nose, moving the head, do affect olfactory stimulation -- Steriade, 2001). High alerting capacity is provided by the fact that sudden changes in visual, tactile, auditory or olfactory stimulation provoke immediate orienting behaviors that peremptorily modify cognitive processing.

What characterizes pain is its particularly large amount of alerting capacity. Here it is virtually impossible to prevent oneself from attentively focussing on the noxious stimulation. Pain also has corporality, but to a lesser extent. Moving one's body can generally modify the pain (one can remove one's finger from the fire; rub the aching limb and change the incoming sensations), but there are cases like headaches or toothaches which are more problematic. Headaches and toothaches are characterized by the fact that associated sensory input changes only moderately as a function of things that one can do such as press on the head, chew with one's teeth. This lack of an ability to easily modulate the sensory stimulation by body motions, that is, a reduced corporality, could possibly correspond to a particular aspect of pains such as headaches which distinguishes them from vison, touch hearing and smell, namely that they have an interior quality, often not clearly localized.

We have plotted thinking and recalling from memory at the other extreme, because they have neither corporality or alerting capacity, as we pointed out above.

Proprioception is the neural input that signals mechanical displacements of the muscles and joints. Motor commands which give rise to movements necessarily produce proprioceptive input, so proprioception has a high degree of corporality. On the other hand, proprioception has no alerting capacity: body position changes do not peremptorily cause attentional ressources to be diverted to them. We therefore expect that proprioception should not appear to have an experienced sensory quality. Indeed it is true that though we generally know where our limbs are, this position sense does not have a sensory nature.

The vestibular system detects the position and motion of the head, and so vestibular inputs have corporality. However they have no alerting capacity. This is because although sudden changes in body orientation immediately result in re-adjusting reactions, these do not per se interfere with current cognitive processing. Coherent with our expectations therefore, the vestibular sense is not perceived as corresponding to an experience. We know we are standing vertical, but we do not have the experience of this in the same sense as we have the experience of hearing a bell or seeing a red patch.

Speculatively we suggest our plot also can track phenomena intermediate between sensory and mental states. One of several examples very tentatively included as points in Figure 1 is richness. The feeling of being rich is a case where there is a limited form of corporality (there are things one can do when one is rich like get the money from the bank teller, buy an expensive car, but this is nothing like the immediate and intimate link that action has on visual perception, for example), and little alerting capacity (there is no warning signal when one’s bank account goes empty). As a consequence, the feeling of being rich is somewhat, though not entirely sensory.

Consciousness

The argument made in this paper has concerned the nature of sensation: What gives sensation its “experienced” quality, what makes sensory qualities the way they are. But note: we have purposefully not touched upon the question of why and when sensations are conscious.

Our claim would now be that a sensation is conscious when a person is poised to cognitively make use of the sensation in their judgments, decisions, and rational behavior.

Why does this constitute progress towards answering the question of the explanatory gap, namely the problem of how a physico-chemical mechanism in the brain could ever give rise to an experience?

The answer is that first, having cognitive access to a fact is something that is  generally considered to not to offer particular problems with scientific description and explanation (see Dennett,1978, Baars, 1988). It amounts to what Block (1995) has called Access Consciousness, and is something which, though it may constitute a difficult thing to implement in a machine, is nevertheless describable in broadly functionalist terms. There is no a priori logical difficulty (although there may be practical difficulties) in using scientific methods to understand Access Consciousness.

Second, we have defined sensation in a way which doesn't seem problematic from a scientific point of view, namely in terms of sensorimotor skills. The different types of sensations and their experienced characteristics -- their similarities and differences, their experienced "presence", can all be accounted for in terms of the differences between the skills, and in terms of way the neural channels are tuned to the environment, namely by the properties of corporality and alerting capacity.

If having a conscious experience amounts to having cognitive access to sensations, then what has previously been considered mysterious, namely what Block has called Phenomenal Consciousness, can now be decomposed into two scientifically tractable components: conscious experience would in our approach consist in having Access Consciousness of sensations. Since Access Consciousness is amenable to scientific methods, and since sensations, being sensorimotor skills, are also amenable to scientific methods, under our approach Phenomenal Conscious now also comes within the domain of science.

Description or Explanation?

It is interesting to consider finally the explanatory status of the concepts put forward in this paper. The question of accounting for the experienced quality of sensation is the question of accounting for why certain mental processes are taken to have a sensory nature, while others, like doing arithmetic or making a decision, are not. If one does not espouse a sensorimotor approach, one could claim that saying that sensations  involve neural channels possessing corporality and alerting capacity is simply describing something about sensations, and has no explanatory status.

But if one espouses the sensorimotor approach, then the question of accounting for the experienced quality of sensation becomes tractable by the scientific method, since we can see that each of the aspects of the experienced quality of sensory experience which previously seemed difficult to explain, actually correspond to objectively describable aspects of the skills that are involved. The most important such aspect, and the one which has posed the most problem to classical approaches to phenomenal consciousness, is the problem of “presence”. We have dealt with this in the sensorimotor approach by noting that sensory stimulation possesses corporality and alerting capacity, thereby providing the skills involved in exploring sensory stimulation with its particular intimate, vivid, inescapable quality. These seem to deal adequately with what we mean by "presence".

We also think our approach holds the promise of accounting for further fine-grained features of sensation which have been noticed by various theorists (see for example the list of features in (Humphrey, 1992, 2001;  O'Regan & Noë, 2001b; Myin & O'Regan, 2002). Consider for example ineffability and subjectivity: Under an approach where sensation is neurally generated, it would be difficult to explain why certain neural processes generate qualities which are felt, but which cannot be described (ineffability); equally, it would be difficult to explain why certain neural processes appear to generate subjective quality, whereas others do not.

Within the sensorimotor approach, on the other hand, the appearance of both properties is predicted and is thus explainable: sensory experiences are subjective, and are the sole property of the experiencer because they involve the experiencer himself potentially undertaking actions and exercising sensorimotor skills (see Humphrey, 1992, 2001, for a similar explanation). Similarly, sensory experiences are ineffable because they involve exercising implicit, practical skills. Like tying one's shoelaces, exercising the sensorimotor contingencies associated, say, with red, involves putting into practise a practical skill that one cannot describe with words, but that one knows one possesses.

While it may at first sight be unclear how we have made the passage from description to explanation by changing our view of what sensation is, it should be noted that such a shift in theoretical paradigm occurred in the 20th century as regards the question of life. Whereas at the beginning of the 20th Century, cell division, metabolism, respiration, etc. were considered to be caused by an as yet unexplained vital essence, today we consider these phenomena to be constitutive of life. The notion of life has been re-defined: instead of being caused by some underlying mechanism, it is considered now to be constituted by all the various ways the organism can act within its environment. In the same way, by changing one's viewpoint on what sensation is, and espousing the sensorimotor, skill-based approach, one can avoid the issue of generation and thus of the explanatory gap, and immediately see how each of the characteristics that people attribute to sensation arise from aspects of neural machinery and their interaction with the environment.

Thus, we think we have shown that, contrary to the idea that there is an unbridgeable gap between neural processes and 'sensory consciousness', a connection may be made between the two domains if neural systems are conceived not as generating sensations, but as allowing organisms to deploy sensorimotor skills.

APPLICATION 1: Intra- and intermodal differences in sensory quality

One important aspect of sensory experience concerns the differences and the similarities between sensations of a same modality. Why, for example, is the sensation of red different from the sensation of green? It seems that any account in terms of different neural processes correlated with red and green immediately encounters an insurmountable problem: why should this particular neural process, say (whatever its specification in neural terms) provide the red sensation, rather than the green sensation?

In the main text, it was claimed, with reference to the example of softness, that  an account in terms of sensorimotor contingencies sidesteps such difficulties. This same approach can now be applied to color. The incoming sensory data concerning a fixated patch of color depends on eye position. Because of the non-uniform distribution of retinal cones, eye movements provoke different patterns of change in sensory input, depending on which colors are being fixated. Such sensorimotor contingencies are part of what constitute the sensations of the different colors. Another type of sensorimotor contingency associated with colors depends on body motions. Consider the light reflected from a colored piece of paper. Depending on where the observer is positioned with respect to ambient illumination, the paper can, for example, reflect more bluish sky light, more yellowish sunlight, or more reddish lamplight. Such laws of change constitute another type of sensorimotor contingency that constitute the sensations of different colors. The fact that color sensation can indeed depend on body motions has been suggested by Broackes (1992) and further work on color from a related perspective is reported in Myin (2001; cf. also Pettit, in press).

Research done by Ivo Kohler (Kohler, 1951) provides empirical confirmation for this application of the sensorimotor approach to color. Kohler’s subjects wore goggles in which one side of the field was tinted one color (e.g. blue) and the other one another color (e.g. yellow). Within a period of some days the subjects came to see colors as normal again. The sensorimotor theory  would indeed predict such an adaptation to the new sensorimotor contingencies associated with each color. Unfortunately Kohler’s experiments have only been replicated with variable success (McCollough, 1965),  so further examination of the possibility of such adaptations is necessary (O'Regan, Clark, Bompas, 2001).

 

A second important aspect of sensory experience concerns intermodal differences in sensory quality: The fact that for example, hearing involves a different quality as compared to seeing, which has a different quality as compared to tactile sensation.

We propose to again apply the idea that sensation involves the exercising of sensorimotor contingencies : differences between modalities come from the different skills that are exercised. The difference between hearing and seeing amounts to the fact that among other things, one is seeing if, when one blinks, there is a large change in sensory input; one is hearing if nothing happens when one blinks, but, there is a left/right difference when one turns one's head, etc. Some other modality specific sensorimotor contingencies are specified in Table 1.

In addition to providing a more principled account of sensory modality the sensorimotor approach leads to an interesting prediction. According to this approach, the quality of a sensory modality does not derive from the particular sensory input channel or neural circuitry involved in that modality, but from the laws of sensorimotor contingency that are implicated. It should therefore be possible to obtain a visual experience from auditory or tactile input, for example, provided the sensorimotor laws that are being obeyed are the laws of vision (and provided the brain has the computing resources to extract those laws).

 

 

Table 1. Some sensorimotor contingencies associated with seeing and hearing

Action:

SEEING

HEARING

Blink

Big change

No change

Move eyes

Translating

flowfield

No change

Turn head

Some changes

in flow

Left/right ear phase and

amplitude difference

Move forward

Expanding 

flowfield.

Increased amplitude in

both ears

 

The phenomenon of sensory substitution is coherent with this view. Sensory substitution has been experimented with since Bach-y-Rita constructed a device to allow blind people to "see" via tactile stimulation provided by a matrix of vibrators connected to a video camera (Bach-y-Rita, 1967) . Today there is renewed interest in this field , and a number of new devices are being tested with the purpose of substituting different senses: visual-to-tactile (Sampaio, Maris & Bach-y-Rita, 2001; visual-to-auditory: Veraart, Cremieux & Wanet-Defalque, 1992;  auditory to visual: Meijer, 1992; auditory-to-tactile: Richardson & Frost, 1977). One particularly interesting finding is that the testimonials of users of such devices at least sometimes come framed in terms of a transfer of modalities. For example, a woman wearing a visual-to-auditory substitution device will explicitly describe herself as seeing through it (cf. the presentation by Pat Fletcher at the Tucson 2002 Consciousness conference, available on http://www.seeingwithsound.com/tucson2002f.ram). Sensory substitution devices are still in their infancy. In particular, no systematic effort has been undertaken up to now to analyze the laws of sensorimotor contingency that they provide. From the sensorimotor approach, it will be the similarity in the sensorimotor laws that such devices recreate which determines the degree to which users will really feel they are having sensations in the modality being substituted.

Related phenomena which also support the idea that the experience associated with a sensory modality is not wired into the neural hardware, but is rather a question of sensorimotor contingencies, comes from the experiment of (Botvinick & Cohen, 1998), where the “feel” of being touched can be transfered from one's own body to a rubber replica lying on the table in front of one (see also related work on the body image in tool use: Yamamoto, & Kitazawa, 2001; Iriki, Tanaka & Iwamura,1996; Farne & Ladavas, 2000). The finding of the Sur group (Roe, Pallas, Hahm & Sur, 1990), according to which ferrets can see with their auditory cortex can also be interpreted within the context of the present theory (Hurley & Noë, in press).

APPLICATION 2: Dreaming and imagery

Dreams are characterised by the the fact that while people are dreaming they seem to assume that they are having the same full-blown perceptual experiences that they have in real life. Clearly however dreams do not involve corporality or alerting capacity in the normal fashion, since there is no sensory input at all.

On the other hand it is also clear that it is precisely corporality which ultimately allows people to realise that they are actually dreaming -- the classic way of knowing that you are dreaming is to try to switch on the light: this kind of "reality-checking" is nothing more than testing for corporality  -- checking that your actions produce the normal sensory changes expected when you are having real sensory experiences.

It is important to note however that what counts in giving the particular "sensory" feel of sensation is not the actual sensory input itself, but the knowledge that the sensory input possesses corporality and alerting capacity. This means that an observer can have a sensation even though he is, at a given moment, doing nothing at all, and even though he is receiving no sensory input at all. It suffices for this that he be in the same mental state that he would usually be in when he has implicit knowledge that the sensorimotor contingencies associated with a sensation are currently applicable.

We can therefore understand how it might come about that a person would have experience as of reality without sensory input, and therefore no corporality and alerting capacity. The person merely has implicitly to think (in point of fact incorrectly) that if he were to move, then those changes would occur that normally occur when he moves. He just has to implicitly think (incorrectly) that were there to be a sudden event, his attention would be automatically attracted to it.

Dreaming therefore poses no problem for the sensorimotor approach that we are proposing. Indeed the approach actually makes it easier to envisage brain mechanisms that engender convincing sensory experiences without any sensory input, since the sensation of richness and presence and ongoingness can be produced in the absence of sensory input merely by the brain implicitly 'supposing' (in point of fact incorrectly) that if the eyes were to move, say, they would encounter more detail. This state of 'supposing one can get more detail' would be a much easier state to generate than having to actually recreate all the detail somewhere in the brain. In dreaming, furthermore, the state would be particularly easy to maintain because what characterizes dreaming would seem to be a lack of attention to the absence of disconfirming evidence, which is quite unsurprising, since one is asleep. This lowering of epistemic standards implies that, while dreaming, one is easily led into thinking one is perceiving, while — if only one were to pay attention — it would be obvious that one is not. Thus one can remain convinced for the whole duration of one's dream that one is experiencing reality. A whole series of different bizarre dream events may be taken at face value simply because nothing disconfirms them.

Similar remarks apply to mental imagery. As for dreams, mental imagery would correspond to a kind of perceptual action without an actual stimulus and without 'going through' the motions — it would involve having implicit expectancies without these being actually fulfilled by worldly responses (for a detailed account of mental imagery along roughly 'sensorimotor' lines, see Thomas, 1999).

APPLICATION 3: Spatial and temporal completeness of the visual world: change blindness.

When one looks out upon the world, one has the impression of seeing a rich, continuously present  visual panorama. Under the sensorimotor theory however, the richness and continuity of this sensation are not due to the activation in the brain of a neural representation of the outside world. On the contrary, the ongoingness and richness of the sensation derive from implicit knowledge of the many different things one can do (but need not do) with one's eyes, and the sensory effects that result from doing them. Having the impression of seeing a whole scene comes, not from every bit of the scene being present in the mind, but from every bit of the scene being immediately available for handling by the slightest flick of the eye. In terms of the core concepts of this paper: the “feeling of seeing everything” comes from exercise of implicitly knowing one is in a relation with the visually perceived part of the environment which has a high degree of both corporality (moving the body causes changes in sensory input coming from the visual field) and alerting capacity (if something suddenly changes inside the visual field, attention will immediately be drawn to it).

But now a curious prediction can be made. Only one aspect of the scene can be being “handled” at any one moment. The vast majority of the scene, although perceived as present, is actually not being “handled”. If such currently “un-handled” scene areas were to be surrepticiously replaced, such changes should go unnoticed.

Under normal circumstances the alerting capacity of visual input ensures that any change made in a scene will provoke an eye movement to the locus of the change. This is because low-level movement detectors are hard-wired into the visual system which detect any sudden change in local contours. Attention is peremptorily focussed on the change, and visual “handling” is the immediate result. But if the alerting capacity could be inactivated, then we predict that it should indeed be possible to make big changes without this being noticed.

An extensive current literature on “change blindness” confirms this prediction (for a review see Simons, 2000). By inserting a blank screen or “flicker”, or else an eye movement, a blink, “mudsplashes”, or a film cut between successive images in a sequence of images or movie sequence, the local transients that would normally grab attention and cause perceptual “handling” of a changing scene aspect are drowned out. Under such conditions, observers remain unaware of very large changes. Another method of obviating the usual alerting action of local changes is to make them so slow that they are not detected by the low-level transient detectors in the visual system. Demonstrations of change blindness phenomena can be found on the web sites: http://nivea.psycho.univ-paris5.fr and http://viscog.beckman.uiuc.edu/change/.

A related phenomenon is the phenomenon of “inattentional blindness” pioneered by Neisser & Becklen (1975) and Mack and Rock (1999) and recently convincingly extended by Simons and co-workers (Simons & Chabris, 1999; Simons, Franconeri & Reimer, 2000). In this, a movie sequence of a complex scene is shown to observers, and they are told to engage in an attentionally demanding task, like counting the number of ball exchanges made in a ball game. An unexpected event (like an actor dressed in a gorilla suit) can go totally unnoticed in such circumstances, even though the event is perfectly visible and in the very center of the visual scene. Demonstrations can be seen on http://nivea.psycho.univ-paris5.fr and http://www.wjh.harvard.edu/~viscog/lab/demos.html.

 

 

 

 

 

 

References

Auvray, M. & O'Regan, J.K. (in press) L'influence des facteurs sémantiques sur la cécité aux changements progressifs dans les scènes visuelles. Année Psychologique.

 

Baars, B. 1988, A Cognitive Theory of Consciousness, Cambridge University Press.

 

Bach-y-Rita Rita, P. (1967). Sensory plasticity. Applications to a vision substitution system. Acta Neurologica Scandinavia, 43(4), 417-426.

 

Botvinick, M., & Cohen, J. (1998). Rubber hands 'feel' touch that eyes see [letter]. Nature, 391(6669), 756.

 

Block, N. (1995). "On a confusion about a function of consciousness", Behavioral and Brain Sciences, 18 (2), 227-247.

 

Broackes, J. (1992). The autonomy of colour. In K. Lennon & D. Charles (Eds.), Reduction, Explanation, and Realism (pp. 421-465). Oxford: Oxford University Press.

 

Dennett, D.(1978). Brainstorms: Philosophical Essays on Mind and Psychology, Cambridge MA: MIT Press.

 

Dennett, D. (1988). Quining qualia, in A. Marcel and E. Bisiach (Eds.) Consciousness in contemporary science.  Oxford: Clarendon Press.

 

Farne A, Ladavas E (2000). Dynamic size-change of hand peripersonal space following tool use, Neuroreport,  11 (8): 1645-1649.

 

Gregory, R. (1990). How do we interpret images? In H.Barlow et al. (Eds.), Images and Understanding (pp. 310-330). Cambridge University Press.

 

Hume, D. (1777/1975), Enquiries concerning human understanding and concerning the principles of morals. Oxford University Press.

 

Humphrey, N. (1992). A history of the mind. London: Chatto & Windus.

 

Humphrey, N. (2000), How to solve the mind-body problem,  Journal of Consciousness Studies 7 (4), 5-20.

 

Hurley & Noë (in press). Neural plasticity and consciousness. Biology and Philosophy.

 

Husserl, E. (1907/1973), Ding und Raum. Vorlesungen 1907. Hussserliana 16. Edited by U. Claesges. The Hague: M. Nijhoff.

 

Intraub, H., & Richardson, M. (1989). Wide-angle memories of close-up scenes. Journal of Experimental Psychology: Learning, Memory, & Cognition,

15(2), 179-187.

 

Iriki, A., Tanaka, M., & Iwamura, Y. (1996). Coding of modified body schema during tool use by macaque postcentral neurones. Neuroreport, 7(14), 2325-2330

 

Kohler, I. (1951) Über Aufbau und Wandlungen der Wahrnehmungswelt. Österreichische Akademie der Wissenschaften. Sitzungsberichte, philosopohisch-historische Klasse, 227, 1-118.

 

Levine, J. (1983). Materialism and qualia: The explanatory gap. Pacific Philosophical Quarterly, 64(354-361).

 

McCollough, C. (1965). The conditioning of color perception. American Journal of Psychology, 78, 362-368.

 

Mack, A., & Rock, I. (1998). Inattentional blindness. Cambridge, MA, USA: The Mit Press.

 

MacKay, D.M. (1962) Theoretical models of space perception. In: Aspects of the theory of artificial intelligence, ed. C.A. Muses. Plenum Press.

 

Meijer, P. B. L. (1992). An experimental system for audiory image representations. IEEE Transactions on Biomedical Engineering, 39(2), 112-121.

 

Merleau-Ponty, M. (1945) Phénoménologie de la Perception. Paris: Gallimard.

 

Myin, E. (2001) Color and the duplication assumption. Synthese 129(1), 61-77.

 

Myin, E. (in press) An account of color without a subject? (commentary on Byrne and Hilbert), Behavioral and Brain Sciences.

 

Myin, E. & O'Regan, J.K. (2002) Perceptual consciousness, access to modality and skill theories. Journal of Consciousness Studies 9 (1), 27-45

 

Neisser, U., & Becklen, R. (1975). Selective looking: attending to visually-specified events. Cognitive Psychology, 7, 480-494

 

Noë, A. (2002). Is the visual world a grand illusion? Journal of Consciousness Studies 9 (5/6),  1-12.

 

Noë, A. (2002). On what we see. Pacific Philosophical Quarterly. 83 (1),

 

Noë, A. (forthcoming). Action in Perception. Cambridge MA: MIT Press.

 

Noë, A.  and O’Regan, J.K. (2000), Perception, attention and the grand illusion. Psyche 6 (15), 2000. URL: http://psyche.cs.monash.edu.au/v6/psyche-6-15-noe.html

 

 O'Regan, J.K., Clark, J., Bompas, A. (2001) Implications of a sensorimotor theory of vision for scene perception and colour sensation. (Abstract). Perception, , 30, supplement, 94.

 

O'Regan, J.K.  & Noë, A. 2001). A sensorimotor account of vision and visual consciousness. Behavioral and Brain Sciences 24 (5), 2001: 883-917.

 

O'Regan, J.K.  & Noë, A. (2001). Acting out our sensory experience: Authors' Response to Commentary. Behavioral and Brain Sciences 24 (5), 2001: 955-975.

 

O'Regan, J.K.  & Noë, A. (2001). What it is like to see: a sensorimotor theory of perceptual experience. Synthese 129 (1), 79-103.

 

O'Regan, J.K., Myin, E.  & Noë, A. (in press). Towards an analytic phenomenology: The concepts of bodiliness and grabbiness. In A. Carsetti (Ed.). Seeing and Thinking. Reflections on Kanizsa's Studies in Visual Cognition. Kluwer.

 

Pacherie, E. (1999). Leibhaftigkeit and representational theories of perception. In J. Petitot et al. Naturalizing phenomenology: issues in contemporary phenomenology and cognitive science. Stanford University Press,  pp. 148-160.

 

Pettit, P.(in press) Looks as powers. Philosophical Issues.

 

Philipona, D., O'Regan, J.K.,  & Nadal, J.-P. (in press) Is there anything out there? Inferring space from sensorimotor dependencies. Neural Computation.

 

Rees, G., Kreiman, G. and Koch, C. (2002). Neural correlates of consciouosness in humans. Nature Reviews Neuroscience 3, 261-270.

 

 Richardson, B. L., & Frost, B. J. (1977). Sensory substitution and the design of an artificial ear. Journal of  Psychology, 96(2d Half), 259-285.

 

Roe, A. W., Pallas, S. L., Hahm, J. O., & Sur, M. (1990). A map of visual space induced in primary auditory cortex. Science, 250(4982), 818-820.

 

Sampaio, E., Maris, S., & Bach-y-Rita, P. (2001). Brain plasticity: 'visual' acuity of blind persons via the tongue. Brain Research, 908(2), 204-207.

 

Simons, D.J. (2000). Current approaches to change blindness. Visual Cognition, 7, 1-15.

 

Simons, D. J., & Chabris, C. F. (1999). Gorillas in our midst: Sustained inattentional blindness for dynamic events. Perception, 28(9), 1059-1074

 

Simons, D. J., Franconeri, S. L., and Reimer, R. L. (2000). Change Blindness in the Absence of Visual Disruption,Perception, 29, 1143-1154.

 

Steriade, M. (2001). The intact and sliced brain. Cambridge MA: MIT press.

 

Thomas, N. (1999). Are theories of imagery theories of imagination? An Active Perception Approach to Conscious Mental Content. Cognitive Science, 23, (1999), 207-245

 

Torrance, S. (2002) The skill of seeing: beyond the sensorimotor account?, Trends in Cognitive Sciences,  6 (12), 495-496.

 

Veraart, C., Cremieux, J., & Wanet-Defalque, M. C. (1992). Use of an ultrasonic echolocation prosthesis by early visually deprived cats. Behavioral Neuroscience, 106(1), 203-216.

 

Yamamoto, S., & Kitazawa, S. (2001). Sensation at the tips of invisible tools. Nature Neuroscience, 4(10), 979-980