This is an excerpt from a report on the Sensory Substitution and Augmentation Conference at the British Academy in March of 2013, written by Kevin Connolly, Diana Acosta Navas, Umut Baysan, Janiv Paulsberg, and David Suarez, available at http://networksensoryresearch.utoronto.ca/Events_%26_Discussion.html 4. Can normal non-sensory feelings be generated through sensory substitution? Perception provides us with sensory contents like shapes, colors, flavors, and smells. In ordinary perception, these are accompanied by a variety of non-sensory feelings as well. Perception arouses noetic feelings concerning the presence or familiarity of what one is perceiving, emotional affects such as love and hate, hedonic responses such as pleasure and pain, and aesthetic feelings of beauty and ugliness. As Jerome Dokic noted in his presentation, although some non-sensory feelings seem to be naturally associated with specific sensory contents, non-sensory feelings do not seem to be supervenient on the sensory contents they accompany, since non-sensory feelings can vary while sensory contents remain the same. It has been reported that the use of sensory substitution devices (SSDs) fails to elicit some of the non-sensory feelings expected in normal cases of perception. Though many users of SSDs are eventually able to attribute stimuli in substituting modalities to distal objects, it takes training-and, in many cases, explicit instruction in the mapping between substituting and substituted modalities-to generate the noetic feeling that substituted stimuli are due to the presence of real objects. In addition, Bach-y-Rita et al. report an "absence of qualia" (i.e., absence of affective responses) in trained, adult users of tactile-visual sensory substitution systems (TVSS): [W]e found that while experienced blind TVSS subjects could perceive faces and printed images, they were very disappointed when perception was not accompanied by qualia: A Playboy centerfold carried no emotional message, and the face of a girl-friend or a wife created an unpleasant response since it did not convey an affective message. (2003, p. 293; see also, Bach-y-Rita, 2002) 2 Deroy and Auvray (2012), citing Lenay et al. (2003), report that hedonic responses also fail to transfer from substituted modalities: "[S]hapes perceived in one sensory modality are not directly associated to pleasures or pains felt while perceiving the same shape in another sensory modality" (p. 4). Finally, it seems likely that SSDs will face difficulties in transferring aesthetic responses between modalities. Mohan Matthen raised the question of whether visual-to-auditory sensory substitution could preserve the aesthetic properties of visual stimuli. Would a visual-toauditory SSD allow us to hear as beautiful, what is normally seen as beautiful? In response, Malika Auvray speculated that even though such a transfer of responses has not yet been found, it is possible that emotional responses to stimuli could be generated through further training. This suggestion follows Bach-y-Rita's (2003; 2002) hypothesis that SSDs fail to generate normal non-sensory feelings because the developmental and learning processes which normally result in the association of non-sensory feelings with sensory contents has not taken place. Moreover, he suggests that such associations could be developed through further training and experience in the use of SSDs. Bach-y-Rita suggests that lack of normal affective responses in SSD-use, may be compared to the acquisition of a second language as an adult. The emotional aspects of the new language are often lacking, especially with emotionally charged words and expressions, such as curse words. It appears that both spoken language and other sensory messages require long experience within the context of other aspects of cultural and emotional development to be able to contain qualia. (Bach-y-Rita, 2002, p. 510) In support of this, Bach-y-Rita reports that "Systems for blind babies ... have already provided some suggestive evidence for the development of qualia, such as the infant's smile upon perceiving the mother's approach" (Bach-y-Rita, 2002, p. 510). And, as Deroy and Auvray note, "there are similar reports of the absence of emotion and meaning felt by persons blind from birth who recover sight following the removal of cataracts" (2012, p. 4). 3 But is insufficiency of training or experience really the only stumbling block for generating normal non-sensory responses? In his commentary on Jonathan Cohen's presentation, Charles Spence pointed out that flavor and olfactory substitution devices may lack a point if they cannot duplicate the normal hedonic responses to tasty or fragrant stimuli. The problem is that it seems plausible to think that delivering the relevant information (through, say, vision, audition, or touch) would not generate the pleasure we associate with the flavor of chocolate or the smell of a rose. Moreover, it seems unlikely that further training with such a device could succeed in associating visual, auditory, or tactile representations of "tasty" or "fragrant" stimuli with a normal hedonic response. All of this suggests that merely receiving the information normally provided by a sensory modality may not be sufficient for the duplication of that modality's affective/hedonic components. And, in the case of flavor and olfaction, in particular, the hedonic component seems quite central to the normal functioning of the modality, such that sensory substitution would lose much of its point if the normal hedonic responses could not be duplicated in the substituting modality. This raises the question of how to conceive of the relationship between affective/hedonic responses and sensory modalities. Are they essential to the phenomenal feel of a sensory modality? In the absence of such responses, would we still have the same modality? What one says in response to these questions may bear on the truth of representationalism. Suppose one holds the representationalist view that sensory modalities are to be distinguished by the information they carry about the external world. It would follow that all that is needed for the substitution of sensory modality is the right kind of informational input stream. But if duplicating the informational stream doesn't duplicate the affective/hedonic responses that normally accompany the functioning of a sense, then have we really substituted for that sense? 4 References: Bach-y-Rita, P. (2002). Sensory substitution and qualia. In A. Noë & E. Thompson (Eds.), Vision and mind: selected readings in the philosophy of perception (pp. 497–514). Cambridge, MA: MIT Press. Bach-y-Rita, P., Tyler, M. E., & Kaczmarek, K. A. (2003). Seeing with the brain. International Journal of Human-Computer Interaction, 15(2), 285–295. Deroy, O., & Auvray, M. (2012). Reading the world through the skin and ears: a new perspective on sensory substitution. Frontiers in Psychology, 3(457), 1–13. doi:10.3389/fpsyg.2012.00457 Lenay, C., Gapenne, O., Hanneton, S., Marque, C., & Genouelle, C. (2003). Sensory substitution: limits and perspectives. In Y. Hatwell, A. Streri, & E. Gentaz (Eds.), Touching for Knowing (pp. 275–292). Amsterdam: John Benjamins. ual experience are true.