This is an excerpt from a report on the Perceptual Learning and Perceptual Recognition II Workshop at the University of Toronto, Mississauga in May of 2012, written by Kevin Connolly, John Donaldson, David M. Gray, Emily McWilliams, Sofia Ortiz-Hinojosa, and David Suarez, and available at http://networksensoryresearch.utoronto.ca/Events_%26_Discussion.html 2. What Are the Origins of Multimodal Associations? What are the origins of multimodal associations, that is, associations between features detected by distinct sense modalities? There are two main possibilities: (1) multimodal associations are the result of neural hardwiring, or (2) multimodal associations are learned responses to environmental correlations. In her talk, Cecilia Heyes discussed a specific kind of multimodal association, namely, the association between when a person performs an action and when that person observes that same type of action. Mirror neurons are neurons that fire in both cases. Heyes argues that these neurons are a byproduct of learned visual/tactile associations. That is, she thinks that correlated sensorimotor experience forges mirror neurons, such that without such experience, we would not have such neurons. On Heyes' view, mirror neurons arise from associative learning (see Heyes, 2009). She rejects the nativist hypothesis that mirror neurons are present at or shortly after birth such that the role for experience in the development of these neurons is limited to 'tuning'. If Heyes is right, then this may be taken as evidence in favor of the hypotheses that at least some multimodal associations are learned responses to environmental conditions. This associative hypothesis may or may not extend more broadly, to sensory integration in general. Daphne Maurer argued in her talk that in neurological development, infants begin life with lots of connections between areas of the brain that seem later to handle discrete modalities (see Maurer, Gibson, and Spector, 2012). For instance, the visual cortex initially receives input from many different sense modalities. Connections from the visual input end up getting reinforced, while the other connections are pruned away. Experience plays a role in the pruning 2 of connections, strengthening those that correspond to the environment, and doing away with most of the rest. If this is right, then it is evidence that some multimodal associations may be the result of neural hardwiring. Though these connections are reinforced and strengthened by the environment, they are not learned. That is, the connections do not arise anew as the result of a learning process. Louise Richardson pointed out in her comments on Maurer's talk that although it is tempting to conclude from the evidence that infant perception is multimodal, in infants, the sensory cortices are not yet specialized. So, evidence of (what in adults is) an auditory area responding to visual input is not evidence that the infant's experience of that input is partly auditory. Nonetheless, if these early connections persist once the sensory cortices become specialized, then it seems reasonable to conclude that some multimodal associations are the result of neural hardwiring. From all this, we might conclude that both (1) and (2) have a role to play. Some multimodal associations are the result of neural hardwiring that is present at birth, and reinforced/strengthened by exposure to environmental correlations. And some such associations are the result of associative learning. In particular, the mirror neurons that associate visual with tactile experience come about as a result of associative learning. References: Heyes, C. M. (2010). "Where do mirror neurons come from?" Neuroscience and Biobehavioural Reviews, 34, 575-583. Maurer, D., Gibson, L. C., and Spector, F. (2012). "New insights into the development of multisensory perception." In Bremner, Lewkowicz, and Spence (Eds.) Multisensory Development. Oxford: Oxford University Press.