How do people learn multisensory, or amodal, representations, and what consequences do these representations have for perceptual performance? We address this question by performing a rational analysis of the problem of learning multisensory representations. This analysis makes use of a Bayesian nonparametric model that acquires latent multisensory features that optimally explain the unisensory features arising in individual sensory modalities. The model qualitatively accounts for several important aspects of multisensory perception: (a) it integrates information from multiple sensory sources in such a (...) way that it leads to superior performances in, for example, categorization tasks; (b) its performances suggest that multisensory training leads to better learning than unisensory training, even when testing is conducted in unisensory conditions; (c) its multisensory representations are modality invariant; and (d) it predicts ‘‘missing” sensory representations in modalities when the input to those modalities is absent. Our rational analysis indicates that all of these aspects emerge as part of the optimal solution to the problem of learning to represent complex multisensory environments. (shrink)

Computational theories of vision typically rely on the analysis of two aspects of human visual function: (1) object and shape recognition (2) co-calibration of sensory measurements. Both these approaches are usually based on an inverse-optics model, where visual perception is viewed as a process of inference from a 2D retinal projection to a 3D percept within a Euclidean space schema. This paradigm has had great success in certain areas of vision science, but has been relatively less successful in understanding perceptual (...) representation, namely, the nature of the perceptual encoding. One of the drawbacks of inverse-optics approaches has been the difficulty in defining the constraints needed to make the inference computationally tractable (e.g. regularity assumptions, Bayesian priors, etc.). These constraints, thought to be learned assumptions about the nature of the physical and optical structures of the external world, have to be incorporated into any workable computational model in the inverse-optics paradigm. But inference models that employ an inverse optics plus structural assumptions approach inevitably result in a na. (shrink)

1. Sometimes perceptual experience is crystal clear, as when one inspects an object close-up in bright light with corrective lenses. But experience can be less clear. To illustrate how experiences...

Bringing about desirable collisions (making interceptions) and avoiding unwanted collisions are critically important sensorimotor skills, which appear to require us to estimate the time remaining before collision occurs (time-to-collision). Until recently the theoretical approach to understanding time-to-collision estimation has been dominated by the tau-hypothesis, which has its origins in J.J. Gibson’s ecological approach to perception. The hypothesis proposes that a quantity (tau), present in the visual stimulus, provides the necessary time-to-collision information. Empirical results and formal analyses have now accumulated to (...) demonstrate conclusively that the tau-hypothesis is false. This article describes an alternative approach that is based on recent data showing that the information used in judging time-to-collision is task- and situation-dependent, is of many different origins (of which tau is just one) and is influenced by the information-processing constraints of the nervous system. (shrink)

Two experiments were performed to explore the mechanisms of human 3D shape perception. In Experiment 1, the subjects’ performance in a shape constancy task in the presence of several cues (edges, binocular disparity, shading and texture) was tested. The results show that edges and binocular disparity, but not shading or texture, are important in 3D shape perception. Experiment 2 tested the effect of several simplicity constraints, such as symmetry and planarity on subjects’ performance in a shape constancy task. The 3D (...) shapes were represented by edges or vertices only. The results show that performance with or without binocular disparity is at chance level, unless the 3D shape is symmetric and/or its faces are planar. In both experiments, there was a correlation between the subjects’ performance with and without binocular disparity. Our study suggests that simplicity constraints, not depth cues, play the primary role in both monocular and binocular 3D shape perception. These results are consistent with our computational model of 3D shape recovery. (shrink)

In this paper we contrast bounded and ecological rationality with a proposed alternative, generative rationality. Ecological approaches to rationality build on the idea of humans as “intuitive statisticians” while we argue for a more generative conception of humans as “probing organisms.” We first highlight how ecological rationality’s focus on cues and statistics is problematic for two reasons: the problem of cue salience, and the problem of cue uncertainty. We highlight these problems by revisiting the statistical and cue-based logic that underlies (...) ecological rationality, which originate from the misapplication of concepts in psychophysics. We then work through the most popular experimental task in the ecological rationality literature—the city size task—to illustrate how psychophysical assumptions have informally been linked to ecological rationality. After highlighting these problems, we contrast ecological rationality with a proposed alternative, generative rationality. Generative rationality builds on biology—in contrast to ecological rationality’s focus on statistics. We argue that in uncertain environments cues are rarely given or available for statistical processing. Therefore we focus on the psychogenesis of awareness rather than psychophysics of cues. For any agent or organism, environments “teem” with indefinite cues, meanings and potential objects, the salience or relevance of which is scarcely obvious based on their statistical or physical properties. We focus on organism-specificity and the organism-directed probing that shapes awareness and perception. Cues in teeming environments are noticed when they serve as cues-for-something, requiring what might be called a “cue-to-clue” transformation. In this sense, awareness toward a cue or cues is actively “grown.” We thus argue that perception might more productively be seen as the presentation of cues and objects rather than their representation. This generative approach not only applies to relatively mundane organism interactions with their environments—as well as organism-object relationships and their embodied nature—but also has significant implications for understanding the emergence of novelty in economic settings. We conclude with a discussion of how our arguments link with—but modify—Herbert Simon’s popular “scissors” metaphor, as it applies to bounded rationality and its implications for decision making in uncertain, teeming environments. (shrink)

How can an incomplete and uncertain model of the environment be used to perceive, infer, decide and act efficiently? This is the challenge that both living and artificial cognitive systems have to face. Symbolic logic is, by its nature, unable to deal with this question. The subjectivist approach to probability is an extension to logic that is designed specifically to face this challenge. In this paper, we review a number of frequently encountered cognitive issues and cast them into a common (...) Bayesian formalism. The concepts we review are ambiguities, fusion, multimodality, conflicts, modularity, hierarchies and loops. First, each of these concepts is introduced briefly using some examples from the neuroscience, psychophysics or robotics literature. Then, the concept is formalized using a template Bayesian model. The assumptions and common features of these models, as well as their major differences, are outlined and discussed. (shrink)

I develop a Russellian representationalist account of size experience that draws importantly from contemporary vision science research on size perception. The core view is that size is experienced in ‘body-scaled’ units. So, an object might, say, be experienced as two eye-level units high. The view is sharpened in response to Thompson’s (forthcoming) Doubled Earth example. This example is presented by Thompson as part of an argument for a Fregean view of size experience. But I argue that the Russellian view I (...) develop handles the Doubled Earth example in a natural and illuminating way, thereby avoiding the need to posit irreducible experiential ‘modes of presentation’. I also address a kind of neo-Fregean ‘reference-fixing’ view of size experience, that shares features with the Russellian view developed. I give reasons for favoring the latter. Finally, I argue that Peacocke’s claim that spatial experience is ‘unit free’ is not persuasive. (shrink)