Active and passive scene recognition across views

Abstract

Recent evidence suggests that scene recognition across views is impaired when an array of objects rotates relative to a stationary observer, but not when the observer moves relative to a stationary display [Simons, D.J., Wang, R.F., 1998. Perceiving real-world viewpoint changes. Psychological Science 9, 315–320]. The experiments in this report examine whether the relatively poorer performance by stationary observers across view changes results from a lack of perceptual information for the rotation or from the lack of active control of the perspective change, both of which are present for viewpoint changes. Three experiments compared performance when observers passively experienced the view change and when they actively caused the change. Even with visual information and active control over the display rotation, change detection performance was still worse for orientation changes than for viewpoint changes. These findings suggest that observers can update a viewer-centered representation of a scene when they move to a different viewing position, but such updating does not occur during display rotations even with visual and motor information for the magnitude of the change. This experimental approach, using arrays of real objects rather than computer displays of isolated individual objects, can shed light on mechanisms that allow accurate recognition despite changes in the observer's position and orientation.

Introduction

Real-world object and scene recognition faces a fundamental problem: the retinal projection of the environment changes whenever the observer or objects in the environment move. Changes to the relative positions of the observer and objects can lead to size and orientation changes in the retinal projection of the environment. Yet our visual system somehow finds stability in these changing images. Two distinct approaches to achieving stability across view changes have been proposed in the literature. The system may selectively encode features of the scene that are invariant to perspective changes and use those features in object and scene recognition. For example, we may represent the object-centered spatial relationships among the parts of an object. Alternatively, our system may employ some transformation rules to compensate for changes in the retinal projection, thereby providing a common basis for comparing two different views. For example, we may mentally rotate an object until it is aligned with a previous representation or we could interpolate between two or more views to recognize objects from new perspectives.

Research on object recognition across views has provided some support for each of these possibilities. For example, Biederman and colleagues (Ellis et al., 1989; Biederman and Cooper, 1991, Biederman and Cooper, 1992; Cooper et al., 1992; see also Bartram, 1976) used a priming paradigm and measured the response latency to name line drawings of familiar objects. In their studies, the amount of priming was unaffected by changes in the retinal size of the object from study to test (scaling invariance). Furthermore, naming latency was impervious to changes to the position of the object in the visual field and to the object's orientation in depth. Biederman and Gerhardstein (1993)showed similar orientation invariance when observers were asked to match individual shapes (Geons), name familiar objects, and classify unfamiliar objects.

In contrast, many other studies suggest that object recognition performance is view-dependent; recognition accuracy and latency differ as the test views deviate from the studied view (e.g. Shepard and Metzler, 1971; Shepard and Cooper, 1982; Rock et al., 1989). With wire-frame or blob-like objects in same-different judgment tasks (Bülthoff and Edelman, 1992; Tarr, 1995; Tarr et al., 1997), subjects typically show fast, accurate recognition for test views within a small distance of the studied view and impaired performance for novel views. Furthermore, the impairment seems to be systematically related to the magnitude of the difference between studied and tested views, particularly for changes to the in-depth orientation of an object. The greater the rotation in depth away from the studied view, the longer the response latency (see also Tarr and Pinker, 1989). Such findings necessarily imply that object representations are viewer-centered.

Another critical piece of evidence in support of viewer-centered representations is that when two or more views of the same object are provided at study, subjects subsequently generalize to intermediate views but not to other views (Bülthoff and Edelman, 1992; Kourtzi and Shiffrar, 1997). A number of models of object recognition have attempted to account for this finding by positing mechanisms that operate on viewer-centered representations. For example, linear combinations of 2D views (Ullman and Basri, 1991) and view approximation (Poggio and Edelman, 1990; Vetter et al., 1995) are both consistent with these data. However, in order to interpolate between two or more views, the initial views must first be linked to the same object. That is, subjects must recognize that the same object is being viewed in the first and second studied views even though those views differ. It is unclear from these models how this initial matching is accomplished, particularly if the views are relatively far apart and the object is not symmetrical (see Vetter and Poggio, 1994). Although these models may not fully account for the nature of object recognition for novel views, the empirical data seems to support the claim that representations of individual objects are view-dependent.

Both view-independent and view-dependent models of object recognition seem to capture some aspects of how the visual system accommodates view changes. For example, when the learning period is relatively long and the object is relatively complicated and difficult to name, recognition may rely on viewer-centered representations. On the other hand, when objects are made of distinct parts whose spatial relationship can be coded easily, and when the task concerns more abstract knowledge such as naming or classification, recognition may rely on view-independent representations. Nevertheless, studies comparing these models typically test recognition for isolated objects and they ignore extra-retinal information that is available in real-world object recognition. Thus, neither model is likely to explain all aspects of object representation.