Beyond the classical receptive field: The effect of contextual stimuli Lothar Spillmann Department of Psychology, The University of Hong Kong, Pok Fu Lam, Hong Kong Neurology Clinic, University of Freiburg, Freiburg im Breisgau, Germany Birgitta Dresp-Langley University of Strasbourg, ICube UMR 7357 CNRS, Strasbourg, France Chia-huei Tseng $ Department of Psychology, The University of Hong Kong, Pok Fu Lam, Hong Kong Following?1 the pioneering studies of the receptive field (RF), the RF concept gained further significance for visual perception by the discovery of input effects from beyond the classical RF.?2 These studies demonstrated that neuronal responses could be modulated by stimuli outside their RFs, consistent with the perception of induced brightness, color, orientation, and motion. Lesion scotomata are similarly modulated perceptually from the surround by RFs that have migrated from the interior to the outer edge of the scotoma and in this way provide filling-in of the void. Large RFs are advantageous to this task. In higher visual areas, such as the middle temporal and inferotemporal lobe, RFs increase in size and lose most of their retinotopic organization while encoding increasingly complex features. Whereas lowerlevel RFs mediate perceptual filling-in, contour integration, and figure–ground segregation, RFs at higher levels serve the perception of grouping by common fate, biological motion, and other biologically relevant stimuli, such as faces. Studies in alert monkeys while freely viewing natural scenes showed that classical and nonclassical RFs cooperate in forming representations of the visual world. Today, our understanding of the mechanisms underlying the RF is undergoing a quantum leap. What had started out as a hierarchical feedforward concept for simple stimuli, such as spots, lines, and bars, now refers to mechanisms involving ascending, descending, and lateral signal flow. By extension of the bottom-up paradigm, RFs are nowadays understood as adaptive processors, enabling the predictive coding of complex scenes. Top-down effects guiding attention and tuned to task-relevant information complement the bottom-up analysis. Introduction In a previous paper (Spillmann, 2014), the early history of the receptive field (RF) concept was reviewed, recounting the seminal studies of optic nerve responses in the frog (Hartline 1938, 1940; Barlow, 1953) and cat (Kuffler, 1953; Barlow, Fitzhugh, & Kuffler, 1957) as well as the systematic studies of the functional architecture of cortical neurons in cat and monkey by Nobel Prize laureates Hubel and Wiesel (1962, 1965, 1968). ?3In these experiments, simple stimuli, such as dots, lines, and bars, were used to explore RF properties. In this paper, we extend the history of RF research to experiments in striate and extrastriate cortex, using contextual stimuli, including movie clips and natural scenes. Over the years, theoretical accounts for RF properties have progressively shifted from classic bottom-up processing toward contextual processing with top-down and horizontal modulation contributing. These latter effects provide evidence for long-range interactions between neurons relevant to figure–ground segregation and pop-out by brightness, color, orientation, texture, motion, and depth. How the neuronal mechanisms underlying these attributes generate large-scale surface properties from local features-indeed, how they construct the surfaces themselves from complex natural scenes-is one of the most pressing questions in contemporary visual neuroscience. Feed-forward projections (retino–geniculo– cortical), horizontal interactions (cortico–cortical), and backward propagation (re-entrant from MT, V4, V3, and V2 to V1) have been suggested to underlie the Citation: Spillmann, L., Dresp-Langley, B., & Tseng, C.-H. (2015). Beyond the classical receptive field: The effect of contextual stimuli. Journal of Vision, XX(XX):X, XX–XX, doi:10.1167/XX.XX.XX. //titan/production/j/jovi/live_jobs/jovi-15-09/jovi-15-09-04/layouts/jovi-15-09-04.3d  19 June 2015  7:13 pm  Allen Press, Inc.  MS#: JOV-04687-2015 Page 1 Journal of Vision (2015) 15(9):0, 1–22 1 doi: 10 .1167 /XX.XX.XX ISSN 1534-7362  2015 ARVOReceived February 15, 2015; published Month XX, XXXX perception of extended areas segregated from their surrounds. Together they account for phenomena such as the perception of uniform surfaces, filling-in, and grouping (Grossberg & Mingolla, 1985; Spillmann & Werner, 1996; Lamme, Super, & Spekreijse, 1998). These mechanisms have been proposed to enable the transition from local to global processing by using information from beyond the classical RF. Surfaces enclosed by boundaries, rather than edge effects, have become the main focus of interest. A cellular basis for these mechanisms will have to be sought in cortical visual areas rather than in interactive processes in the retina (Spillmann, 1997, 1999). Detailed laminar cortical models of figure–ground segregation have been proposed to unify the explanation of many perceptual and neurobiological data about such boundary–surface interactions (e.g., Cao & Grossberg, 2005). Long-range interaction between RFs serves not only unperturbed everyday perception, it also provides for ''repair'' mechanisms when sensory information is incomplete or ambiguous, such as in the perceptual filling-in of scotomata (Spillmann & DeWeerd, 2003; Spillmann, 2011),?4 the completion across the physiological blind spot (Kawabata, 1984; Komatsu, 2006, 2011), and in predictive scene coding of natural images.?5 Beyond the classical RF The earliest attempts to relate visual physiology to perception were severely limited by the techniques available. Wolfgang Köhler (Köhler & Held, 1949), one of the fathers of Gestalt psychology, together with Held, invoked ''electric field'' effects to explain how the perception of patterns would be produced in the brain. In their study, they set out to demonstrate an isomorphic shape correlate of pattern vision (see the review by Wurtz, 2009), but what was missing at the time was an appropriate technology for recording interpretable brain signals that could lead to an understanding of the neuronal mechanisms underlying perception. Yet, Köhler and Held's (1949) concluding remarks in that paper, suggesting that access to the cortical correlates of complex pattern vision would have an immediate impact on any theory of psychophysics and perception, turned out to be prophetic. Hardly a decade later, single-cell recordings from the cat and monkey brains (Hubel & Wiesel, 1959, 1962, 1965, 1968) were to produce exactly such an impact, marking the beginnings of a deeper understanding of the ways in which information is passed on from one processing stage to another in the brain. Systematic investigation of RF properties did not stop there. Input effects from outside the classical RF were soon discovered (for review, see Allman, Miezin, & McGuiness, 1985b), leading to a distinction between local and global percepts; the definition of contextual stimuli; and, in the 1990s, the concept of an association field (Field, Hayes, & Hess, 1993). This latter concept was a psychophysical blueprint for linking up stimulus elements lying on a common path. Therewith, Köhler and Held's (1949) idea that shape perception could find an explanation in terms of a global brain field theory was taken to the next level. This review deals with this higher level by recounting how the study of long-range signal interaction between cortical neurons has produced theoretical developments beyond the classical RF with new concepts for understanding the neural basis of complex scene integration in the brain. Long-range interaction and contextual neurons Our perception relies on the interaction between proximal and distant points in visual space, requiring shortand long-range neural connections among neurons responding to different regions within the retinotopic map. Evidently, the classical center-surround RF can only accommodate short-range interactions; for long-range interactions, more powerful mechanisms are needed. Accordingly, the hitherto established local RF properties had to be extended to take distant global inputs into account. The idea of an extended (called nonclassical or extraclassical today) RF was not new. Kuffler (1953, p. 45) already wrote, ''. . . not only the areas from which responses can actually be set up by retinal illumination may be included in a definition of the receptive field but also all areas which show a functional connection, by an inhibitory or excitatory effect on a ganglion cell. This may well involve areas which are somewhat remote from a ganglion cell and by themselves do not set up discharges.'' The first evidence for a distant modulation of a neuron came from McIlwain (1964), who demonstrated in the cat that a moving stimulus in the far periphery of the RF enhanced the response to a stimulus localized within the RF, i.e., the periphery effect. Next C. Blakemore, Carpenter, and Georgeson (1970) and C. B. Blakemore and Tobin (1972) in the cat showed that lines of different orientation interacted antagonistically, suggesting mutual inhibition between neighboring cortical columns. In a follow-up paper, Rose and Blakemore (1974) targeted a specific inhibitory neurotransmitter (bicuculline) to account for this effect. Thereafter, Fischer and Krüger (1974) in the lateral geniculate nucleus (LGN) demonstrated that a grating jerk in the far surround of an RF produced a brisk neuronal excitation in the center, i.e., the shift effect. This discovery was followed by reports in the cat //titan/production/j/jovi/live_jobs/jovi-15-09/jovi-15-09-04/layouts/jovi-15-09-04.3d  19 June 2015  7:13 pm  Allen Press, Inc.  MS#: JOV-04687-2015 Page 2 Journal of Vision (2015) 15(9):0, 1–22 Spillmann, Dresp-Langley, & Tseng 2 cortex of an unresponsive or silent surround (Maffei & Fiorentini, 1976) and, more importantly, a region beyond the classical RF, generating interactive effects between coaxial lines (Nelson & Frost, 1978). Yet, von der Heydt, Peterhans, and Baumgartner (1984) were the first to find neurons in V2 of the monkey cortex that responded to an ''incomplete'' bar as if receiving input from outside the classical RF. The authors interpreted this response as a mechanism designed to bridge a gap in a discontinuous contour. Figure 1 (right) illustrates how a neuron in monkey area V2 responds to a continuous bar moving across the RF (small oval). The response is vigorous in both directions (Figure 1A). When the bar was split into upper and lower segments, no response was expected because the RF was fully contained within the gap, yet there was a weak but undeniable response (Figure 1B). This response suggested that the neuron must have received information from outside its classical RF. There was no response when the upper and lower segments stopped short of the gap, separated from it by only a thin barrier (Figure 1C). These results prompted Peterhans and von der Heydt (1991) to propose an explanation in terms of amodal completion by illusory contours as perceived in the Schumann (Figure 1, top left) and well-known Kanizsa triangle illusions (Figure 1, bottom left). Following these early discoveries, researchers started using contextual stimuli to study contextsensitive neurons (Allman, Miezin, & McGuiness, 1985a; Gilbert & Wiesel, 1990; Gilbert, 1992; Knierim & van Essen, 1992; Sillito, Grieve, Jones, Cudeiro, & Davis, 1995). ?6Figure 2 illustrates two examples. On the top (left) is shown a pattern with a small vertical bar embedded within a textural background of horizontal bars, i.e., orientation contrast. A neuron in cat area 17 responded much more strongly to this pattern than to the uniform control pattern on the right, in which all bars have the same orientation (Kastner, Nothdurft, & Pigarev, 1999). Evidently, cross-orientation between the central bar and the bars in the surround enhanced the response whereas iso-orientation inhibited it. The same relationship is obtained for the pattern shown on the left (bottom), in which the center bar moved in one direction while the bars in the surround moved in the opposite direction, i.e., motion contrast. Again, the neuronal response to this pattern was much stronger than the response to the control pattern on the right, in which all bars moved in the same direction. In both examples, the difference in relative rather than absolute response level enables the pop-out (Z. Li, 1999, 2002). Note that the stimulus surround for both kinds of patterns in Figure 2 exceeded the size of the classical RF. Neurons therefore must have received input from beyond this area (see also Jones, Grieve, Wang, & Sillito, 2001; Jones, Wang, & Sillito, 2002). To illustrate the various ways in which RF surrounds influence their centers, we here present some of the most compelling examples of contextual modulation from the ever-growing literature. In a psychophysical experiment on contour integration in human observers, Field et al. (1993) tested the detectability of a string of Gabor patches (Figure 3, left) aligned on a background of randomly oriented Gabor patches (Figure 3, right). The authors varied (a) the angle of element rotation relative to the path, (b) the angle of path deviation from collinearity, and (c) the distance of neighboring Gabor patches from each other. Deviation from collinearity affected detectability the most, suggesting that the Gestalt factor of good continuation was critical for contour integration. Remarkably, the string could still be detected when the distance between the aligned elements was five times the Figure 1. Left (top): Schumann illusion eliciting perception of an illusory bright bar. Left (bottom): Kanizsa triangle eliciting perception of an illusory triangle defined by illusory contours and enhanced brightness. Right: Response of a V2 neuron in the monkey sensitive to stimuli eliciting perception of an illusory contour. (A) Response to a continuous bar sweeping across the RF, (B) response to the same but discontinuous bar sparing the RF, and (C) response to the same bar when both bar segments were fully contained within the white background. (D) Response to two abutting gratings (not discussed here). (E) Response to an empty field used as a control. (From Peterhans & von der Heydt, 1989.) //titan/production/j/jovi/live_jobs/jovi-15-09/jovi-15-09-04/layouts/jovi-15-09-04.3d  19 June 2015  7:13 pm  Allen Press, Inc.  MS#: JOV-04687-2015 Page 3 Journal of Vision (2015) 15(9):0, 1–22 Spillmann, Dresp-Langley, & Tseng 3 length of an individual Gabor patch. Field et al. interpreted their findings in terms of long-range interaction within an association field. Their data confirmed the prediction of how bipole receptive fields may complete boundaries (Grossberg, 1984; Grossberg & Mingolla, 1985). Importantly, trained rhesus monkeys produced psychophysical thresholds similar to those of human observers (Mandon & Kreiter, 2005). Such long-range interaction likely involves contextual RFs at different retinal locations, which are grouped together by higherorder neurons in the extrastriate cortex. Another experiment demonstrates the influence of contextual modulation by using collinear facilitation. Figure 4 (top) shows that the response of a V1 neuron to a low-contrast test line in the RF was enhanced when a high-contrast collinear flanker was presented outside this area (Nelson & Frost, 1985; Kapadia, Ito, Gilbert, Figure 2. Contextual patterns for orientation contrast (top left) and motion contrast (bottom left). In the experiment, bright bars were used on a dark background. The center bar was located inside the classical RF (small rectangle) of a neuron in cat area 17, and the surround bars were positioned outside the RF. Mean responses for motion contrast are also shown (right). The response to the contrast patterns (columns 2 and 6) was in the same order of magnitude as that to the center presented in isolation (columns 0 and 4), and there was hardly any response to the surround shown by itself (columns 3 and 7) and no response either to the uniform patterns (columns 1 and 5) serving as a control. (Modified from Kastner et al., 1999.) Figure 3. Grouping of aligned elements according to the Gestalt factor of good continuation. A string composed of iso-oriented Gabor elements is easily perceived when shown in isolation (left) but is hard to detect when embedded in a background of randomly oriented Gabor patches (right). In the experiment, the ''snake'' on the left had not previously been shown to the observer. (From Field et al., 1993.) //titan/production/j/jovi/live_jobs/jovi-15-09/jovi-15-09-04/layouts/jovi-15-09-04.3d  19 June 2015  7:14 pm  Allen Press, Inc.  MS#: JOV-04687-2015 Page 4 Journal of Vision (2015) 15(9):0, 1–22 Spillmann, Dresp-Langley, & Tseng 4 & Westheimer, 1995; Kapadia, Westheimer, & Gilbert, 2000). There was no response to the flanker alone. What may be the biological role of such a mechanism? In our world, most objects are given incompletely. Transforming local line segments into global contours is therefore crucial for object recognition. To recover a whole from its parts, the visual system must achieve contour integration through RFs that lie along a collinear path. Consistent with their neuronal results, Kapadia et al. (1995) also showed in human observers that the psychophysical threshold for raising a subliminal line to visibility was lowered by the presence of a collinear flanker, i.e., facilitation (Figure 4, bottom). Stimulus conditions were critical: A small lateral offset of the two lines from collinearity, a difference in relative orientation, or a short orthogonal line between the target line and the flanker weakened and ultimately abolished the facilitating effect. These findings are compelling evidence that contextual modulation works similarly at both the single neuron level and the population level, proving Köhler's early ''field'' intuitions right. Several authors reported comparable effects (e.g., Dresp, 1993; Polat & Sagi, 1993, 1994; Yu & Levi, 1997, 2000; Chen, Kasamatsu, Polat, & Norcia, 2001; Tzvetanov & Dresp, 2002; Dresp & Langley, 2005; Huang, Chen, & Tyler, 2012). Some of the effects shown therein were found to depend on the contrast intensity of the stimuli with facilitating interactions at low flanker contrast and inhibitory interactions at higher flanker contrast intensities (e.g., Polat & Norcia, 1996; Wehrhahn & Dresp, 1998; Chen & Tyler, 2001, 2008). Horizontal interactions in area V1 are known to be of shorter range than in V2, and although the neurophysiological data summarized in Figure 4 show that such interactions can enhance neuronal responses to short aligned stimuli, long-range boundary completion as demonstrated for illusory contour formation in monkey (von der Heydt et al., 1984; Figure 1) is limited predominantly to neuronal processing in area V2 (V1 in cat, Redies, Crook, & Creutzfeldt, 1986). A model by Grossberg, Mingolla, and Ross (1997), simulating the von der Heydt et al. (1984) and Kapadia et al. (1995) data, illustrates this distinction between V1 and V2. The third experiment exhibiting contextual modulation from the surround is based on neuronal processing of orientation contrast as a means for figure–ground segregation in the monkey (Lamme, 1995). Figure 5 (left) shows the stimulus display: two test patches of line segments with opposite or same orientation to the background. Whereas the patch on the right is barely discernible, the one on the left merges with the background and is invisible. The same stimuli are illustrated schematically by isoor crosshatched windows (located within the four boxes in the middle). The RF (small black rectangle) of a V1 neuron was always fully enclosed within the test patch. Thus, the neuron should not have received any input to inform it of the orientation of the surrounding background. The results indicated otherwise: Lamme found that the neuron readily discriminated between cross-orientation and iso-orientation of the test patch. The response to crossorientation (graphs on the right) was always stronger. This finding implies that the first steps of figure–ground segregation may already be built into the responses of the earliest, retinotopically mapped, cortical area. The figure–ground enhancement effect, however, occurred with a delay of 30–40 ms, suggesting that feedback from higher visual areas may play a role in this mechanism. A last experiment to be mentioned here involves a phenomenon called border ownership. According to Rubin (1915/1921), a figure occludes the ground and ''owns'' the borders separating it from them. ?7In an experiment tapping the neuronal mechanism of border Figure 4. Facilitation of contextual sensitivity in an alert monkey and a human observer. Top: A complex neuron in V1 responded much more strongly to a bar within the classical RF when it was presented together with a collinear flanker outside the RF (right). The flanker itself (middle) did not elicit a response. Bottom: Psychophysical threshold of a human observer in the absence (thin curve) and presence of the flanker (thick curve). The leftward shift of the response curve (arrow) indicates facilitation. (Modified from Kapadia et al., 1995.) //titan/production/j/jovi/live_jobs/jovi-15-09/jovi-15-09-04/layouts/jovi-15-09-04.3d  19 June 2015  7:14 pm  Allen Press, Inc.  MS#: JOV-04687-2015 Page 5 Journal of Vision (2015) 15(9):0, 1–22 Spillmann, Dresp-Langley, & Tseng 5 ownership, Zhou, Friedman, and von der Heydt (2000) found neurons predominantly in V2 (but also V1) of the monkey that responded selectively to the location of a figure relative to the RF. For example, an edgeselective neuron responded strongly to a contour when it was owned by a figure on the neuron's preferred side and significantly less to a contour that was owned by a figure on the other side (Figure 6). Note that the contrast step in the RF is the same for all six configurations shown, but the direction in terms of figure–ground is opposite in Figure 6A and B. The neuron illustrated has a preference for border ownership to the lower left, but other neurons with the same RF location showed the opposite preference. This suggests that any contour is represented by two groups of neurons with opposite border ownership preferences. These studies are among the clearest demonstrations of contextual influences from beyond the classical RF. By varying the distance of the remote contours from the RF (e.g., by varying the size of the squares), it is possible to measure the extent of the contextual influence (Zhang & von der Heydt, 2010, their figure 5). Border ownership selectivity and side preference are intrinsic properties of the individual neuron, possibly based on modulatory feedback from hypothetical ''grouping cells'' at a higher level (Craft, Schütze, Niebur, & von der Heydt, 2007; Mihalas, von der Heydt, & Niebur, 2011). ?8A recent study finding elevated spike synchrony between border ownership neurons when activated by the same object, which provides strong evidence for such feedback (Martin & von der Heydt, 2015). ?9Selective attention to a figure was found to enhance the responses representing border ownership (Qiu, Sugihara, & von der Heydt, 2007). These examples show that neuronal responses depend not only on local stimulus analysis within the Figure 5. Figure–ground segregation in area V1 of the trained, alert monkey. (Left) Stimulus display. Only the patch on the right-hand side is visible due to orientation contrast to the ground, and the patch on the left merges with the background. (Middle) Schematic representation of the RF (small black rectangle) within a 48 3 48 window whose hatching is either crossor iso-oriented to that of the background. Black and white demarcations were not shown. Fixation was on the center dot. Note that the orientation of the hatching in (a) and (b) is the same as is the hatching in (c) and (d). (Right) Neuronal responses were significantly larger when the hatching within the window was cross-oriented to the background than when it was iso-oriented. (Modified from Lamme, 1995.) //titan/production/j/jovi/live_jobs/jovi-15-09/jovi-15-09-04/layouts/jovi-15-09-04.3d  19 June 2015  7:14 pm  Allen Press, Inc.  MS#: JOV-04687-2015 Page 6 Journal of Vision (2015) 15(9):0, 1–22 Spillmann, Dresp-Langley, & Tseng 6 classical RF, but rather on global feature integration, and these contextual influences can extend over relatively large regions of the visual field (Gilbert & Li, 2013). This adds further proof to the Gestalt credo that a whole is not reducible to the sum of its parts. Likewise, the response of a cell to a complex stimulus cannot be fully predicted from the responses to its elements (Gilbert, 1992). Suddenly, the RF was recognized as fulfilling an important role for figure– ground segregation and surface perception, which are critical steps of processing for object perception and recognition. Contextual influences in vision and visual perception have attracted increasing interest in psychophysical and neurophysiological research (Li & Chen, 2001; Albright & Stoner, 2002; Series, Lorenceau, & Frégnac, 2003), prompting the question: How large is the outer field of such contextual neurons??10 Zipser, Lamme, and Schiller (1996), using contextual modulation for a textural figure, found that the spatial extent of contextual modulation of a parafoveal RF was approximately 88– 108 of visual angle. Measurements by Bringuier, Chavane, Glaeser, and Frégnac (1999) based on intracellular recordings reported similar orders of magnitude. These authors demonstrated that the visually evoked synaptic integration field in cat cortex extends over an area four to 15 times the size of the classical RFs of Hubel andWiesel (1962). An even larger figure comes from a study of Angelucci, Levitt, and Lund (2002a), who suggest on anatomical grounds that the field of contextual influence is 20 times larger than the classical RF. Figure 7 illustrates the classical RF center, the (classical) near surround, and the (extraclassical) outer surround. The authors attribute the first to feed-forward from the LGN, the second to horizontal input from within V1 (cortico–cortical), and the third to feedback from extrastriate areas (Hupé et al., 1998; Angelucci et al., 2002a, 2002b). RF size varies not only by virtue of contextual interaction with the outer surround; it also varies with retinal eccentricity (Daniel & Whitteridge, 1961; Drasdo, 1977) and location within the visual system. Smith, Singh, Williams, and Greenlee (2001) have compiled average data from the literature on single-unit recordings in the monkey, showing that classical RFs increase in size from near foveal to peripheral locations (for computational modeling, see Schwartz, 1980) and from V1 to higher areas in the extrastriate cortex (Freeman & Simoncelli, 2011). RFs are smallest in the primary visual cortex (V1), larger in V2, larger again in V3/VP, and largest of all in areas V3A and V4 (Figure Figure 6. Border ownership in a neuron of the macaque. Left: In all six panels, a purple-to-gray (light-to-dark) edge stimulates the RF (small ellipse) of a V2 neuron. Top: In (A), the edge is owned by the light square on the lower left; in (B), it is owned by the dark square on the upper right. Middle: Here, the shape of the contours next to the RF is the same as above, but the direction of border ownership is reversed. Bottom: Stimulation by the border between two overlapping figures. Right: The black columns labeled (A) and (B) show the neuronal responses elicited by each of the stimuli on the left. In each case, the response is consistently stronger when the stimulating edge is owned by the figure on the lower left. This asymmetry is taken as evidence for a neuronal correlate of border ownership. (Modified from Zhou et al., 2000.) Figure 7. Classical RF and outer surround (schematic). RF center (white disk), near inhibitory surround (light gray zone), and far outer surround (dark gray annulus). (From Angelucci et al., 2002a, 2002b.) //titan/production/j/jovi/live_jobs/jovi-15-09/jovi-15-09-04/layouts/jovi-15-09-04.3d  19 June 2015  7:14 pm  Allen Press, Inc.  MS#: JOV-04687-2015 Page 7 Journal of Vision (2015) 15(9):0, 1–22 Spillmann, Dresp-Langley, & Tseng 7 8A). Also the slope of the functions describing the increase in size with eccentricity increases progressively from lower to higher visual areas. Dumoulin and Wandell (2008) likewise present average data for neurophysiological RFs (singleand multiunit) from the literature and arrive at the same stacking order although the slopes of their regression lines for visual areas V1, V2, and V3 are less steep (Figure 8B). For processing visual information, RFs are not isolated entities of single neurons, but part of an interrelated network, in which one input affects another. Functional magnetic resonance imaging (fMRI) has recently been used to map neuronal responses to complex patterns and estimate the population receptive field (pRF) in various visual field locations. These quantitative estimates of pRF size in occipital regions of the human visual cortex are shown in Figures 8C through F. Overall data for V1–V3 compare reasonably well with single-cell RF measurements obtained at corresponding eccentricities and locations in monkey visual cortex (Figure 8A, B), and RF sizes in the lateral occipital are much greater. Ordinates and abscissas in Figure 8 have been scaled appropriately for better comparison except for Figure 8C, the ordinate of which is given in duty cycles (percentage). Compared with RFs of neurons in V1–V3, RFs in yet higher visual areas, such as the inferotemporal (IT) and middle temporal (MT) lobes are considerably larger, covering as much as 258 of visual angle (Felleman & Kaas, 1984; Rolls, Aggelopoulos, & Zheng, 2003); they also lose much of their retinotopic organization although this has been disputed for human brains (Wandell & Winawer, 2011). At the same time, such neurons encode increasingly complex stimuli. For example, although the RFs of neurons in areas V1–V3 mediate perceptual filling-in, contour integration, and figure–ground segregation, neurons in IT respond to faces (Perrett, Rolls, & Caan, 1982) and in MT to coherently moving patterns (Desimone, Albright, Gross, & Bruce, 1984; Singer, 1989) and biological motion (Oram & Perrett, 1994). For a summary of visual percepts and their presumed level of origin in the brain, see table 1 in Spillmann (2009). Furthermore, RF properties of cells in lower cortical areas are rather fixed compared to those in the temporal and parietal cortex, which are more malleable (Ben Hamed, Duhamel, Bremmer, & Graf, 2002; Quraishi, Heider, & Siegel, 2007). Dynamic RF topography: Changes in RF size and location Ever since Hartline's (1938, p. 410) first description, RFs of single cells were assumed to be invariant in size and location. Although this is generally true, it does not hold for RFs of cells that are deprived of their input. Gilbert (1992; see also Gilbert & Wiesel, 1992) reported that in the cat following a focal retinal lesion, RFs of cortical neurons fell silent immediately after deafferentation as was expected. However, within minutes, these same neurons responded again when light fell on the regions next to the lesion. At the same time, RFs near the lesion boundary expanded (by a factor of up to five) and shifted outward from the lesion site, implying dynamic changes in both RF size and location. This is shown in Figure 9. The change in cortical topography of RFs suggests that neurons can be quickly ''rewired,'' presumably by recruiting collaterals through disinhibition. Long-range interaction would then enable them to respond to input from outside the lesion area for which they were previously unresponsive. Gilbert and Wiesel (1992) in the monkey (Lund et al., 1993) and cat have reported long axonal connections capable of propagating information from the edge to the interior of a given area. ?11 The remapping of RFs from positions inside the lesion area to locations partly outside has been proposed as a possible mechanism for perceptual filling-in across a scotoma (Spillmann & Werner, 1996). In normal vision, horizontal interactions of this kind might also underlie the induction of brightness and color contrast, assimilation (neon color, watercolor effect) and their relationship to perceived stratification and transparency (Gilbert, 1992). For example, there is evidence that neurons stimulated by the edge of a surface actively propagate their information to neurons representing the interior of that surface via long-range interaction. In this way, filling-in from the border may sustain the brightness of the enclosed surface area  Figure 8. Comparison of RF and pRF measurements by various authors. Sizes are plotted as a function of retinal eccentricity for visual cortices V1–V4. Top row (A): Single cell RFs in the monkey, average data from the literature. (B) Estimates derived from singleand multiunit activity and local field potentials in nonhuman primates. Solid lines indicate averages from the literature. Middle and bottom rows (C–F): pRFs derived from fMRI measurements in human subjects. Sources: (A) Smith et al., 2001; (B) Dumoulin & Wandell, 2008; (C) Smith et al., 2001; (D) Dumoulin & Wandell, 2008; (E) Amano,Wandell, & Dumoulin, 2009; (F) Harvey & Dumoulin, 2011. Axes of ordinates and abscissas are scaled to the same axis units, except for (C), the ordinate of which is given in duty cycles (percentage) and is not directly comparable to degrees. Nomenclature: V1⁄4 ventral, D1⁄4 dorsal, VP1⁄4 ventral posterior, h1⁄4 human; LO 1⁄4 lateral occipital. (Courtesy of Dr. Franz Aiple.) //titan/production/j/jovi/live_jobs/jovi-15-09/jovi-15-09-04/layouts/jovi-15-09-04.3d  19 June 2015  7:14 pm  Allen Press, Inc.  MS#: JOV-04687-2015 Page 8 Journal of Vision (2015) 15(9):0, 1–22 Spillmann, Dresp-Langley, & Tseng 8 //titan/production/j/jovi/live_jobs/jovi-15-09/jovi-15-09-04/layouts/jovi-15-09-04.3d  19 June 2015  7:14 pm  Allen Press, Inc.  MS#: JOV-04687-2015 Page 9 Journal of Vision (2015) 15(9):0, 1–22 Spillmann, Dresp-Langley, & Tseng 9 (Huang & Paradiso, 2008).?12 Also, when a steady stimulus of uniform luminance was shown within the classical RF while the background luminance was modulated well beyond the RF area, the response of the neuron to the uniform stimulus could be modified (Rossi, Rittenhouse, & Paradiso, 1996). DeValois, Webster, DeValois, and Lingelbach (1986) studied this effect psychophysically (see also Paradiso & Hahn, 1996). Spatial interactions between distant patches of retina reinforce Mach (1865) and Hering's (1878) assertion that knowing what is locally activated is not sufficient for predicting what is globally perceived (Spillmann, 1997). ?13 Gilbert and Wiesel (1992) found that several months after the lesion, the RFs had migrated even further to the outside of the lesion scotoma. This lesion-induced shift in location suggests that the cortex of the adult cat possesses considerable plasticity and is capable of ''repairing'' a hole (scotoma) in the visual field although at the cost of geometrical topography (Figure 10). Spillmann and Werner (1996) suggested that the perception of brightness, color, texture, and stereo depth in and across a scotoma might conceivably be restored from the surround by virtue of such a mechanism. Interpolation of image features from the surround may also account for perceptual completion across the physiological blind spot. Fiorani, Rosa, Gattas, and Rocha-Miranda (1992) obtained responses in area V1 of the monkey when stimulating two regions on opposite sides of the optic disk that were 158 apart. There was no response with a stimulus on one side only. These retinal distances are several times the spatial extent of conventional RFs, implying a functional (and structural) convergence much larger than hitherto thought. Yet, in terms of cortical magnification, the same distances might be greatly reduced. Komatsu (2011) has proposed a hypothetical wiring diagram, which accounts for spreading information in the blind spot region by intracortical circuitry (Figure 11). According to this account, retinal signals from the region surrounding the BS are conveyed from layers 2/3 of V1 to layer 6, where they form a large RF (solid circle), providing completion and filling-in of the void. Whether this diagram also accounts for oriented fillingin (Kawabata, 1984) remains to be shown. Cortical models (Grossberg, 1994, 1997; Cao & Grossberg, 2005; Grossberg & Yazdanbakhsh, 2005) exploiting the functional properties of laminar cortical organization, first demonstrated by the pioneering Figure 9. Migration of RFs in cat area 17, following binocular retinal lesions at retinotopically corresponding sites. The dashed circle encloses the RF locations prior to the lesion. Hatched rectangles give the size and location of RFs shortly after the lesion. Arrows show the direction and amplitude of RF migration. X 1⁄4 postlesional unresponsive positions. (From Gilbert & Wiesel, 1992.) Figure 10. Hypothetical ''repair'' of a retinal scotoma caused by retinal laser coagulation. (a) Visual hemifield with hole representing the scotoma. (b) Cortical representation of scotoma. (c) Scotoma in V1 is gradually closed at the cost of retinal and visual field topography. (From Gilbert & Wiesel, 1992.) //titan/production/j/jovi/live_jobs/jovi-15-09/jovi-15-09-04/layouts/jovi-15-09-04.3d  19 June 2015  7:15 pm  Allen Press, Inc.  MS#: JOV-04687-2015 Page 10 Journal of Vision (2015) 15(9):0, 1–22 Spillmann, Dresp-Langley, & Tseng 10 work of Ramòn y Cajal (1899), have provided physiologically inspired mechanistic models to account for the completion and filling-in of gaps across stimuli.?14 Therein the authors distinguish between two distinct, but complementary, mechanisms: (a) perceptual completion of boundaries, which is orientation-selective, and (b) surface filling-in, which is unoriented. A good example of this fundamental distinction is the Kanizsa triangle, which is perceived both by virtue of its amodal contours and the brightness enhancement of the enclosed surface. Similar mechanisms of neuronal activity may also apply to the filling-in of an artificial scotoma, i.e., a uniform surface with no lesion applied. Pettet and Gilbert (1992) recorded dynamic changes in cat RF size when they covered the RF of a cortical neuron with a uniform mask on a jittering background. In analogy to the retinal lesion condition, the neuron responded to stimulation from outside the mask with a fivefold increase in size. This is illustrated by Figure 12, in which the RF size originally corresponded to frame size #1 but expanded to frame size #2 when an occluder covered the RF. When the occluder was removed, the RF shrank to frame size #3, just to re-expand to frame size #4 when it was put back on again. Finally, without the occluder, the RF collapsed to frame size #5, slightly smaller than its original extent (frame #1). Such changes in size occurred within a span of only 15 min after applying the mask, suggesting unmasking of preexisting connections. These results show that RF size adjusts itself to stimulus demands and, thereby, challenging two established beliefs in neuroscience: (a) that the RF would correspond to an invariant set of photoreceptors funneling their inputs onto higher-level sensory neurons and (b) that there would be a fixed RF map based on retinal topography. Dilks, Baker, Liu, and Kanwisher (2009) recently reported similar results in psychophysics. These authors ''deprived'' the region of the visual cortex (V1) corresponding to the blind spot in one eye by patching an observer's contralateral eye. Within seconds of this deprivation, observers reported a white square 0.58 away from the boundary of the blind spot to extend (''stretch'') into the blind spot. This perceptual elongation is suggestive of rapid RF expansion within the deprived blind spot area in V1, analogous to findings from single-cell recordings after a retinal lesion (Gilbert & Wiesel, 1992). A similar effect was observed in a patient who had suffered a stroke that destroyed the fibers that normally provide input to the upper left visual field in V1 and who described a black square presented to the lower left visual field as a ''finger'' reaching toward and into the upper blind visual field (Dilks, Serences, Rosenau, Yantis, & McCloskey, 2007). Similarly, a circle was described as cigar-like and a triangle as pencil-like. Figure 11. Hypothetical wiring diagram for the perceptual filling in of the blind spot. (A) Cortical representation of the blind spot of the right eye when both eyes are open. The blind spot, clearly visible by the empty notch at the level of the LGN, is completely obscured in V1 due to input from the left eye. (B) Cortical representation of the same area, when the left eye is closed. Here the area corresponding to the blind spot is ''filled in'' at the level of V1 by feedback from neighboring layers 2/3 onto layer 6. In both cases, the blind spot is not perceived. (From Komatsu, 2011.) //titan/production/j/jovi/live_jobs/jovi-15-09/jovi-15-09-04/layouts/jovi-15-09-04.3d  19 June 2015  7:15 pm  Allen Press, Inc.  MS#: JOV-04687-2015 Page 11 Journal of Vision (2015) 15(9):0, 1–22 Spillmann, Dresp-Langley, & Tseng 11 These fleeting perceptual elongations were confirmed by fMRI (Dilks et al., 2007) and are consistent with the assumption of a cortical reorganization in V1 due to long-standing deafferentation. In this context, it is noteworthy that Ricco's area for complete spatial summation (the psychophysical equivalent of a neuronal RF) becomes larger by 30% when retinal cell density decreases with age (Malania et al., 2011). On the other hand, no evidence for cortical remapping has been found in patients suffering from age-dependent macular degeneration (AMD) (Baseler et al., 2011). This may be because, here, lateral separation in cortical tissue space is exceptionally large due to the foveal magnification factor. On the other hand, filling-in of a line, grating, or regular dot pattern in AMD patients has been found to occur across several degrees of visual angle (Zur & Ullman, 2003), suggesting long-range cortical reorganization in V1. Feature discontinuities, saliency maps, and predictive coding Since the early discoveries of extraclassical RF effects, their functional characteristics have been studied further to explore how bottom-up mechanisms, such as end-stopping, would account for the long-range coding of feature discontinuities in visual stimuli. Findings revealed that the firing rates of cat cortical neurons in area 17 exposed to edges perpendicular to their preferred orientation were enhanced when a ''feature border'' was presented outside and close to the RF (Z. Li 1999, 2002; Schmid, 2008). The ''feature borders'' were defined by discontinuities in phase, orientation, or motion direction of the stimulus. A comparison with control measures led to the conclusion that the enhanced firing rates were due to a release in suppression (i.e., disinhibition). Model accounts of the observations suggest that center–surround interactions, contextual modulation, and end-stopping are part of a single brain mechanism for representing spatial discontinuities in visual image analysis, with which, quite often, several goals must be achieved simultaneously as in orientation-based texture segmentation (Schmid & Victor, 2014). Although visual RFs are typically considered bottom-up detectors, or neuronal filters, selective only to given stimulus parameters (Spillmann, 2014), contextual neurons have recently been found in area V4 that are modifiable by attention, i.e., top-down processing (Krause & Pack, 2014). Specifically, the allocation of spatial attention may be understood as a behavioral characteristic of visual RFs (Treue, 2012) whose sensitivity to spatial stimuli is dynamically modulated by the attentional spotlight. ?15There are two modes of attention: passive and active. For example, a perceptual object in the visual field may capture attention in a stimulus-driven fashion, or it may become subject to goal-directed top-down attentional control (e.g., Yantis & Jonides, 1990, Conci et al., 2001). ?16 Attention is one modifier of RFs; the choice of stimuli is another. Most studies cited so far used laboratory stimuli. With the advent of studies in alert monkeys using free viewing and natural stimuli, an increase in information transmission efficiency has been found for natural scenes in V1 (Gallant, Connor, & van Essen, 1998; Vinje & Gallant, 2002). These studies suggest that extraclassical RF effects may be linked to Figure 12. Dynamic change of RF size in cat area 17 in response to an artificial scotoma on a jittering background. (Left) The small empty diamond illustrates the original size of the RF. The large square surrounding it gives the size of a mask used to occlude the RF. (Right) Individual frames depict the dynamic expansion and contraction of the RF when the mask was alternatively applied (2, 4) or removed (3, 5). Time for conditioning 15 min. (From Pettet & Gilbert, 1992.) //titan/production/j/jovi/live_jobs/jovi-15-09/jovi-15-09-04/layouts/jovi-15-09-04.3d  19 June 2015  7:15 pm  Allen Press, Inc.  MS#: JOV-04687-2015 Page 12 Journal of Vision (2015) 15(9):0, 1–22 Spillmann, Dresp-Langley, & Tseng 12 the predictive coding of natural images. Neural circuits would learn the statistical regularities of the natural world and communicate deviations from regularity to higher processing centers. Such selective signaling would reduce redundancy by discarding the predictable, hence redundant, components of the input signal (Rao & Ballard, 1999). More recently, interpretations of extraclassical RF effects have been extended even further in computational models, suggesting that V1 firing patterns may deliver universal signatures of visual saliency irrespective of their intrinsic feature preferences, e.g., contrast polarity (Z. Li, 1999, 2002). What is more salient is likely to figure and to attract visual attention first. Despite these advances into uncharted territory, much of the response variance in V1 still remains unexplained, and it is likely that one of the central functions of cortical processing is to predict upcoming stimulus events based on the spatial and temporal context of a scene. Muckli and colleagues (Muckli, Vetter, & Smith, 2011; Muckli & Petro, 2013) investigated the information content of feedback projections using the apparent motion path between two alternating stimulus locations (Wertheimer, 1912) or an illusory shape suggesting a partially occluded triangle (Kanisza, 1955) to probe for a response in retinotopic regions of the brain.?17 This is reminiscent of Ginsburg (1975), who used spatial filtering in an attempt to isolate and enhance the illusory triangle in a Kanizsa figure, thereby demonstrating, for the first time, that the relevant information (i.e., the illusory contours) generated by the incomplete stimulus pattern was implicit in the overall spatial relationships of that pattern (for review, see Dresp, 1997). Similarly, Muckli et al. (2011) and Muckli and Petro (2013) analyzed fMRI activity patterns corresponding to incomplete stimulus parts and found that they generated objectrelated percepts. The authors concluded that extraclassical RFs of neurons in V1 carry information relevant to the conscious interpretation of an incomplete stimulus as a meaningful whole. This kind of predictive coding introduces a conceptual shift in visual neuroscience, with which the brain is seen as continually generating models of the world based on information from memory in order to give meaning to incomplete sensory input. As pointed out already by Helmholtz (1867/ 1924), MacKay (1956), and Gregory (1980), our perception is guided by inferences, or object hypotheses, by which it seeks to resolve ambiguities in the stimulus in the most plausible manner (Spillmann & Dresp, 1995). In the brain, such predictive models would be created in higher cortical areas and communicated to lower areas through feedback connections (Muckli & Petro, 2013). Outlook and perspectives Research on RFs started 75 years ago and is moving on swiftly. During the last 20 years, the RF concept in neuroscience research has undergone a complete revision from that of the earlier years (for review, see Löffler, 2008; Spillmann, 2014), showing that functional properties of RFs depend (a) on the visual context in which a target stimulus is embedded and (b) on the method of analysis used. This article does not attempt to be complete in reviewing all the extraclassical RF effects reported in this rapidly evolving field. Rather, we restricted ourselves to describing some of the major findings in the literature. Developments clearly do not stop here, and further exciting discoveries will undoubtedly come up in the near future. In a nutshell, although RFs were formerly believed to have invariant response characteristics, they are, in fact, modifiable by intracortical (lateral, recurrent) interactions (e.g., Bair, 2005; Yeh, Xing, Williams, & Shapley, 2009; Neri, 2011; see also Grossberg & Raizada, 2000). Also, Hubel and Wiesel's (1962, 1965) initial distinction between complex and hypercomplex cells in the functional hierarchy of the primary visual cortex had to be reconsidered (Mechler & Ringach, 2002; Bair, 2005). Simple stimuli, such as flashed spots, oriented bars, and drifting gratings, used in the early studies of RF properties only revealed the most basic response properties. Our knowledge of RF size and location in different parts of the visual field has since evolved considerably (see Figures 7 and 8). Hartline's dictum that RFs are ''fixed'' has been shown to be untrue. ?18Furthermore, it was found that one and the same stimulus feature elicits a stronger response when embedded in a natural scene rather than in a random field (Field, 1987). Complex stimuli, such as natural images or movie clips (Olshausen & Field, 1996; Gallant et al., 1998; Vinje & Gallant, 2003; Felsen & Dan, 2005), have revealed new RF substructures (see also Ringach, Hawken, & Shapley, 2002; Carandini et al., 2005; Schwartz et al., 2012). ?19 The recent proposal that V1 responses constitute visual saliency maps (e.g., Z. Li, 1999, 2002; Zhaoping, 2008, 2014) adds to the early intuitions by Köhler and Held (1949) relative to the existence of a Gestalt field at the level of neural representation. The research on predictive coding (e.g., Muckli & Petro, 2013) discussed above has provided us with new accounts of the functional role of complex intracortical feedback and top-down processing. The RFs of what were formerly called feature detectors are influenced by spatiotemporal context, selective attention, and memory. Why this is so can be understood on the basis of the brain's need to constantly update knowledge. Not only do familiar objects need to be detected and recognized quickly, new objects never before encountered need to be learned as //titan/production/j/jovi/live_jobs/jovi-15-09/jovi-15-09-04/layouts/jovi-15-09-04.3d  19 June 2015  7:15 pm  Allen Press, Inc.  MS#: JOV-04687-2015 Page 13 Journal of Vision (2015) 15(9):0, 1–22 Spillmann, Dresp-Langley, & Tseng 13 well and added to the memory inventory. This requires that the already learned visual representations are stable and accessible and that there is capacity for the processing and acquisition of new and not yet learned input. Grossberg (1983) called this the stability– plasticity dilemma, which is addressed by his adaptive resonance theory (ART). In ART networks, top-down projections generate a hypothesis for the recognition of objects from the sensory input. If such a hypothesis is recurrently reinforced and consolidated, it is believed to lead to conscious recognition (Grossberg, 1983, 1997). RFs thus have acquired an important role in providing knowledge about the visual world. The ecological relevance of RF behavior was first touched upon by Barlow (1953), Lettvin, Maturana, McCulloch, & Pitts (1959), Jung (1961), and Baumgartner (1990), and it is coming into the focus of contemporary neuroscience. The key questions here are (a) how do RFs change dynamically to enhance their contribution to visual perception in different tasks, and (b) how does the visual brain integrate local cues to form global representations within a dynamically changing world (see von der Heydt & Peterhans, 1989; Spillmann, 1999; Pan et al., 2012). Thus, the revised RF concept takes into consideration not only functional plasticity and a bottom-up saliency map, but also top-down processes, such as spatial attention, the detection of irregularities, scene recognition, and priming. Such modulation by higherlevel input becomes plausible if one considers that- surprisingly-far more fibers descend from the primary visual cortex (V1) of the monkey to the LGN than ascend in the opposite direction (Peters, Payne, & Budd, 1994). The results of a systematic study using localized tissue cooling (Payne, Lomber, Villa, & Bullier, 1996) are consistent with massive feedback from higher visual areas (V4, V3) to lower ones (V2, V1). Spatial attention, for example, would appear to act like a gain control mechanism, enhancing the perceptual salience of the object under consideration and suppressing information from outside the focus of interest (Itti & Koch, 2001). Recent research (e.g., Z. Li, 1999; Gilbert & Li, 2013; Schmid & Victor, 2014) suggests that, in addition to responding to select physical properties of local stimuli, RFs and their associated neurons avail themselves of information from global stimuli that are relevant to the ongoing perceptual task. Rather than possessing fixed functional properties, as suggested by Hartline (1938), RFs are therefore conceived as dynamic processors whose tuning changes according to stimulus context, expectation, and attention. For example, in a behavioral curve-tracing task, it was demonstrated that neurons, whose RFs lay along a curved contour, responded more strongly when the contour was attended to by the monkey rather than when it was unattended (Roelfsema, Lamme, & Spekreijse, 1998). These data show that attention and perceptual grouping interact in the interest of boundary formation as predicted by laminar cortical models of vision (e.g., Grossberg & Raizada, 2000; Raizada & Grossberg, 2001). How cortical processes subserving boundary formation interact with the top-down processes that control attention is one of the core issues addressed by these models. Poort et al. (2012), recording from V1 and V4 in the monkey, conclude that boundary detection is an early process based on bottom-up computation whereas surface filling occurs later, requires feedback, and is facilitated by visual attention. It thus appears that the RFs of extrastriate neurons behave like matched filters, or templates, that are dynamically tuned to optimize visual processing and visual search (David, Hayden, Mazer, & Gallant, 2008; Schmid & Victor, 2014). Their selectivity for searched patterns is enhanced by attention (Itti & Koch, 2001; Ipata, Gee, & Goldberg, 2012). It has long been known that a cell in the superficial layers of the superior colliculus responds more robustly when a stimulus that falls within its RF becomes the target for a subsequent saccade (Goldberg & Wurtz, 1972). ?20We now know that attention, in conjunction with goal-directed saccades, modulates the RFs of neurons in macaque V4 and MT by shifting their centers toward attended locations, not unlike a flashlight (Colby, Duhamel, & Goldberg, 1996; Tolias et al., 2001; Womelsdorf, Anton-Erxleben, & Treue, 2008,Womelsdorf, Anton-Erxleben, Pieper, & Treue, 2006). This spatiotemporal dynamics in predictive remapping demonstrate that, already in the extrastriate cortex, RF properties are not invariant but highly adaptable, enhancing perceptual processing whenever a task requires it. The attentional spotlight tells the brain where in the restricted space of the visual field a change has occurred. This is called detection. But for recognition (is it a line, a dot, or a small animal?), top-down processes are needed to identify the perceived objects. Beyond the level of signal detection, perception relies in part on information stored in memory representations (e.g., Churchland, 2002). Future research on extraclassical RF properties will have to include studies on processes of perceptual learning and memory. The temporal firing characteristics of neurons are critical in these processes (e.g., Jensen, Idiart, & Lisman, 1996; Churchland, 2002) as most of perceptual learning is temporal rather than spatial (see Wang, Cong, & Yu, 2013). Résumé Since Hartline's (1938, 1940) original studies in the frog, the RF concept has evolved in several ways. Table //titan/production/j/jovi/live_jobs/jovi-15-09/jovi-15-09-04/layouts/jovi-15-09-04.3d  19 June 2015  7:15 pm  Allen Press, Inc.  MS#: JOV-04687-2015 Page 14 Journal of Vision (2015) 15(9):0, 1–22 Spillmann, Dresp-Langley, & Tseng 14 1 includes the most important discoveries: (a) lateral inhibition; (b) DOG filters and Fourier channels; (c) perceptive fields; (d) simple, complex, and hypercomplex (end-stopped) RFs; (e) RFs beyond the classical RF with contextual sensitivity; (f) large outer surrounds; (g) RFs sensitive to selective rearing and deprivation; (h) RF dynamics; and (i) RFs responsive to natural scenes.?21 Hubel and Wiesel (e.g., Hubel, 1963) in the early 1960s advanced the hypothesis that RFs of cells at a given level of the visual system emerged by combining a number of lower-level RFs. Sustained and transient channels in the cat were added to their hierarchical feed-forward concept of simple, complex, and hypercomplex cells in the 1970s. The feedforward concept was challenged in the mid-1980s, when researchers discovered that neuronal responses could be modified by stimulation from the extended outer surround (i.e., beyond the classical RF). In the 1990s, further research reported long-range horizontal interactions as well as reciprocal projections from higher visual areas, enabling higher level neurons to modulate neuronal responses at lower levels by feedback (Hupé et al., 1998; Cudeiro & Sillito, 2006). Our understanding of visual perception has gained immeasurably from each of these steps. Population perceptive fields (pPFs) have become the psychophysical and perceptual counterparts of RFs. ?22A next step is the application of the RF concept to natural stimuli and natural scenes, including cognitive strategies, such as attention, search, and perceptual learning (W. Li, Piëch, & Gilbert, 2004, 2008; Seitz & Dinse, 2007; Wang et al., 2013). It thus seems that after 75 years of research, bottom-up finally meets top-down and feature detection gives way to perception and cognition. Keywords: nonclassical receptive fields, contextual neurons, association field, attention, predictive coding Acknowledgments This work was supported by a General Research Fund from the Research Grants Council of Hong Kong, China, and the Hong Kong University Seed Funding Program for Basic Research to Dr. Tseng. It represents an expanded version of an invited talk given at APCV 2013 in Su-zhou, China. The talk was sponsored, in part, by a Teaching Excellence Fellowship of the University of Hong Kong (to Prof. Chia-huei Tseng), research funds from the PekingTsinghua Center for Life Sciences, Peking University (to Prof. Cong Yu), and Chinese Academy of Sciences, Shanghai (to Prof. Wei Wang). Profs. Horace Barlow, Charles Gilbert, Barbara Heider, Wu Li, Susana Martinez-Conde, Jerry Nelson, Leo Peichl, James Thomas, John Troy, Wei Wang, John S. Werner, and Li Zhaoping offered valuable comments on earlier versions of the manuscript. Moeka Komachi, Jing Ting Huang, and Dr. Franz Aiple helped with the figures, Matt Oxner and Diederick Niehorster with editorial work. The librarians of the Medical Library at the University of Freiburg provided valuable assistance in bibliographical research. We thank them all. ?23 Hartline, 1938, 1940 Summation area in frog optic nerve Barlow, 1953 Lateral surround in frog optic nerve, fly detector Kuffler, 1953 Lateral surround in cat optic nerve Barlow et al., 1957 Loss of lateral inhibition in scotopic vision Lettvin et al., 1959 Bug detector Enroth-Cugell & Robson, 1966 Campbell & Robson, 1968 Fourier channels Jung, 1961; Jung, Baumgarten, & Baumgartner, 1952 Jung & Spillmann, 1970; Ransom-Hogg & Spillmann, 1980 Psychophysical correlates, perceptive fields Baumgartner et al., (1984) Oehler, 1985 Illusory contour responses Westheimer function in monkeys Hubel & Wiesel, 1962 Simple cells Hubel & Wiesel, 1965, 1968 Complex cells, end-stopped cells Blakemore and colleagues?43 Effects of deprivation, plasticity Allman et al., 1985b Knierim & van Essen, 1992 Sillito et al., 1995 Kastner et al., 1999 Contextual neurons Field et al., 1993 Kapadia et al., 1995 Lamme, 1995 Contextual stimuli Angelucci et al., 2002a, 2002b Large outer surround Wiesel & Hubel, 1963, 1965, 1966 Effects of selective rearing Gilbert & Wiesel, 1992 Filling-in of lesion scotoma Pettet & Gilbert, 1992 Filling-in of artificial scotoma Olshausen & Field, 1996 RFs and natural stimuli Gilbert & Li, 2013 Dynamic processors, perceptual tasks Table 1. Major steps in RF research.?42 //titan/production/j/jovi/live_jobs/jovi-15-09/jovi-15-09-04/layouts/jovi-15-09-04.3d  19 June 2015  7:15 pm  Allen Press, Inc.  MS#: JOV-04687-2015 Page 15 Journal of Vision (2015) 15(9):0, 1–22 Spillmann, Dresp-Langley, & Tseng 15 Commercial relationships: none. Corresponding author: Chia-huei Tseng. E-mail: CH_Tseng@alumni.uci.edu. Address: Department of Psychology, The University of Hong Kong, Pok Fu Lam, Hong Kong References Albright, T. D., & Stoner, G. R. (2002). Contextual influences on visual processing. Annual Review of Neuroscience, 25, 339–379. Allman, J., Miezin, F., & McGuinness, E. (1985a). Direction-and velocity-specific responses from beyond the classical receptive field in the middle temporal visual area (MT). Perception, 14, 105–126. Allman, J., Miezin, F., & McGuiness, E. (1985b). Stimulus specific responses from beyond the classical receptive field: Neurophysiological mechanisms for local–global comparisons in visual neurons. Annual Review of Neuroscience, 8, 407– 430.?24 Amano, K., Wandell, B. A., & Dumoulin, S. O. (2009). Visual field maps, population receptive field sizes, and visual field coverage in the human MTþ complex. Journal of Neurophysiology, 102, 2704– 2718. Angelucci, A., Levitt, J. B., & Lund, J. S. (2002a). Anatomical origins of the classical receptive field and modulatory surround field of single neurons in macaque visual cortical area V1. Progress in Brain Research, 136, 373–388. Angelucci, A., Levitt, J. B., Walton, E., Hupé, J. M., Bullier, J., & Lund, J. S. (2002b). Circuits for local and global signal integration in primary visual cortex. The Journal of Neuroscience, 22, 8633–8646. Attneave, F. (1954). Some informational aspects of visual perception. Psychological Review, 61, 183– 193.?25 Bair, W. (2005). Visual receptive field organization. Current Opinion in Neurobiology, 15, 459–464. Barlow, H. B. (1953). Summation and inhibition in the frog's retina. Journal of Physiology, 119, 69–88. Barlow, H. B., Fitzhugh, R., & Kuffler, W. S. (1957). Change of organization in the receptive fields of the cat's retina during dark adaptation. Journal of Physiology, 137, 338–354. Baseler, H. A., Gouws, A., Haak, K. V., Racey, C., Crossland, M. D., Tufail, A., . . . Morland, A. B. (2011).Large-scale remapping of visual cortex is absent in adult humans with macular degeneration. Nature Neuroscience, 14, 649–655. Baumgartner, G. (1990). Where do visual signals become a perception?. In J. Eccles & O. Creutzfeldt (Eds.), The principles of design and operation of the brain, vol. 78 (pp. 99–114). Vatican City: Pontificiae Academiae Scientiarum Scripta Varia. Baumgartner, G., von der Heydt, R., & Peterhans, E. (1984). Anomalous contours: A tool for studying the neurophysiology of vision. Experimental Brain Research, 9(Suppl.), 413–419. Ben Hamed, S., Duhamel, J.-R., Bremmer, F., & Graf, W. (2002). Visual receptive field modulation in the lateral intraparietal area during attentive fixation and free gaze. Cerebral Cortex, 12, 234–245. Blakemore, C., Carpenter, R. H. S., & Georgeson, M. A. (1970). Lateral inhibition between orientation detectors in the human visual system. Nature, 228, 37–39. Blakemore, C. B., & Tobin, E. A. (1972). Lateral inhibition between orientation detectors in cat's visual cortex. Experimental Brain Research, 15, 439–3440. ?26 Bringuier, V., Chavane, F., Glaeser, L., & Frégnac, Y. (1999). Horizontal propagation of visual activity in the synaptic integration field of area 17 neurons. Science, 283, 695–699. Campbell, F. W., & Robson, J. G. (1968). Application of Fourier analysis to the visibility of gratings. Journal of Physiology, 197, 551–566. Cao, Y., & Grossberg, S. (2005). A laminar cortical model of stereopsis and 3D surface perception: Closure and da Vinci stereopsis. Spatial Vision, 18, 515–578. Carandini, M., Demb, J. B., Mante, V., Tolhurst, D. J., Dan, Y., Olshausen, B. A., . . . Rust, N. C. (2005). Do we know what the early visual system does? Journal of Neuroscience, 25, 10577–10597. Chen, C. C., Kasamatsu, T., Polat, U., & Norcia, A. M. (2001). Contrast response characteristics of long-range lateral interactions in cat striate cortex. Neuroreport, 12, 655–661. Chen, C. C., & Tyler, C. W. (2001). Lateral sensitivity modulation explains the flanker effect in contrast discrimination. The Proceedings of the Royal Society (London) Series B, 268, 509–516. Chen, C. C., & Tyler, C. W. (2008). Excitatory and inhibitory interaction fields of flankers revealed by contrast-masking functions. Journal of Vision, 8(4): 10, 1–14, doi:10.1167/8.4.10. Churchland, P. S. (2002). Brain-wise. Studies in neurophilosophy. Cambridge, MA: MIT Press. Colby, C. L., Duhamel, J. R., & Goldberg, M. E. (1996). Visual, presaccadic, and cognitive activa- //titan/production/j/jovi/live_jobs/jovi-15-09/jovi-15-09-04/layouts/jovi-15-09-04.3d  19 June 2015  7:15 pm  Allen Press, Inc.  MS#: JOV-04687-2015 Page 16 Journal of Vision (2015) 15(9):0, 1–22 Spillmann, Dresp-Langley, & Tseng 16 tion of single neurons in monkey lateral intraparietal area. Journal of Neurophysiology, 76, 2841– 2852. Conci, M., Tollner, T., Leszczynski, M., & Muller, H. J. (2011). The time-course of global and local attentional guidance in Kanizsa-figure detection. Neuropsychologia, 9, 2456–2464. Craft, E., Schuetze, H., Niebur, E., & von der Heydt, R. (2007). A neural model of figure-ground organization. Journal of Neurophysiology, 97, 4310– 4326. Cudeiro, J., & Sillito, A. M. (2006). Looking back: Corticothalamic feedback and early visual processing. Trends in Neurosciences, 29, 298–306. Daniel, P. M., & Whitteridge, D. (1961). The representation of the visual field on the cerebral cortex in monkeys. Journal of Physiology, 159, 203–221. David, S. V., Hayden, B. Y., Mazer, J. A., & Gallant, J. L. (2008). Attention to stimulus features shifts spectral tuning of V4 neurons during natural vision. Neuron, 59, 509–521. Desimone, R., Albright, T. D., Gross, C., & Bruce, C. (1984). Stimulus-selective properties of inferior temporal neurons in the macaque. Journal of Neuroscience, 4, 2051–262. DeValois, R. L., Webster, M. A., DeValois, K. K., & Lingelbach, B. (1986). Temporal properties of brightness and color induction. Vision Research, 26, 887–897. Dilks, D. D., Baker, C. I., Liu, Y., & Kanwisher, N. (2009). Referred ''visual sensations'': Rapid perceptual elongation after visual cortical deprivation. Journal of Neuroscience, 29, 8960–8964. Dilks, D. D., Serences, J. T., Rosenau, B. J., Yantis, S., & McCloskey, M. (2007). Human adult cortical reorganization and consequent visual distortion. Journal of Neuroscience, 27, 9585–9594. Drasdo, N. (1977). The neural representation of visual space. Nature, 266, 554–556. Dresp, B. (1993). Bright lines and edges facilitate the detection of small light targets. Spatial Vision, 7, 213–225. Dresp, B. (1997). On 'illusory' contours and their functional significance. Current Psychology of Cognition, 16, 489–517. Dresp, B., & Bonnet, C. (1991). Psychophysical evidence for low-level processing of illusory contours. Vision Research, 10, 1813–1817.?27 Dresp, B., & Langley, O. K. (2005). Long-range spatial integration across contrast signs: A probabilistic mechanism? Vision Research, 45, 275–284. Dumoulin, S. O., & Wandell, B. A. (2008). Population receptive field estimates in human visual cortex. NeuroImage, 39, 647–660. Enroth-Cugell, C., & Robson, J. G. (1966). The contrast sensitivity of retinal ganglion cells of the cat. Journal of Physiology, 187, 517–552. Felleman, D. J., & Kaas, J. H. (1984). Receptive-field properties of neurons in middle temporal visual area (MT) of owl monkeys. Journal of Neurophysiology, 52, 488–513. Felsen, G., & Dan, Y. (2005). A natural approach to studying vision. Nature Neuroscience, 8, 1643–1646. Field, D. J. (1987). Relations between the statistics of natural images and the response of cortical cells. Journal of the Optical Society of America A, 4, 2379–2393. Field, D. J., Hayes, A., & Hess, R. F. (1993). Contour integration by the human visual system: Evidence for a local ''association field.'' Vision Research, 33, 173–193. Fiorani, M., Rosa, M. G. P., Gattas, R., & RochaMiranda, C. E. (1992). Dynamic surrounds of receptive fields in primate striate cortex: A physiological basis for perceptual completion. Proceedings of the National Academy of Sciences, USA, 89, 8547–8551. Fischer, B., & Krüger, J. (1974). The shift effect in the cat's lateral geniculate neurones. Experimental Brain Research, 21, 225–227. Freeman, J., & Simoncelli, E. P. (2011). Metamers of the ventral stream. Nature Neuroscience, 14, 1195– 1201. Gallant, J. L., Connor, C. E., & van Essen, D. C. (1998). Neural activity in areas V1, V2, and V4 during free viewing of natural scenes compared to controlled viewing. Neuroreport, 9, 2153–2158. Gattass, R., Sousa, A. P., & Gross, C. G. (1988). Visuotopic organization and extent of V3 and V4 of the macaque. Journal of Neuroscience, 8, 1831– 1845. ?28 Gilbert, C. D. (1992). Horizontal integration and cortical dynamics. Neuron, 9, 1–13. Gilbert, C. D., & Li, W. (2013). Top-down influences on visual processing. Nature Reviews, Neuroscience, 14, 350–363. Gilbert, C. D., & Wiesel, T. (1992). Receptive field dynamics in adult primary visual cortex. Nature, 356, 150–152. Gilbert, C. D., & Wiesel, T. N. (1990). The influence of contextual stimuli on the orientation selectivity of cells in primary visual cortex of the cat. Vision Research, 30, 1689–1701. //titan/production/j/jovi/live_jobs/jovi-15-09/jovi-15-09-04/layouts/jovi-15-09-04.3d  19 June 2015  7:16 pm  Allen Press, Inc.  MS#: JOV-04687-2015 Page 17 Journal of Vision (2015) 15(9):0, 1–22 Spillmann, Dresp-Langley, & Tseng 17 Ginsburg, A. P. (1975). Is the illusory triangle physical or imaginary? Nature, 257, 219–220. Gregory, R. L. (1980). Perceptions as hypotheses. Philosophical Transactions of the Royal Society of London, B, 290, 181–197. Grossberg, S. (1983). The quantized geometry of visual space: The coherent computation of depth, form, and lightness. Behavioral & Brain Sciences, 6, 625– 657. Grossberg, S. (1984). Outline of a theory of brightness, color, and form perception. In E. Degreef & J. van Buggenhaut (Eds.), Trends in mathematical psychology (pp. 59–85). Amsterdam: North-Holland. Grossberg, S. (1994). 3D vision and figure-ground separation by visual cortex. Perception & Psychophysics, 55, 48–120. Grossberg, S. (1997). Cortical dynamics of 3-D figureground perception of 2-D pictures. Psychological Review, 104, 618–658. Grossberg, S., & Mingolla, E. (1985). Neural dynamics of perceptual grouping: Textures, boundaries, and emergent segmentations. Perception & Psychophysics, 38, 141–171. Grossberg, S., Mingolla, E., & Ross, W. D. (1997). Visual brain and visual perception: How does the cortex do perceptual grouping? Trends in Neurosciences, 20, 106–111. Grossberg, S., & Raizada, R. D. S. (2000). Contrastsensitive perceptual grouping and object-based attention in the laminar circuits of primary visual cortex. Vision Research, 40, 1413–1432. Grossberg, S., & Yazdanbakhsh, A. (2005). Laminar cortical dynamics of 3D surface perception: Stratification, transparency, and neon color spreading. Vision Research, 45, 1725–1743. Hartline, H. K. (1938). The response of single optic nerve fibers of the vertebrate eye to illumination of the retina. American Journal of Physiology, 121, 400–415. Hartline, H. K. (1940). The receptive fields of optic nerve fibers. American Journal of Physiology, 130, 690–699. Harvey, B. M., & Dumoulin, S. O. (2011). The relationship between cortical magnification factor and population receptive field size in human visual cortex: Constancies in cortical architecture. Journal of Neuroscience, 31(38), 13604–13612. Helmholtz, H. (1924). Treatise on physiological optics (J. P. C. Southall, Trans.) Washington DC: Optical Society of America. (Original work published 1867, Leipzig, Germany: Voss). Huang, P. C., Chen, C. C., & Tyler, C. W. (2012). Collinear facilitation over space and depth. Journal of Vision, 12(2):20, 1–9, doi:10.1167/12.2.20. Hubel, D. H. (1963). Integrative processes in central visual pathways of the cat. Journal of the Optical Society of America, 53, 58–66. Hubel, D. H., & Wiesel, T. N. (1959). Receptive fields of single neurones in the cat's striate cortex. The Journal of Physiology, 148, 574–591. Hubel, D. H., & Wiesel, T. N. (1962). Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. Journal of Physiology, 160, 106–154. Hubel, D. H., & Wiesel, T. N. (1965). Receptive fields and functional architecture in two nonstriate visual areas (18 and 19) of the cat. Journal of Neurophysiology, 28, 229–289. Hubel, D. H., & Wiesel, T. N. (1968). Receptive fields and functional architecture of monkey striate cortex. Journal of Physiology, 195, 215–243. Hupé, J. M., James, A. C., Payne, B. R., Lomber, S. G., Girard, P., & Bullier, J. (1998). Cortical feedback improves discrimination between figure and background by V1, V2 and V3 neurons. Nature, 394, 784–787. Ipata, A. E., Gee, A. L., & Goldberg, M. E. (2012). Feature attention evokes task-specific pattern selectivity in V4 neurons. Proceedings of the National Academy of Sciences, USA, 10, 16778– 16785. Itti, L., & Koch, C. (2001). Computational modeling of visual attention. Nature Reviews Neuroscience, 2, 194–203. Jensen, O., Idiart, M. A. P., & Lisman, J. E. (1996). Physiologically realistic formation of auto-associative memory in networks with theta/gamma oscillations Role of fast NMDA channels. Learning and Memory, 3, 243–256. Jones, H., Grieve, K., Wang, W., & Sillito, A. (2001). Surround suppression in primate V1. Journal of Neurophysiology, 86, 2011–2028. Jones, H., Wang, W., & Sillito, A. (2002). Spatial organisation and magnitude of orientation contrast interactions in primate V1. Journal of Neurophysiology, 88, 2796–2808. Jung, R. (1961). Korrelationen von Neuronentätigkeit und Sehen. In R. Jung & H. Kornhuber (Eds.), Neurophysiologie und psychophysik des visuellen systems (pp. 410–434). Berlin, Germany: Springer. ?29 Jung, R., Baumgarten, R. von, & Baumgartner, G. (1952). Mikroableitungen von einzelnen nervenzellenimoptischen cortex der katze: Die lichtaktiviert- //titan/production/j/jovi/live_jobs/jovi-15-09/jovi-15-09-04/layouts/jovi-15-09-04.3d  19 June 2015  7:16 pm  Allen Press, Inc.  MS#: JOV-04687-2015 Page 18 Journal of Vision (2015) 15(9):0, 1–22 Spillmann, Dresp-Langley, & Tseng 18 en B-neurone. Archiv für Psychiatrie und Nervenheilkunde, 189, 521–539.?30 Jung, R., & Spillmann, L. (1970). Receptive-field estimation and perceptual integration in human vision. In F. A. Young & D. B. Lindsley (Eds.), Early Experience and Visual Information Processing in Perceptual and Reading Disorders. Proceedings of the National Academy of Sciences, USA (pp. 181– 197). Washington, DC.?31 Kanizsa, G. (1955). Margini quasi-percettivi in campi con stimolazione omogenea. Rivista di Psicologia, 49, 7–30.?32 Kapadia, M. K., Ito, M., Gilbert, C. D., & Westheimer, G. (1995). Improvement in visual sensitivity by changes in local context: Parallel studies in human observers and in V1 of alert monkeys. Neuron, 15, 843–856. Kapadia, M. K., Westheimer, G., & Gilbert, C. D. (2000). Spatial contribution of contextual interactions in primary visual cortex and in visual perception. Journal of Neurophysiology, 84, 2048– 2062. Kastner, S., Nothdurft, H. C., & Pigarev, I. N. (1999). Neuronal responses to orientation and motion contrast in cat striate cortex. Visual Neuroscience, 16, 587–600. Kawabata, N. (1984). Perception at the blind spot and similarity grouping. Perception & Psychophysics, 36, 151–158. Knierim, J. J., & Van Essen, D. C. (1992). Neuronal responses to static texture patterns in area V1 of the alert macaque monkey. Journal of Neurophysiology, 67, 961–980. Köhler, W., & Held, R. (1949). The cortical correlate of pattern vision. Science, 110, 414–419. Komatsu, H. (2011). Bridging gaps at V1: Neural responses for filling-in and completion at the blind spot. Chinese Journal of Psychology, 53, 413–420. Krause, M. R., & Pack, C. W. (2014). Contextual modulation and stimulus selectivity in extrastriate cortex. Vision Research, 104, 36–46. Kuffler, S. W. (1953). Discharge patterns and functional organization of mammalian retina. Journal of Neurophysiology, 16, 37–68. Lamme, V. A. (1995). The neurophysiology of figureground segregation in primary visual cortex. The Journal of Neuroscience, 15, 1605–1615. Lamme, V. A. F., Super, H., & Spekreijse, H. (1998). Feedforward, horizontal, and feedback processing in the visual cortex. Current Opinion in Neurobiology, 8, 529–535. Lee, S., Papanikolaou, A., Logothetis, N. K., Smirnakis, S. M., & Keliris, G. A. (2013). A new method for estimating population receptive field topography in visual cortex. Neuroimage, 81, 144–157. ?33 Lettvin, J. Y., Maturana, H. R., McCulloch, W. S., & Pitts, W. H. (1959). What the frog's eye tells the frog's brain. Proceedings of the IRE, November, 1940–1951. ?34 Li, H. H., & Chen, C. C. (2011). Surround modulation of global form perception. Journal of Vision, 11(1): 17, 1–9, doi:10.1167/11.1.17. Li, W., Piëch, V., & Gilbert, C. D. (2004). Perceptual learning and top-down influences in primary visual cortex. Nature Neuroscience, 7, 650–657. Li, W., Piëch, V., & Gilbert, C. D. (2008). Learning to link visual contours. Neuron, 57, 442–451. Li, Z. (1999). Contextual influences in V1 as a basis for pop out and asymmetry in visual search. Proceedings of the National Academy of Sciences, USA, 96, 10530–10535. Li, Z. (2002). A salience map in primary visual cortex. Trends in Cognitive Sciences, 6, 9–16. Löffler, G. (2008). Perception of contours and shapes: Low and intermediate stage mechanisms. Vision Research, 48, 2106–2172. MacKay, D. (1956). Towards an information-flow model of human behaviour. British Journal of Psychology, 47, 30–43. Maffei, L., & Fiorentini, A. (1976). The unresponsive regions of visual cortical receptive fields. Vision Research, 16, 1131–1139. Malania, M., Devinck, F., Knoblauch, K., Delahunt, P. B., Hardy, J. L., & Werner, J. S. (2011). Senescent changes in photopic spatial summation. Journal of Vision, 11(10):15, 1–15, doi:10.1167/11. 10.15. Mandon, S., & Kreiter, A. K. (2005). Rapid contour integration in macaque monkeys. Vision Research, 45, 291–300. Martin, A. B., & von der Heydt, R. (2015). Spike synchrony reveals emergence of proto-objects in visual cortex. The Journal of Neuroscience, 35, 6860–6870. McIlwain, J. T. (1964). Receptive fields of optic tract axons and lateral geniculate cells: Peripheral extent and barbiturate sensitivity. Journal of Neurophysiology, 27, 1154–1173. Mechler, F., & Ringach, D. L. (2002). On the classification of simple and complex cells. Vision Research, 42, 1017–1033. Mihalas, S., Dong, Y., von der Heydt, R., & Niebur, E. (2011). Mechanisms of perceptual organization provide auto-zoom and auto-localization for at- //titan/production/j/jovi/live_jobs/jovi-15-09/jovi-15-09-04/layouts/jovi-15-09-04.3d  19 June 2015  7:16 pm  Allen Press, Inc.  MS#: JOV-04687-2015 Page 19 Journal of Vision (2015) 15(9):0, 1–22 Spillmann, Dresp-Langley, & Tseng 19 tention to objects. Proceedings of the National Academy of Sciences. USA, 108, 7583–7588. Muckli, L., & Petro, L. S. (2013). Network interactions: Non-geniculate input to V1. Current Opinion in Neurobiology, 23, 195–201. Muckli, L., Vetter, P., & Smith, F. (2011). Predictive coding Contextual processing in primary visual cortex V1. Journal of Vision, 11(11):25, doi:10.1167/ 11.11.25.?35 Nelson, J. I., & Frost, B. J. (1978). Orientation-selective inhibition from beyond the classic visual receptive field. Brain Research, 139, 359–365. Nelson, J. I., & Frost, B. J. (1985). Intracortical facilitation among co-oriented, co-axially aligned simple cells in cat striate cortex. Experimental Brain Research, 61, 54–61. Neri, P. (2011). Global properties of natural scenes shape local properties of human edge detectors. Frontiers in Psychology, Perception Science, 2, 172. Oehler, R. (1985). Spatial interactions in the rhesus monkey retina: A behavioural study using the Westheimer paradigm. Experimental Brain Research, 59, 217–225. Olshausen, B. A., & Field, D. J. (1996). Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature, 380, 607– 609. Oram, M. W., & Perrett, D. I. (1994). Responses of anteriorsuperior temporal polysensory (STPa) neurons to 'biological motion' stimuli. Journal of Cognitive Neuroscience, 6, 99–116. Pan, Y., Chen, M., Yin, J., An, X., Zhang, X., Lu, Y., . . . Wang, W. (2012). Equivalent representation of real and illusory contours in macaque V4. Journal of Neuroscience, 32, 6760–6770. Paradiso, M. A., & Hahn, S. (1996). Filling-in percepts produced by luminance modulation. Vision Research, 36, 2657–2663. Payne, B. R., Lomber, S. G., Villa, A. E., & Bullier, J. (1996). Reversible deactivation of cerebral network components. Trends in Neuroscience, 19, 535–542. Perrett, D. I., Rolls, E. T., & Caan, W. (1982). Visual neurons responsive to faces in the monkey temporal cortex. Experimental Brain Research, 47, 329–342. Peterhans, E., & von der Heydt, R. (1989). Mechanisms of contour perception in monkey visual cortex. II. Contours bridging gaps. Journal of Neuroscience, 9, 1749–1763. Peterhans, E., & von der Heydt, R. (1991). Subjective contours-Bridging the gap between psychophysics and physiology. Trends in Neurosciences, 14, 112– 119. Peterhans, E., von der Heydt, R., & Baumgartner, G. (1986). Neuronal responses of illusory contour stimuli reveal stages of visual cortical processing. In J. D. Pettigrew, K. J. Sanderson, & W. R. Levick (Eds.), Visual neuroscience (pp. 343–351). Cambridge, UK: Cambridge University Press. ?36 Peters, A., Payne, B. R., & Budd, J. A. (1994). A numerical analysis of the geniculocortical input to striate cortex in the monkey. Cerebral Cortex, 3, 69–78. Pettet, M. W., & Gilbert, C. D. (1992). Dynamic changes in receptive-field size in cat primary visual cortex. Proceedings of the National Academy of Sciences, USA, 89, 8366–8370. Polat, U., & Norcia, A. M. (1996). Neurophysiological evidence for contrast dependent long-range facilitation and suppression in human visual cortex. Vision Research, 36, 2099–2109. Polat, U., & Sagi, D. (1993). Lateral interactions between spatial channels: Suppression and facilitation revealed by lateral masking experiments. Vision Research, 33, 993–999. Polat, U., & Sagi, D. (1994). The architecture of perceptual spatial interaction. Vision Research, 34, 73–78. Poort, J., Raudies, F., Wannig, A., Lamme, V., Neumann, H., & Roelfsema, P. (2012). The role of attention in figure-ground segregation in areas V1 and V4 of the visual cortex. Neuron, 75, 143–156. Qiu, F. T., & von der Heydt, R. (2005). Figure and ground in the visual cortex: V2 combines stereoscopic cues with Gestalt rules. Neuron, 47, 155–166. ?37 Qiu, F. T., Sugihara, T., & von der Heydt, R. (2007). Figure-ground mechanisms provide structure for selective attention. Nature Neuroscience, 10, 1492– 1499. Quraishi, S., Heider, B., & Siegel, R. M. (2007). Attentional modulation of receptive field structure in area 7a of the behaving monkey. Cerebral Cortex, 17, 1841–1857. Raizada, R. D. S., & Grossberg, S. (2001). Contextsensitive binding by the laminar circuits of V1 and V2: A unified model of perceptual grouping, attention, and orientation contrast. Visual Cognition, 8, 431–466. Ramón y Cajal, S. (1899). Comparative study of the sensory areas of the human cortex. Harvard University Press. ?38 Ransom-Hogg, A., & Spillmann, L. (1980). Perceptive field size in fovea and periphery of the lightand dark-adapted retina. Vision Research, 20, 221–228. Rao, R. P. N., & Ballard, D. H. (1999). Predictive //titan/production/j/jovi/live_jobs/jovi-15-09/jovi-15-09-04/layouts/jovi-15-09-04.3d  19 June 2015  7:16 pm  Allen Press, Inc.  MS#: JOV-04687-2015 Page 20 Journal of Vision (2015) 15(9):0, 1–22 Spillmann, Dresp-Langley, & Tseng 20 coding in the visual cortex: A functional interpretation of some extra-classical receptive fields. Nature Neuroscience, 2, 79–87. Redies, C., Crook, J. M., & Creutzfeldt, O. D. (1986). Neuronal responses to borders with and without luminance gradients in cat visual cortex and dorsal geniculate nucleus. Experimental Brain Research, 61, 469–481. Ringach, D. L., Hawken, M. J., & Shapley, R. (2002). Receptive field structure of neurons in monkey primary visual cortex revealed by stimulation with natural image sequences. Journal of Vision, 2(1):2, 12–24, doi:10.1167/2.1.2. Rodieck, R. W. (1965). Quantitative analysis of cat retinal ganglion cell response to visual stimuli. Vision Research, 5, 583–601. Rodiek, R. W., & Stone, J. (1965). Analysis of receptive fields of cat retinal ganglion cells. Journal of Neurophysiology, 28, 833–849.?39 Roelfsema, P. R., Lamme, V. A. F., & Spekreijse, H. (1998). Object-based attention in the primary visual cortex of the macaque monkey. Nature, 395, 376– 381. Rolls, E. T., Aggelopoulos, N. C., & Zheng, F. (2003). The receptive fields of inferior temporal cortex neurons in natural scenes. Journal of Neuroscience, 23, 339–348. Rose, D., & Blakemore, C. (1974). Effects of bicuculline on functions of inhibition in visual cortex. Nature, 249, 375–377. Rossi, A. F., Rittenhouse, C. D., & Paradiso, M. A. (1996). The representation of brightness in primary visual cortex. Science, 273, 1104–1107. Schmid, A. (2008). The processing of feature discontinuities for different cue types in primary visual cortex. Brain Research, 1238, 59–74. Schmid, A., & Victor, J. D. (2014). Possible functions of contextual modulations and receptive field nonlinearities: Pop-out and texture segmentation. Vision Research, 104, 57–67. Schwartz, E. L. (1980). Computational anatomy and functional architecture of striate cortex: A spatial mapping approach to perceptual coding. Vision Research, 20, 645–669. Schwartz, G. W., Okawa, H., Dunn, F. A., Morgan, J. L., Kerschensteiner, D., Wong, R. O., & Rieke, F. (2012). The spatial structure of a nonlinear receptive field. Nature Neuroscience, 15, 1572–1580. Seitz, A. R., & Dinse, H. R. (2007). A common framework for perceptual learning. Current Opinion in Neurobiology, 17, 1–6. Series, P., Lorenceau, J., & Frégnac, Y. (2003). The ''silent'' surround of V1 receptive fields: Theory and experiments. Journal of Physiology-Paris, 97, 453– 474. Sillito, A. M., Grieve, K. L., Jones, H. E., Cudeiro, J., & Davis, J. (1995). Visual cortical mechanisms detecting focal orientation discontinuities. Nature, 378, 492–496. Singer, W. (1989). Search for coherence: A basic principle for cortical self-organization. Concepts in Neuroscience, 1, 1–26. Smith, A. T., Singh, K. D., Williams, A. L., & Greenlee, M. W. (2001). Estimating receptive field size from fMRI data in human striate and extrastriate visual cortex. Cerebral Cortex, 11, 1182–1190. Spillmann, L. (1997). Colour in a larger perspective: The rebirth of Gestalt psychology. Perception, 26, 1341–1352. Spillmann, L. (1999). From elements to perception. Local and global processing in visual neurons. Perception, 28, 1461–1492. Spillmann, L. (2009). Phenomenology and neurophysiological correlations: Two approaches to perception research. Vision Research, 49, 1507–1521. Spillmann, L. (2011). Fading, filling-in and the perception of extended surfaces. Chinese Journal of Psychology, 53, 399–411. Spillmann, L. (2014). Receptive fields of visual neurons: The early years. Perception, 43, 1145– 1176. Spillmann, L., & De Weerd, P. (2003). Mechanisms of surface completion: Perceptual filling-in of texture. In L. Pessoa & P. De Weerd (Eds.), Filling-in: From perceptual completion to cortical reorganization (pp. 81–105). Oxford, UK: Oxford University Press. Spillmann, L., & Dresp, B. (1995). Phenomena of illusory form: Can we bridge the gap between levels of explanation? Perception, 24, 1333–1364. Spillmann, L., & Werner, J. S. (1996). Long-range interactions in visual perception. Trends in Neurosciences, 19, 428–434. Tolias, A. S., Moore, T., Smirnakis, S. M., Tehovnik, E. J., Siapas, A. G., & Schiller, P. H. (2001). Eye movements modulate visual receptive fields of V4 neurons. Neuron, 29, 757–767. Treue, S., & Martinez-Trujillo, J. C. (2012). The spotlight of attention: Shifting, resizing and splitting receptive fields when processing visual motion. E-Neuroforum, 3, 74–79. Tzvetanov, T., & Dresp, B. (2002). Shortand longrange effects in line contrast detection. Vision Research, 42, 2493–2498. //titan/production/j/jovi/live_jobs/jovi-15-09/jovi-15-09-04/layouts/jovi-15-09-04.3d  19 June 2015  7:16 pm  Allen Press, Inc.  MS#: JOV-04687-2015 Page 21 Journal of Vision (2015) 15(9):0, 1–22 Spillmann, Dresp-Langley, & Tseng 21 Vinje, W. E., & Gallant, J. L. (2002). Natural stimulation of the nonclassical receptive field increases information transmission efficiency in V1. The Journal of Neuroscience, 22, 2904–2915. von der Heydt, R., & Peterhans, E. (1989). Mechanisms of contour perception in monkey visual cortex: I. Lines of pattern discontinuity. Journal of Neuroscience, 9, 1731–1748. von der Heydt, R., Peterhans, E., & Baumgartner, G. (1984). Illusory contours and cortical neuron responses. Science, 224, 1260–1262. Wandell, B. A., & Winawer, J. (2011). Imaging retinotopic maps in the human brain. Vision Research, 51, 718–737. Wang, R., Cong, L.-J., & Yu, C. (2013). The classical TDT Perceptual learning is mostly temporal learning. Journal of Vision, 13(5):9, 1–9, doi:10. 1167/13.5.9. Wehrhahn, C., & Dresp, B. (1998). Detection facilitation by collinear stimuli in humans: Dependence on strength and sign of contrast. Vision Research, 38, 423–428. Wiesel, T. N. (1982). Postnatal development of the visual cortex and the influence of environment. Nature, 299, 583–591.?40 Wiesel, T. N., & Hubel, D. H. (1963). Single-cell responses in striate cortex of kittens deprived of vision in one eye. Journal of Neurophysiology, 26, 1003–1017. Wiesel, T. N., & Hubel, D. H. (1965). Binocular interaction in striate cortex of kittens reared with artificial squint. Journal of Neurophysiology, 28, 1041–1059. Wiesel, T. N., & Hubel, D. H. (1966). Spatial and chromatic interactions in the lateral geniculate body of the rhesus monkey. Journal of Neurophysiology, 29, 1115–1156. Womelsdorf, T., Anton-Erxleben, K., Pieper, F., & Treue, S. (2006). Dynamic shifts of visual receptive fields in cortical area MT by spatial attention. Nature Neuroscience, 9, 1156–1159. Womelsdorf, T., Anton-Erxleben, K., & Treue, S. (2008). Receptive field shift and shrinkage in macaque middle temporal area through attentional gain modulation. The Journal of Neuroscience, 28, 8934–8944. Wurtz, R. H. (2009). Recounting the impact of Hubel and Wiesel. The Journal of Physiology, 587, 2817– 2823. Yantis, S., & Jonides, J. (1990). Abrupt visual onsets and selective attention: Voluntary versus automatic allocation. Journal of Experimental Psychology: Human Perception and Performance, 16, 121–134. Yeh, C. I., Xing, D., Williams, P. E., & Shapley, R. M. (2009). Stimulus ensemble and cortical layer determine V1 spatial receptive fields. Proceedings of the National Academy of Sciences, USA, 106, 14652–14657. Yu, C., & Levi, D. M. (1997). Spatial facilitation predicted with end-stopped spatial filters. Vision Research, 37, 3117–3128. Yu, C., & Levi, D. M. (2000). Surround modulation in human vision unmasked by masking experiments. Nature Neuroscience, 3, 724–728. Zhang, N. R., & von der Heydt, R. (2010). Analysis of the context integration mechanisms underlying figure-ground organization in the visual cortex. Journal of Neuroscience, 30, 6482–6496. Zhaoping, L. (2008). Attention capture by eye of origin singletons even without awareness-A hallmark of a bottom-up saliency map in the primary visual cortex. Journal of Vision, 8(5):1, 1–18, doi:10.1167/ 8.5.1. Zhaoping, L. (2014). Understanding vision: Theory, models, and data. Oxford University Press. ?41 Zhou, H., Friedman, H. S., & von der Heydt, R. (2000). Coding of border ownership in monkey visual cortex. Journal of Neuroscience, 20, 6594– 6611. Zipser, K., Lamme, V. A., & Schiller, P. H. (1996). Contextual modulation in primary visual cortex. Journal of Neuroscience, 16, 7376–7389. Zur, D., & Ullman, S. (2003). Filling-in of retinal scotomas. Vision Research, 43, 971–982. //titan/production/j/jovi/live_jobs/jovi-15-09/jovi-15-09-04/layouts/jovi-15-09-04.3d  19 June 2015  7:16 pm  Allen Press, Inc.  MS#: JOV-04687-2015 Page 22 Journal of Vision (2015) 15(9):0, 1–22 Spillmann, Dresp-Langley, & Tseng 22 Queries for jovi-15-09-04 1. Author: This article has been lightly edited for grammar, style, and usage. Please compare against your original document and make changes on these pages. Please limit your corrections to substantive changes that affect meaning. If no change is required in response to a question, please write ''OK as set'' in the margin. Copy editor 2. Author: Please include email addresses and websites for all authors if available. Copy editor 3. Author: In the paragraph beginning ''In a previous paper (Spillmann, 2014), the early history. . .'' please confirm that the Hubel & Wiesel reference is to the 1962, 1965, and 1968 publications. Copy editor 4. Author: In the paragraph beginning ''Long-range interaction between RFs serves not only unperturbed. . .'' there is a spelling discrepancy between the text and the reference list: DeWeerd/De Weerd. Please confirm the spelling and ensure it is correct throughout. Copy editor 5. Author: In the paragraph beginning, ''Long-range interaction between RFs serves not only unperturbed. . .'' the Komatsu, 2006, reference is not included on the reference list. Please add it. Copy editor 6. Author: In the paragraph beginning, ''Following these early discoveries, researchers started using. . .'' there is a capitalization discrepancy between the text and the reference list: van Essen/Van Essen. Please confirm the name and ensure it is correct throughout.'' Copy editor 7. Author: In the paragraph beginning, ''A last experiment to be mentioned here involves a phenomenon. . .'' the Rubin, 1915/1921, reference is not included on the reference list. Please add it. Copy editor 8. Author: In the paragraph beginning, ''Border ownership selectivity and side preference are intrinsic properties. . .'' there is a spelling discrepancy between the text and the reference list: Schütze/Schuetze. Please confirm the spelling and ensure it is correct throughout.'' Copy editor 9. Author: In the paragraph beginning, ''Border ownership selectivity and side preference are intrinsic properties. . .'' the Mihalas, von der Heydt, & Niebur, 2011, reference is not included on the reference list. Could it be Mihalas, Dong, von der Heydt, & Niebur, 2011? If not, please add the correct reference to the list. Copy editor 10. Author: In the paragraph beginning, ''Contextual influences in vision and visual perception. . .'' the Li & Chen, 2001, reference is not included on the reference list. Could it be H. H. Li & Chen, 2011? If not, please add the correct reference to the list. Copy editor 11. Author: In the paragraph beginning, ''Ever since Hartline's (1938, p. 410) first description. . .'' the Lund et al., 1993, reference is not included on the reference list. Please add it. Copy editor 12. Author: In the paragraph beginning, ''The remapping of RFs from positions inside the lesion area. . .'' the Huang & Paradiso, 2008, reference is not included on the reference list. Please add it. Copy editor 13. Author: In the paragraph beginning, ''The remapping of RFs from positions inside the lesion area. . .'' the Mach, 1865, and Hering, 1878, references are not included on the reference list. Please add them. Copy editor 14. Author: In the paragraph beginning, ''Cortical models (Grossberg, 1994, 1997; Cao & Grossberg, 2005. . .'' there is a spelling (accent) discrepancy between the text and the reference list: Ramòn y Cajal/Ramón y Cajal. Please confirm the spelling and ensure it is correct throughout.'' Copy editor 15. Author: In the paragraph beginning, ''Although visual RFs are typically considered bottom-up detectors. . .'' the Treue, 2012, reference is not included on the reference list. Could it be Treue & Martinez-Trujillo, 2012? If not, please add the correct reference to the list. Copy editor 16. Author: In the paragraph beginning, ''Although visual RFs are typically considered bottom-up detectors. . .'' the Conci et al., 2001, reference is not included on the reference list. Could it be Conci, Tollner, Leszczynski, & Muller, 2011? If not, please add the correct reference to the list. Copy editor 17. Author: In the paragraph beginning, ''Despite these advances into uncharted territory. . .'' the Wertheimer, 1912, reference is not included on the reference list. Please add it. Copy editor 18. Author: In the paragraph beginning ''In a nutshell, although RFs were formerly believed to have. . .'' please include a date for the Hartline reference. Copy editor 19. Author: In the paragraph beginning, ''In a nutshell, although RFs were formerly believed to have. . .'' there is a date discrepancy between the text and the reference list: Vinje & Gallant, 2003/2002. Please confirm the date and ensure it is correct throughout.'' Copy editor 20. Author: In the paragraph beginning, ''It thus appears that the RFs of extrastriate neurons behave. . .'' the Goldberg & Wurtz, 1972, reference is not included on the reference list. Please add it. Copy editor 21. Author: In the paragraph beginning ''Since Hartline's (1938, 1940) original studies in the frog. . .'' please spell out the acronym ''DOG.'' Copy editor 22. Author: In the paragraph beginning ''Our understanding of visual perception has gained immeasurably. . .'' should ''Population perceptive fields (pPFs)'' be ''Population receptive fields (pRFs)''? Copy editor 23. Author: Please confirm that the ''Acknowledgments'' section is correct and complete. Make sure that ALL funding/financial support is listed in the Acknowledgments section. Copy editor 24. Author: For the Allman et al. references, please ensure the correct spelling of ''McGuinness/McGuiness'' on the reference list and throughout. Copy editor 25. Author: The Attneave reference is not cited in the text. Please add it where appropriate or delete it from the list. Copy editor 26. Author: For the Blakemore & Tobin reference, please clarify the page range of the cited article. Copy editor //titan/production/j/jovi/live_jobs/jovi-15-09/jovi-15-09-04/layouts/jovi-15-09-04q.3d Friday, 19 June 2015 7:16 pm Allen Press, Inc. Page 1 27. Author: The Dresp & Bonnet reference is not cited in the text. Please add it where appropriate or delete it from the list. Copy editor 28. Author: The Gattass et al. reference is not cited in the text. Please add it where appropriate or delete it from the list. Copy editor 29. Author: For the Jung reference, please provide an English translation for the chapter and book titles. Copy editor 30. Author: For the Jung et al. reference, please provide an English translation for the article and journal titles. Copy editor 31. Author: For the Jung & Spillmann reference, please include the name of the publisher. Copy editor 32. Author: For the Kanizsa reference, please provide an English translation for the article and journal titles. Copy editor 33. Author: The Lee et al. reference is not cited in the text. Please add it where appropriate or delete it from the list. Copy editor 34. Author: For the Lettvin et al. reference, please include the volume and issue number if appropriate for the cited article. Copy editor 35. Author: For the Muckli et al. reference, please include the page range of the cited article. Copy editor 36. Author: The Peterhans et al. reference is not cited in the text. Please add it where appropriate or delete it from the list. Copy editor 37. Author: The Qui & von der Heydt reference is not cited in the text. Please add it where appropriate or delete it from the list. Copy editor 38. Author: For the Ramón y Cajal reference, please include the city of publication. Copy editor 39. Author: The two Rodieck/Rodiek references are not cited in the text. Please add them where appropriate-and reconcile the spelling discrepancy-or delete them from the list. Copy editor 40. Author: The Wiesel reference is not cited in the text. Please add it where appropriate or delete it from the list. Copy editor 41. Author: For the Zhaoping, 2014, reference, please include the city of publication. Copy editor 42. Author: Please include column heads for the table. Copy editor 43. Author: Please specify the Blakemore references. Copy editor //titan/production/j/jovi/live_jobs/jovi-15-09/jovi-15-09-04/layouts/jovi-15-09-04q.3d Friday, 19 June 2015 7:16 pm Allen Press, Inc. Page