On the relations between action planning, object identification, and motor representations of observed actions and objects
Introduction
Grasp planning can be primed by observing graspable objects (e.g., Tucker & Ellis, 2001) and by observing grasps performed by another individual (e.g., Vainio, Tucker, & Ellis, 2007). The objective of the present study was to investigate the influence of grasp observation on (a) the identification of graspable objects and (b) the activation of the motor program typically elicited by these graspable objects.
Electrophysiological evidence suggests that neurons in monkey premotor cortex (F5) fire during grasping and additionally discharge on the presentation of graspable objects, even when no immediate action upon the object is allowed (Murata et al., 1997). This canonical neuron system (CNS) appears to play an important role in representing action knowledge of graspable objects in monkey (Fagg and Arbib, 1998, Taira et al., 1990). In the CNS, cells in the anterior intraparietal sulcus of parietal cortex (AIP) appear to extract from the visual stream those feature codes of the object that are relevant to grasping (i.e., object affordances), and send these on to area F5. Moreover, it has been shown that viewing and naming pictures of tools selectively activate the left ventral premotor cortex in human (e.g., Chao & Martin, 2000). These have led to the suggestion that the representation of graspable objects is formed by a network that includes motor-based knowledge of object utilization (action knowledge) mediated by the ventral premotor cortex.
Behavioural research (Craighero et al., 1998, Gentilucci, 2002, Goodale et al., 1994, Grezes and Decety, 2002, Jakobson and Goodale, 1991, Jeannerod, 1988, Tucker and Ellis, 1998) has supported the view that when an individual views an object, its action attributes are processed automatically at the motor level. Of relevance to our present study are data reported by Tucker and Ellis, 2001, Ellis and Tucker, 2000, which reflect how grasp selection processes are affected by the size of an observed object. Tucker and Ellis demonstrated that grasp selection is affected by the size of the target object even when the experimental task does not require any processing of size-related information. For convenience, we shall refer to this finding as the “object-size effect”. Specifically, participants were instructed to make either a precision or power grasp response with their dominant hand, depending on the category (natural or man-made) of the target object. Precision grasp responses were faster and more accurate (relative to power grasp responses) when the target object was optimally graspable using a precision grasp, and vice versa for power grasp responses. It is likely that human AIP neurons underlie the grasp potentiation in the object-size effect, since these neurons are activated in the object-size effect task (Grezes, Tucker, Armony, Ellis, & Passingham, 2003). In monkey, activation in AIP neurons that were stated to play a fundamental role in CNS appear to reflect coding of object affordances at the motor level. Thus, it is likely that in the object-size effect an object’s grasp-relevant attribute (size) is automatically processed when the object is attended, which in turn activates a grasp program that is compatible with the size.
Helbig, Graf, and Kiefer (2006) have recently demonstrated that when participants are asked to name an object that is preceded by a briefly displayed masked object, participants show superior object naming if the preceding masked object shares the same motor affordance as the target object (e.g., frying pan – dust pan) rather than if they do not share the same motor affordance (e.g., frying pan – banjo). This research suggests that action representations, activated by action-relevant information of a prime object, influence the identification of graspable objects, thereby suggesting a role of action representations in object recognition.
Similar priming effects can also be observed when the participant is presented with stimuli that contain action-related rather than object-related information. For instance, viewing static and dynamic presentations of a grasping hand influences the viewer’s own grasp responses (Craighero et al., 2002, Edwards et al., 2003, Stürmer et al., 2000, Vogt et al., 2003). Of particular relevance to the current study, Vainio et al. (2007) observed that selection of a grasp type is affected if the participant has to analyse the hand that is making a similar grasp. The observed hand was making either accurate or inaccurate precision or power grasps and participants signalled the accuracy of the observed grip by making one or other response depending on instructions. Grasp responses were faster when they matched with the observed grip type. The data were interpreted as behavioural evidence of the automatic imitation coding of observed actions.
Similarly, action-based information (as activated by observing a hand that is grasping or using an object), can affect the processing of object-based information. For example, Yoon and Humphreys (2005) observed that when participants were asked to verify whether the displayed objects were typically used in the way described by a written label (displayed below the object), the action verification responses were made faster if the object was shown with a hand adopting a grasp that was congruent rather than incongruent with how the object is commonly used. A similar effect was observed when the observed hand moved these objects in a manner that was congruent (rather than incongruent) with the standard action associated with the object. However, when participants were required to identify the object in order to verify whether it matched its display name, the grasp and movement information did not have a similar effect on reaction times. It was suggested that the identification task did not involve processing action-relevant stimulus information to the same extent that the action verification tasks did. Moreover, Tipper, Paul, and Hayes (2006) observed that left or right hand response superiority elicited by the left or right orientation of a door handle is significantly increased if the action aspects of the object are highlighted by showing participants video clips, prior to the object onset, in which this object is used in its usual context (a hand opening a door by its handle). This suggests that congruent action-based information not only influences the processing of action-relevant object properties, but also influences action planning processes associated with the object’s affordance.
The effects of action observation on action execution have been commonly attributed to processes within the mirror neuron system (MNS). The “mirror” neurons were first discovered in monkey premotor areas (F5). These neurons fire when the animal makes a particular action as well as when it observes similar actions made by others (Rizzolatti, Fadiga, Fogassi, & Gallese, 1996). Traditionally this research has highlighted the importance that movement of the observed action has in activating the MNS (see Rizzolatti & Craighero, 2004, for a review). Furthermore, this research suggests that mirror neurons are more selective for object-directed actions than for observation of a moving hand without any goal object (see Rizzolatti, Fogassi, & Gallese, 2001). For instance, Umiltà et al. (2001) have shown that if actions are object-directed, many mirror neurons are involved in representing them, even when crucial parts of these actions are hidden. When the same dynamic action stimuli were observed without any object this activation in MNS was diminished.
Evidence suggests that a mirror neuron system similar to that discovered in monkeys also exists in humans. Fadiga, Fogassi, Pavesi, and Rizzolatti (1995) used transcranial magnetic stimulation (TMS) to test whether the observation of actions elicits an enhancement of the motor evoked potentials (MEPs) induced by TMS of the motor cortex. They observed that MEPs recorded from the hand muscles increased during grasping observation. The increase of excitability was observed only in those muscles that participants would use when performing the observed movements. This study provided evidence that humans have a system which has properties similar to the mirror neurons found in monkeys. Similar supporting evidence for a human MNS is provided by several neuroimaging studies (e.g., Hari et al., 1998, Newman-Norlund et al., 2007, Nishitani and Hari, 2000). In human, posterior inferior frontal and rostral inferior parietal areas appear to have mirror properties (Iacoboni & Mazziotta, 2007). These studies indicate that when we are looking at another individual performing an action, the same motor circuits that are recruited when we ourselves perform that action are activated. However, these TMS and neuroimaging experiments do not “prove” the existence of mirror neurons in human. More research needs to be carried out to explore similarities and differences between the MNS in monkey and similar action/observation matching system in human.
Of most relevance to the current study, Borghi et al., 2005, Borghi et al., 2007 investigated how action-based information influences the processing of object-based information. In this study, participants classified target objects as natural or man-made. Target objects were preceded by a prime stimulus of a hand configured in a grasp that was either congruent or incongruent with the target object (e.g., the prime depicted a power grasp that was incongruent for grasping the target object – a coin). Borghi et al., 2005, Borghi et al., 2007 observed that the prime hand influenced the identification of congruent objects. Although these results appear to demonstrate that object identification involves the processing of motor aspects of an object, they do not present a straightforward picture of the extent of this involvement, and under what conditions it occurs. For instance, the effect was only associated with identification of precision grasp compatible objects (Borghi et al., 2005). Moreover, the prime hand was only observed to affect the identification of congruent objects following a training phase in which the precision and power grasp compatible objects were associated with the precision and power grasp postures (during this training phase participants were required to reproduce the hand gestures shown as primes). It was suggested that this training might be a necessary pre-condition for the re-activation of gestures, primed by the observed hand, to affect object identification. According to this view, the training phase induced motor resonance behaviour, such that the participant used their own body representation to simulate the other persons’ actions. Alternatively, it was suggested that the effect of motor activation on object identification, as elicited by the observed hand, may require the observation of a dynamic action, and that a static grasp image does not sufficiently activate the gesture representation (and in turn it is not capable of influencing object identification processes).
The current study therefore attempted to further clarify and explore to what extent, and under what conditions, object identification involves the processing of motor aspects of an object. All three experiments demonstrated that even in the absence of any training phase, a hand prime influences the identification of precision and power grasp compatible objects. Moreover, Experiment 3 suggested that the object-based activation of precision and power grasp motor programs can be affected by an action-related prime.
Section snippets
Experiment 1
Experiment 1 used a modified version of the paradigm of Borghi et al. (2005) to explore the influence of action-related motor priming on the identification of a grasp compatible object. Specifically, in Borghi et al.’s (2005) study participants were primed, before target object onset, with an image of a static hand that was configured in a precision grasp, power grasp or an open hand. Participants were required to analyse this hand in order to decide whether the trial was a “go” trial (i.e., a
Experiment 2
As mentioned earlier, mirror neurons in monkey appear to be more selective for object-directed actions than for observation of a moving hand without any goal object (e.g., Rizzolatti et al., 2001, Umiltà et al., 2001). In fact, Arbib (2005) has proposed that a match between motion or preshape information of an observed hand and the affordances of a potential target object may play an important role in action-related activation of MNS. Moreover, in human, a motor action effect that is assumed to
Experiment 3
Experiments 1 and 2 appeared to demonstrate that object identification processes can be influenced by the pre-activation of a motor representation that is primed by observing an action. Experiment 3 investigated whether the motor representation that is primed by the observed hand influences the generation of a motor program associated with an observed object. As mentioned in the Introduction, we know that the selection of precision and power grasp responses is affected by the size of the
General discussion
As discussed in Section 1, Yoon and Humphreys (2005) have demonstrated that (task-irrelevant) action information associated with the grasp and movement type of a hand that is holding an object influences analysing how that object is used. In our study, three experiments demonstrated that action information primed by observing an unfolding grasp influenced the time taken to identify a target object’s category. Those objects that were congruent with the grasp of the prime hand (e.g., power grasp
Acknowledgement
This work was supported by ESRC research grant.
References (41)
- et al.
Are visual stimuli sufficient to evoke motor information? Studies with hand primes
Neuroscience Letters
(2007) - et al.
Representation of manipulable man-made objects in the dorsal stream
NeuroImage
(2000) - et al.
Hand action preparation influences the responses to hand pictures
Neuropsychologia
(2002) - et al.
Motor facilitation following action observation: A behavioural study in prehensile action
Brain and Cognition
(2003) - et al.
Modelling parietal–premotor interactions in primate control of grasping
Neural Networks
(1998) Object motor representation and reaching-grasping control
Neuropsychologia
(2002)- et al.
Separate neural pathways for the visual analysis of object shape in perception and prehension
Current Biology
(1994) - et al.
Does visual perception of object afford action? Evidence from a neuroimaging study
Neuropsychologia
(2002) - et al.
Premotor cortex and the recognition of motor actions
Brain Research: Cognitive Brain Research
(1996) - et al.
I know what you are doing: A neurophysiological study
Neuron
(2001)
Precision and power grip priming by grasp observation
Brain & Cognition
Visuomotor priming by pictures of hands: Perspective matters
Neuropsychologia
From monkey-like action recognition to human language: An evolutionary framework for neurolinguistics
Behavioral and Brain Sciences
Focusing on body sites: The role of spatial attention in action perception
Experimental Brain Research
From the perception of action to the understanding of intention
Nature Reviews Neuroscience
Visual hand primes and manipulable objects
Visuomotor priming
Visual Cognition
Micro-affordance: The potentiation of components of action by seen objects
British Journal of Psychology
Motor facilitation during action observation: A magnetic stimulation study
Journal of Neurophysiology
Cited by (83)
Electrophysiological alterations during action semantic processing in Parkinson's disease
2024, Neurobiology of AgingTouchable pixels: Examining the affordance effect between an on-screen object and a user-elicited gesture on the touchscreen
2023, Computers in Human BehaviorCitation Excerpt :Specifically, this correspondence effect allows subjects faster and more accurate responses in a corresponding condition between an object and an action, than a non-corresponding condition. For instance, several corresponding effects have been reported, such as the alignment between a handle grip orientation and the left or right hand (Azaad & Laham, 2019; Bub & Masson, 2010; Tucker & Ellis, 1998), volumetric compatibility between object size and grip type (Tucker & Ellis, 2001; Vainio, Symes, Ellis, Tucker, & Ottoboni, 2008), and other correspondences between an object's properties and a subject's actions. Thus, the main study investigates a corresponding effect between a visual property of interface objects and a touch gesture on the screen.
Influence of colour on object motor representation
2022, NeuropsychologiaNeutral affordances: Task conflict in the affordances task
2022, Consciousness and CognitionSmash the dichotomy of Skeuomorphism and flat design: Designing an affordable interface to correspond with the human perceptuomotor process
2020, International Journal of Human Computer Studies