A combination of thematic and similarity-based semantic processes confers resistance to deficit following left hemisphere stroke
- 1 Moss Rehabilitation Research Institute, Philadelphia, PA, USA
- 2 Laboratoire Unité de REcherche en sciences Cognitives et Affectives, Université Charles-de-Gaulle Lille 3, Lille, France
Semantic knowledge may be organized in terms of similarity relations based on shared features and/or complementary relations based on co-occurrence in events. Thus, relationships between manipulable objects such as tools may be defined by their functional properties (what the objects are used for) or thematic properties (e.g., what the objects are used with or on). A recent study from our laboratory used eye-tracking to examine incidental activation of semantic relations in a word–picture matching task and found relatively early activation of thematic relations (e.g., broom–dustpan), later activation of general functional relations (e.g., broom–sponge), and an intermediate pattern for specific functional relations (e.g., broom–vacuum cleaner). Combined with other recent studies, these results suggest that there are distinct semantic systems for thematic and similarity-based knowledge and that the “specific function” condition drew on both systems. This predicts that left hemisphere stroke that damages either system (but not both) may spare specific function processing. The present experiment tested these hypotheses using the same experimental paradigm with participants with left hemisphere lesions (N = 17). The results revealed that, compared to neurologically intact controls (N = 12), stroke participants showed later activation of thematic and general function relations, but activation of specific function relations was spared and was significantly earlier for stroke participants than controls. Across the stroke participants, activation of thematic and general function relations was negatively correlated, further suggesting that damage tended to affect either one semantic system or the other. These results support the distinction between similarity-based and complementarity-based semantic relations and suggest that relations that draw on both systems are relatively more robust to damage.
Introduction
Growing evidence indicates that several types of semantic relationships between objects inform conceptual structure. Both similarity relations based on shared features (also referred to as taxonomic relations, e.g., hammer–screwdriver) and complementary relations based on co-occurrence in events or situations (also referred to as thematic relations, e.g., hammer–nail, see Estes et al., 2011) influence conceptual processing. The degree of feature overlap between concepts predicts the magnitude of semantic priming and semantic competition effects (e.g., Cree et al., 1999; Vigliocco et al., 2004; Mirman and Magnuson, 2009). Similarly, thematic relationships affect semantic priming and categorization behaviors (e.g., Moss et al., 1995; Lin and Murphy, 2001; Hare et al., 2009). Furthermore, recent data support the idea that similarity-based and thematic knowledge are subserved by two functionally distinct systems (Kalénine et al., 2009; Crutch and Warrington, 2010; Schwartz et al., 2011; Mirman and Graziano, 2012). In healthy adults, taxonomic and thematic relationship processing efficiency differs as a function of individual preferences (Mirman and Graziano, 2012) and object kinds (Kalénine and Bonthoux, 2008; Kalénine et al., 2009). Dissociations between taxonomic and thematic knowledge have also been reported in brain-damaged patients, suggesting that the two systems rely on distinct neuroanatomical substrates (Schwartz et al., 2011). The thematic knowledge system would selectively involve areas of the posterior temporal and parietal cortices (Kalénine et al., 2009; Schwartz et al., 2011). The similarity-based system would recruit areas of the anterior temporal lobes (Schwartz et al., 2011) and possibly cerebral regions associated with perceptual similarity processing in the visual cortex (Kalénine et al., 2009). Although the neural delimitations of the two systems have not been fully identified yet and may depend on stimulus and task characteristics, previous evidence suggests that semantic processing could draw on two functionally and neuroanatomically separate systems based on feature similarity and thematic relation computation.
However, the contribution of the two systems to processing of different semantic relationships is not always clear a priori. Many relevant semantic relationships may exist for a single object that do not strictly map onto the taxonomic/thematic distinction. This complexity can be easily illustrated with the multiple semantic relationships associated with manipulable object functional use. There are thematic relations that bind objects that are directly used together (e.g., broom is used with dustpan). Their processing has been differentiated from taxonomic (feature-based) processing in explicit categorization tasks (Kalénine and Bonthoux, 2008; Kalénine et al., 2009), although thematically related objects may in certain cases share some functional features (e.g., broom and dustpan are used for cleaning the floor). There are also functional similarity relationships between objects that share functional features (e.g., broom and vacuum cleaner are used for cleaning the floor), which are not necessarily used together directly but might be, occasionally. Moreover, as evidenced by feature generation studies (e.g., Cree and McRae, 2003; McRae, Cree, et al., 2005), a given object may have several functional features of different generality levels (e.g., used for cleaning the floor, used for cleaning the house), which could lead to the computation of several functional similarity relationships at different levels of generality (broom and vacuum cleaner are used for cleaning the floor; and broom, vacuum cleaner, and sponge are used for cleaning the house). In this context, it is difficult to determine whether processing functional similarity and thematic relationships would be systematically dissociable, and what would be the contribution of similarity-based and thematic knowledge system to semantic processing of the different relationships associated with object functional use. One may assume that regardless of the level generality of functional features shared by objects, functional similarity relationships would bear upon feature similarity computation (in the present case, functional feature similarity), and would be therefore equally dissociable from thematic knowledge processing. Recent evidence indicates, however, that processing specific functional similarity relationships is likely to involve both similarity-based and thematic knowledge systems.
Results come from a recent study in healthy adults using the “visual world” paradigm (VWP; Kalénine et al., 2012). In the VWP, a set of pictures with experimentally controlled relationships are presented to a participant, and eye movements are recorded while the participant locates the target given an auditory prompt. A key feature of the VWP is that, prior to target identification, distractor pictures that are related or similar to the target in some way compete for attention and are fixated more compared to unrelated distractor pictures. The relation between target and related distractor can be semantic (Huettig and Altmann, 2005; Yee and Sedivy, 2006; Mirman and Magnuson, 2009), phonological (Allopenna et al., 1998), visual (Dahan and Tanenhaus, 2005), or motor (Myung et al., 2006). For an example in the semantic domain, when participants hear the target word “key” and are presented with a four-picture display including the target object (key), a semantically related distractor (lock), and two unrelated distractors (deer and apple), they look more to the lock than to the unrelated distractors before clicking on the key. This pattern reflects the activation of the information shared by the target and related distractors (keys are used on locks) when identifying the target word (key). VWP has several major advantages. The shape of the competition effect can reveal the precise temporal dynamics of conceptual activation, in addition to the magnitude of conceptual activation (Allopenna et al., 1998; Mirman and Magnuson, 2009). The task is very simple and highly sensitive, so it can reveal subtle differences in conceptual activation in both directions (e.g., greater vs. smaller or earlier vs. later competition effect), without facing ceiling or floor effect limitations and without introducing complex task demands. These characteristics make the paradigm optimal for the assessment of semantic processing differences in various populations, including very young children and cognitively impaired participants (Huang and Snedeker, 2009; Myung et al., 2010; Silverman et al., 2010; Mirman et al., 2011).
In our study comparing thematic and functional similarity processing in healthy adults, a target word (e.g., broom) was presented in three conditions: with a thematically related distractor picture (e.g., dustpan) in the Thematic condition, with a distractor picture that shares the same specific function (vacuum cleaner, cleaning the floor) in the Specific Function condition, and with a distractor picture that shares the same general function (sponge, cleaning the house) in the General Function condition. Results showed a competition effect for each of the three types of related distractors of approximately equal magnitude, indicating that thematic, specific function, and general function relationships were all activated approximately equally when performing a word–picture matching task. However, the time courses of activation differed across the three types of relations. Thematic distractors produced an early transient competition effect, whereas General Function distractors produced a later transient competition effect, suggesting a difference in the time course of activation of thematic and function information. Interestingly, the competition effect in the Specific Function condition exhibited an intermediate pattern: relatively extended competition that started early like the thematic competitors and continued late like the general function competitors. These findings suggest that thematic and general functional relationships rely mostly on somewhat distinct thematic and similarity-based processes, respectively. In contrast, objects sharing a specific function may involve a combination of both processes, causing a mixture of earlier and later activation.
The main goal of the present study was to assess the effect of mild-to-moderate left hemisphere stroke on activation of thematic, specific functional, and general functional relationships. We aimed at identifying patterns of behavioral dissociations in a diverse group of stroke participants that were not selected according to specific lesion location. This approach has been proven to be successful in elucidating different patterns of performance related to differences in neuroanatomic substrate (e.g., Buxbaum et al., 2005; Jax et al., 2006). Its main advantage is avoiding statistically underpowered comparisons of very small groups of participants selected on putative lesion location criteria.
In a sample of diverse individuals with brain-damage, we assumed that, regardless of specific lesion location, a single stroke would be less likely to affect both thematic and similarity-based semantic systems simultaneously than one. Thus, if thematic knowledge or functional similarity computation is sufficient to activate specific function relationships, then competition in the Specific Function condition should be more robust to damage in stroke. This account parallels the assumptions of dual-coding theories (Paivio, 1986). For example, dual-coding theories explain the greater robustness of concrete than abstract concepts as a result of concrete concepts’ capacity to rely on either linguistic or sensory-motor representations. A somewhat less likely alternative is that specific function relations require both thematic knowledge and functional similarity computation, which would predict that competition in the Specific Function condition would be most affected following left hemisphere stroke. A similar hypothesis was formulated to account for evidence that word and face recognition can be impaired separately, but object recognition is impaired when either face or word recognition is impaired (Farah, 1991; but see Buxbaum et al., 1999).
In the VWP described above, we predicted that, compared to a group of neurologically intact participants, left hemisphere stroke participants would show reduced and/or later competition between objects that mostly rely on a single semantic process (Thematic and General Function conditions). A corollary prediction is that there should be a negative correlation between impaired activation of Thematic and General Function relations because individuals will tend to have damage to either one or the other. In contrast, competition should be relatively spared when relationships involve a combination of the two semantic processes (Specific Function condition). A less likely outcome is that Specific Function competition would be most affected by stroke, suggesting that it requires both thematic and functional knowledge to be intact. These predictions were tested in the VWP experiment described below.
Materials and Methods
Participants
Seventeen left hemisphere stroke participants (eight females, nine males) took part in the study. Participants were recruited from the Neuro-Cognitive Rehabilitation Research Registry at the Moss Rehabilitation Research Institute (Schwartz et al., 2005) and were at least 6 months post-stroke. Participants over the age of 80 and/or with histories of co-morbid neurologic disorders, alcohol or drug abuse, or psychosis were excluded. The mean age for this group was 57 (SD = 11 years) and mean years of education was 14 (SD = 3). All participants had cortical lesions and showed some phonological, lexical, and/or semantic difficulties as reflected by their scores on the Philadelphia Naming Test (PNT; Roach et al., 1996), the comprehension subtest of the Western Aphasia Battery (WABcomp: Kertesz, 1982), and the Camel and Cactus Test (CCT; Bozeat et al., 2000). Demographic, lesion, and neuropsychological data are reported in Table 1. For comparison, we report data from 12 neurologically intact control subjects selected from Kalénine et al. (2012) such that the control group was matched on age (M = 63, SD = 5) and education (M = 14, SD = 2) to the group of participants with left hemisphere stroke.
Table 1. Demographic, neuropsychological, and lesion data from the 17 stroke participants.
All participants gave informed consent to participate in the behavioral testing in accordance with the guidelines of the IRB of Albert Einstein Healthcare Network, were paid $15/h for their participation, and reimbursed for travel expenses.
Stimuli
Stimuli were 96 color photographs of objects, including 16 reference object pictures, 48 semantically related pictures (16 Thematic, 16 Specific Function, and 16 General Function), and 32 unrelated pictures. All 96 critical pictures had at least 90% name agreement. An additional set of 139 pictures was also used for practice and filler trials. Eight 4-picture displays were derived for each reference object. Three displays were used for critical trials, one in each semantic relationship condition. Three other displays were used for composed filler trials and two served as unrelated filler trials. A complete list of the critical items is provided in Table A1 in Appendix.
On critical trials, the reference object (e.g., BROOM) was always the target, one object was related to the target (i.e., the competitor) and the last two objects were semantically and phonologically unrelated to both the target and the competitor. The competitor was thematically related to the target in the Thematic displays (e.g., DUSTPAN; used with broom), shared a specific function with the target in the Specific Function displays (e.g., VACUUM CLEANER; clean the floor), or shared a general function in the General Function displays (e.g., SPONGE; clean the house). Composed filler trials were added to allow the related objects to be targets so that participants would not be able to guess which object was the target based on prior exposure. On those trials, the pictures used for critical trials were rearranged and one of the related pictures became the target. Unrelated filler trials involved novel pictures unrelated to each other, one of them being presented twice as the target.
A large norming procedure was conducted on the stimuli. Results are provided in Table 2. Visual and manipulation similarity between the reference objects and their corresponding related and unrelated objects was assessed by asking healthy adults to rate on a 7-point scale to what extent the two object pictures were visually similar and the objects displayed could be manipulated in the same way. Visual similarity ratings were low and equivalent between conditions. Manipulation similarity was slightly higher in the Specific Function relationship condition compared to other conditions. Thus, manipulation similarity ratings were used as a covariate when comparing conditions in the analysis of gaze data.
Table 2. Mean values and standard deviations of normative ratings and COALS measures for the thematic, specific function, and general function related and unrelated object pairs.
The type of semantic relatedness between reference and distractor objects was evaluated with in three rating blocks. In the Thematic block, participants had to judge on a 7-point scale to what extent the object on the left (reference object) could be used to act with or upon the object on the right (competitor or unrelated object). In the Function Similarity blocks, participants had to judge “to what extent the two objects are similar if one wants to (specific of general similarity).” For example, they had to evaluate to what extent the broom and vacuum cleaner are similar if ones wants to clean the floor (specific similarity) and if one wants to clean the house (general similarity). The ratings confirmed that related objects in the Thematic relationship condition were consistently used to act with/upon each other (M = 6.6). In the same way, related objects in the Specific Function and General Function relationship conditions were judged highly similar in the Specific and General Similarity blocks, respectively (M = 6.1 and 5.7). Unrelated objects were not associated with the reference objects in any of the three situations: ratings were very low for the unrelated pairs in the Thematic, Specific Similarity, or General Similarity blocks (M = 1.5, 1.25, and 1.35, respectively). Moreover, the data indicated that objects in the Specific Function relationship condition (e.g., broom and vacuum cleaner) were judged equally similar in the Specific and General Similarity blocks (p = 0.12), while objects in the General Function relationship condition (e.g., broom and sponge) received systematically higher ratings in the General Similarity block compared to the Specific Similarity block (p < 0.001). These data confirmed the hierarchical relation between specific and general functional similarities.
Finally, a corpus-based semantic similarity measure (COALS) was used to assess overall degree of semantic relatedness (Rohde, under review). As clearly demonstrated in the presentation of Rohde et al.’s model, COALS is a measure of semantic similarity based on word co-occurrence computation in large text corpora. The measure reflects the fact that words appearing in similar linguistic contexts convey similar meanings. It accounts for over 70% of the variance in word-pair similarity and synonym judgment tasks – more than HAL, LSA, or WordNet. For this reason, we regard it as a good experiment-external measure of overall semantic similarity. Averaged COALS measures for the word pairs used in this experiment indicate that the related object noun pairs were more semantically similar than unrelated pairs, and the degree of semantic relatedness between the reference object noun and the related object nouns did not significantly differ between conditions. Together with the normative ratings collected, this confirmed that Thematic, Specific Function, and General Function conditions differ in the type of semantic relatedness between targets and competitors, not in the degree or amount of overall semantic relatedness.
Overall, there were 16 × 8 = 128 trials, including 48 critical trials: 16 Thematic displays, 16 Specific Function displays, and 16 General Function displays. Ten practice trials were also designed on the same model.
Apparatus
Gaze position and duration were recorded using an EyeLink 1000 desktop eyetracker at 250 Hz. Stimulus presentation and response recording were conducted by E-Prime software (Psychological Software Tools, Pittsburgh, PA, USA).
Procedure
Participants were seated with their eyes approximately 27′′ from a 17′′ screen with resolution set to 1,024 × 768 pixels. Since left hemisphere stroke participants often cannot use their contralesional paretic hand, all participants used their left hand to respond. Participants clicked on a central fixation cross to begin each trial. Then they saw four images; each image was presented near one of the screen corners. Images had a maximum size of 200 × 200 pixels and were scaled such that at least one dimension was 200 pixels. Therefore, each picture subtended about 3.5° of visual angle. The position of the four pictures was randomized. The display was presented for a 1-s preview to allow for initial fixations that are driven by random factors or visual salience rather than word processing. Two hundred and fifty milliseconds before the offset of the preview, a red circle appeared in the center of the screen in order to drive attention back to the neutral central location. Then participants heard the target word through speakers and had to click on the image that corresponded to the target word (Figure 1). Eye movements were recorded starting from when the display appeared on the screen and ending when the participant clicked on the target picture. The same procedure was followed for the 10 practice trials and the 128 test trials. The test trial order was randomized.
Figure 1. Example of trial used in the eye-tracking experiment. The display presents the target object (e.g., broom), a semantic competitor (e.g., sponge), and two unrelated objects (e.g., phone and ruler). Target words were delivered after a 1000-ms preview of the display.
Data Analysis
Fixation data averaging
Four areas of interest (AOI) associated with the four object pictures were defined in the display. Each AOI corresponded to a 400 × 300 pixel quadrant situated in one of the four corners of the computer screen. Accordingly, fixations that fell into one of these AOI were considered object fixations, while fixations that fell out of any of the AOI were non-object fixations. At any moment on a single trial, a participant can either fixate an object or not; thus, fixation proportion of each AOI can be either 0 or 1 at any point in time. For each trial of each participant, we computed the proportion of time spent fixating each AOI for each 50 ms time bin. Critical trial data were averaged over items and participants in order to obtain a time course estimate of the fixations on the target, related, and unrelated objects. Data from filler trials were not analyzed. The proportion of fixations on the two unrelated objects was averaged.
Growth curve analysis statistical approach
Growth curve analysis (GCA) is a multi-level modeling framework specifically designed to analyze change over time and adapted for analysis of fixation time course (Magnuson et al., 2007; Mirman et al., 2008). GCA allows simultaneous quantification of fine-grained time course differences between groups and/or conditions of interest as well as between individuals within a group or condition. This is particularly relevant for neuropsychological studies that commonly aim at both comparing a small patient sample to a control group and comparing patients with one another (e.g., Mirman et al., 2008, 2011).
Growth curve analysis of gaze data typically captures the data pattern with two model levels. The first submodel, called Level-1, captures the effect of Time on fixation proportions using fourth-order orthogonal polynomials. A fourth-order polynomial is necessary to capture the rise and fall of fixation probabilities over the course of a trial. Specifically, the intercept term reflects average overall fixation proportion, the linear term reflects a monotonic change in fixation proportion (similar to a linear regression of fixation proportion as a function of time), the quadratic term reflects the symmetric rise and fall rate around a central inflection point, and the cubic and quartic terms similarly reflect the steepness of the curve around inflection points. In our paradigm, these higher order terms (i.e., cubic and quartic) appear to distinguish particularly well between early-rising/transient vs. later-rising/longer-lasting fixation time courses (Kalénine et al., 2012).
The second set of submodels, called Level-2, capture the experimental effects of group, condition, etc. on the Level-1 time terms. They describe each level-1 model term as a function of population means, fixed effects, and random effects. Fixed effects correspond to the effects of the experimental manipulations (group and/or conditions). Random effects can express (a) the deviation for one subject (or item) from the grand mean of fixation proportion (quantification of general individual differences), and (b) the deviation of one subject (or item) in a particular condition from the mean of this participant and the mean of this condition (quantification of individual differences for a particular manipulation). Thus, while fixed effects evaluate the effect of the experimental manipulations at the group level, random effects provide a way to quantify individual participant (or item) effect sizes. Individual effect sizes can then be used to assess individual differences.
Using this multi-level modeling approach, we conducted two separate sets of analyses. First, we compared the patterns of competition for the three Display Types (Thematic, Specific Function, and General Function) within the group of left hemisphere stroke participants. If, as suggested by the results from our first study (Kalénine et al., 2012), functional similarity and thematic processes are somewhat distinct and the Specific Function pairs draw on a combination of both, we should find (1) differences in the amount and/or time course of competition between General Function and Thematic displays on the one hand, and Specific Function displays on the other hand and (2) a negative correlation between degree of General Function and Thematic competition across stroke participants.
The second analysis compared the two groups – left hemisphere stroke participants and neurologically intact controls – in each of the Display Types. Extending the logic of hypothesis 1, we may find (3) distinct patterns of competition effect differences between stroke and control participants in Thematic and General Function displays on the one hand, and Specific Function displays on the other hand. These predictions are described in more detail below.
Within-group analysis: Comparison of the time course of thematic, specific function, and general function competition in stroke participants
In the by-subject analysis, fixation probabilities over time were modeled as a function of Object Relatedness (competitor, unrelated), Display type (Thematic, Specific Function, General Function), and the Object Relatedness × Display Type interaction as fixed effects, with Subject and Subject × Object × Display Type as random effects. In the by-item analysis, the Subject factor was replaced by the Item factor. In addition, since manipulation similarity between objects was known to differ between display types, this factor was introduced as a control variable in the Level-2 model before the factors of interest in the item analysis.
Fixed effects were incorporated in the Level-2 submodels incrementally in three (by-subject) or four (by-item) steps. In this way, it was possible to test the improvement of the model fit after adding each factor of interest and, thus, evaluate the overall effects of Object Relatedness, Display Type and the interaction between Object Relatedness and Display Type on the time course of the gaze data, while controlling for differences in manipulation similarity between conditions. Models were fit using Maximum Likelihood Estimation and compared using the −2LL deviance statistic (−2 times the log-likelihood), which is distributed like χ2 with k degrees of freedom corresponding to the k parameters added.
If activation of Specific Function relations can draw either upon the thematic system or the similarity-based system, then there should be more robust competition in this condition than in the Thematic and General Function displays in stroke participants. In contrast, if both systems are required, then Specific Function competition should be the most impaired condition in stroke participants. As illustrated in Kalénine et al. (2012), we anticipated that the earlier-rise vs. later-rise of competition effects should be visible on higher order terms (cubic and/or quartic). Thus, we expected significant competition effect differences between Display Types on these time terms.
Moreover, we used the random effects of this analysis to quantify individual effect sizes in each of the three Display Type conditions. We then examined the correlations of individual participant effects sizes between conditions in order to test the relationships among the competition effect time courses in the Thematic, Specific Function, and General Function conditions (for an example of this approach in the phonological domain, see Mirman et al., 2011). Specifically, the hypothesis that there are distinct thematic and similarity-based processes predicts a negative correlation between individual competition effect sizes in the Thematic and General function displays (because left hemisphere stroke participants will tend to have one kind of damage or the other). In contrast, competition effect sizes in the Specific Function displays should overall not be related to effect sizes in the other conditions.
Between group analysis: Comparison of the time course of competition in each display type between stroke and control participants
In the by-subject analysis, fixation probabilities over time were modeled as a function of Object Relatedness (competitor, unrelated), Group (stroke participants, controls), and Object Relatedness × Group as fixed effects, with Subject and Subject × Object as random effects. In the by-item analysis, the Subject factor was replaced by the Item factor. Fixed effects were incorporated incrementally in three Level-2 submodels. Using the same model comparison approach as in the within-group analysis, we assessed the overall effects of Object Relatedness, Group, and the interaction between Object Relatedness and Group on the time course of fixations, in each condition. We expected an overall effect of group and, more importantly, an interaction between Group and Object Relatedness, which would indicate differences in the competition effect time course between groups. Again, we then tested this interaction on the different time terms.
As described above, the core hypothesis being tested was that Thematic and General Function competition relies mostly on distinct semantic processes, i.e., thematic or feature similarity processing, whereas Specific Function competition draws on both. As a result, we predicted that Thematic and General Function competition would be vulnerable to left hemisphere stroke. Accordingly, we expected stroke participants to show later-rising competition effects compared to controls in the General Function and Thematic displays. In the Specific function displays, if either one process or the other is sufficient to activate the semantic relationship, stroke participants should demonstrate close-to-normal competition effects. Alternatively, if Specific Function relations require both processes, then the Specific Function competition should be later-rising in stroke participants compared to controls. As in the within-group analysis, differences in competition time courses between groups should be particularly obvious on the cubic and/or quartic terms.
Results
All participants, left hemisphere stroke participants and neurologically intact controls, were highly accurate in identifying the target object among distractors in all three conditions, performing on average between 95 and 99% correct (no significant difference between groups or conditions, all F < 1). Mean mouse click reaction times from display onset was 3081 ms for the control group and 4536 ms for the stroke participant group [F(1,78) = 3.90, p = 0.052]. There was no effect of Display Type and no interaction between Group and Display Type on mouse click reaction times (F < 1).
Gaze data were collected from the onset of each trial (i.e., the presentation of the four-picture display) to the end of the trial (i.e., the mouse click). No trial had to be excluded because of a lack of gaze data (track loss or off-screen fixations). Each trial received between 2 and 27 fixations in controls (M = 9, SD = 2.6), and between 1 and 55 fixations in stroke participants (M = 11.8, SD = 4.6). Trials where participants made an incorrect response or the reaction time was more three standard deviations from the participant’s condition mean (1.8% of control data; 3.8% from stroke participant data) were excluded from the fixation analysis.
Figure 2 shows the averaged time course of fixations to the target, competitor and unrelated objects from target word onset for the participants with left hemisphere stroke (top) and for the control participants (bottom). The statistical analysis was restricted to the competition effects driven by the linguistic input. Accordingly, we compared fixation proportion between related and unrelated distractors from 500 ms until 2000 ms after word onset. This analysis window was chosen because it starts slightly before target fixation proportions begin to rise above distractor fixations (i.e., when fixations start to be driven by processing of the target word) and ends when the competition has been resolved and target fixation proportions have reached their ceiling1.
Figure 2. Averaged time course of fixations to the target, competitor and unrelated objects from word onset in each display type for stroke (top) and control (bottom) participants.
Within-Group Analysis: Comparison of the Time Course of Thematic, Specific Function, and General Function Competition in Left Hemisphere Stroke Participants
Overall, there was neither an effect of Object Relatedness [by-subject: χ2(5) = 5.89, p = 0.31; by-item: χ2(5) = 4.90, p = 0.42] nor an effect of Display Type [by-subject: χ2(10) = 12.15, p = 0.27; by-item: χ2(5) = 14.43, p = 0.15]. However, there was a reliable Object Relatedness × Display Type interaction [by-subject: χ2(10) = 24.17, p < 0.01; by-item: χ2(10) = 25.98, p < 0.005] indicating differences in the time course of competition across the three types of competitors.
Significance tests on the individual parameter estimates revealed that there was no difference in the overall amount of competition between the display types (intercept term: all p > 0.30). However, the Specific Function display significantly differed from the other two display types on the cubic term (Specific Function–General Function: Estimate = 0.182, SE = 0.065, p < 0.01 by-subject, and Estimate = 0.173, SE = 0.059, p < 0.01 by-item; Specific Function–Thematic: Estimate = 0.173, SE = 0.067, p < 0.05 by-subject, and Estimate = 0.164, SE = 0.062, p < 0.01 by-item), reflecting the earlier-rising and more transient competition effect in this condition compared to the other two (Figure 2, top row).
The correlation analysis between individual competition effects sizes in the three conditions indicated that competition effect time courses in the Thematic and General Function displays were negatively correlated (Figure 3). In particular, stroke participants who showed a greater amount and rise of fixations to the competitor in the Thematic condition also tended to have a reduced amount and rise of fixations to the competitor in the General Function (Intercept: r = −0.50, p < 0.05, Figure 3A; Linear: r = −0.65, p < 0.05, Figure 3B). Individual competition effect sizes in the Specific Function condition were not reliably correlated with effect sizes in either of the other conditions.
Figure 3. Relationship between individual competition effect estimates in the thematic and general function displays on the intercept (A) and linear (B) terms of the model for the group of stroke participants.
Between Group Analysis: Comparison of the Time Course of Competition in Each Display Type between Left Hemisphere Stroke and Control Participants
In the General Function displays (Figure 2, left column), there was an effect of group on the overall time course of fixations, regardless of object relatedness [by-subject: χ2(5) = 16.14, p < 0.01; by-item: χ2(5) = 256.29, p < 0.0001]. The interaction between Group and Object Relatedness failed to reach significance in the by-subject analysis [χ2(5) = 8.76, p = 0.11], but was highly reliable in the by-item analysis [χ2(5) = 41.48, p < 0.0001], suggesting differences in the time course of the competition effect between stroke participants and neurologically intact controls. Significance tests on the parameter estimates showed a later-rising but longer-lasting competition effect for stroke participants compared to controls, as indicated by a reliable difference between groups on the cubic term (Estimate = −0.171, SE = 0.062, p < 0.01 by-subject; Estimate = −0.142, SE = 0.027, p < 0.0001 by-item).
The same pattern was observed in the Thematic displays (Figure 2, right column). There was an overall effect of Group [by-subject: χ2(5) = 22.64, p < 0.001; by-item: χ2(5) = 451.33, p < 0.0001], and an interaction between Group and Object Relatedness, highly significant by-item [by-subject: χ2(5) = 5.50, p = 0.35; by-item: χ2(5) = 60.17, p < 0.0001]. This interaction was clearly visible on the cubic term (Estimate = −0.123, SE = 0.057, p < 0.05 by-subject; Estimate = −0.112, SE = 0.027, p < 0.001 by-item). As in the General Function displays, the Thematic competition effect was later-rising for stroke participants compared to control participants.
In contrast, stroke participants did not show later competition effects than controls in the Specific Function displays (Figure 2, middle column). In this condition, there was a reliable effect of Group [by-subject: χ2(5) = 19.98, p < 0.005; by-item: χ2(5) = 397.64, p < 0.0001], and a significant Group × Object Relatedness interaction by-item [by-subject: χ2(5) = 3.96, p = 0.55; by-item: χ2(5) = 26.33, p < 0.0001]. This interaction tended to be significant on the cubic term (Estimate = 0.110, SE = 0.070, p = 0.11 by-subject; Estimate = 0.123, SE = 0.027, p < 0.0001 by-item]. Critically, the difference between the competition effect time courses in the two groups was in the opposite direction (positive estimate here, negative estimates in the other display types). That is, the activation of specific function relations tended to be earlier-rising and more transient in stroke participants compared to neurologically intact controls.
Discussion
To sum up, results from the present study showed that (1) Left hemisphere stroke participants exhibited later activation of Thematic and General Function relations than Specific Function relations during the identification of a manipulable artifact object among distractors. (2) Across stroke participants, there was a negative relationship between the competition effects sizes in the Thematic and General Function conditions. (3) Stroke participants exhibited later Thematic and General Function competition effects and earlier Specific Function competition effects compared to a group of age- and education-matched neurologically intact control participants.
We propose that the different temporal dynamics between the three types of semantic relationships reflect the relative involvement of distinct thematic and similarity-based processes in semantic processing of manipulable objects. Processing thematically related objects (e.g., broom–dustpan) mostly relies on thematic knowledge about the roles of objects in events (Nelson, 1983, 1985; McRae, Hare, et al., 2005; Bonthoux and Kalénine, 2007), regardless of object property overlap. In contrast, processing objects related by a general function (e.g., broom–sponge) mainly relies on the computation of the features shared by the two objects. Because these two kinds of knowledge/processing are functionally and neuroanatomically distinct, a given stroke is unlikely to disrupt both processes. Thus, participants with weaker thematic knowledge activation tend to show preserved feature similarity processing, and vice-versa (Schwartz et al., 2011; Mirman and Graziano, 2012; under review).
Recent data from stroke participants suggest that anterior temporal lobe structures are particularly important for taxonomic semantic knowledge and temporo-parietal cortex is particularly important for thematic semantic knowledge (Schwartz et al., 2011; Mirman and Graziano, under review), but in the present study we failed to find any systematic association between percentage damage to these locations and competition effect sizes in the different conditions. It is hard to interpret this null result. It is possible that we did not have enough statistical power to detect this association, but one cannot distinguish between lack of an effect and lack of power. Further studies will be needed to investigate the neuroanatomical bases of thematic and feature similarity processing while considering the various similarity-based relationships a single object may have.
The main novel finding of the present study concerns the relative sparing of specific function relations – the condition that we hypothesized to involve both thematic and functional similarity processes. The reasons for this putative combination in the Specific Function condition in both healthy adults and stroke participants are not clear. One possibility is that the thematic system strongly involves action knowledge processing, especially for manipulable objects. In contrast, we may speculate that the similarity-based system at play in computing functional similarities is less likely to recruit action knowledge. This is consistent with the dissociation observed in certain situations between action and function knowledge (e.g., Buxbaum and Saffran, 2002; Boronat et al., 2005; Canessa et al., 2008; Pelgrims et al., 2011). However, when objects are functionally similar at the specific level, action and function may become more interconnected in a computational sense, which would be reflected by activation of both similarity-based and thematic knowledge systems in processing of specific-level concepts. This interpretation requires further investigation.
More importantly for the present issue, results showed that semantic processing of specific functional similarities is more likely to be preserved after stroke. This argues in favor of the assumption that the two semantic processes are somewhat redundant, and that either can be used in this condition, as in classic dual-coding theories of cognitive processes (e.g., Paivio, 1986). In dual-coding theories, knowledge or processes that are supported by a single code (e.g., linguistic or associative representations of abstract concepts) are more vulnerable to damage than those that are supported by two or more codes (e.g., both linguistic and sensory-motor representations of concrete concepts). In these dual-code situations (e.g., recall a concrete word from memory), one code or the other is sufficient to achieve good performance on the cognitive task. Similarly, the findings reported here suggest that in some situations where both thematic and similarity-based processes are involved, only one or the other is sufficient to ensure object semantic processing. Semantic processing of manipulable objects benefits from the involvement of both thematic and feature similarity processing, which leads to close-to-normal performance in the Specific Function condition in the group of stroke participants. Interestingly, competition between objects related by a specific function was even exaggerated in stroke participants compared to controls. It is tempting to speculate that this may be the result of an impairment in a cognitive process that normally manages competition, a frequently observed deficit following stroke (Gotts and Plaut, 2002; Novick et al., 2005; Jefferies et al., 2008). However, such an account cannot explain why earlier competition is only observed in the Specific Function displays and not the others. We also investigated whether the pattern of competition observed in stroke participants was related to other linguistic/semantic neuropsychological measures (i.e., PNT, CCT, and WAB) and did not find any systematic correlations relationships between individual scores on language and semantic tests and competition effect sizes. Reasons for these earlier Specific Function competition effects, then, remain unclear.
The alteration vs. preservation of the efficiency of thematic and functional similarity processing after stroke was evident in the time course of competition effects between semantically related distractors in a word-to-matching task. It was not highlighted in explicit object identification measures (mouse click accuracy or reaction times) or in the magnitude of the competition effects between semantically related objects. The ability to detect such subtle abnormalities was made possible by the use of a simple experimental paradigm that is sensitive to time course and a statistical technique well-suited to quantifying group, condition, and individual participant effects. We believe that such methods are particularly useful for the study of fine-grained differences in semantic processes in both cognitively intact and impaired populations.
In conclusion, we have provided evidence supporting a relative involvement of two distinct mechanisms in the processing of semantic relationships between objects. Comparison of the temporal dynamics of conceptual activation between different semantic relationships and between left hemisphere stroke and neurologically intact participants suggests that conditions that rely on both mechanisms are more resistant to brain-damage. Semantic richness may be considered in many ways: in terms of multiplicity of sensori-motor modalities involved in a concept (e.g., Campanella and Shallice, 2011), number of contexts associated (e.g., Yap et al., 2011), density of semantic neighborhoods (e.g., Mirman and Magnuson, 2008; Mirman, 2011), multiplicity of semantic processes at play (e.g., Crutch and Warrington, 2010), etc. The present findings provide additional support to the critical role of semantic richness as a predictive dimension of semantic processing in brain-damaged populations.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Acknowledgments
This research was supported by NIH R01-NS065049 and James S. McDonnell Foundation #220020190 grant awards to Laurel J. Buxbaum, by NIH grant R01-DC010805 to Daniel Mirman, and by the Moss Rehabilitation Research Institute.
Footnote
- ^For a meaningful comparison with the gaze data from stroke participants, the time window used in the prior analysis of the neurologically intact participant data (Kalénine et al., 2012) has been extended from 1300 to 2000 ms after word onset.
References
Allopenna, P. D., Magnuson, J. S., and Tanenhaus, M. K. (1998). Tracking the time course of spoken word recognition using eye movements: evidence for continuous mapping models. J. Mem. Lang. 38, 419–439.
Bonthoux, F., and Kalénine, S. (2007). Preschoolers’ superordinate taxonomic categorization as a function of individual processing of visual vs. contextual/functional information and object domain. Cogn. Brain Behav. 11, 713–731.
Boronat, C. B., Buxbaum, L. J., Coslett, H. B., Tang, K., Saffran, E. M., Kimberg, D. Y., and Detre, J. A. (2005). Distinctions between manipulation and function knowledge of objects: evidence from functional magnetic resonance imaging. Cogn. Brain Res. 23, 361–373.
Bozeat, S., Lambon Ralph, M. A., Patterson, K. E., Garrard, P., and Hodges, J. R. (2000). Non-verbal semantic impairment in semantic dementia. Neuropsychologia 38, 1207–1215.
Buxbaum, L. J., Glosser, G., and Coslett, H. B. (1999). Impaired face and word recognition without object agnosia. Neuropsychologia 37, 41–50.
Buxbaum, L. J., Johnson-Frey, S. H., and Bartlett-Williams, M. (2005). Deficient internal models for planning hand-object interactions in apraxia. Neuropsychologia 43, 917–929.
Buxbaum, L. J., and Saffran, E. M. (2002). Knowledge of object manipulation and object function: dissociations in apraxic and nonapraxic subjects. Brain Lang. 82, 179–199.
Campanella, F., and Shallice, T. (2011). Manipulability and object recognition: is manipulability a semantic feature? Exp. Brain Res. 208, 369–383.
Canessa, N., Borgo, F., Cappa, S. F., Perani, D., Falini, A., Buccino, G., Tettamanti, M., and Shallice, T. (2008). The different neural correlates of action and functional knowledge in semantic memory: an fMRI study. Cereb. Cortex 18, 750–751.
Cree, G. S., McRae, K., and McNorgan, C. (1999). An attractor model of lexical conceptual processing: simulating semantic priming. Cogn. Sci. 23, 371–414.
Cree, G. S., and McRae, K. (2003). Analyzing the factors underlying the structure and computation of the meaning of chipmunk, cherry, chisel, cheese, and cello (and many other such concrete nouns). J. Exp. Psychol. Gen. 132, 163–201.
Crutch, S. J., and Warrington, E. K. (2010). The differential dependence of abstract and concrete words upon associative and similarity-based information: complementary semantic interference and facilitation effects. Cogn. Neuropsychol. 27, 46–71.
Dahan, D., and Tanenhaus, M. K. (2005). Looking at the rope when looking for the snake: conceptually mediated eye movements during spoken-word recognition. Psychon. Bull. Rev. 12, 453–459.
Estes, Z., Golonka, S., and Jones, L. L. (2011). Thematic thinking: the apprehension and consequences of thematic relations. Psychol. Learn. Motiv. 54, 249–294.
Farah, M. J. (1991). Patterns of co-occurrence among the associative agnosias: implications for visual object representation. Cogn. Neuropsychol. 8, 1–19.
Gotts, S. J., and Plaut, D. C. (2002). The impact of synaptic depression following brain damage: a connectionist account of “access/refractory” and “degraded-store” semantic impairments. Cogn. Affect. Behav. Neurosci. 2, 187–213.
Hare, M., Jones, M., Thomson, C., Kelly, S., and McRae, K. (2009). Activating event knowledge. Cognition 111, 151–167.
Huang, Y. T., and Snedeker, J. (2009). Semantic meaning and pragmatic interpretation in 5-year-olds: evidence from real-time spoken language comprehension. Dev. Psychol. 45, 1723–1739.
Huettig, F., and Altmann, G. T. M. (2005). Word meaning and the control of eye fixation: semantic competitor effects and the visual world paradigm. Cognition 96, 23–32.
Jax, S. A., Buxbaum, L. J., and Moll, A. D. (2006). Deficits in movement planning and intrinsic control in ideomotor apraxia. J. Cogn. Neurosci. 18, 2063–2076.
Jefferies, E., Patterson, K., and Lambon Ralph, M. A. L. (2008). Deficits of knowledge versus executive control in semantic cognition: insights from cued naming. Neuropsychologia 46, 649–658.
Kalénine, S., and Bonthoux, F. (2008). Object manipulability affects children and adults’ conceptual processing. Psychon. Bull. Rev. 15, 667–672.
Kalénine, S., Mirman, D., Middleton, E. L., and Buxbaum, L. J. (2012). Temporal dynamics of activation of thematic and functional knowledge during conceptual processing of manipulable artifacts. J. Exp. Psychol. Learn. Mem. Cogn. [Epub ahead of print].
Kalénine, S., Peyrin, C., Pichat, C., Segebarth, C., Bonthoux, F., and Baciu, M. (2009). The sensory-motor specificity of taxonomic and thematic conceptual relations: a behavioral and fMRI study. Neuroimage 44, 1152–1162.
Lin, E. L., and Murphy, G. L. (2001). Thematic relations in adults’ concepts. J. Exp. Psychol. Gen. 130, 3–28.
Magnuson, J. S., Dixon, J. A., Tanenhaus, M. K., and Aslin, R. N. (2007). The dynamics of lexical competition during spoken word recognition. Cogn. Sci. 31, 133–156.
McRae, K., Cree, G. S., Seidenberg, M. S., and McNorgan, C. (2005). Semantic feature production norms for a large set of living and nonliving things. Behav. Res. Methods 37, 547–559.
McRae, K., Hare, M., Elman, J. L., and Ferretti, T. (2005). A basis for generating expectancies for verbs from nouns. Mem. Cognit. 33, 1174–1184.
Mirman, D. (2011). Effects of near and distant semantic neighbors on word production. Cogn. Affect. Behav. Neurosci. 11, 32–43.
Mirman, D., Dixon, J. A., and Magnuson, J. S. (2008). Statistical and computational models of the visual world paradigm: growth curves and individual differences. J. Mem. Lang. 59, 475–494.
Mirman, D., and Graziano, K. M. (2012). Individual differences in the strength of taxonomic versus thematic relations. J. Exp. Psychol. Gen.
Mirman, D., and Magnuson, J. S. (2008). Attractor dynamics and semantic neighborhood density: processing is slowed by near neighbors and speeded by distant neighbors. J. Exp. Psychol. Learn. Mem. Cogn. 34, 65–79.
Mirman, D., and Magnuson, J. S. (2009). Dynamics of activation of semantically similar concepts during spoken word recognition. Mem. Cogn. 37, 1026–1039.
Mirman, D., Yee, E., Blumstein, S. E., and Magnuson, J. S. (2011). Theories of spoken word recognition deficits in aphasia: evidence from eye-tracking and computational modeling. Brain Lang. 117, 53–68.
Moss, H. E., Ostrin, R. K., Tyler, L. K., and Marslen-Wilson, W. D. (1995). Accessing different types of lexical semantic information: evidence from priming. J. Exp. Psychol. Learn. Mem. Cogn. 21, 863–883.
Myung, J.-Y., Blumstein, S. E., and Sedivy, J. C. (2006). Playing on the typewriter, typing on the piano: manipulation knowledge of objects. Cognition 98, 223–243.
Myung, J.-Y., Blumstein, S. E., Yee, E., Sedivy, J. C., Thompson-Schill, S. L., and Buxbaum, L. J. (2010). Impaired access to manipulation features in apraxia: evidence from eyetracking and semantic judgment tasks. Brain Lang. 112, 101–112.
Nelson, K. (1983). “The derivation of concepts and categories from event representations,” in New Trends in Conceptual Representation: Challenges to Piaget’s Theory?, ed. E. K. Scholnick (Hillsdale, NJ: Erlbaum), 129–149.
Novick, J. M., Trueswell, J. C., and Thompson-Schill, S. L. (2005). Cognitive control and parsing: reexamining the role of Broca’s area in sentence comprehension. Cogn. Affect. Behav. Neurosci. 5, 263–281.
Pelgrims, B., Olivier, E., and Andres, M.. (2011). Dissociation between manipulation and conceptual knowledge of object use in the supramarginalis gyrus. Hum. Brain Mapp. 32, 1802–1810.
Roach, A., Schwartz, M. F., Martin, N., Grewal, R. S., and Brecher, A. R. (1996). The Philadelphia Naming Test: scoring and rationale. Clin. Aphasiol. 24, 121–133.
Schwartz, M. F., Brecher, A. R., Whyte, J., and Klein, M. G. (2005). A patient registry for cognitive rehabilitation research: a strategy for balancing patients’ privacy rights with researchers’ need for access. Arch. Phys. Med. Rehabil. 86, 1807–1814.
Schwartz, M. F., Kimberg, D. Y., Walker, G. M., Brecher, A. R., Faseyitan, O. K., Dell, G. S., Mirman, D., and Coslett, H. B. (2011). Neuroanatomical dissociation for taxonomic and thematic knowledge in the human brain. Proc. Natl. Acad. Sci. U.S.A. 108, 8520–8524.
Silverman, L. B., Bennetto, L., Campana, E., and Tanenhaus, M. K. (2010). Speech-and-gesture integration in high functioning autism. Cognition 115, 380–393.
Vigliocco, G., Vinson, D. P., Lewis, W., and Garrett, M. F. (2004). Representing the meanings of object and action words: the featural and unitary semantic space hypothesis. Cogn. Psychol. 48, 422–488.
Yap, M. J., Tan, S. E., Pexman, P. M., and Hargreaves, I. S. (2011). Is more always better? Effects of semantic richness on lexical decision, speeded pronunciation, and semantic classification. Psychon. Bull. Rev. 18, 742–750.
Yee, E., and Sedivy, J. C. (2006). Eye movements to pictures reveal transient semantic activation during spoken word recognition. J. Exp. Psychol. Learn Mem. Cogn. 32, 1–14.
Appendix
Table A1. List of critical items in the Thematic, Specific Function and General Function conditions, and the corresponding functions evaluated in the norms.
Keywords: semantic processing, thematic knowledge, functional similarity, eye-tracking, stroke
Citation: Kalénine S, Mirman D and Buxbaum LJ (2012) A combination of thematic and similarity-based semantic processes confers resistance to deficit following left hemisphere stroke. Front. Hum. Neurosci. 6:106. doi: 10.3389/fnhum.2012.00106
Received: 27 December 2011; Accepted: 10 April 2012;
Published online: 04 May 2012.
Edited by:
Penny M. Pexman, University of Calgary, CanadaReviewed by:
Ken McRae, University of Western Ontario, CanadaLara Jones, Wayne State University, USA
Copyright: © 2012 Kalénine, Mirman and Buxbaum. This is an open-access article distributed under the terms of the Creative Commons Attribution Non Commercial License, which permits non-commercial use, distribution, and reproduction in other forums, provided the original authors and source are credited.
*Correspondence: Solène Kalénine, Laboratoire Unité de REcherche en sciences Cognitives et Affectives, Université Charles-de-Gaulle Lille 3, Domaine universitaire du Pont de Bois, BP 149, 59653 Villeneuve d’Ascq Cedex, France. e-mail: solene.kaleni@univ-lille3.fr