Abstract
The aim of the research was to find out whether participants completing an SCM questionnaire to assess attitudes towards the Roma would give different answers in response to different sets of instructions. Three sets of instructions were tested using cognitive interviews: answer from your personal viewpoint, from the viewpoint of the majority of Slovaks, from the viewpoint of those close to you. The research sample comprised 24 respondents, of whom 12 were upper secondary school students and 12 working adults. Responses from the personal viewpoint differed markedly from responses from the viewpoint of the majority of Slovaks, but were very similar to responses from the viewpoint of those close to the person. In the research, internal and external motivation to respond with/without prejudice was also investigated. Participants with internalised unbiased beliefs showed a preference for assessing the Roma minority from their own viewpoint, while participants with internalised biased beliefs thought the instructions were unimportant.
Introduction
Use of the Stereotype Content Model (SCM; Fiske, Cuddy, Glick & Xu, 2002; Cuddy, Fiske, & Glick, 2008; Cuddy et al., 2009) obtains in-depth information on the formation of stereotypes in intergroup relations and social cognition. The SCM is based on the intergroup functioning of stereotypes and on four hypotheses (Fiske et al., 2002): (1) the two basic stereotype dimensions are warmth and competence; (2) frequent clusters are a combination of high warmth and low competence (paternalistic stereotype) and low warmth and high competence (envious stereotype); (3) four combinations of warmth and competence (high—low, low—high, high—high, low—low) are linked to distinct emotions (pity, envy, admiration, contempt); (4) status predicts high competence and competition predicts low warmth. The relationship between the degree of warmth and competence on the one hand and the emotions accompanying the different stereotypes on the other can be displayed on two orthogonal scales (Fig. 1).
Degree of warmth and competence and emotions accompanying stereotypes
Therefore people can like out-groups even when they do not respect them, and they can respect them even when they do not like them (Cuddy et al., 2009). The basic principle underpinning the SCM is that most groups do not experience one-dimensional, hostile prejudice (Cuddy, Fiske & Glick, 2008). On the contrary, many stereotypes can lead to positive or ambivalent attitudes (Durante, 2008). However, people need to be able to predict others’ intentions towards them, and they assess these using the dimension of warmth. This contains traits such as morality, trustworthiness, sincerity, kindness and friendliness. They also want to assess others’ abilities to fulfil their intentions, and to do so they use the competence dimension (efficacy, cleverness, skill, creativity and intelligence) (Cuddy, Fiske, & Glick, 2008). If out-groups wish to cooperate and share the goals and values of group membership, then the in-group sees them as mature, trustworthy and friendly. If, however, their aim is to compete, take advantage of or threaten the goals and values of the in-group, then they are considered cold, untrustworthy and potentially dangerous (Fiske, 2012).
The Stereotype Content Model has been repeatedly tested by numerous researchers (Fiske, Cuddy, Glick & Xu, 2002; Cuddy, Fiske, & Glick, 2007; Cuddy et al., 2009; Fiske & North, 2015) using a questionnaire[2] designed around the model. To validate[3] the Slovak version of the SCM questionnaire we used cognitive interviews, where the aim was to achieve a more meaningful interpretation of the questionnaire items (Popper & Petrjánošová, 2016).
We also decided to use cognitive interviews to investigate the instructions on how to complete the questionnaire. The original instructions asked participants to give their opinion on a particular stigmatised minority from the viewpoint of most members of the majority group. In our case they were specifically asked to give their opinions on the Roma minority from the viewpoint of “the majority of Slovaks”. We compared this instruction with two alternatives, one of which asked participants to express their “own opinion” and the other that asked them to express the opinions of those “close to them”. The objective was to ascertain the degree of consistency; that is, if there was any change in the opinions expressed about the Roma minority according to the instruction used. We were particularly interested in whether the implicit assumption that the original instruction reduces social desirability would be substantiated. We also wanted to see which instruction was preferred by people with different sources of motivation (internal vs. external) to (not) respond in a prejudiced manner.
Socially desirable activities include behaviours that exhibit normative conformity (i.e. that adjust in relation to social norms) and that are associated with anticipated gains (e.g. the researcher’s approval). Social desirability is a frequent source of bias, which affects the validity of results obtained using experimental and/or survey methods (e.g. Nederhof, 1985), and this is particularly true of questionnaire items that participants consider sensitive (Krumpal, 2013). In an attempt to eliminate social desirability bias, indirect questions such as “what do others/most people think about the topic” are considered preferable to directly asking participants for their own opinions. It is also assumed respondents find it easier to express their own opinions and attitudes when they are asked indirect and impersonal questions (Kidder & Judd in Jo, Nelson, & Kiecker, 1997). Fisher (1993) has empirically demonstrated that indirect questioning reduces social desirability and that respondents project their own beliefs into the indirect responses. However, he also pointed out that respondents may try too hard to produce accurate predictions as to what the group in question would say and construct their responses on the basis of general knowledge/awareness. Indirect questions may reduce bias but that does not necessarily mean the views are a valid and accurate representation of the respondents’ real opinions. The extent to which people includes themselves (i.e. their own beliefs) in their representations of typical opinion depends on the degree of similarity between the respondent and the typical representative, and the smaller this is, the greater the risk the information obtained will be irrelevant (Fisher & Tellis, 1998).
Social desirability can relate to internal and external motivation to respond without prejudice (Devine & Plant, 1998) and to internal and external motivation to respond with prejudice (Forscher, Cox, Graetz, & Devine, 2015).
Internal motivation to respond without prejudice (IMS) is rooted in internalised and personally important unbiased beliefs and attitudes, while external motivation to respond without prejudice (EMS) occurs out of a fear of being judged by others or out of a desire to avoid negative reactions from others (Devine & Plant, 1998).
Research by Devine and Plant (2002) has shown that those scoring high on IMS and low on EMS are less likely to express racial prejudice. By contrast participants scoring high on both IMS and EMS are more likely to contravene their own personal standards (not responding in a biased manner). Regardless of their EMS scores, participants with low IMS scores do not regulate racial prejudice, which means they respond in a biased manner.
The overall probability of stereotypical or prejudicial responses being produced depends not only on the source of motivation but also on the situational variables. Differences have been recorded in whether participants who score highly on EMS only (and therefore low on IMS) give prejudiced responses based on whether the responses are anonymous (Devine & Plant, 1998). Responding in front of an audience (e.g. the researcher) motivates some participants to strategically alter their responses to prevent their true biased responses being detected. In the absence of such motivation, such as in front of a biased audience, the participants would undoubtedly have responded in a prejudiced manner (Devine & Plant, 1998). “Those without either type of motivation (i.e. low IMS, low EMS) are not particularly concerned with responding without prejudice and, thus, should not regulate bias under any circumstances” (Devine & Plant, 2002, p. 838).
Openly negative attitudes to the outgroup are exhibited most by people with low IMS and who lack the values that would lead them to treat other people equally (Forscher, Cox, Graetz, & Devine, 2015). Moreover some people not only lack the internal motivation to respond without prejudice but equally have the motivation to respond in a biased manner (Forscher, Cox, Graetz, & Devine, 2015). People can therefore be motivated to (not) respond in a prejudiced manner either for internal reasons (i.e. personal beliefs) or for external reasons (e.g. social norms). Depending on the degree of internal and external motivation for (not) responding in a biased manner, one could therefore reasonably expect different responses to different types of instructions (directly or indirectly addressing the participants).
Research aim and research problem
The research aim was to ascertain whether differences emerged in the responses participants gave when completing an SCM questionnaire designed to assess the marginalised Roma group in Slovakia depending on whether they had been asked to give their personal viewpoint, the majority viewpoint or the viewpoint of those close to them regarding other minority groups. Our primary interest was whether these three different types of instruction would affect social desirability when responding to survey questions, and whether they would affect subjective assessments of how agreeable/disagreeable and how difficult/not difficult the activity was. The seven research questions were:
What view do the majority of Slovaks have of the Roma, and how do the respondents interpret “the majority of Slovaks”?
What view do those close to the respondent have of the Roma, and how do the respondents interpret “those close to you”?
What is the degree of consistency between personal opinions and the perceived opinions of the majority of Slovaks?
What is the degree of consistency between personal opinions and the perceived opinions of those close to the respondents?
How agreeable was it for the respondents to assess the marginalised group from their own viewpoint, from the viewpoint of those close to them and from the viewpoint of the majority of Slovaks?
How difficult did they find it to assess the marginalised group in terms of their own opinion, the opinions of those close to them and the opinion of the majority of Slovaks?
Did the respondents answer the SCM survey questions honestly, or did they consider the social desirability of replying in a certain way?
Methods
Research sample
The respondents (N = 24) were recruited using purposive sampling (Palinkas et al., 2015; Ritchie, Lewis & El am, 2003), where the aim is to select a sample that best reflects the research goals. As the SCM questionnaire was primarily designed to test the efficacy of various prejudice-reducing interventions among upper secondary school students, half the sample comprised upper secondary school students (N = 12). The students were aged 18–19, and half were male and half female. The SCM questionnaire can, however, also be used with older populations as well, and so the second half of the sample comprised working adults (N = 12), again with equal male/female representation. The participants’ ages ranged from 27 to 50 (M = 40.5 years). Six of these participants had completed secondary school education and had obtained their leaving certificates, while six had studied at the higher education level. The sample size met the recommended 5–25 respondents for cognitive interviews (Willis, 2005; Sheatsley, 1983).
Cognitive interviews
The cognitive interview method enables the researcher to study the manner in which the target group understands, mentally processes and responds to the material presented, such as a survey (Willis, 2005). It can help researchers identify and analyse the sources of response error, by focusing on the cognitive processes the respondents use when answering survey questions (Haeger, Lambert, Kinzie, & Gieser, 2012). The aim is to discover whether the participants understand the questions in the way the researcher intended and whether there is consistency among the participants on what the question means (Collins, in Haeger et al., 2012). It is also used to study judgement/estimation processes in sensitive areas or regarding social desirability so as to ascertain whether respondents wish to tell the truth or say something that will make them look “better” (Willis, 2005). There are two basic techniques of cognitive interviewing: thinking-aloud and verbal probing (Willis, 1999; Willis & Artino, 2013). We used verbal probing so as to gain control over the discussion and target particular cognitive subprocesses (Willis, 2005). In our research the respondents first completed the SCM questionnaire and then retrospectively answered questions about the various types of questionnaire instruction (giving their personal view, the view of others or the view of those close to them).
The original questionnaire began with the instructions:
In this research we are looking at how people [in Slovakia] perceive different groups of people. We are interested in how you think other people generally view this group. We are not asking how you personally view this group. We are asking how you think the majority of people [in Slovakia] view this group.
According to Fiske, Cuddy, Glick, and Xu (2002) this type of instruction should reduce socially desirable answers whilst capturing perceived cultural stereotypes. However, nowhere is it stated whether this instruction has been tested to see if it in fact reduces social desirability.
We distributed the SCM questionnaire with the original type of instruction to one third of the participants in each sample. A second third had to complete the questionnaire after being instructed to do so “from the viewpoint of those close to you” rather than “from the viewpoint of the majority of people in Slovakia”. The last third were given instructions emphasising that they had to respond “from your own point of view”. We assigned the respondents to groups in turn, that is, the first respondent was instructed to answer from the “majority viewpoint”, the second from the viewpoint of “those close to you” and the third from “your own viewpoint”, and so on until all the respondents had been assigned to one of the three groups.
The SCM questionnaire used in this research contained 28 questions concerning Roma stereotypes, divided into four sections. The first section contained six questions aimed at assessing the competence and warmth dimension among the Roma. The second section contained three questions exploring the status of the Roma and three questions investigating whether the participants viewed the Roma as competitors. The third section had eight questions designed to establish which emotions the Roma elicited among the participants. In this section participants had to say whether they envied the Roma, had contempt for them, felt sorry for them or whether they admired them. The last section also contained eight questions on behaviour towards the Roma population, where participants had to say whether this was helpful or harmful. The responses were recorded on a 5-point Likert scale, where 1 meant “not at all” and 5 meant “very”.
Immediately after they had completed the questionnaire we investigated how the respondents had interpreted the instruction to answer “from the viewpoint of the majority of Slovaks” and from “the viewpoint of close friends and family”, and who they thought about when doing so (e.g. those around them, classmates). We then used open questions to explore how the participants had arrived at their survey responses, whether, if the other two sets of instructions had been used, their own answers would have been the same as the representative viewpoints of those close to them and of the majority of Slovaks. We also asked how they felt/ would have felt (whether the process was/would have been agreeable) and whether it was/ would have been difficult. We looked at whether social desirability changed depending on whether they were giving their own beliefs, the beliefs of those close to them or the beliefs of the majority of Slovaks regarding the minority group. We investigated social desirability by asking questions that asked whether they had given an honest response or whether they had thought about it and replied in a socially appropriate/politically correct manner and to think about whether they had responded as they thought the researcher expected them to. We also explored whether their responses to the survey questions would have differed if it had been anonymous, that is, the respondents were asked hypothetically to state whether they would have completed the questionnaire in the same way if, at the beginning, they had been told their responses would be made public. Asking participants about how honest they were could seem problematic given that those who responded in a socially desirable manner will not necessarily admit that to the researcher. On the other hand, it may be that if the respondents are asked to participate in the research for the express purpose of improving the measuring tool, they will be so engaged in the process that they will reply honestly. In any case, as stated above, the cognitive interview is recommended for use in situations where the researchers wish to investigate participant reactions to sensitive or socially desirable survey questions (Willis, 2005). Once the cognitive interviews had been held, all that remained was for the participants to assess their internal and external motivation to respond with/without prejudice.
Internal and external motivation was assessed on the basis of the following four statements, taken from the construct (tools) devised by Devine and Plant (1998) and Forscher, Cox, Graetz, & Devine (2015):
My personal values lead me to believe that using stereotypes about the Roma is bad.
I try to hide negative thoughts about the Roma so as to avoid disapproving reactions from other people.
I express negative opinions on the Roma because other people expect me to.
My personal belief is that I should express negative feelings about the Roma.
Data analysis
Once the interviews had been transcribed we subjected the data to a content analysis (Hsieh & Shannon, 2005; Zhang & Wildemuth, 2009; Krippendorff, 2004), and then coded the results into categories. We also employed a quantitative approach (where certain tendencies had been identified in the participants’ responses) which enabled us to report coding frequencies (Willis, 2015). We analysed the degree of consistency between (1) the participant’s own opinions and the perceived opinions of the majority, (2) the participant’s own opinions and the perceived opinions of those close to them. We observed whether there was a high, moderate or low degree of consistency in the opinions. A high degree of consistency indicated great similarity or only minimal differences in the opinions expressed (e.g. the responses would have been the same). If there were great differences or few similarities between the opinions we categorised the responses as moderately consistent (e.g. in some questions they would have been consistent but not in others). A low degree of consistency indicated large differences or very few similarities in the opinions given (e.g. the opinions would have been extremely different). We also decided to use this coding method and coding frequencies when investigating how agreeable and how difficult it had been for the participants to give their own opinion, the opinion of the majority of Slovaks and the opinion of those close to them. Degree of agreeability was assessed on the basis of responses such as “worse”, “uncomfortable, “disagreeable”, “weirder”, “embarrassing”, “I didn’t want to give the viewpoint of”, versus “better”, “more agreeable”, “more comfortable”, “more normal”, “I would rather have given the viewpoint of”. The degree of difficulty was assessed on the basis of responses such as: “harder”, “I had to think more”, “more challenging” versus “easier”, “better”, “less challenging”, “I didn’t have to think so much”. Finally we applied this method for analysing high, moderate, low degrees to our social desirability findings.
Results
At this stage, and without further empirical testing, it is important to note that the results from the cognitive interviews were obtained from a sample of 24 participants and should therefore be considered more in indicative terms than as claims of general validity.
Majority Slovak view of Roma
When asked about the majority Slovak view of the Roma minority, almost all the participants envisaged very negative opinions. The students who were internally motivated not to respond with prejudice thought the majority of Slovaks exhibited prejudices such as homophobia, racism, intolerance, radicalness, negative judgements and that the Roma had less value. By contrast the adults who were primarily internally motivated to respond with prejudice thought Slovaks (or they themselves) would have negative opinions and attitudes, but that this was because the Roma are anti-social, leave a mess outside their homes, are not very hygienic, fail to bring up their children properly, destroy their own homes, abuse the social system, are lazy, steal and so forth. This indicates that people with internal motivation not to respond in a prejudiced manner tend to explain majority-population prejudice against the Roma by pointing out that more general intolerance can be found in the views of the majority of Slovaks‘, while individuals with internal motivation to respond in a prejudiced manner tend to see the problem as lying with the minority. Most frequently the participants interpreted the term “the majority of Slovaks” as meaning the media and people they know.
Views people close to them have on Roma
When asked how those close to them viewed the Roma, the participants did not envisage such negative opinions and attitudes as they had for the majority of Slovaks; in fact in some cases they thought those close to them would have neutral or positive views of the Roma. They also thought some of the people close to them would have conflicting opinions, some of which would be positive opinions and experiences, but some of which would be more negative, a more radical view and negative experience of the Roma minority. Therefore answers given in response to the instruction to convey “the viewpoint of those close to you” leads to ambivalent opinions on the Roma minority and there is no comparative systematic tendency that would indicate a difference in opinion between people internally motivated to (not) respond in a prejudiced manner. The participants most frequently interpreted “those close to you” as meaning family and friends.
Degree of consistency between personal opinions, majority opinions and opinions of those close to the participant
The degree of consistency in personal opinions and majority Slovak opinions, and personal opinions and the opinions of those close to the participant are compared in Table 1.
Frequency of consistency of opinion
| Consistency of opinion | High | Moderate | Low |
|---|---|---|---|
| I – majority | 7 | 7 | 10 |
| I – close to person | 12 | 12 | 0 |
a z
Overall there is a low degree of consistency of opinion when personal responses are compared to majority Slovak opinions, especially among the eight students internally motivated to not respond in a biased manner, as is illustrated in this extract: “They would have differed in the extreme, for we are a homophobic nation…in this respect homophobic but also racist”. Seven participants from the adult sample exhibited a high degree of consistency between personal opinions and perceived majority opinions. Five of these were primarily internally motivated to respond in a prejudiced manner: “the responses would be the same … for the majority of people say they steal, they don’t work and they do only bad things, they take advantage of and abuse the social system in this country and I agree with that”. Moderate consistency between personal opinions and perceived majority opinions was also found in seven participants with different kinds of motivation: “well on some questions definitely yeah [they would be consistent], but on some I think the majority Slovak view would be more critical than mine…it would probably be even more negative than my own opinion”.
When we compared personal responses with those from the view of people close to the participants, we found the same frequency (12 responses) for high and moderate consistency of opinion. An example is the following statement: “I tend to spend time with people close to me and friends who have the same opinion as me”. A moderate degree of consistency of opinion is illustrated by this statement:
I think a bit yes but they wouldn’t differ hugely but some yes probably…my aunt in fact has a better opinion than mine…and my parents I would say have a somewhat worse opinion than me, so probably somewhere in the middle.
We did not identify a low degree of consistency between personal opinions and those of people close to the participant in any of the participants.
Comparing degree of agreeability and difficulty
In Table 2 we give the frequency of results for degree of agreeability and difficulty of responding to the instruction.
Frequency of degree of agreeability and difficulty of responding
| Viewpoint | Highly agreeable | Moderately agreeable | Slightly agreeable | Slightly difficult | Moderately difficult | Very difficult |
|---|---|---|---|---|---|---|
| Personal | 21 | 0 | 0 | 7 | 3 | 0 |
| Those close to participant | 8 | 5 | 1 | 6 | 7 | 2 |
| Majority of Slovaks | 6 | 3 | 9 | 5 | 3 | 2 |
Where the instruction was to give a personal response, the participants most frequently reported that the process was highly agreeable and only slightly difficult, but there were greater differences in degree of agreeability than in difficulty, depending on the instruction given. Ten students and eleven adults stated the activity was highly agreeable. We recorded seven instances where the process was not difficult. This is a typical example: “better and easier [to give my opinion]…I would have replied intuitively and mechanically…it would be best from my viewpoint”. Table 3 shows the most frequent reasons participants provided for considering giving their own opinion to be agreeable or not difficult.
Reasons why responding was agreeable and not difficult according to type of instruction
| Viewpoint | Very agreeable/ not very difficult | Not very agreeable/ very difficult |
|---|---|---|
| Personal |
|
|
| Those close to the participant |
|
|
| Majority |
|
|
Where participants were instructed to respond from the viewpoint of “those close to you”, they most frequently reported the process was highly agreeable:
I would know how to respond so it would be fine to do it from the viewpoint of those close to me…from the viewpoint of those close to me would be best and from the viewpoint of the majority would be worst… [from my viewpoint] that would be better again than those close to me.
Responding in accordance with the second instruction was most frequently thought to be moderately difficult. Perhaps this statement reflects that: “I would feel normal… responding from that viewpoint wouldn’t cause any problems… I would answer best for myself, I can’t read other people’s minds completely, they could say one thing and think another, I don’t know”. In Table 3 we show typical reasons for why responding from the viewpoint of people close to the participant was or would be (dis)agreeable and (not) difficult.
Unlike the previous two sets of instructions, responding “from the majority Slovak viewpoint” was most frequently associated with a low level of agreeability. This is a characteristic excerpt: “[the majority view] disagreeable …sometimes I had to think about whether I wasn’t simplifying people or degrading them too much…whether the majority really have that opinion…so sometimes I wasn’t really sure”. All the participants who were found to have low levels of agreeability were primarily internally motivated to respond without prejudice. Next came highly agreeable, which was found in six participants, who were mostly internally motivated to respond in a prejudiced manner. However, these participants stated that answering from any viewpoint was highly agreeable (or not difficult) because the instructions were unimportant: “it would have been easy, with no feeling of guilt for all three sets of instructions”.
Five respondents reported finding it very slightly difficult to respond from the majority Slovak viewpoint. For example:
the majority of Slovaks, well I think we all know what that general view is and it’s not hard to respond from that viewpoint…so it wouldn’t be hard but then again it wouldn’t be very agreeable responding from that viewpoint because it would be so horribly negative.
Table 3 shows how agreeable/difficult participants found responding from the majority Slovak viewpoint and why.
Based on these results we can say that participants who are internally motivated not to respond in a prejudiced manner tend to prefer being instructed to give their own opinion, while participants who are internally motivated to respond in a prejudiced manner do not think the type of instruction matters.
Sincerity and social desirability
When analysing the interviews in relation to the social desirability, we found that participants exhibited high levels of sincerity and did not think about whether it was appropriate to respond in a particular way. The following statement is typical: “I tried to be honest…I didn’t try to respond in a politically correct manner, because I think that’s pointless… pretending something… I’m not afraid to give my own opinion, I would definitely be honest”.
We also asked participants about how honest their answers would have been if the questionnaire had not been anonymous. Again we recorded high levels of honesty among all participants (excluding two) considering this hypothetical situation (especially in relation to the instruction to give personal views). This is illustrated in the following statement: “my opinion…I’d give it under any conditions and anywhere”. The only example of a low level of honesty is this statement:
“Well that would be a big problem [if the questionnaire was not anonymous], the more people the better, so I don’t look like a racist or rather so that people wouldn’t know I’m a racist… if I had to put my name to it.
Discussion
The instructions to the original SCM questionnaire emphasised that the researchers are not interested in the participants’ personal beliefs but in how they thought different minorities were viewed by others (society) (in our case, how the Roma were judged by the majority of Slovaks). Fiske et al. (2002) selected these instructions on the grounds they would reduce social desirability and capture cultural stereotypes. Where surveys are concerned it is often implicitly assumed that asking for the majority opinion instead of personal opinions reduces social desirability and enhances validity (Nederhof, 1985; Krumpal, 2013) and that it is easier to give personal responses to indirect and impersonal questions (Kidder & Judd in Jo, Nelson & Kiecker, 1997). However, the findings of our cognitive interviews indicate that the majority of participants responded differently when giving their own view compared to when giving the view of the majority of Slovaks. This confirms Fisher’s (1993) assumption that participants can give accurate predictions of what the typical individual thinks, in our case the majority of Slovaks, and give responses based on a general knowledge of social stereotypes (regarding the Roma minority) but without necessarily projecting their own opinions into the opinions of the majority. The only exception is those who are internally motivated to respond with prejudice, which supports the assumption of Forscher, Cox, Graetz, & Devine (2015).
In the cognitive interviews differences were found between participants who were internally motivated to respond without prejudice and those who were internally motivated to respond with prejudice. Participants with unbiased personal beliefs (would have) preferred to respond from their own viewpoint, that is, in accordance with being instructed to give their own personal opinions and attitudes. By contrast participants with internalised biased beliefs thought the instructions were/would have been unimportant. These responses contradict Devine and Plant’s (2002) assumption that participants with high EMS scores and low IMS scores only reveal their true biased attitudes when responding anonymously, as, apart from two responses, all the respondents either gave or would have given very sincere responses whatever the conditions.
The participants thought giving their own viewpoint was most agreeable and not difficult. In the cognitive interviews we also tested a third, type of instruction not generally used, which was responding from the viewpoint of those close to the person. We found that these responses corresponded far more closely to the participants’ personal responses than to the responses from the majority viewpoint.
These findings on participants’ responding differently to different types of instructions could be systematically taken into account when designing surveys.
The limitations of the research include the fact that respondents gave their responses directly to the researchers asking the questions, which meant it was impossible to check whether they would in fact give honest responses regardless of whether the survey was anonymous, or whether they were simply trying to portray themselves that way to the researcher. Another limitation is that the conclusions relate to results obtained using a relatively small number of participants. As others have noted this is generally an issue with cognitive interviews as they are conducted using small samples and consequently this may affect the reliability of the results (Tourangeau, Rips, & Rasinski, 2000). The results of the cognitive interview should therefore be seen as just one, albeit very useful, part of the jigsaw. For these preliminary findings to be subjected to rigorous empirical verification, research would have to be performed using a representative sample and comparing the responses with the different sets of instructions.
Conclusion
When participants were instructed to respond from the “majority [Slovak] viewpoint”, the answers differed from when they were given in response to the instruction to give their “personal viewpoint”. Researchers whose goal is to investigate prevailing cultural stereotypes regarding minorities should therefore use the original instruction to respond from the majority viewpoint. However, if the SCM is being used to ascertain for example the success of prejudice-reduction interventions, then researchers would be advised to use the instructions asking respondents to give their own viewpoint. Most participants also found responding to this instruction to be more agreeable and less difficult. Thus far it seems that differences between responses given from a personal viewpoint and from the majority viewpoint are not necessarily primarily caused by social desirability, but tend to be the consequence of differences in the person’s own beliefs and the beliefs of the majority. Further empirical testing is essential if a clearer picture is to be obtained.
References
Cuddy, A. J. C., Fiske, S. T., & Glick, P. (2007). The BIAS map: Behaviors from intergroup affect and stereotypes. Journal of Personality and Social Psychology, 92(4), 631-648.10.1037/0022-3514.92.4.631Search in Google Scholar
Cuddy, A. J. C., Fiske, S. T., & Glick, P. (2008). Warmth and competence as universal dimensions of social perception: The stereotype content model and the BIAS map. In M. P. Zanna (Ed.), Advances in Experimental Social Psychology, 40 (pp. 61-149). San Diego, CA, US: Elsevier Academic Press.10.1016/S0065-2601(07)00002-0Search in Google Scholar
Cuddy, A. J. C., Fiske, S. T., Kwan, V. S. Y., Glick, P., Demoulin, S., Leyens, J.-P., … Ziegler, R. (2009). Stereotype content model across cultures: Towards universal similarities and some differences. British Journal of Social Psychology, 48, 1-33.10.1348/014466608X314935Search in Google Scholar
Devine, P. G., & Plant, A. (2002). The regulation of explicit and implicit race bias: The role of motivations to respond without prejudice. Journal of Personality and Social Psychology, 82(5), 835-848.10.1037/0022-3514.82.5.835Search in Google Scholar
Devine, P. G., & Plant, A. (1998). Internal and external motivation to respond without prejudice. Journal of Personality and Social Psychology, 75(3), 811-832.10.1037/0022-3514.75.3.811Search in Google Scholar
Durante, F. (2008). Testing and extending the Stereotype Content Model (Unpublished doctoral dissertation). Università degli Studi di Padova, Padova. Retrieved from http://paduaresearch.cab.unipd.it/341/1/F.Durante_Dissertation.pdfSearch in Google Scholar
Fisher, R. J. (1993). Social desirability bias and the validity of indirect questioning. Journal of Consumer Research, 20, 303-315.10.1086/209351Search in Google Scholar
Fisher, R. J., & Tellis, G. J. (1998). Removing social desirability bias with indirect questioning: is the cure worse than the disease? Advances in Consumer Research, 25, 563-567.Search in Google Scholar
Fiske, S. T. (2012). Warmth and competence: Stereotype content issues for clinicians and researchers. Canadian Psychology, 53(1), 14-20.10.1037/a0026054Search in Google Scholar
Fiske, S. T., Cuddy, A. J. C., Glick, P., & Xu, J. (2002). A model of (often mixed) stereotype content: Competence and warmth respectively follow from perceived status and competition. Journal of Personality and Social Psychology, 82(6), 878-902.10.1037/0022-3514.82.6.878Search in Google Scholar
Fiske, S. T., & North, M. S. (2015). Measures of stereotyping and prejudice. In G. J. Boyle, D. H. Saklofske, & G. Matthews (Eds.), Measures of personality and social psychological constructs (pp. 684-718). Elsevier. https://doi.org/10.1016/B978-0-12-386915-9.00024-310.1016/B978-0-12-386915-9.00024-3Search in Google Scholar
Forscher, P. S., Cox, W. T., Graetz, N., & Devine, P. G. (2015). The motivation to express prejudice. Journal of Personality and Social Psychology, 109(5), 791-812.10.1037/pspi0000030Search in Google Scholar
Haeger, H., Lambert, A. D., Kinzie, J., & Gieser, J. (2012). Using cognitive interviews to improve survey instruments (pp. 1-15). New Orleans: Association for Institutional Research Forum.Search in Google Scholar
Hsieh, H. F., & Shannon, S. E. (2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15(9).10.1177/1049732305276687Search in Google Scholar
Jo, M., Nelson, J. E., & Kiecker, P. (1997). A model for controlling social desirability bias by direct and indirect questioning. Marketing Letters, 8(4), 429-437.10.1023/A:1007951313872Search in Google Scholar
Krippendorff, K. (2004). Content analysis: an introduction to its methodology. California: Sage Publications, Inc.Search in Google Scholar
Krumpal, I. (2013). Determinants of social desirability bias in sensitive surveys: A literature review. Quality and Quantity, 47(4), 2025-2047.10.1007/s11135-011-9640-9Search in Google Scholar
Lášticová, B., Popper, M., Findor, A., Kanovský, M., & Petrjánošová, M. (2018). (Under review). Cross-cultural validation of Stereotype Content Model questionnaire on Roma in Slovakia: The case for a mixed-methods approach.Search in Google Scholar
Nederhof, A. J. (1985). Methods of coping with social desirability bias: A review. European Journal of Social Psychology, 15, 263-280.10.1002/ejsp.2420150303Search in Google Scholar
Palinkas, L. A., Horwitz, S.M., Green, C. A., Wisdom, J. P., Duan, N., & Hoagwood, K. (2015). Purposeful sampling for qualitative data collection and analysis in mixed method implementation research. Administration and Policy in Mental Health Services Research, 42(5), 533-544.10.1007/s10488-013-0528-ySearch in Google Scholar
Popper, M., & Petrjánošová, M. (2016). Do we know what we are asking? Individual and group cognitive interviews. Human Affairs, 26, 253-270.10.1515/humaff-2016-0023Search in Google Scholar
Ritchie, J., Lewis, J., & El am, G. (2003). Designing and selecting samples. In J. Ritchie & J. Lewis (Eds.), Qualitative Research Practice. A Guide for Social Science Students and Researchers (pp. 77-108). London: Sage Publications.Search in Google Scholar
Sheatsley, P. B. (1983). Questionnaire construction and item writing. In P. Rossi, J. D. Wright, & A. B. Anderson, Handbook of Survey Research (pp. 195-230). New York: Academic Press, Inc.10.1016/B978-0-12-598226-9.50012-4Search in Google Scholar
Tourangeau, R., Rips, L. J., Rasinski, K. (2000). The psychology of survey response. Cambridge: Cambridge University Press.10.1017/CBO9780511819322Search in Google Scholar
Willis, G. B. (1999). Cognitive interviewing: A “how to” guide (pp. 1-37). Research Triangle Institute: Meeting of the American Statistical Association.Search in Google Scholar
Willis, G. B. (2005). Cognitive interviewing: A tool for improving questionnaire design. California: Sage Publications, Inc.10.1037/e538062007-001Search in Google Scholar
Willis, G. B. (2015). Analysis of the cognitive interview in questionnaire design. New York: Oxford University Press.Search in Google Scholar
Willis, G. B., & Artino, A. R. (2013). What do our respondents think we’re asking? Using cognitive interviewing to improve medical education surveys. Journal of Graduate Medical Education, September, 353-356.10.4300/JGME-D-13-00154.1Search in Google Scholar
Zhang, Y., & Wildemuth, B. M. (2009). Qualitative analysis of content. In B. M. Wildemuth, Applications of Social Research Methods to Questions in Information and Library Science (pp. 308-319). Connecticut: Libraries Unlimited.Search in Google Scholar
© 2018 Miroslav Popper, Veronika Kollárová, published by De Gruyter
