Verbal protocol methodology is used to examine how fourth-grade students construct meaning as they read and respond to two informational social studies texts. Results indicate most students are active readers, often engaging in higher-level comprehension strategies and critical thinking as they read independently. However, critical thinking and comprehension processes are not often captured in their responses to end-of-reading questions (ERQ), which as a result have limited scope and utility for guiding social studies instruction. Results also indicate that when (...) students change their patterns of strategy use for reading and responding to text in response to a change in text, they are more successful on the ERQ than when their strategy use remains stagnant. (shrink)

The standard interpretation of cognitive reflection tests assumes that correct responses are reflective and lured responses are unreflective. However, prior process-tracing of mathematical reflection tests has cast doubt on this interpretation. In two studies (N = 201), we deployed a validated think-aloud protocol in-person and online to test how this assumption is satisfied by the new, validated, less familiar, and less mathematical verbal Cognitive Reflection Test (vCRT). Importantly, thinking aloud did not disrupt test performance compared to a control (...) group. Moreover, verbalized thoughts in both studies revealed that most (but not all) correct responses involved reflection and that most (but not all) lured responses lacked reflection. These data suggest that the vCRT usually satisfies the standard interpretation of the reflection tests (albeit not without exceptions) and that the vCRT can be a good measure of the construct theorized by the two-factor explication of ‘reflection’ (as deliberate and conscious). (shrink)

The central problem which cognitive scientists face in studying thinking is that thinking cannot be observed directly by other people. The traditional solution has been to rely on introspective methods, where individuals observe their own thinking and reflect on its characteristics. In everyday life, the most common technique involves asking people questions about their thinking, knowledge, and strategies. Psychologists have refined the methods for questioning individuals by designing questionnaires and structured interviews. However, these two ways of obtaining information about thinking (...) share two fundamental methodological problems. First, the accuracy of the reports cannot be assessed, because in naturally occurring situations in everyday life the investigator doesn't have any other empirical evidence against which to evaluate the validity of the subjects' reported information. Hence, psychologists are forced to trust the subjects to provide valid information. The second issue concerns whether even those individuals who strive to give accurate reports are able to access and supply valid information on the cognitive processes that mediate their behavior. (shrink)

Recent developments in pattern analysis research have made this methodology suitable for the study of the processes that are set in motion in psychological interventions. Outcome research, based on the comparison between clinical results from treatment and control groups, has leveraged our empirical knowledge about the efficacy of psychological interventions. However, these methods of research are not precise enough for the analysis of these processes. On the contrary, pattern analysis could be a powerful tool to study moment-to-moment interactions typical of (...) psychological interventions. This is methodology is relevant because clinical psychology is experiencing a paradigm shift from a protocol for syndrome perspective to a principle-based and person-centered intervention. This evidence-based, theory-grounded, and process-oriented paradigm of clinical intervention needs new research methods to thrive. The analysis of the therapeutic relationship built into the verbal interaction between the clinician and the client is one of the cornerstones of this new era of research. So, the purpose of this article is three-fold: to discuss the role of the verbal interaction pattern analysis in the clinical context to the development of the principle-based clinical psychology, to analyze the patterns of verbal interaction in a clinical case, and to compare the results using two different methods. To reach these purposes, using the observational methodology, we have coded the verbal interaction of 16 clinical sessions with a person diagnosed with a borderline personality disorder. We have analyzed the data using sequential analysis and pattern recognition algorithms. We have been able to detect typical patterns during different phases of psychological intervention. Finally, the conceptual, methodological, and empirical implications of this study will be discussed within the realms of pattern analysis research and principle-based clinical psychology. (shrink)

Two studies examined the use of think aloud (TA) protocol as a means for collecting data of cognitive processes during performance in golf. In study 1, TA was employed to examine if different verbalisation (Level 2 or Level 3 TA) instructions influence performance of high and low skilled golfers. Participants performed 30 putts using TA at either Level 2, Level 3, or no verbalization condition. Although Level 3 verbalization produced a higher volume of verbal data than Level 2, (...) TA at either level 2 or 3 did not impair putting performance compared to no verbalization. Study 2 examined the congruence between data collected via TA at Level 3 and cued retrospective recall of cognitive processes during golf performance. Experienced golfers performed six holes of golf whilst engaging in Level 3 TA. After performance, three semi-structured retrospective interviews were conducted (ten minutes after performance, 24 hours after performance and 48 hours after performance). A comparison of the themes identified large discrepancies between the information reported during TA and at interview, with only 38-41% similarity in variables reported to influence decision making on each hole. Both studies suggest TA is a valuable method for recording cognitive processes of individuals during task performance. TA provides richer verbal data regarding decisions than cued retrospective recall, and TA does not negatively impact performance. (shrink)

Most countries compile evidence from witnesses and victims manually, whereby the interviewer assimilates what the interviewee says during the course of an interview to produce an evidential statement. This exploratory research examined the quality of evidential statements generated in real world investigations. Transcribed witness/victim interviews were compared to the resultant written statements produced by the interviewing officer and signed as an accurate record by the interviewee. A coding protocol was devised to assess the consistency of information between what was (...) said by the interviewee in the verbal interview and what was reported in the written statement. Statements contained numerous errors including omissions, distortions, and the inclusion of information not mentioned in the verbal interview. This exploratory work highlights an important area for future research focus. (shrink)

This article introduces the Verbal Irony Procedure (VIP), a first systematic method for identifying irony in natural discourse. The first section discusses previous operationalizations of irony and demonstrates that these are not explicit about which criteria were used to separate irony from non-irony. The second section argues why irony can be defined as an “utterance with a literal evaluation that is implicitly contrary to its intended evaluation.” This section also explains why ironic utterances can be placed on an evaluation (...) scale. In the third section, clauses are proposed as a good unit of analysis when looking at irony in natural discourse. The different steps of the VIP are then introduced in the fourth section and subsequently applied in a sample analysis of a natural text in the fifth section. The sixth section discusses a reliability analysis of the VIP protocol. The article ends with an outlook on how future research on verbal irony might benefit from applying the VIP. (shrink)

ObjectivesMany older adults with visual impairment also have significant hearing loss. The aim was to investigate the effectiveness of a newly developed Dual Sensory Loss protocol on communication and wellbeing of older persons with DSL and their communication partners in the Netherlands and Belgium.MethodsParticipants and their communication partners were randomized in the “DSL-protocol” intervention group or a waiting-list control group. The intervention took 3 to 5 weeks. Occupational therapists focused on optimal use of hearing aids, home-environment modifications and (...) effective communication strategies. The primary outcome was the Communication Strategies domain of the Communication Profile for the Hearing Impaired. Secondary outcomes measured in participants were the Low Vision Quality Of Life Adjustment subscale, the Center for Epidemiological Studies - Depression Scale, De Jong Gierveld Loneliness Scale and the Fatigue Assessment Scale. The Hearing Handicap and Disability Inventory - Reaction of Others subscale and the Care-related Quality of Life - 7 Dimensions was measured in communication partners. Measurements were taken at baseline and 3-month follow-up. Linear mixed models were used to analyze effects between groups over time for every outcome measure.ResultsIntention-to-treat analyses showed a significant effect of the DSL-protocol on the use of verbal strategies in favor of the control group, however, this effect was non-significant after adjustment for confounding. Effect sizes of other outcomes varied between −0.23 [−0.57, 0.12] and 0.30 [−0.05, 0.64]. The LMM showed a significant effect on the HHDI-Reaction of others scale in favor of communication partners in the treatment group, however, the effect did not remain significant at a 0.01 significance level and the effect size was very small and non-significant 0.12, 95% CI [−0.27 to 0.51]. Adjusted analyses did not reveal treatment effects.ConclusionThe DSL-protocol did not clearly contribute to the enhancement of communication and wellbeing in DSL-patients. Possible reasons for the lack of effects are OTs not being comfortable giving advice on communication and psychosocial issues or the short-term treatment and follow-up period. Further study is warranted to find out how the protocol may be adapted or whether it is necessary to involve mental healthcare professionals.Clinical Trial Registrationwww.ClinicalTrials.gov, identifier NTR2843. (shrink)

Background: The risk of surgery in eloquent areas is related to neuropsychological dysfunctions. Maximizing the extent of resection increases the overall survival. The onco-functional balance is mandatory when surgery involves cognitive areas, and maximal information on the cognitive status of patients during awake surgery is needed. This can be achieved using direct cortical stimulation mapping and, in addition to this, a neuropsychological monitoring technique called real-time neuropsychological testing. The RTNT includes testing protocols based on the area where the surgery is (...) performed. We reported on tests used for left temporal lobe surgery and our RTNT decision tree.Case Report: We reported our RTNT experience with a 25-year-old right-handed man with 13 years of schooling. He reported daily partial seizures. MRI revealed the presence of a low-grade glioma involving the temporo-insular cortex. The neuropsychological status presurgery which was within the normal range was combined with functional magnetic resonance imaging and diffusion tensor imaging information. Awake surgery plus RTNT was performed. Direct electrical stimulation during object naming elicited a motor speech arrest. Resection was continuously accompanied by the RTNT. The RTNT provided enriched information to the surgeon. Performance never dropped. A slight decrement in accuracy emerged for pseudoword repetition, short-term memory and working memory, phonological processing, and verbal comprehension. Total resection was performed, and the histological examination confirmed the nature of the lesion. Immediate postsurgery performance was within the normal range as it was the fMRI and DTI assessment.Conclusion: The RTNT provides essential information that can be used online, during surgery, for clinical aims to provide the surgeon with useful feedback on the cognitive status of patients. (shrink)

Obtaining prospective written consent from women to participate in trials when they are experiencing an obstetric emergency is challenging. Alternative consent pathways, such as gaining verbal consent at enrolment followed, later, by obtaining written consent, have been advocated by some clinicians and bioethicists but have received little empirical attention. We explored women’s and staff views about the consent procedures used during the internal pilot of a trial, where the protocol permitted staff to gain verbal consent at recruitment. (...) Interviews with staff and participating women. Data were analysed thematically and interviews were cross-compared to identify differences and similarities in participants’ views about the consent procedures used. Women and some staff highlighted benefits to obtaining verbal consent at trial enrolment, including expediting recruitment and reducing the burden on those left exhausted by their births. However, most staff with direct responsibility for taking consent expressed extreme reluctance to proceed with enrolment until they had obtained written consent, despite being comfortable using verbal procedures in their clinical practice. To account for this resistance, staff drew a strong distinction between research and clinical care and suggested that a higher level of consent was needed when recruiting into trials. In doing so, staff emphasised the need to engage women in reflexive decision-making and highlighted the role that completing the consent form could play in enabling and evidencing this process. While most staff cited their ethical responsibilities to women, they also voiced concerns that the absence of a signed consent form at recruitment could expose them to greater risk of litigation were an individual to experience a complication during the trial. Inexperience of recruiting into peripartum trials and limited availability of staff trained to take consent also reinforced preferences for obtaining written consent at recruitment. While alternative consent pathways have an important role to play in advancing emergency medicine research, and may be appreciated by potential recruits, they may give rise to unintended ethical and logistical challenges for staff. Staff would benefit from training and support to increase their confidence and willingness to recruit into trials using alternative consent pathways. This qualitative research was undertaken as part of the GOT-IT Trial. Date of registration 26/03/2014. (shrink)

Verbal autopsy presents the opportunity to understand the disease burden in many low-income countries where vital registration systems are underdeveloped and most deaths occur in the community. Advances in technology have led to the development of software that can provide probable cause of death information in real time, and research considering the ethical implications of these advances is necessary to inform policy. Our research explores these ethical issues in rural Nepal using a public health ethics framework. We considered the (...) burdens and benefits of VA and giving cause of death information to families of the deceased through qualitative research with VA interviewers, community members, national policy stakeholders and international academics. Burdens can be experienced differently, and it is important to balance the emotional burden of VA with utilization of the data to inform planning and increased access to health services. The training, support and supervision of VA interviewers should be prioritized if VA is taken to scale. Initial and ongoing community engagement is recommended in addition to engaging ethical, legal, health and policy personnel in developing protocols and systems. Integrating rigorous research while cautiously moving forward is recommended to ensure systems and responses to concerns are relevant to contexts. (shrink)

The use of remember–know judgments to assess subjective experience associated with memory retrieval, or as measures of recollection and familiarity processes, has been controversial. In the current study we had participants think aloud during study and provide verbal reports at test for remember–know and confidence judgments. Results indicated that the vast majority of remember judgments for studied items were associated with recollection from study , but this correspondence was less likely for high-confidence judgments . Instead, high-confidence judgments were more (...) likely than remember judgments to be associated with incorrect recollection and a lack of recollection. Know judgments were typically associated with a lack of recollection , but still included recollection from the study context . Thus, although remember judgments provided fairly accurate assessments of retrieval including contextual details, know judgments did not provide accurate assessments of retrieval lacking contextual details. (shrink)

Background: Post-traumatic stress disorder is a prevalent, debilitating, and costly psychiatric disorder. Evidenced-based psychotherapies, including Cognitive Processing Therapy, are effective in treating PTSD, although a fair proportion of individuals show limited benefit from such treatments. CPT requires cognitive demands such as encoding, recalling, and implementing new information, resulting in behavioral change that may improve PTSD symptoms. Individuals with PTSD show worse cognitive functioning than those without PTSD, particularly in acquisition of verbal memory. Therefore, memory dysfunction may limit treatment gains (...) in CPT in some individuals with PTSD. Methods and Analysis: Here, we present a protocol describing the Cognition and PsychoTherapy in PTSD study, a prospective, observational study examining how cognitive functioning affects treatment response in CPT for PTSD. The study aims to recruit 105 outpatient veterans with PTSD between the ages of 18 and 70 years. Prior to beginning 12 sessions of CPT, Veteran participants will have standardized assessments of mood and functioning and complete a comprehensive neurocognitive battery assessing episodic learning, attention and speed of processing, language ability, executive control, and emotional functioning. This study aims to fill gaps in the current literature by: examining the specificity of memory effects on treatment response; exploring how baseline cognitive functioning impacts functional outcomes; and examining potential mechanisms, such as memory for treatment content, that might explain the effects of baseline memory functioning on PTSD symptom trajectory. Discussion: If successful, this research could identify clinically relevant neurocognitive mechanisms that may impact PTSD psychotherapy and guide the development of individualized treatments for PTSD. (shrink)

Preoperative mapping of language and declarative memory functions in temporal lobe epilepsy patients is essential since they frequently encounter deterioration of these functions and show variable degrees of cerebral reorganization. Due to growing evidence on language and declarative memory interdependence at a neural and neuropsychological level, we propose the GE2REC protocol for interactive language-and-memory network mapping. GE2REC consists of three inter-related tasks, sentence generation with implicit encoding and two recollection memory tasks: recognition and recall. This protocol has previously (...) been validated in healthy participants, and in this study, we showed that it also maps the LMN in the left TLE. Compared to healthy controls, left TLE showed widespread inter- and intra-hemispheric reorganization of the LMN through reduced activity of regions engaged in the integration and the coordination of this meta-network. We also illustrated how this protocol could be implemented in clinical practice individually by presenting two case studies of LTLE patients who underwent efficient surgery and became seizure-free but showed different cognitive outcomes. This protocol can be advantageous for clinical practice because it is short and easy to perform; allows brain mapping of essential cognitive functions, even at an individual level; engages language-and-memory interaction allowing to evaluate the integrative processes within the LMN; provides a more comprehensive assessment by including both verbal and visual modalities, as well as various language and memory processes. Based on the available postsurgical data, we presented preliminary results obtained with this protocol in LTLE patients that could potentially inform the clinical practice. This implies the necessity to further validate the potential of GE2REC for neurosurgical planning, along with two directions, guiding resection and describing LMN neuroplasticity at an individual level. (shrink)

Despite the importance of knowing the cognitive capabilities of children with neurodevelopmental conditions, less than one-third of children with cerebral palsy participate in standardized assessments. Globally, approximately 50% of people with cerebral palsy have an intellectual disability and there is significant risk for domain-specific cognitive impairments for the majority of people with cerebral palsy. However, standardized cognitive assessment tools are not accessible to many children with cerebral palsy, as they require manual manipulation of objects, verbal response and/or speeded response. (...) As such, standardised assessment may result in an underestimation of abilities for children with significant motor and/or speech impairment. The overall aim of the project is to examine and compare the psychometric properties of standardised cognitive assessment tools that have been accommodated for use with either a switch device or eye-gaze control technologies, with the specific aims to: Examine the psychometric properties of accommodated assessment tools by comparing the performance of typically developing children on six cognitive assessment tools administered via standardised versus accommodated administration; Describe and compare the performance and user experience of children with cerebral palsy on six accommodated cognitive assessments administered via switch or eye-gaze control technologies. Secondary aims are to: Describe the completion rates and time to complete assessments of participants in each group; Within the group with cerebral palsy, examine the effects of condition-specific characteristics and demographics on participation. This protocol paper describes a two-phase validation and acceptability study that utilizes a mixed-model design. This study will collect concurrent data from 80 typically developing children and 40 children with cerebral palsy, who use switch or eye-gaze control technology as alternate access communication methods. The set of instruments will measure receptive vocabulary, fluid reasoning, sustained attention, vision perception, visuospatial working memory and executive functions. Data analyses will be conducted using SPSS v. 25 and R v 4.1.0. SPSS Sample Power 3 was used for power computation and allows for a 10% drop out rate. Quantitative descriptive statistics, measurement agreement data plotting, bivariate and multiple regressions analysis will be conducted using appropriate methods. (shrink)

Research in the fields of complex problem solving (CPS) and dynamic decision making using microworlds has been mainly conducted in Western industrialized countries. This study analyzes the CPS process by investigating thinking‐aloud protocols in five countries. Participants were 511 students from Brazil, Germany, India, the Philippines, and the United States who worked on two microworlds. On the basis of cultural‐psychological theories, specific cross‐national differences in CPS strategies were hypothesized. Following theories of situatedness of cognition, hypotheses about the specific frequency of (...) problem‐solving strategies in the two microworlds were developed. Results of the verbal protocols showed (a) modification of the theoretical CPS model, (b) task dependence of CPS strategies, and (c) cross‐national differences in CPS strategies. Participants’ CPS processes were particularly influenced by country‐specific problem‐solving strategies. (shrink)

Video games are ideal platforms for the study of skill acquisition for a variety of reasons. However, our understanding of the development of skill and the cognitive representations that support skilled performance can be limited by a focus on game scores. We present an alternative approach to the study of skill acquisition in video games based on the tools of the Expert Performance Approach. Our investigation was motivated by a detailed analysis of the behaviors responsible for the superior performance of (...) one of the highest scoring players of the video game Space Fortress. This analysis revealed how certain behaviors contributed to his exceptional performance. In this study, we recruited a participant for a similar training regimen, but we collected concurrent and retrospective verbal protocol data throughout training. Protocol analysis revealed insights into strategies, errors, mental representations, and shifting game priorities. We argue that these insights into the developing representations that guided skilled performance could only easily have been derived from the tools of the Expert Performance Approach. We propose that the described approach could be applied to understand performance and skill acquisition in many different video games and help reveal mechanisms of transfer from gameplay to other measures of laboratory and real-world performance. (shrink)

Video games are ideal platforms for the study of skill acquisition for a variety of reasons. However, our understanding of the development of skill and the cognitive representations that support skilled performance can be limited by a focus on game scores. We present an alternative approach to the study of skill acquisition in video games based on the tools of the Expert Performance Approach. Our investigation was motivated by a detailed analysis of the behaviors responsible for the superior performance of (...) one of the highest scoring players of the video game Space Fortress. This analysis revealed how certain behaviors contributed to his exceptional performance. In this study, we recruited a participant for a similar training regimen, but we collected concurrent and retrospective verbal protocol data throughout training. Protocol analysis revealed insights into strategies, errors, mental representations, and shifting game priorities. We argue that these insights into the developing representations that guided skilled performance could only easily have been derived from the tools of the Expert Performance Approach. We propose that the described approach could be applied to understand performance and skill acquisition in many different video games and help reveal mechanisms of transfer from gameplay to other measures of laboratory and real-world performance. (shrink)

Thinking aloud is the concurrent verbalization of thoughts while performing a task. The study of thinking-aloud protocols has a long tradition in cognitive psychology, the field of education, and the industrial-organizational context. It has been used rarely in cultural and cross-cultural psychology. This paper will describe thinking aloud as a useful method in cultural and cross-cultural psychology referring to a few studies in general and one study in particular to show the wide applications of this method. Thinking-aloud protocols can be (...) applied for a) improving the validity of cross-cultural surveys, b) process analysis of thoughts and the analysis of changes over time, c) theory development across cultures, d) the study of cultural meaning systems, and e) individual as well as group level analyses allowing hypothesis testing cross-culturally. Limitations of the thinking-aloud method are also discussed. (shrink)

Toulmin’s (1958) model of argument was employed in the analysis of verbal protocols obtained during the solving of ill-structured problems. The participants were experts in the domain under study. For the analysis the Toulmin model was extended in order to enable description of lines of argument found in protocols as long as 10 paragraphs. Results included: (1) That while the protocol was comprised of a large number of specific arguments, the analysis provided for tracing a solver’s line of (...) argument. (2) On occasion datum and backing were difficult to distinguish. (3) Warrants essentially were not stated, although substantial backing was provided. However, as perhaps would be expected, the Toulmin model did not provide for delineation of components of the problem-solving process. A second analysis assuming a “higher level” problem-solving structure and a “lower level” argument structure produced an integrated problem-solving – argumentation structure depicting how reasoning is used in relation to particular task goals. Finally, at a more general level, problem solving was considered as a classical rhetorical structure. (shrink)

Research on expert‐novice differences has mainly focused on how experts solve familiar problems. We know far less about the skills and knowledge used by experts when they are confronted with novel problems within their area of expertise. This article discusses a study in which verbal protocols were taken from subjects of various expertise designing an experiment in an area with which they were unfamiliar. The results showed that even when domain knowledge is lacking, experts solve a novel problem within (...) their area of expertise by dividing the problem into a number of subproblems that are solved in a specified order. The lack of domain knowledge is compensated for by using abstract knowledge structures and domain‐specific heuristic strategies. However, the quality of their solutions is considerably lower than the quality attained by experts who were familiar with the type of problem to be solved. The results suggest that when experts are confronted with novel problems as compared with familiar problems, their form of reasoning remains intact, but the content of their reasoning suffers due to lack of domain knowledge. (shrink)

Six characteristics of effective representational systems for conceptual learning in complex domains have been identified. Such representations should: (1) integrate levels of abstraction; (2) combine globally homogeneous with locally heterogeneous representation of concepts; (3) integrate alternative perspectives of the domain; (4) support malleable manipulation of expressions; (5) possess compact procedures; and (6) have uniform procedures. The characteristics were discovered by analysing and evaluating a novel diagrammatic representation that has been invented to support students' comprehension of electricity—AVOW diagrams (Amps, Volts, Ohms, (...) Watts). A task analysis is presented that demonstrates that problem solving using a conventional algebraic approach demands more effort than AVOW diagrams. In an experiment comparing two groups of learners using the alternative approaches, the group using AVOW diagrams learned more than the group using equations and were better able to solve complex transfer problems and questions involving multiple constraints. Analysis of verbal protocols and work scratchings showed that the AVOW diagram group, in contrast to the equations group, acquired a coherently organised network of concepts, learnt effective problem solving procedures, and experienced more positive learning events. The six principles of effective representations were proposed on the basis of these findings. AVOW diagrams are Law Encoding Diagrams, a general class of representations that have been shown to support learning in other scientific domains. (shrink)

When a prominent attribute looms larger in one response procedure than in another, a violation of procedure invariance occurs. A hypothesis based on compatibility between the structure of the input information and the required output was tested as an explanation of this phenomenon. It was also compared with other existing hypotheses in the field. The study had two aims: (1) to illustrate the prominence effect in a selection of preference tasks (choice, acceptance decisions, and preference ratings); (2) to demonstrate the (...) processing differences in a matching procedure versus the selected preference tasks. Hence, verbal protocols were collected in both a matching task and in subsequent preference tasks. Silent control conditions were also employed. The structure compatibility hypothesis was confirmed in that a prominence effect obtained in the preference tasks was accompanied by a lower degree of attention to the attribute levels in these tasks. Furthermore, as predicted from the structure compatibility hypothesis, it was found that fewer comparisons between attribute levels were performed in the preference tasks than in the matching task. It was therefore concluded that both these processing differences may explain the occurrence of the prominence effects. (shrink)

A study is reported which focused on the problem-solving strategies employed by expert electronics engineers pursuing a real-world task: integrated-circuit design. Verbal protocol data were analysed so as to reveal aspects of the organisation and sequencing of ongoing design activity. These analyses indicated that the designers were implementing a highly systematic solution-development strategy which deviated only a small degree from a normatively optimal top-down and breadth-first method. Although some of the observed deviation could be described as opportunistic in (...) nature, much of it reflected the rapid depth-first exploration of tentative solution ideas. We argue that switches from a predominantly breadth-first mode of problem solving to depth-first or opportunistic modes may be an important aspect of the expert's strategic knowledge about how to conduct the design process effectively when faced with difficulties, uncertainties, and design impasses. (shrink)

Six characteristics of effective representational systems for conceptual learning in complex domains have been identified. Such representations should: (1) integrate levels of abstraction; (2) combine globally homogeneous with locally heterogeneous representation of concepts; (3) integrate alternative perspectives of the domain; (4) support malleable manipulation of expressions; (5) possess compact procedures; and (6) have uniform procedures. The characteristics were discovered by analysing and evaluating a novel diagrammatic representation that has been invented to support students' comprehension of electricity—AVOW diagrams (Amps, Volts, Ohms, (...) Watts). A task analysis is presented that demonstrates that problem solving using a conventional algebraic approach demands more effort than AVOW diagrams. In an experiment comparing two groups of learners using the alternative approaches, the group using AVOW diagrams learned more than the group using equations and were better able to solve complex transfer problems and questions involving multiple constraints. Analysis of verbal protocols and work scratchings showed that the AVOW diagram group, in contrast to the equations group, acquired a coherently organised network of concepts, learnt effective problem solving procedures, and experienced more positive learning events. The six principles of effective representations were proposed on the basis of these findings. AVOW diagrams are Law Encoding Diagrams, a general class of representations that have been shown to support learning in other scientific domains. (shrink)

Current conceptions of shame emphasize its negative communication value as a phenomenon of conscious experience. A tendency in our contemporary society is to view this phenomenon as an extremely disparaging and undesirable experience that every person should avoid or eliminate. It has become a cultural norm now that shame, perceived as human failure or sickness, is to be rejected, hidden, and not discussed. It is believed to stand in the way of personal progress and self-realization. The research literature mirrors not (...) only the lack of interest in understanding but the ignorance of this central-to-human-life experience. The present study examines the meaning and communicative structure of shame through an application of the phenomenological method of Merleau-Ponty and Lanigan. My analysis is grounded in empirical phenomenology and focuses on meaning as reflected in verbal protocols. The results obtained dispel the misleading notion of shame as primarily a negative, to-be-avoided experience. Rather, reflection on the empirical data indicates that the subjects do not adopt the negative theoretical model of shame but accept it as a universal positive experience of communication that is fundamental in their self-improvement process. (shrink)

In this article we describe research methods that are used for the study of individual multiattribute evaluation processes. First we explain that a multiattribute evaluation problem involves the evaluation of a set of alternatives, described by their values on a number of alternatives. We discuss a number of evaluation strategies that may be applied to arrive at a conclusion about the attractiveness or suitability of the alternatives, and next introduce two main research paradigms in this area, structural modelling and process (...) tracing. We argue that the techniques developed within these paradigms all have their advantages and disadvantages, and conclude that the most promising technique to detect the true nature of the evaluation strategy used by a judge seems to be the analysis of verbal protocols. At the same time we think it is wise not to rely on just one technique, but to use a multimethod approach to the study of multiattribute evaluation processes whenever that is possible. (shrink)

This study investigated the cognitive processes involved in inductive reasoning. Sixteen undergraduates solved quadratic function–finding problems and provided concurrent verbal protocols. Three fundamental areas of inductive activity were identified: Data Gathering, Pattern Finding, and Hypothesis Generation. These activities are evident in three different strategies that they used to successfully find functions. In all three strategies, Pattern Finding played a critical role not previously identified in the literature. In the most common strategy, called the Pursuit strategy, participants created new quantities (...) from x and y, detected patterns in these quantities, and expressed these patterns in terms of x. These expressions were then built into full hypotheses. The processes involved in this strategy are instantiated in an ACT‐based model that simulates both successful and unsuccessful performance. The protocols and the model suggest that numerical knowledge is essential to the detection of patterns and, therefore, to higher‐order problem solving. (shrink)

We explore the extent to which Anderson's (1987) theory of knowledge compilation can account for the relationship between instructions and practice in learning to use a simple device. Bibby and Payne (1993) reported experimental support for knowledge compilation in this domain. This article replicates the finding of a performance cross‐over between instruction type and task type that disappears with practice on the tasks. The research is extended by using verbal protocols to model the strategies of novice and more experienced (...) individuals. Production system models of these strategies suggest that knowledge compilation only provides an adequate account of practice for one of two Instruction groups. To model the strategy shifts for the second group, we employ a “procedure modification heuristic” (after Neches, 1987), which relies on access to a declarative model of the configuration of the device. This suggests that instructionally derived declarative knowledge may have a subtle ongoing effect on the changes in procedural knowledge with practice. (shrink)

When, how, and why students use conceptual knowledge during math problem solving is not well understood. We propose that when solving routine problems, students are more likely to recruit conceptual knowledge if their procedural knowledge is weak than if it is strong, and that in this context, metacognitive processes, specifically feelings of doubt, mediate interactions between procedural and conceptual knowledge. To test these hypotheses, in two studies (Ns = 64 and 138), university students solved fraction and decimal arithmetic problems while (...) thinking aloud; verbal protocols and written work were coded for overt uses of conceptual knowledge and displays of doubt. Consistent with the hypotheses, use of conceptual knowledge during calculation was not significantly positively associated with accuracy, but was positively associated with displays of doubt, which were negatively associated with accuracy. In Study 1, participants also explained solutions to rational arithmetic problems; using conceptual knowledge in this context was positively correlated with calculation accuracy, but only among participants who did not use conceptual knowledge during calculation, suggesting that the correlation did not reflect “online” effects of using conceptual knowledge. In Study 2, participants also completed a nonroutine problem‐solving task; displays of doubt on this task were positively associated with accuracy, suggesting that metacognitive processes play different roles when solving routine and nonroutine problems. We discuss implications of the results regarding interactions between procedural knowledge, conceptual knowledge, and metacognitive processes in math problem solving. (shrink)

In recent literature there is unanimous agreement about children's pragmatic competence in drawing scalar implicatures about some , if the task is made easy enough. However, children accept infelicitous some sentences more often than adults do. In general their acceptance is assumed to be synonymous with a logical interpretation of some as a quantifier. But in our view an overlap with some as a determiner in under-informative sentences cannot be ruled out, given the ambiguity of the experimental instructions and the (...) attitude of trust by children in adults. Our study investigated this hypothesis with different experimental manipulations. We found that when the experimenter's intentions are clear (Experiment 1, all / some order effect; Experiments 2 and 4, conditions 2 and 3), under-informative sentences are usually rejected; otherwise (Experiment 1, some / all order effect; Experiments 3 and 4, control condition) they are accepted. However, analysis of verbal protocols indicated that pragmatically infelicitous sentences are accepted, with some interpreted mostly as a determiner, irrespective of the function of some as a quantifier. Acceptance is not in itself synonymous with a logical interpretation of some as a quantifier. (shrink)

This article attends to “questionnaires” in linguistic fieldwork defined by the inclusion of sensory stimuli. It shows that such non-verbal protocols have been used to help elucidate and compare semantic content, which has generally been subordinated to formal analysis in the history of linguistics. To explain and exemplify this relationship, I target the color questionnaire developed by Hugo Magnus, which included ten standardized color chips and a long list of interview questions on language use. Magnus’s questionnaire (Fragebogen) decoupled perception (...) and denotation in its day, though similar protocols have recently been employed to restore the connection between meaning and form. (shrink)

Why games? How could anyone consider action games an experimental paradigm for Cognitive Science? In 1973, as one of three strategies he proposed for advancing Cognitive Science, Allen Newell exhorted us to “accept a single complex task and do all of it.” More specifically, he told us that rather than taking an “experimental psychology as usual approach,” we should “focus on a series of experimental and theoretical studies around a single complex task” so as to demonstrate that our theories of (...) human cognition were powerful enough to explain “a genuine slab of human behavior” with the studies fitting into a detailed theoretical picture. Action games represent the type of experimental paradigm that Newell was advocating and the current state of programming expertise and laboratory equipment, along with the emergence of Big Data and naturally occurring datasets, provide the technologies and data needed to realize his vision. Action games enable us to escape from our field's regrettable focus on novice performance to develop theories that account for the full range of expertise through a twin focus on expertise sampling and longitudinal studies of simple and complex tasks. (shrink)

One of the most influential studies in all expertise research is de Groot’s (1946) study of chess players, which suggested that pattern recognition, rather than search, was the key determinant of expertise. Many changes have occurred in the chess world since de Groot’s study, leading some authors to argue that the cognitive mechanisms underlying expertise have also changed. We decided to replicate de Groot’s study to empirically test these claims and to examine whether the trends in the data have changed (...) over time. Six Grandmasters, five International Masters, six Experts, and five Class A players completed the think-aloud procedure for two chess positions. Findings indicate that Grandmasters and International Masters search more quickly than Experts and Class A players, and that both groups today search substantially faster than players in previous studies. The findings, however, support de Groot’s overall conclusions and are consistent with predictions made by pattern recognition models. (shrink)

As one approach to examining the way ethical decisions are made, we asked experts and novices to review a set of scenarios that depict some important ethical tensions in research. The method employed was “protocol analysis,” a talk-aloud technique pioneered by cognitive scientists for the analysis of expert performance. The participants were asked to verbalize their normally unexpressed thought processes as they responded to the scenarios, and to make recommendations for courses of action. We found that experts spent more (...) time working through the decision-making process than novices and also raised substantially more concerns than novices. Differences also exist among the three groups of experts. (shrink)

This article presents an exploratory analysis of the metaphoric structure of five artistic paintings within “Think aloud” protocols, in which a group of 14 English speakers with a low self-rated level of expertise in art and history of art expertise were asked to verbalize all their thoughts, ideas and impressions of the artworks. The main findings of this study can be summarized as follows: (1) multiple interpretations for the same artwork are possible, (2) the interpretations of the metaphorical structures described (...) by the participants often diverge from those advanced by the researchers. These findings challenge the methods by which metaphor identification and analysis in pictorials is currently approached. As a matter of fact, most of the research in pictorial metaphors tends to reduce stimuli such as artistic paintings to unique metaphoric interpretations generally produced by a single researcher by means of introspection. By addressing this methodological problem in metaphor research, this article contributes to the development of a theoretical and operational participant-based framework that takes into account the role of metaphoric conceptualization within the domain of art and art cognition. (shrink)

We designed a new protocol requiring French adult participants to group a large number of Munsell colour chips into three or four groups. On one, relativist, view, participants would be expected to rely on their colour lexicon in such a task. In this framework, the resulting groups should be more similar to French colour categories than to other languages categories. On another, universalist, view, participants would be expected to rely on universal features of perception. In this second framework, the (...) resulting groups should match colour categories of three and four basic terms languages. In this work, we first collected data to build an accurate map of French colour terms categories. We went on testing how native French speakers spontaneously sorted a set of randomly presented coloured chips and, in line with the relativist prediction, we found that the resulting colour groups were more similar to French colour categories than to three and four basic terms languages. However, the same results were obtained in a verbal interference condition, suggesting that participants rely on language specific and nevertheless perceptual, colour categories. Collectively, these results suggest that the universalist/relativist dichotomy is a too narrow one. (shrink)

We designed a new protocol requiring French adult participants to group a large number of Munsell colour chips into three or four groups. On one, relativist, view, participants would be expected to rely on their colour lexicon in such a task. In this [ramework, the resulting groups should be more similar to French colour categories than to other languages categories. On another, universalist, view, participants would be expected to rely on universal features of perception. In this second framework, the (...) resulting groups should match colour categories of three and four basic terms languages. In this work, we first collected data to build an accurate map of French colour terms categories (Experiment 1). We went on testing how native French speakers spontaneously sorted a set of randomly presented coloured chips and, in line with the relativist prediction, we found that the resulting colour groups were more similar to French colour categories than to three and four basic terms languages (Experiment 2). However, the same results were obtained in a verbal interference condition (Experiment 3), suggesting that participants rely on language specific and nevertheless perceptual, colour categories. Collectively, these results suggest that the universalist/relativist dichotomy is a too narrow one. (shrink)

In this paper, the theory of syllogistic reasoning proposed by Johnson-Laird is shown to be inadequate and an alternative theory is put forward. Protocols of people attempting to solve syllogistic problems and explaining to another person how they reached their conclusions were obtained. Two main groups of subjects were identified. One group represented the relationship between classes in a spatial manner that was supplemented by a verbal representation. The other group used a primarily verbal representation. A detailed theory (...) of the processes for both groups is given. (shrink)

BackgroundApproval of the research proposal by an ethical review committee from both sponsoring and host countries is a generally agreed requirement in externally sponsored research.However, capacity for ethics review is not universal. Aim of this study was to identify opinions and views of the members serving in ethical review and ethics committees in Sri Lanka on informed consent, essential components in the information leaflet and the consent form.MethodsWe obtained ethical approval from UK and Sri Lanka. A series of consensus generation (...) meetings on the protocol were conducted. A task oriented interview guide was developed. The interview was based on open-ended questionnaire. Then the participants were given a WHO checklist on informed consent and requested to rate the items on a three point scale ranging from extremely important to not important.ResultsTwenty-nine members from ethics committees participated. Majority of participants (23), believed a copy of the information leaflet and consent form, should accompany research proposal. Opinions about the items that should be included in the information leaflets varied. Participants identified 18 criteria as requirements in the information leaflet and 19 for the consent form.The majority, 20 (69%), believed that all research need ethical approval but identified limited human resource, time and inadequate capacity as constraints. Fifteen (52%) believed that written consent is not required for all research. Verbal consent emerged as an alternative to written consent. The majority of participants rated all components of the WHO checklist as important.ConclusionThe number of themes generated for the consent form (N = 18) is as many as for the information leaflet (N = 19) and had several overlaps. This suggests that the consent form should be itemized to reflect the contents covered in the information leaflet. The participants' opinion on components of the information leaflets and consent forms proved to be similar with WHO checklist on informed consent. (shrink)

Current theories of reasoning such as mental models or mental logic assume a universal cognitive mechanism that underlies human reasoning performance. However, there is evidence that this is not the case, for example, the work of Ford (1995), who found that some people adopted predominantly spatial and some verbal strategies in a syllogistic reasoning task. Using written and think-aloud protocols, the present study confirmed the existence of these individual differences. However, in sharp contrast to Ford, the present study found (...) few differences in reasoning performance between the two groups, in terms of accuracy or type of conclusion generated. Hence, reasoners present an outward appearance of ubiquity, despite underlying differences in reasoning processes. These findings have implications for theoretical accounts of reasoning, and for attempts to model reasoning data. Any comprehensive account needs to account for strategic differences and how these may develop in logically untrained individuals. (shrink)

OBJECTIVES: To identify the factors that influence the assessment of reported cases of physician-assisted death by members of the public prosecution. DESIGN/SETTING: At the beginning of 1996, during verbal interviews, 12 short case-descriptions were presented to a representative group of 47 members of the public prosecution in the Netherlands. RESULTS: Assessment varied considerably between respondents. Some respondents made more "lenient" assessments than others. Characteristics of the respondents, such as function, personal-life philosophy and age, were not related to the assessment. (...) Case characteristics, i.e. the presence of an explicit request, life expectancy and the type of suffering, strongly influenced the assessment. Of these characteristics, the presence or absence of an explicit request was the most important determinant of the decision whether or not to hold an inquest. CONCLUSIONS: Although the presence of an explicit request, life expectancy and the type of suffering each influenced the assessment, each individual assessment was dependent on the assessor. The resulting danger of legal inequality and legal uncertainty, particularly in complicated cases, should be kept to a minimum by the introduction of some form of protocol and consultation in doubtful or boundary cases. The notification procedure already promotes a certain degree of uniformity in the prosecution policy. (shrink)