Elsevier

Cognition

Volume 166, September 2017, Pages 407-417
Cognition

Original Articles
Desirable difficulties during the development of active inquiry skills

https://doi.org/10.1016/j.cognition.2017.05.021Get rights and content

Abstract

This study explores developmental changes in the ability to ask informative questions, hypothesizing a link between the ability to update beliefs in light of evidence and the ability to ask informative questions. Five- to ten-year-old children played an iPad game asking them to identify a hidden insect. Learners could either ask about individual insects, or make a series of feature queries (e.g., “Does the hidden insect have antenna?”) that could more efficiently narrow the hypothesis space. Critically, the task display either helped children integrate evidence with the hypothesis space or required them to perform this operation themselves. Our prediction was that assisting children with belief updating would help them formulate more informative queries. This assistance improved some aspects of children’s active inquiry behavior; however, despite making some updating mistakes, children required to update their own beliefs asked questions that were more context-sensitive and thus informative. The results show how making a task more difficult can improve some aspects of children’s active inquiry skills, thus illustrating a type of “desirable difficulty” for reasoning.

Introduction

A skill of central importance during development is learning how to ask informative questions in order to make sense of the world. The roots of these abilities are observable even in the early preschool years. For example, in simple causal reasoning tasks, preschool-aged children can distinguish confounded from unconfounded evidence to draw causal inferences (Gopnik et al., 2001, Kushnir and Gopnik, 2005, Kushnir and Gopnik, 2007, Schulz and Gopnik, 2004). Preschool-aged children also selectively explore confounded evidence in their own exploratory play (Cook et al., 2011, Gweon and Schulz, 2008, Schulz and Bonawitz, 2007). Despite these early emerging abilities, many of the cognitive skills required for self-guided, active inquiry seem to follow protracted developmental trajectories. For example, in tasks designed to assess scientific reasoning abilities, children in the older elementary school years (ages 8–10) often have difficulty adopting systematic strategies, such as testing the effects of one variable at a time or selecting interventions that will lead to determinate evidence (Chen & Klahr, 1999). Although children in the older elementary school years can be taught to engage in these strategies via direct instruction (Klahr and Nigam, 2004, Kuhn and Dean, 2005), it is notable how difficult it is for them to discover and implement them on their own.

One reason for the difficulties children exhibit in these types of inquiry tasks may be that active inquiry depends on the coordination of a variety of component cognitive processes (Bonawitz and Griffiths, 2010, Coenen and Gureckis, 2015). For example, according to one popular view (Klein et al., 2006a, Klein et al., 2006b, Russell et al., 1993), active inquiry unfolds as a sequence of mental steps (see Fig. 1). Learners must generate possible hypotheses to explain their environment. They then must engage in decision making to ask questions or gather additional information to decide which of these hypotheses is most likely. They then must understand the results of these inquiry behaviors and update their beliefs accordingly, and so on. The various stages of this loop closely mirror the process of scientific reasoning engaged by scientists (Klein et al., 2006a, Klein et al., 2006b, Russell et al., 1993). Inefficiencies in any or all of these interrelated processes may serve as developmental limitations. For example, young learners may be able to search efficiently for information given a particular set of hypotheses but have trouble updating their beliefs correctly given new evidence. In this sense active inquiry behavior is like a bicycle: when all the elements are properly functioning and aligned the bike moves forward. However, misalignment of even one component can be catastrophic.

Understanding the integrated nature of these cognitive processes is important not just for our scientific understanding of the development of the human mind, but also because of broader educational implications. For example, many educational philosophies emphasize relatively unstructured, self-guided learning environments (Bruner, 1961, Kolb, 1984, Steffe and Gale, 1995). Understanding limitations in children’s active inquiry abilities and how each component of such abilities evolves across age can be used to design more effective learning environments for children of various ages. For example, evidence that younger children benefit from assistance in updating their beliefs in response to new evidence would suggest that learning environments for younger children need to provide support for this component of their learning.

The present study attempts to decompose the component processes involved in active inquiry, specifically focusing on the role of belief updating. We tasked five- to ten-year old children to identify a hidden insect in a simple iPad variant of the classic “Guess Who?” game. Children sequentially asked questions to try to identify the hidden target and received truthful answers. Based on prior work reviewed below (e.g., Mosher & Hornsby, 1966), we expected younger children to have difficulty formulating informative queries and thus sought to explore what types of automated assistance might aid children’s reasoning strategies. Specifically, we manipulated whether the computer program helped children to use the new evidence that resulted from their queries to narrow down the hypothesis space, or whether they had to reconcile the revealed evidence and the hypothesis space on their own. Our expectation was that helping children to update their beliefs accurately following the receipt of new information would free up cognitive resources and lead to higher quality question-asking. Interestingly, our results opposed this initial hypothesis in that elements which ostensibly made our task more difficult actually improved the quality of children’s inquiry behavior and suggest an important refinement of the information processing model summarized in Fig. 1.

Active inquiry fundamentally depends on the ability of learners to construct actions or queries which gain information (e.g., asking a question of a knowledgeable adult). A now classic way to study this behavior is through experimental tasks based on the 20-questions or ‘Guess Who?” game. In the game, the asker (participant) tries to determine a hidden object known only to the answerer (experimenter) by asking a series of yes-or-no questions. Mosher and Hornsby (1966) identified two broad question types commonly used in the game: hypothesis-scanning questions test a single hypothesis or specific instance (e.g., “Is it a monkey?”), whereas constraint-seeking questions attempt to constrain the hypothesis space faster by querying features that are present or absent in multiple objects (e.g., “Is it soft?”), but that do not directly identify the answer except by virtue of elimination.

A classic finding in this literature is that younger children (e.g., aged 6) tend to ask more hypothesis-scanning questions, while older children (e.g., aged 11) use more constraint-seeking questions, and also tend to find the answer after fewer questions (Mosher & Hornsby, 1966). One explanation is that only older children have developed the ability to focus on the high-level features that group the hypotheses, whereas younger children focus on individual stimuli. Consistent with this viewpoint, manipulations that help children focus on these higher-level features, such as cuing them with basic level category labels instead of exemplar names (Ruggeri & Feufel, 2015), increase the likelihood that young children will generate constraint-seeking questions (see also Herwig, 1982). Further, although young children are often relatively less likely than older children to ask constraint-seeking questions, even younger children (ages 7–9) are more likely to do so when such questions are particularly informative, such as when the hypothesis space is large and there are several equally probable solutions remaining (Ruggeri and Lombrozo, 2014, Ruggeri and Lombrozo, 2015). These results reinforce the viewpoint described above: having the right set of hypotheses in mind, or being primed with the right level of category information seems to drive more efficient information search.

The behavioral distinction between constraint-seeking and hypothesis-scanning questions can also be studied from the perspective of normative models (Oaksford and Chater, 1994, Nelson, 2005, Tsividis et al., 2013). These models attempt to objectively define the “quality” of a question and to see how people’s choices compare (see below for a larger discussion). A number of recent studies have explored how children’s question asking compared to such models. For example, Nelson, Divjak, Gudmundsdottir, Martignon, and Meder (2014) found that 8–10 year-old children can search a familiar structured domain (people with varying gender, hair color, etc.) fairly efficiently, tending to ask about frequent real-world features that roughly bisected the search space (e.g., gender first). Likewise, Ruggeri, Lombrozo, Griffiths, and Xu (2015) found that children’s patterns of search decisions were well-explained in terms of expected information gain (EIG), one popular model from this class which is described below. Perhaps most importantly, these models are highly context sensitive. Rather than arguing that either constraint-seeking or hypothesis-scanning questions are universally “better,” these models take into account the current context including the learner’s prior belief and the past evidence that has been revealed. This allows much more fine grained predictions. For example, on a given trial a hypothesis-scanning question might be equally informative compared to a constraint-seeking question (e.g., when only two hypotheses remain). In our study we will analyze children’s question asking with respect to these models to allow an objective measurement of the quality of their information seeking behavior.

While it is clear that there are developmental changes in how children formulate questions, less work has considered developmental changes in how children make use of the new evidence that their questions reveal (but see Denison, Reed, & Xu, 2013). However, there are many reasons to think that these two behaviors might be deeply entwined. The active inquiry loop in Fig. 1 suggests one obvious interaction because if questions or information gathering actions are made on the basis of current beliefs, and those beliefs are wrong, then a query may not have the expected effects (c.f., research on the hot stove effect, Denrell and March, 2001, Rich and Gureckis, 2015). There are certainly many examples where scientific progress has been derailed by incorrect interpretation of evidence, as in the case of experiments thought to support the theory of spontaneous generation of life (Needham, 1745).

Coenen and Gureckis (2015) describe a more fundamental reason for why belief updating and information search might be related. In particular, they focus on a popular computational model of active inquiry called Expected Information Gain (EIG). As mentioned above, this model has been widely used in both the adult and developmental literature to understand how people decide between different queries (Oaksford and Chater, 1994, Coenen et al., 2014, Gureckis and Markant, 2009, Nelson, 2005, Nelson et al., 2014, Markant and Gureckis, 2012, Ruggeri et al., 2015, Steyvers et al., 2003). Intuitively, EIG evaluates the quality of a question by considering how much is expected to be learned from each possible answer to that question. For example, in the constrained 20-questions game “Guess Who?”, a child might ask “Does your character have a hat?” or “Is your character male?”. To decide between these two queries EIG considers each possible answer (“yes” or “no” for each) and how much each answer would alter the learner’s current beliefs given the question. If all the remaining characters in the game were wearing hats then the answerer would never respond “no” to the hat question, and the received “yes” would not normatively alter the learner’s beliefs; no information would be gained by asking about hats. Even if one of the dozen remaining characters had a hat, asking about hats would have low EIG, since it would be unsurprising that the answer is “no”–only in one of twelve possible worlds does the hidden character happen to be wearing a hat, while in 11 of 12 worlds the character is not. In contrast, if half the remaining characters were male and half were female, then either answer to the gender question would strongly shift what the learner knows, eliminating half of the candidates (either the males, or the females). Thus, the more valuable question according to EIG would be “Is your character male?”. In this model, belief updating is fundamental to judging the information quality of a possible query: it is only by imagining how one’s beliefs would change given different answers that a question derives meaning and value. On the basis of this observation, Coenen and Gureckis (2015) reported a study aiming to relate individual differences in belief updating during a causal reasoning task to patterns of information seeking behaviors. Subjects that showed clear evidence of biased belief updating (e.g., incorrectly interpreting ambiguous evidence as unambiguous) also showed biased patterns of information gathering in a causal intervention learning task. This study highlights the strongly interactive nature of belief-updating and information seeking behaviors.

Interestingly, past work on the development of question asking abilities in children has tended not to emphasize belief updating as a dependent measure, or precluded studying updating beliefs by the design of the study. For example, Herwig (1982) presented children with a series of two-alternative forced choice decisions between hypothesis-scanning or constraint-seeking question but did not actually give feedback (and therefore could not detect errors in belief updating). In the 20-questions task of Nelson et al. (2014), 8- to 10-year-olds were asked to identify which of 18 people was the hidden target, and played the game to completion several times for different targets. Children eliminated hypotheses (flipping over cards) based on acquired evidence, but were given help by the experimenter if needed, which presumably means they were not allowed to make errors. In Ruggeri and Lombrozo (2014), the experimenters did not explicitly represent the hypothesis space for participants in Experiment 1’s causal reasoning task (e.g., “Why was a man late to work yesterday?”), and when ten explicit reasons for being late were given in Experiment 2, they remained in view. That is to say, the process of hypothesis updating was not scrutinized in these prior studies.

In the present study, we hypothesize that biases in the way children search for information (e.g., by favoring hypothesis scanning questions over constraint seeking questions) may stem from difficulties in coordinating the belief updating and search process. There are a variety of specific reasons for this prediction. First, although the components of the sensemaking model described in Fig. 1 above are sequential, they likely rely on a common pool of cognitive and attentional resources, and are thus not completely independent. At a minimum, learners have constant and limited capacities for working memory and reasoning during the task, and may come to avoid strategies that tax these resources if they run into difficulty during the course of the experiment. In this case we hypothesize that the cognitive load from planning questions, or from updating beliefs, may impair performance on either task. Second, hypothesis scanning questions might be easier for young children in that they produce evidence that applies to a single hypothesis. If instead children ask constraint-seeking questions, they must eliminate from the hypothesis space any possibilities that are ruled out by the new information. This process could be cognitively taxing, and also prone to errors. Thus, although constraint-seeking questions are often more informative in theory, we posit that they might not always be so to young children, particularly if children have difficulty using the obtained information to update their representation of the hypothesis space accurately.

To test this hypothesis, in the present study we manipulated whether children received assistance in integrating evidence with the hypothesis space or had to undertake this process on their own. Our expectation was that aiding children in coordinating evidence and beliefs would enable more sophisticated, and informative, inquiry behavior. To evaluate this prediction we evaluated the quality of children’s question asking ability against an objective standard of informativeness given by the EIG model described in more detail below. We additionally analyze our data specifically in terms of constraint-seeking and hypothesis-scanning questions. Our central prediction was that assistance in belief updating should increase the relative EIG of children’s questions and the relative utilization of constraint-seeking questions. Given that older children (8–10 years) have previously been found to use more constraint-seeking questions than younger children (5–7 years), we tested across these two age groups, expecting that younger children would benefit more from the assistance in hypothesis updating than would older children.

Section snippets

Experiment

The purpose of the experiment is to investigate how children utilize hypothesis- scanning and constraint-seeking questions when trying to discover a hidden object. To that end we created a tablet-based game based on the popular “Guess Who?” paradigm. The study was conducted in the context of a children’s science museum and the materials and design of the study were selected to integrate with museum content. Our hope was that insights from the study might be used to help museum curators design

General discussion

In the present study, we manipulated the support children were given while updating a hypothesis space during a self-directed learning task. After making a feature (or constraint-seeking) query, participants in the automatic update condition were shown which insects were effectively ruled out at the press of a button, whereas manual update participants were required to select the insects that were consistent with the feedback themselves.

In line with previous research (Mosher and Hornsby, 1966,

Acknowledgments

This paper is an extended and modified version of a paper that was presented at the Annual Conference of the Cognitive Science Society in 2016. This work was supported by the John Templeton Foundation “Varieties of Understanding” grant to TMG and MR and Grant No. BCS-1255538 from the National Science Foundation to TMG. We are grateful to Kathryn Yee, Aja Blanco, Christina Chu, and Talena Smith for data collection, and to Daniel Zeiger and the staff of the Discovery Room at the American Museum

References (36)

  • S. Denison et al.

    The emergence of probabilistic reasoning in very young infants: Evidence from 4.5 and 6-month-olds

    Developmental Psychology

    (2013)
  • J. Denrell et al.

    Adaptation as information restriction: The hot stove effect

    Organization Science

    (2001)
  • A. Gopnik et al.

    Causal learning mechanisms in very young children: Two, three, and four-year-olds infer causal relations from patterns of variation and covariation

    Developmental Psychology

    (2001)
  • T.M. Gureckis et al.

    Active learning strategies in a spatial concept learning game

  • H. Gweon et al.

    Stretching to learning: Ambiguous evidence and variability in preschoolers’ exploratory play

  • D. Klahr et al.

    The equivalence of learning paths in early science instruction: Effects of direct instruction and discovery learning

    Psychological Science

    (2004)
  • G. Klein et al.

    Making sense of sensemaking i: A macrocognitive model

    IEEE Intelligent Systems

    (2006)
  • G. Klein et al.

    Making sense of sensemaking ii: Alternative perspectives

    IEEE Intelligent Systems

    (2006)
  • Cited by (22)

    • Do curious students learn more science in an immersive virtual reality environment? Exploring the impact of advance organizers and epistemic curiosity

      2022, Computers and Education
      Citation Excerpt :

      We have observed that the participants in the current study were younger than those of the aforementioned studies. Research has shown that younger participants could be easily distracted by the VR features (Selzer et al., 2019) and they could also still be developing the various cognitive abilities for active learning, such as decision making or exploration, and metacognitive abilities (Kachergis et al., 2017). It is possible that the students had not developed the required meta-cognitive abilities that lead to effective learning in IVR.

    • On searching and finding: The development of information search abilities

      2021, Cognitive Development
      Citation Excerpt :

      Given the importance of information search behaviors for mastering everyday challenges as well as for cognitive development and maturing, it is not surprising that they are addressed in many different fields of research. Information search behavior is studied, for example, in the context of problem-solving (e.g., Kachergis, Rhodes, & Gureckis, 2017), memorization (e.g., Miller & Weiss, 1981), and decision making (e.g., Mata, von Helversen, & Rieskamp, 2011). For these cognitive tasks, there is a long tradition in handling information search as an important component part.

    • Explanation-seeking curiosity in childhood

      2020, Current Opinion in Behavioral Sciences
      Citation Excerpt :

      With increasing proficiency with age, children preferred questions that targeted an individual explanation when that explanation was more likely than others, but chose questions that eliminated several candidate explanations when all were equally likely. Other studies using similar methods have also shown age-related improvements in question-asking efficiency [96–98, but see Ref. 99], and that question efficiency can be influenced by certain structural supports [96]. To summarize, children use both causal intervention and question asking to seek explanations, with both strategies improving in efficiency over the preschool and early elementary school years, and with the former continuing to develop through adolescence.

    • Stepwise versus globally optimal search in children and adults

      2019, Cognition
      Citation Excerpt :

      whereas an example of a constraint-seeking question is “Is the target monster green?” Because this qualitative distinction does not fully capture the varying usefulness of questions, recent research has used stepwise information gain as a graded measure for questions’ informativeness and as a benchmark for evaluating human search behavior (Kachergis, Rhodes, & Gureckis, 2017; Nelson et al., 2014; Ruggeri et al., 2016; Ruggeri et al., 2017). The close relationship between splithalfiness and stepwise information gain provides a rational basis for the split-half heuristic (Nelson et al., 2014), which characterizes much of older children’s and adults’ behavior when selecting among given questions in the 20-questions game (Denney & Denney, 1973; Eimas, 1970; Nelson et al., 2014; Siegler, 1977; Thornton, 1982).

    View all citing articles on Scopus
    View full text