Abstract

Introduction

Some aspects of number knowledge involve an awareness of the meaning of somewhat arbitrary symbols (such as Arabic numerals and number words) that are used on a daily basis. This knowledge is an important predictor of later mathematics achievement (Rousselle and Noël, 2007; De Smedt et al., 2009; Krajewski and Schneider, 2009), which makes it a useful indicator of risk for mathematics learning difficulties (Mazzocco and Thompson, 2005; Jordan et al., 2007; Stock et al., 2010; Desoete et al., 2012). Although symbolic number skills begin to develop prior to schooling, they depend on instruction and typically become established in approximately first grade (Bugden and Ansari, 2011) to third grade (Girelli et al., 2000), at least for small whole numbers. Accordingly, early childhood educators' attention has been drawn to this aspect of “number sense” as a target of formal and informal learning and assessment. A challenge for educators is knowing what observable behaviors (such as counting or labeling sets) map on to important elements of number sense and how these behaviors are typically manifested in early childhood. In other words, educators may want to know what a weak number sense looks like, and which numerical behaviors reflect typical or atypical development.

One challenge in responding to this need lies in the limited delineation of number sense skills identified to date (Purpura and Lonigan, 2013), even within the subset of number skills classified as symbolic representations (e.g., verbal number words, written notation, physical number lines), which collectively differ from non-symbolic arrays (e.g., visual or audible sets). Measures of number sense often represent a conglomeration of numerical tasks that vary in the degree to which they overlap with each other and with non-numerical domain-general skills such as verbal memory, working memory, or spatial reasoning (e.g., Geary, 2004). Composite standardized test scores are useful for determining broad categories of mathematics difficulties, and extreme scores may also help differentiate between children with dyscalculia—a specific mathematical learning disability—and other sources of mathematics difficulties (Rubinsten and Henik, 2009). However, the dichotomous (pass/fail) nature of the item scores used to generate composites may fail to capture meaningful differences in mathematically relevant skills between individuals at a point in time when such differentiation can aid identification and instructional priorities. Indeed, broad mathematics achievement scores may underestimate the contributions of these early foundational skills (Geary et al., 2013).

In this retrospective longitudinal study, we focus on whole number knowledge in primary school as an example of foundational skills. We focus on behaviors that would be readily assessable in informal environments, and evaluate whether early indicators of atypical number concepts are associated with future computational fluency (as proposed also by Moeller et al., 2011). While recognizing that the number concepts we focus on are broad, we hypothesize that atypical errors on number knowledge tasks can meaningfully represent incomplete number concepts that persist throughout the school age years.

Accordingly, we propose that knowledge of such differences may be revealed through a qualitative analysis of responses to mathematics problems, with the goal of elucidating early number concepts that predict specific mathematics difficulties. We use this approach to assess aspects of performance failure rather than dichotomous pass/fail scoring, using frequency data from our completed longitudinal study to guide classification of typical and atypical errors that can then be evaluated as indicators of whole number concepts, and predictors of future math performance. The motivation for this approach is threefold: the aforementioned growing recognition that the number sense construct needs further delineation, the corresponding gaps in knowledge of developmental norms for fine grained numerical skills, and the high likelihood of behavioral differences in number skills given the heterogeneous nature of mathematical difficulties (Mazzocco, 2007; Rubinsten and Henik, 2009). We propose that the differences to emerge using this approach are likely to be meaningful indicators of later pervasive difficulties in specific areas of mathematics, because conceptual differences in number knowledge have been shown to persist well beyond the primary school age years (e.g., Mazzocco and Devlin, 2008; Geary et al., 2013).

A qualitative approach to assessing early number knowledge has both practical and theoretical significance. In practice, this approach is a complement to composite test scores for informal or formal assessments (e.g., differential diagnosis), and may be a more sensitive indicator of specific future mathematics outcomes. Theoretical contributions of qualitative error analyses provide for a more detailed understanding of developmental and individual differences in children's number sense and concepts. Although we do not claim that a qualitative approach is novel in research or assessment (e.g., Ginsburg, 2003), we do propose that it is an overlooked source of meaningful insights in the search for individual differences in number skills that do not necessarily conform to variation along a continuum. We use this approach to test the hypothesis that atypical errors that reflect numerical misconceptions in primary school are linked to aberrant responses observed late in middle school (Grade 8). This tests the broader notion that when early number concepts go awry, the consequences can persist.

Materials and methods

Results

Analyses related to whole number knowledge

Developmental trends and effects of mathematics ability group on total score

We ran preliminary analyses to verify anticipated effects of grade and math achievement group (TA, LA, or MLD) on overall accuracy, using a 3 (Group) × 2 (Grade) repeated measures ANOVA on the total number of correct responses (range = 0–6). Main effects were confirmed for Grade, F_{(1, 163)} = 60.90, p < 0.0001, η² = 0.272, with overall accuracy increasing from Grades 2 to 3 (Table ​(Table1);1); and for math achievement group, F_{(2, 163)} = 69.08, p < 0.0001, η² = 0.418. Children with MLD made fewer correct responses relative to the LA or TA groups, even at Grade 3, ps < 0.009. The small but significant Grade × Group interaction, F_{(2, 163)} = 7.96, p = 0.001, η² = 0.089, reflected larger increases in accuracy over time for the MLD group, likely due to ceiling effects.

Table 1

Mean (and SD) number of correct responses out of 6 on Written Numbers Task among children with TA, LA, or MLD.

Participant group	Grade 2	Grade 3
TA (n = 123)	5.29 (0.84)	5.76 (0.67)
LA (n = 26)	4.81 (1.17)	5.50 (0.71)
MLD (n = 17)	2.59 (1.80)	4.18 (1.78)
All groups (N = 166)	4.93 (1.30)	5.56 (0.97)

Open in a separate window

These analyses of how many errors children made on the Written Numbers Task reveal normal developmental trends in accuracy and quantitative differences in mathematics performance across mathematics achievement groups. Figure ​Figure11 further illustrates developmental trends across easier-to-harder items (that is, one-, two-, and three-digit numbers) and the exaggeration of this effect in children with MLD. Whereas the effect of grade appears driven primarily by gains in knowledge of the largest three-digit number achieved between Grades 2 to 3, the main effect of group appears largely driven by the much larger proportion of children with MLD who do not make this shift at this time period.

Open in a separate window

Figure 1

The percentage of all children in the study who correctly or incorrectly responded to each of six items on the Written Numbers Task. This performance summary reveals developmental and group differences from Grades 2 to 3 among children with typical achievement (TA) or low achievement (LA) in mathematics or mathematical learning disability (MLD).

Are these qualitative group differences significant? In both grades, many children with MLD failed even the 2-digit item(s), whereas most children with LA or TA did not. Very few children failed to identify the smallest one-digit number as either 1 or 0, but of the five that did fail, 4 had MLD (vs. 1 of 126 with TA, Fisher's Exact p < 0.01; and 0 of 26 with LA, Fisher's Exact p = 0.055). This pattern of performance veered from the more typical developmental pattern revealed by the data, and justified the following qualitative analyses.

Qualitative analyses of written numbers task errors

Does the type of errors made vary across math achievement group? To address this question, it was necessary to classify error types. Our a priori hypotheses focused on developmentally appropriate vs. idiosyncratic responses, which we operationalized in terms of frequency of errors made in Grade 2 across all trials and all students (210 in the study, plus 14 second graders excluded from the study due to missing data in Grade 3). Of the 1344 individual responses generated by these 224 second graders, 1068 were correct and 276 were errors. (Criteria for correct responses appear in the Methods section and in Table ​Table22).

Table 2

Classification (and counts) of 224 second graders' responses showing correct responses, frequent errors, and a sample of infrequent errors^* on individual Written Numbers Task items.

Response type	One digit				Two digit				Three digit
	Smallest		Largest		Smallest		Largest		Smallest		Largest
Correct	0	(118)	9	(200)	10	(192)	99	(180)	100	(196)	999	(87)
	1	(95)
All frequent errors		(0)		(0)	11	(9)	90	(12)		(0)	199	(20)
											900	(46)
Infrequent errors
Representative infrequent errors	2	(4)	2	(2)	19	(1)	23	(1)	102	(1)	102	(1)
	4	(1)	3	(2)	22	(2)	58	(2)	104	(1)	236	(1)
	6	(1)	12	(1)	51	(1)	89	(1)	109	(2)	308	(1)
									110	(2)	590	(1)
									197	(1)	653	(1)
									532	(1)	800	(1)
									910	(1)	901	(3)
Total infrequent errors made:		(11)		(22)		(23)		(31)		(25)		(67)
“Don't know”		(0)		(2)				(1)		(2)		(3)
No response										(1)		(1)

Open in a separate window

^*By definition, frequent errors were those made by >3% of all participants (≥7/224 participants), and infrequent errors were those made by fewer than 3% (<7) of all participants. Responses of “I don't know” and non-responses were considered errors but were not classified as either frequent or infrequent.

Errors were categorized as frequent or infrequent. A frequent error was produced by more than 3% (≥7) of all second graders in the study. There were four errors classified as frequent, which collectively occurred 87 times across trials and were made by 82 children. The mean number of children making any of the four frequent errors was 21.75 (range = 9–46). Thus, by definition, each frequent error was made by several children.

An error was categorized as infrequent if it was produced by fewer than 3% (<7) of all second graders in the study. Across all trials, 111 unique errors were classified as infrequent, which collectively occurred 179 times and were made by 81 children. The mean number of children making one of the 111 specific infrequent errors was 1.61, (range = 1–6). On each trial, the mean number of children who made any given infrequent error ranged from 1.32 to 2.75. Thus, by definition, infrequent errors were quite idiosyncratic in that each was made by very few children. Table ​Table22 includes a summary of responses observed among all 224 second graders enrolled in the larger longitudinal study.

Very few responses (10 of 1334) were reports of “I don't know” (n = 8) or no response at all (n = 2), collectively made by five children (four of whom also made infrequent errors). “I don't know” responses were neither frequent nor idiosyncratic, so they were omitted from comparisons of frequent vs. infrequent responses, which led to the exclusion of data from one child whose “I don't know” error was his only error.

Do children with MLD make significantly more infrequent responses?

We evaluated the number of infrequent errors made using a 2 (Grade) × 3 (Mathematics Achievement Group) ANOVA with only the 108 children who met criteria for MLD, LA, or TA, were tested in both Grades 2 and 3, and made at least one error in either grade (excluding the child whose only error was an “I don't know” response). There were main effects of Grade, F_{(1, 105)} = 34.68, p < 0.0001, η² = 0.248, and Group, F_{(2, 105)} = 32.55, p < 0.0001, η² = 0.383. The number of infrequent errors decreased from Grades 2 to 3 (from 1.45 to 0.62); across grades, children with MLD made more infrequent errors (2.25) than did children with either LA (0.52) or TA (0.32), ps < 0.0001 (the latter two frequencies did not differ from each other, p = 0.355). There was a small but significant Grade × Group interaction, F_{(2, 105)} = 2.78, p < 0.03, η² = 0.068, reflected in Figure ​Figure2.2. Although the proportion of children making an infrequent error decreased from Grades 2 to 3, most children with MLD still did so in Grade 3. In fact, the number of infrequent errors made in Grade 3 was significantly different from zero for the MLD group only, t₍₁₅₎ = 3.50, p < 0.01.

Open in a separate window

Figure 2

The percentage of error types made by children who made errors on the Written Number Task at Grades 2 and/or 3, reported separately for children with typical achievement (TA) or low achievement (LA) in mathematics or mathematical learning disability (MLD). Repeated infrequent errors indicate making an infrequent error during both years of the study, although not necessarily the same infrequent error.

Perhaps the higher incidence of infrequent errors in the MLD group merely reflects greater errors of any kind among this group. If so, then their frequent errors should also be more prevalent. We repeated the previous 2 × 3 ANOVA, this time with the number of frequent errors as the outcome variable. Neither main effect emerged as statistically significant (ps > 0.15). Although the MLD group had the fewest frequent errors of any group (TA = 0.49; LA = 0.52; MLD = 0.31), these differences were not significant (ps > 0.3). Thus, we conclude that the performance of the MLD group is characterized by infrequent errors rather than simply more errors.

The validity of infrequent errors as indicators of atypical number concepts

Infrequent errors were most common among children with MLD (vs. LA or TA), but some children in each group made infrequent errors, and many of these children repeatedly made infrequent errors in Grades 2 and 3. Our subsequent analyses focused on whether children made these errors regardless of mathematics ability group.

Do infrequent errors merely reflect wild guesses? We examined metacognitive evaluation measures included in the longitudinal study test battery (Garrett et al., 2006) in which children were prompted to report if they were “sure” or “not sure” of their individual response. These prompts were administered after each trial of the Written Numbers Task. Of interest was whether children who made infrequent errors were more likely to indicate uncertainty (i.e., to state that they were “not sure” of their response), relative to children who made frequent errors, and whether this difference was limited to instances of infrequent errors.

Using data for the trial on which infrequent responses were most common (Trial 6, the “largest three-digit number”), we carried out two contingency table analyses, one per grade. We found that nearly half (48%) of the participants who made infrequent errors were confident in their incorrect responses in Grade 2 (as indicated by their report of being “sure.”), and that this rate rose to 60% among the smaller cohort that made infrequent errors in Grade 3. Although the rates were lower than the rate among respondents making frequent errors in Grade 2 (48 vs. 73%; Fisher's exact p = 0.016) this was not the case at Grade 3 (60 vs. 70%; p = 0.166), perhaps due to sample size. Nevertheless, the findings demonstrate that infrequent errors were not always merely guesses, especially in third grade, when children making these errors were more likely to be certain of their response than uncertain.

Discussion

In this retrospective study based on secondary analyses, we focused on qualitative aspects of children's early number concepts. The motivation for this study stemmed, in part, from our observations over time of the persistent difficulties some children in our longitudinal study displayed, on relatively basic arithmetic skills, from kindergarten through Grade 8. The findings show how early (and easily assessed) indicators of number skills predict later performance, thereby validating those skills as potential screening points. These findings also illustrate the value of evaluating MLD based upon specific skills rather than composite performance (e.g., Butterworth, 2005), and the implications of individual differences for persistent mathematics difficulties. Specifically, we show that the occurrence and nature of atypical number concepts at Grade 3 are associated with accuracy and types of errors made on mental calculations at Grade 8. Finally, our findings support claims of qualitative differences in early number skills between children with vs. without MLD (e.g., Mazzocco et al., 2011), although claims counter to this notion have also been supported (e.g., Landerl and Kölle, 2009). We believe these findings have practical and theoretical value, despite their preliminary nature.

We examined the “smallest” and “largest” numbers children generated in second and third grade, a time by which we would expect children's single digit whole number knowledge to be well-mastered, their associations between symbolic numbers and meanings to be automatized (e.g., Girelli et al., 2000), and their familiarity with numbers to apply to 2- and 3-digit numbers. Several interesting patterns emerged from this study related to developmental and individual differences in performance accuracy on this task and to the implications of these differences for future computational fluency. First, we found anticipated developmental trends in how accurately children generated two- and three-digit numbers from Grades 2 to 3. The pattern of heightened accuracy in two- and, later, three-digit numbers parallels findings that two-digit processing is not automatically generalized to three-digit number processing (Mann et al., 2012). Next, and as anticipated, these developmental differences seen in overall accuracy were somewhat exaggerated in children with low achievement, and remarkably exaggerated among children with MLD, who continued to make errors at a much higher rate through Grade 3 (Figure ​(Figure11 depicts both the developmental trends and the group differences). More importantly, interesting individual differences were observed in the nature of children's errors when generating two- and three-digit numbers.

Not all errors on the Written Numbers Task were equivalent. Some implicated developmental trends that have not been reported previously, but which align with recent evidence of a hundreds-place focus when processing three-digit numbers, at least among children in this age group (Mann et al., 2012). For instance, the most frequent incorrect response to prompts for the largest three-digit number was “900,” which was reported by 46 of the 224 s. graders. The number “90” was the only frequent error to prompts for the largest two-digit number (although made by only 12 s graders). Frequent errors were less common in children with MLD, despite the fact that children with MLD made more errors overall. Only one child with MLD reported that “900” was the largest three-digit number, and yet (as seen in Figure ​Figure1),1), most children with MLD made errors on this trial. Although only a small group of children repeatedly made idiosyncratic errors (that is, in both grades), this was a characteristic of most of the children with MLD (Figure ​(Figure2).2). Note, however, that children rarely reported the same idiosyncratic answer in both years.

Beyond the mere emergence of these idiosyncratic errors, of particular interest is the finding that such errors observed at Grade 3 were associated with specific and unusual errors on mathematics computation 5 years later. On the one hand, these results support the notion that idiosyncratic number concepts in early childhood are a meaningful reflection of persistent number concept anomalies which may affect the foundation for later arithmetic computation (as proposed by Mann et al., 2012). On the other hand, our interpretation is far from definitive given the sample size and other limitations associated with our retrospective analysis of longitudinal data collected for a prospective study.

It is possible that the primary predictor variable that we explored here—infrequent errors in written whole numbers—is simply a repackaging of the MLD criteria used in the study. Based on these criteria, children with MLD make more infrequent errors than their primary school peers on the Written Numbers Task, they make more place value errors than their 8th grade peers (Mazzocco et al., 2008), and they are generally less accurate at evaluating their own math performance and thus produce ratings of confidence poorly calibrated with their performance (Garrett et al., 2006). Yet infrequent errors were also made by some (albeit, very few) children with LA or TA, and a few children with MLD did not make any infrequent errors. Additional support that MLD and infrequent error criteria are distinct (even if overlapping) predictors comes from the finding that infrequent errors at Grade 3 did not predict all types of Fast Math Test errors that occur with greater frequency among children with MLD.

Conclusions and future directions

In this retrospective study, we demonstrate how a qualitative error analysis of early symbolic number knowledge reveals potential sources of individual differences that may affect mathematics outcomes 6 years later. This means that misguided early number concepts may have long-term implications. Our goal was not to definitively identify core deficits of dyscalculia or MLD, but rather to illustrate the contribution of qualitative analysis to delineating meaningful aspects of number sense by focusing on one representation of number concepts.

Although we focused on qualitative analysis of errors, correct responses may also be revealing. For instance, most second graders (118 of 224) reported that the smallest 1 digit number was “0,” and fewer than half (95) reported this value was “1.” In contrast, among children with MLD who correctly answered this item, most responded “1” rather than “0.” Both answers are scored as correct, but each may represent different levels of understanding of these small numbers. At issue is how well responses such as these reflect the nature of children's fundamental number concepts.

Such qualitative analyses of responses, including errors, must be evaluated relative to developmental norms. For instance, over time, errors that were considered frequent vs. infrequent must be re-evaluated. In our longitudinal study, we continued to administer the Written Numbers Task to children who failed any of the six trials in a given year, so we were able to discover that of the seven children who continued to err on trial 6 during Grade 5, all answered “900.” [Some children continued to err on this item in Grade 6 (n = 6), Grade 7 (n = 3), and even Grade 8 (n = 1)]. Whereas at Grade 3 this response was categorized as a frequent error, it became infrequent by Grade 5.

Our observations underscore recommendations for the use of thoughtful questioning in mathematics assessments and teaching (Ginsburg, 2003) and when seeking to differentiate delayed vs. deficient mathematics skills (Rubinsten and Henik, 2009), especially when compensatory mechanisms mask otherwise aberrant numerical processing (Murphy and Mazzocco, 2008). Teachers can remain attentive for atypical errors on an informal basis as a source of information used to guide their online and systematic decision-making about students' individual learning needs or difficulties. Information about where trouble may occur down the line increases the specificity of targeted interventions. Since early basic number knowledge deficits can persist throughout the school age years, we must be mindful that their manifestation varies with development, and that the inclusion of specific qualitative assessments of symbolic number knowledge in primary school can provide greater specificity of the types of difficulties likely to emerge among students at risk for poor mathematics outcomes.

Author contributions

Michèle M. M. Mazzocco conceived and designed the study; Michèle M. M. Mazzocco and Melissa M. Murphy carried out the study; Michèle M. M. Mazzocco and Ethan C. Brown analyzed data; Michèle M. M. Mazzocco, Melissa M. Murphy, and Ethan C. Brown wrote the paper. All authors contributed to editing and reviewing the paper.

Acknowledgments

References