MISSIVES Computer-Aided Argument Mapping as a Tool for Teaching Critical Thinking IJLM Martin Davies University of Melbourne/Federation University Australia wmdavies@unimelb.edu.au Visit IJLM.net doi:10.1162/IJLM_a_00106 c© 2014 by the Massachusetts Institute of Technology Published under Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 Unported license Volume 4, Number 3–4 Introduction As individuals we often face complex issues about which we must weigh evidence and come to conclusions. Corporations also have to make decisions on the basis of strong and compelling arguments. Legal practitioners, compelled by arguments for or against a proposition and underpinned by the weight of evidence, are often required to make judgments that affect the lives of others. Medical doctors face similar decisions. Governments make purchasing decisions- for example, for expensive military equipment-or decisions in the areas of public or foreign policy. These issues involve many arguments on all sides of difficult debates. These issues involve understanding the arguments of others and being able to make objections and provide rebuttals to objections. Students in universities deal with arguments all the time. A major purpose of a university education- regardless of subject matter-is to teach students how to read, understand, and respond to complex arguments. The ability to do this makes for highly employable, adaptable, and reflectively critical individuals. We often call the skill of marshaling arguments and assessing them "critical thinking." All universities claim to instill the skill of critical thinking in their graduates and routinely note this in their advertising and promotional documents. Critical Thinking in Universities: The Problem Obtaining well-grounded skills in critical thinking is surprisingly difficult. The human brain simply cannot encompass at one time all aspects of a complex debate. The truth is that we can only ever focus on a small number of things at once. Generally we focus on those few aspects of the debate that are Davies / Computer-Aided Argument Mapping 79 MISSIVES particularly salient to us-usually because they support our prior opinion-and neglect the rest. Research has confirmed what teachers and lecturers already recognize: most people have seriously inadequate critical thinking skills. In an extensive study of students and the general public, Kuhn (1991) found that people formed opinions readily and held to them strongly but that the majority could not provide any genuine evidence or arguments for these beliefs. Further, they did not realize this was a problem. Students are expected to develop critical thinking skills during their years of undergraduate education. However, one review concluded that in attending four years of American university the average student's gain in critical thinking abilities is only somewhere between 0.5 and 0.65 standard deviations (SD) (Hitchcock 2003), or about 0.08 SD per semester on average-and some evidence suggests that much of this is due simply to maturation. Research on the effectiveness of critical thinking courses is perhaps even more worrying. McMillan (1987), Halpern (2002), and van Gelder, Bissett, and Cumming (2004) have all found evidence that conventional critical thinking instruction does not result in gains beyond those due to maturation and university education. In 2000, a leading theorist of critical thinking wrote, "I wish I could say that I had a method or technique that has proved successful. But I do not, and from what I can see, especially by looking at the abundance of textbooks on critical thinking, I don't think anyone else has solved this problem either" (Walton 2000, p. 36). Despite the claims by universities in their promotional documents, many students leave schools and universities "unable to understand, evaluate, or write arguments" (A. Larson, Britt, and Kurby 2009, p. 340). One study, involving 57 native-English-speaking students, found that, without a tutorial on the generic skills of argumentation, college students "frequently failed to distinguish acceptable arguments from structurally flawed arguments" (A. Larson, Britt, and Kurby 2009, p. 358). Another study, involving 76 nativeEnglish-speaking tertiary students, found that students are "not skilled at identifying key elements of an argumentative text" and "were not proficient comprehenders of natural, written arguments" (M. Larson, Britt, and Larson 2004, pp. 205, 220). Only 30 percent of all participants could identify and distinguish between claims (assertions) and reasons in a text. Most selected reasons that could not support the claims being made and mistakenly identified counterclaims as main claims. In a major recent study, Arum and Roksa found, With a large sample of more than 2,300 students, we observe no statistically significant gains in critical thinking, complex reasoning, and writing skills for at least 45 percent of the students in our study. An astounding proportion of students are progressing through higher education today without measurable gains in general skills as assessed by the CLA [Collegiate Learning Assessment]. While they may be acquiring subject-specific knowledge or greater self-awareness on their journeys through college, many students are not improving their skills in critical thinking, complex reasoning, and writing. (Arum and Roksa 2011, p. 36) At the same time, critical thinking skills are being seen as increasingly important for employment. Noting recent survey data from CareerBuilder and O*Net-the U.S. clearinghouse of occupational information-Forbes identified critical thinking as foremost in a list of top-ten skills that help people get hired (Casserly 2012). Among employers and employer groups is a growing recognition that what is needed in graduates is not so much technical knowledge but applied and transformative skills, especially skills in critical thinking (Gannet 2012). Moreover, these skills are also said to be important within companies themselves as drivers of employee comprehension and decisionmaking (Bednarz 2012). The demonstrated need in the U.S. workforce for critical thinking skills among graduates was made clear in a recent large-scale survey of 400 U.S. employers. The survey data were published in a major report by a consortium of U.S. organizations in 2006 (Casner-Lotto and Benner 2006). The skill employers ranked highest was "critical thinking," surpassing "innovation" and "application of information technology," and 92.1 percent of the surveyed employers regarded graduates from four-year colleges as being "deficient" in critical thinking. Paradoxically, while critical thinking skills are increasingly in demand in the workplace, some evidence suggests these skills are not being adequately taught to students in the academy. Part of the reason why critical thinking is hard to teach is because arguments are usually expressed in 80 International Journal of Learning and Media / Volume 4 / Number 3–4 MISSIVES Fig. 1 Argument map of Australians' attitude to government policy. prose. Prose is, by its very nature, open-textured and sometimes vague. The following argument is fairly clearly expressed (compared to complex, conceptually difficult arguments in the disciplines), but to distinguish its premises and conclusion still takes effort. This is partly because of the surrounding text that masks the argument and partly because of the lexical density of the prose itself. It is fairly clear that Australians are not-as some might suggest they are-opposed to government policy, for if they were they would be more critical of the government than they are at present. However, there is no evidence of this; there is no evidence that ordinary Australians are protesting against their elected representatives. The case of the Vietnam War was, of course, different. Then there were mass protests, demonstrations and marching in the streets which indicated a clear opposition to government policy. Compare this to the representation of the same argument in figure 1. For anyone familiar with the basic conventions, this graphical representation is far easier to interpret correctly. Academic discourse compounds the problem of lexical density and the difficulty of ascertaining (from the surrounding verbiage) an author's contention and the reasons leading to it. Take the following authentic example: The move from a structuralist account in which capital is understood to structure social relations in relatively homologous ways to a view of hegemony in which power relations are subject to repetition, convergence, and rearticulation brought the question of temporality into the thinking of structure, and marked a shift from a form of Althusserian theory that takes structural totalities as theoretical objects to one in which the insights into the contingent possibility of structure inaugurate a renewed conception of hegemony as bound up with the contingent sites and strategies of the rearticulation of power.1 Even if this example had a clear argument (that it does is not obvious), it would be exceedingly difficult to "map" as reasons leading to a contention. In reading discipline-specific texts such as this, we assume students will, nonetheless, process complex arguments in prose without being taught to do so, and we assume they will "pick up" critical thinking skills somehow by "osmosis." As the evidence shows, this is clearly not happening. And perhaps this should not be surprising. Critical thinking is hard, even in the case of simple, short, clearly expressed arguments. With difficult texts and long, complex passages of prose it is virtually impossible. So what can be done? Computer-Aided Argument Mapping Argument maps are visual tools that help in understanding, assessing, and evaluating arguments. They differ from "mind maps" and "concept maps" in the level of detail and the degree of precision. Mind maps are largely concerned with associative relationships; concept maps with relational connections; and argument maps with inferential connections (Davies 2011). Argument mapping involves clearly outlining a contention at the top of a map, followed by tiers of reasons and objections. Argument mapping can be augmented with computer-aided argument mapping (CAAM) software that aids the mapping process. Argument mapping can be used in any discipline because every discipline deals with arguments. An example argument map from the discipline of finance is shown in figure 2. This example includes the evidential basis for the terminal premises, which is possible using dedicated software. Web links, snippets of video, and multimedia sources can be easily included in argument Davies / Computer-Aided Argument Mapping 81 MISSIVES Fig. 2 Example of an argument map from the field of finance. Source: http://www.austhink.com. maps, making it a technology likely to appeal to the contemporary student. Empirical Support for CAAM The use of argument mapping tools in enhancing, retaining, and improving knowledge has empirical support. Evidence from the cognitive sciences shows that visual displays do enhance learning (Winn 1991; Vekiri 2002). Argument maps allow the separate encoding of information in memory in visual as well as propositional form, a phenomenon called "conjoint retention" or "dual coding" (Paivio 1983; Kulhavy, Lee, and Caterino 1985; Schwartz 1988). In simple terms, processing information verbally as well as pictorially helps learning by virtue of using more than one modality. This obviates the objection that some learners navigate space best in verbal rather than visual form. Argument mapping employs both verbal and visual modalities. This makes it a method of learning that appeals to both kinds of learner. Map-making also helps us to avoid the problem of "cognitive overload" (van Gelder, Bissett, and Cumming 2004). This is intuitively plausible outside of the educational context as well. Faced with having to travel, for example, from St. Paul's Cathedral to the Museum of London, most people would find that information on a map is easier to process than a long list of complex verbal or written descriptions. The human brain cannot easily cope with excessive information, and that is why maps are useful. Yet, oddly, despite the fact that arguments (i.e., propositions for and against contentions) are the common currency of 82 International Journal of Learning and Media / Volume 4 / Number 3–4 MISSIVES academic discussion in all disciplines, argument maps are seldom used in higher education in presenting complex information. A number of published studies demonstrate empirical support for the use of CAAM in improving critical thinking skills. One study showed that onesubject CAAM-based interventions over a 12-week period achieved a gain in critical thinking abilities of 0.8 SD as measured by preand post-test results in the California Critical Thinking Skills Test (CCTST). This is roughly similar to a shift from the 50th to the 79th percentile and is equivalent to the gains usually achieved in three or four years of an undergraduate degree (van Gelder, Bissett, and Cumming 2004). This CAAM intervention was extensive and involved weekly expert-led lectures on critical thinking and weekly class-based activities involving argument mapping in tutorials. Similar improvements have been found in other studies (Twardy 2004; Butchart et al. 2009). In Twardy's study, the gains were 90 percent of those in van Gelder and colleagues' study under the same conditions, even after controlling for the "teacher effect" and substituting an "inexperienced postdoc" (one who was unused to argument mapping) to teach the class. In the Monash study, the influence of computer-based feedback in an argument mapping class was trialled over a semester involving weekly 30– 40 minute class-based tutorials with regular exercises involving CAAM. The study found a CCTST effect size of 0.45 SD and a 14 percent average improvement in critical thinking skills. Harrell has also found strong gains, especially in students who start out weaker in critical thinking (Harrell 2011). In another study, as little as one hour of instruction in CAAM appeared to yield a modest improvement in critical thinking skills (Carrington et al. 2011). (For a detailed summary of these studies, see Davies 2012, 2013.) Van Gelder recently conducted a meta-analysis of all extant published studies on argument mapping and concluded that (1) argument mapping appears to improve critical thinking; (2) it is more effective than other kinds of critical thinking instruction; and (3) its benefits are dependent on the amount or intensity of argument mapping instruction received (van Gelder, forthcoming). Data emerging from these studies are uniform in showing an improvement in critical thinking skills in using CAAM; however, the effect size differs depending on the nature, type, and extent of the trials. Argument mapping can be taught effectively without computer assistance (i.e., with pen and paper) as Harrell (2008) found. The mapping software itself is incidental and merely assists in doing something that was previously done by hand. Using software, however, may enhance other aspects of learning, not least of which is student engagement. This is no small thing in the contemporary university, with technologically savvy students who are easily bored with conventional teaching techniques (i.e., lectures, or teacher-centered learning). Conclusion This paper suggests sound reasons for making argument mapping a more established part of the curriculum. Argument mapping need not involve the use of computer software dedicated to the purpose, though the use of such software can make for more engaging and interesting classroom activities. Assessment regimes might-as a matter of course-include mapping of arguments (preparatory to a major piece of written work, for example). The inclusion of argument mapping in this way has few apparent disadvantages and many possible advantages. One is that it might allow lecturers to assess students' arguments prior to submission, help students to refine their critical thinking skills, and assist students in producing better, more well-argued written work. Given the documented advantages of argument mapping-and the apparent failure of tertiary institutions to impart critical thinking skills-its inclusion in the curriculum seems to be an appropriate course of action for educational institutions in the 21st century. Note 1. This case of impenetrable text was given first prize in the "Bad Writing Contest" conducted by the journal Philosophy and Literature in 2008. The example is from Judith Butler, professor of rhetoric and comparative literature at the University of California (Dutton 2011). References Arum, R., and J. Roksa. 2011. Academically adrift: Limited learning on college campuses. Chicago: University of Chicago Press. Bednarz, T. F. 2012. Building critical thinking skills to enhance employee comprehension and decision making. Leaders to Leader [blog], April 19. http://blog.majoriumbusinesspress.com/2012/04/19/ building-critical-thinking-skills-to-enhance-employee -comprehension-and-decision-making/ (accessed April 16, 2014). Davies / Computer-Aided Argument Mapping 83 MISSIVES Butchart, S., D. Forster, I. Gold, J. Bigelow, K. Korb, and G. Oppy. 2009. Improving critical thinking using Web based argument mapping exercises with automated feedback. Australasian Journal of Educational Technology 25 (2):268–91. Carrington, M., R. Chen, M. Davies, J. Kaur, and B. Neville. 2011. Enhancing critical thinking: The effectiveness of a single intervention of computer-aided argument mapping in a marketing and a financial accounting subject. Higher Education Research & Development 30 (3):387–402. doi:10.1080/07294360.2011.559197. Casner-Lotto, J., and M. W. Benner. 2006. Are they ready to work? Employers' perspectives on the basic knowledge and applied skills of new entrants to the 21st century U.S. workforce. New York: The Conference Board, Corporate Voices for Working Families, the Partnership for 21st Century Skills, and the Society for Human Resource Management, 2006. http://www.p21.org/storage/documents/FINAL _REPORT_PDF09-29-06.pdf (accessed April 16, 2014). Casserly, M. 2012. The 10 skills that will get you hired in 2013. Forbes (December):10. http://www .forbes.com/sites/meghancasserly/2012/12/10/ the-10-skills-that-will-get-you-a-job-in-2013/ (accessed April 16, 2014). Davies, M. 2011. Mind mapping, concept mapping, argument mapping: What are the differences and do they matter? Higher Education 62 (3):279–301. doi:10.1007/s10734-010-9387-6. Davies, M. 2012. Computer-aided argument mapping and the teaching of critical thinking (part 1). Inquiry: Critical Thinking across the Disciplines 27 (2):15–30. Davies, M. 2013. Computer-aided argument mapping and the teaching of critical thinking (part 2). Inquiry: Critical Thinking across the Disciplines 27 (3):16–28. Dutton, D. 2011. The bad writing contest: Press releases, 1996–1998. http://www.denisdutton.com/bad _writing.htm (accessed April 16, 2014). Gannet, A. K. 2012. Learn to think if you want to get hired. USA Today, June 9. Halpern, D. F. 2002. Thought and knowledge: An introduction to critical thinking. 4th ed. Hillsdale, NJ: Lawrence Erlbaum Associates. Harrell, M. 2008. No computer program required: Even pencil-and-paper argument mapping improves critical thinking skills. Teaching Philosophy 31:351–74. doi:10.5840/teachphil200831437. Harrell, M. 2011. Argument diagramming and critical thinking in introductory philosophy. Higher Education Research & Development 30 (3):371–85. doi:10.1080/07294360.2010.502559. Hitchcock, D. 2003. The effectiveness of computer-assisted instruction in critical thinking. Paper presented at the Informal Logic: Proceedings of the Windsor Conference, Canada. Kuhn, D. 1991. The skills of argument. Cambridge, UK: Cambridge University Press. doi:10.1017/ CBO9780511571350. Kulhavy, R. W., J. B. Lee, and L. C. Caterino. 1985. Conjoint retention of maps and related discourse. Contemporary Educational Psychology 10:683–99. doi:10.1016/0361-476X(85)90003-7. Larson, M., M. A. Britt, and A. A. Larson. 2004. Disfluencies in comprehending argumentative texts. Reading Psychology 25 (3):205–24. doi:10.1080/02702710490489908. Larson, A. A., M. A. Britt, and C. A. Kurby. 2009. Improving students' evaluation of arguments. Journal of Experimental Education 77 (4):339–65. Medline:20174611, doi:10.3200/JEXE.77.4.339-366. McMillan, J. 1987. Enhancing college students' critical thinking: A review of the studies. Research in Higher Education 26:3–29. doi:10.1007/BF00991931. Paivio, A. 1983. The empirical case for dual coding. In Imagery, memory and cognition, ed. J. Yuille, 307–332. Hillsdale, NJ: Lawrence Erlbaum. Schwartz, N. H. 1988. Cognitive processing characteristics of maps: Implications for instruction. Educational Psychology Review 8:93–101. Twardy, C. 2004. Argument maps improve critical thinking. Teaching Philosophy 27 (2):95–116. doi:10.5840/teachphil200427213. van Gelder, T., and G. Cumming. Forthcoming. Impact of argument mapping on critical thinking skills. In Palgrave handbook on critical thinking in higher education, ed. M. Davies and R. Barnett. New York: Palgrave. van Gelder, T., M. Bissett, and G. Cumming. 2004. Enhancing expertise in informal reasoning. Canadian Journal of Experimental Psychology 58:142–52. Medline:15285604, doi:10.1037/h0085794. Vekiri, I. 2002. What is the value of graphical displays in learning? Educational Psychology Review 14 (3):261–312. doi:10.1023/A:1016064429161. Walton, D. 2000. Problems and useful techniques: My experiences in teaching courses in argumentation, informal logic and critical thinking. Informal Logic 20 (Teaching Supplement 2):35–9. Winn, W. 1991. Learning from maps and diagrams. Educational Psychology Review 3 (3):211–47. doi:10.1007/BF01320077. 84 International Journal of Learning and Media / Volume 4 / Number 3–