EVALUATION OF A STUDENT-ORIENTED LOGIC COURSE Aaron Thomas-Bolduc, Richard Zach Department of Philosophy Challenge Courses in formal logic are taught in most English-speaking philosophy departments. Yet very little effort seems to have been made to deliver these courses in a way that improves learning outcomes. A typical introductory course in formal logic still involves a standard lecture format, with tutorials, problem sets and closed book exams. Textbooks tend to be pricey ($100 and up). Logic courses often have a highly skewed gender split, and instructors face the issue of teaching to a wide range of student backgrounds and expectations. Grade distributions are lower than average for philosophy courses, and the DFW (D-F-withdraw) rate is high. Context Most scholarship on the instruction of elementary logic (Croy 2010; Geach 1979; Hedges 1999; Schiller 1913) has assumed the traditional lecture model. One notable exception is Butchart et al. (2009) who investigated the implementation of peer instruction. They found that peer instruction provided statistically significant increases in student satisfaction and learning outcomes as compared to the traditional lecture model of instruction. The meta-analysis in Macpherson (2016) reports that students who score highly for maths anxiety have worse learning outcomes than more confident students, and suggests that cooperative learning decreases maths anxiety and its effects. Cooperative learning is integral to both peer instruction and classroom flipping. New Logic Course In Winter 2017, the first author piloted a course in formal logic in which we aimed to (a) improve student engagement and mastery of the content, and (b) reduce maths anxiety and its negative effects on student outcomes, by adopting student oriented teaching including peer instruction and classroom flipping techniques. The course implemented a partially flipped approach, and incorporated group-work and peer learning elements, while retaining some of the traditional lecture format. By doing this, a wide variety of student learning preferences could be provided for. Open Textbook We revised and remixed a free and open source textbook, forall x (Magnus et al. 2017) which was used as the primary text. We evaluated student satisfaction using a survey instrument based on the Textbook Assessment and Usage Scale (Gurung and Martin 2011), and compared it to student satisfaction with the traditional textbooks in use in other sections of the course (Chellas 1997, Goldfarb 2013). The survey also asked about use patterns, e.g., how often students consulted the textbook, for what purpose the consulted the text, and how they interacted with it. (For forall x, n = 25; Chellas, n = 31; Goldfarb, n = 45.) The survey was approved by the Conjoint Faculties Research Ethics Board. Textbook Use 40% 8% 55% 60% 92% 45% 05. How often do you consult the textbook? 0 25 50 75 100 Goldfarb Chellas ForallX Percentage Response Never Rarely Once every couple of months Once or twice a month Once a week Several times a week Student Satisfaction 18% 8% 38% 39% 60% 19% 43% 32% 43% 25% 4% 43% 43% 52% 26% 32% 44% 31% 21% 12% 48% 36% 52% 24% 43% 36% 29% 11% 0% 36% 68% 76% 26% 21% 24% 38% 25% 0% 29% 54% 68% 36% 21% 32% 36% 43% 16% 52% 36% 52% 26% 21% 32% 21% 54% 29% 64% 25% 33% 12% 21% 38% 24% 36% 8% 54% 32% 64% 22% 32% 28% 24% 50% 12% 58% 29% 75% 19% 21% 12% 23% 23. Is the price of the textbook too high for the amount of learning support it provides? 18. How understandable/clear is the writing? 17. How engaging/interesting is the writing? 16. Do the explanations provide enough detail? 15. Is the text well−organized? 14. How adequate is the number of exercises/problems? 13. How well are examples used to explain the material? 12. How much do figures and diagrams help you understand the text? 11. How well placed are the figures in relation to the material they discuss? 100 50 0 50 100 Goldfarb Chellas ForallX Goldfarb Chellas ForallX Goldfarb Chellas ForallX Goldfarb Chellas ForallX Goldfarb Chellas ForallX Goldfarb Chellas ForallX Goldfarb Chellas ForallX Goldfarb Chellas ForallX Goldfarb Chellas ForallX Percentage Response Very bad Bad Neutral Good Very Good Partially Flipped Classroom The flipped portion of class involved providing short (5– 6 min) screencasts a few days before scheduled groupwork. The groupwork then covered material from the screencast as well as from the previous 2–3 lectures. Students were also expected to read the relevant portions of the textbook. The screencasts themselves were online videos with the instructor's scripted voice recorded over slides, covering central concepts and themes. The students thus learned a good deal of material via screencasts, peer learning during groupwork, and the textbook. Groupwork The students were put into groups of 5 or 6 based on their tutorial sessions, and upper level students were distributed as evenly as possible. Dividing the students by tutorial session allowed the assignments to be completed over the course of one class session plus one tutorial. This way, students always had access to the instructor or TA, and needn't get together outside of class times. Groupwork assignments generally had fewer, but difficult or open-ended questions. This was meant to encourage collaboration and discussion, as well as to expose the students to more advanced material when they were able to ask questions. Other Course Components Generally 1–2 classes per week were traditional lectures, though these were often broken up by other activities, some of which included brief group discussions. There were also 3 in-class tests, and 6 homework assignments. Homework was completed over one week, and generally consisted of many shorter questions. Finally, there was a participation component based primarily on groupwork, which included peer evaluation. D–F–Withdraw Rates The DFW rates for the 10 sections of Logic I in 2014/15 and 2015/16 ranged from 8% to 36.7%, with mean 23.4% and median 23.7%. The DFW rate for this course was 7.9%. This is in line with the two lowest rates (8%, 8.3%), but those were outliers, the next lowest being 18%. This suggests that our approach may help retain students. Limitations Validity of survey results and DFW rates may be limited by a number of factors. Differences in textbooks and instructor between sections were not controlled for, nor were methods of evaluation. This was possibility confounded by the fact that this was the instructor's first time teaching. Surveys were prone to selection bias. Timing of the course (Winter term, afternoons) may have provided an unrepresentative sample of Logic I students. Student Evaluation With CFREB approval, we added questions to the standard faculty teaching evaluation form to gauge student satisfaction with the revised format of the course (n = 55). 4% 2% 7% 16% 23% 18% 31% 35% 23% 91% 88% 82% 61% 50% 45% 36% 27% 25% 5% 11% 11% 23% 27% 38% 33% 38% 52% This course made me more likely to take another philosophy course This course made me more likely to take another logic course I learned best studying material on my own in the textbook I learned best when I simply followed lectures without a screencast before I learned best when I watched a screencast ahead of material covered in class Having lecture slides available electronically is helpful Being able to watch screen casts ahead of time has helped me prepare for class Collaborative work with fellow students has made the class more enjoyable In−class work in groups has improved my understanding of the material 100 50 0 50 100 Percentage Response Strongly Disagree Disagree Neutral Agree Strongly Agree Discussion Despite the mentioned limitations, the very positive responses to both the book and the course structure suggest that this method of teaching logic, and the chosen open textbook forall x, help improve learning outcomes and student retention. In the instructor's opinion, groupwork in particular was genuinely helpful for students. References S. Butchart, T. Handfield, G. Restall (2009). Teaching Philosophy, Logic and Critical Thinking using Peer Instruction. Teaching Philosophy 32:1-40. B. Chellas (1997). Elementary Formal Logic, Penny Lane. M. J. Croy (2010). Teaching the Practical Relevance of Propositional Logic. Teaching Philosophy 33:253-270. P. T. Geach (1979). On Teaching Logic. Philosophy 54:5-17. W. Goldfarb (2013). Deductive Logic. Hackett. R. A. R. Gurung, R. C. Martin (2011). Predicting Textbook Reading. Teaching of Psychology 38:22–28. S. Hedges (1999). Do Students Learn in My Logic Class? Teaching Philosophy 22:141-159. B. Macpherson (2016). Overcoming Instructor-Originated Math Anxiety in Philosophy Students. Metaphilosophy 47:122-146. P. D. Magnus et al. (2017). forall x: Calgary Remix. forallx.openlogicproject.org J. McGivney-Burellee, F. Xue (2013). Flipping Calculus. PRIMUS 23: 477–486. F. C. S. Schiller (1913). The Social Value of Logic Teaching. Hibbert Journal 12:192. Research supported by the Taylor Institute for Teaching and Learning through a Scholarship of Teaching and Learning Grant, and by the Department of Philosophy. Contact Information Aaron Thomas-Bolduc athomasbucalgary.ca