Paper ID #21899 Active Techniques Implemented in an Introductory Signal Processing Course to Help Students Achieve Higher Levels of Learning Dr. Saharnaz Baghdadchi, University of California, San Diego Saharnaz Baghdadchi is an Assistant Teaching Professor at UC San Diego. She is interested in scholarly teaching and uses active learning techniques to help students achieve expert-like level of thinking. She guides students in bridging the gap between facts and usable knowledge to solve complex engineering problems. Rebecca Anne Hardesty, University of California, San Diego Rebecca Hardesty is a PhD Candidate in Communication and Science Studies Mr. Paul Andreas Hadjipieris, University of California, San Diego Paul Hadjipieris is an instructional designer at the University of California San Diego. He holds an MA, in history from Edinboro University of Pennsylvania. His research agenda is on emerging educational technologies and their application to the classroom. He is deeply involved in SoTL research at the University of California San Diego and currently working with faculty on course design and manuscript construction. Dr. Jace Hargis, University of California, San Diego Dr. Jace Hargis currently assists faculty as the Director of the Center for Teaching at the University of California, San Diego. His prior positions include a Professor and Associate Provost in Hawaii; a College Director in Abu Dhabi, UAE; an Associate Professor and Assistant Provost in northern California; and an Assistant Professor and Director of Faculty Development in Florida. He has authored a textbook, an anthology and published over 130 academic articles as well as offered hundreds of academic presentations. He has earned a B.S. in Oceanography from Florida Institute of Technology; an M.S. in Environmental Engineering Sciences and a Ph.D. in Science Education from the University of Florida. Dr. Hargis' research agenda focuses on how people learn while integrating appropriate, relevant and meaningful instructional technologies. c American Society for Engineering Education, 2018 Active Techniques Implemented in an Introductory Signal Processing Course to Help Students Achieve Higher Levels of Learning Abstract Holding students to high standards and assessing, measuring and evaluating their learning with challenging, authentic problems in the midterm and final exams is the goal of the professors who teach core signal processing concepts. However, the heavy reliance of these subjects on mathematics makes it difficult for students to genuinely grasp the concepts and relate to a conceptual framework. Specifically, analyzing the signals and the functionality of systems in Fourier domain; separating the system level analysis from signal level analysis; and understanding how they are related in time domain and frequency domain are among the most challenging concepts. Students' lower grades observed over past years in the introductory signal processing course exposed a potential disconnect between the actual level of learning and the high expectations set by the professors. In this paper, we present the active learning techniques that we implemented in one of the summer session offerings of this course in our department. The research explored Peer Instruction, pre-class reading quizzes and post-lecture quizzes. In addition to the mid and end of the quarter survey results, the comparison analysis of the grades students achieved in the active learning integrated course in the second summer session and the standard course offered in first summer session is discussed. According to our results, the developed techniques helped students in the active classroom perform significantly better than their peers participating in standard lectures when tested by challenging questions in their exams. Keywords: Information Processing, Active Learning, Formative Assessment, Student Response Systems, Peer Instruction, Signal Processing, Electrical Engineering Introduction The study reported here was conducted at a large public research University in the United States. In our university, the culture of collaboration and the dedication to equipping the students with multidisciplinary tools promotes the discoveries that advance the society. Students taking Electrical and Computer Engineering (ECE 101), explore discrete time and continuous time signals and systems and study the properties of these systems. They continue the course by taking a closer look at the linear time-invariant (LTI) class of systems and learn to calculate the output of LTI systems using convolution and the system's impulse response. They explore developing the analysis and synthesis equations for continuous time, and discrete time Fourier Series and learn how to calculate the frequency response of Linear Time Invariant (LTI) systems. In the second half of the quarter, after learning about the Fourier Transform of continuous and discrete time signals, the analyze the effect of filters on the input signals in the Fourier domain. The course finishes after examining the sampling of bandlimited modulated signals and concepts of Laplace Transform. Due to the high mathematical complexity of the topics covered in this course, students in the past reported difficulties in connecting the concepts and using the equations they were taught in the lecture class in solving exam problems. To address these issues, three active learning techniques were selected and implemented in ECE 101 offered in Summer Session 2 in 2016. Peer Instruction, as the first technique, was added to the course by including several multiple choice questions in each lecture to promote student discussions in the lecture class. Students used i-Clickers to answer these questions. The second technique was the guided reading assignments for each lecture accompanied by pre-class reading quizzes that students were asked to take before coming to the lecture class. Each reading assignment included some guides to help the students better understand the assigned reading section. The intention of designing guided reading assignments was for the students to come to each class prepared and to initiate their learning process before coming to class. The third active learning technique used in the course was the guided post-lecture quizzes. These quizzes, assigned after each lecture, consisted of one or two problems related to the topics covered in each class. These problems had step by step instructions on how to solve them. They were used as small practice problems to study the course topics after each lecture further. This paper presents the study of the effect of these three active learning techniques on students' learning of the concepts covered in ECE 101, the introductory signal processing course. Literature Review Information Processing This study seeks to explore the impact active techniques had when implemented in an introductory signal processing course to help students achieve higher levels of learning. To review the extant scholarship in this area, Information Processing Theory will be applied as a theoretical framework to guide our understanding of the application and integration of active peer instruction, and its effect on student learning. Information processing focuses on how learners encode information, attend to environmental events, store new knowledge in memory, and retrieve as required (Schunk,1996). Information Processing Theory examines how new information that travels to the long-term Memory Store does so by connecting and relating to knowledge already stored in the short-term memory store. Richard Atkinson and Richard Shiffrin (1968) proposed that the Information Processing System is also referenced as the "two-store (dual-memory) system" (Hargis,1998) established that there were structural features of the memory system and these components are broken down into three sections: 1. The Sensory Register 2. The Short Term Store 3. The Long Term Store Incoming sensory information enters the sensory register and is present for a very short time before the decaying of that information occurs. At this point, the information is pushed from the sensory register into the short term memory store which is considered the "working memory". Schunk (1996) notes that for learning to take place and information to pass from the short term to the long term memory store (where information is stored in a relatively permanent state), there are three criterions that must be observed: behavioral change, or change in the capacity for behavior; the enduring change over time; and change due to practice or other forms of experience (Schunk, 1996). This knowledge acquisition involves the application of cognitive, linguistic, motor and social skills (Hargis, 1999). In respect to knowledge acquisition, instructional design of a curriculum that incorporates active learning has been shown to improve the learning and understanding of students. In a metaanalysis of 225 studies, it was found that students in classes with lecturing were 1.5 times more likely to fail than classes with active learning (Freeman, 2014). This active learning environment provides a forum for knowledge acquisition to take place, and to pass from short term to long term memory successfully. Marotta and Hargis, (2011) used flip camcorders for active classroom metacognitive reflection. The learning objectives were to increase attention, engagement and subsequent learning. This approach is supported by Albert Bandura (1977) who notes that there are four stages that can be attributed to the memory modeling process, which include (1) Attention; (2) Retention; (3) Reproduction; and (4) Motivation. Attention In the Marotta and Hargis, (2011) study, they discuss how the, "...flip camera engages their senses and holds their attention in a unique and enduring way." Bandura (1977) further discusses the importance of attention for information processing. If the incoming sensory information is multichromatic and marked, we pay more attention. Accordingly, they posed the question of how technology in an active classroom attends to information processing. The findings show that digital cameras, student response systems, and smart phones create the opportunity for students to engage deeply and hooks in their attention, which leads to deeper connections between working and long-term memory. Retention Use of a flip camera in a Civil Engineering class allowed students to watch and reflect upon their progress as they solved a number of problems on the whiteboard (Marotta & Hargis, 2011). This active process allowed, as Bandura (1977) suggests, the ability to remember and retain information that has been accessed by one's attention. Erica Gunn (2014) conducted a study that examined the efficacy of using iClickers (a student response system) to collect formative feedback teaching. Continual feedback leads to student engagement with the material and promotes an active learning environment (Gunn, 2014). This is supported by John Hattie's meta-study on effective size which notes that one of the most powerful of influences is feedback, with a positive effective size of 0.75 (Hattie, 2016). There is a confluence here with Bandura's work on retention in respect to the quick, visual, and personalized nature that is available from using iClickers that can lead to information be stored and brought up, reproduced, and acted upon by students. Gunn's study noted that iClickers were introduced primarily as a method of maintaining student engagement. (Gunn, 2014). Reproduction Bandura describes reproduction as the ability of the student to take the information that has been retained and translates this into action (Bandura,1977). Piaget (1974) discusses how confusion and disequilibrium can be a vital component of deep learning. He notes that when students enter a state of disequilibrium accessed through reproduction (Bandura,1977), two choices are available. The first being to work the new information into the extant plan or methodology. The second choice is to accommodate the new knowledge by replacing the old with the new. Cognitively, order is restored and a balance is returned, allowing information to be processed into students' long-term memory (Atkinson & Shiffrin, 1971). Choudhury and Hargis (2017) explore how formative assessment can identify real time student misconceptions during Bandura's (1974) reproduction phase. Research has shown that Student Response Systems (SRS) can increase engagement and participation (Choudhury & Hargis, 2017). As students are translating newly acquired knowledge into action, any confusion they experience can be identified quickly by the instructor through a "live feed indicator of 'confusion' that notifies a professor when students in their class are not following the material" (Choudhury & Hargis, 2017). This type of intervention by the instructor is made possible by the use of the Student Response System. Ruben Puentedura (2006) would see this as a redefinition of instruction, which without the technology would have been extremely difficult to recreate (Puentedura, 2006). A finding of the Choudhury & Hargis (2017) study is that the availability of a 'Confusion' button allows for students to learn in an interactive environment in which mistakes and confusion are encouraged, that in turn leads to a deeper path of learning and increased student motivation. Motivation The final component of Bandura's (1977) Memory Modelling System is motivation. To be able to model, recreate and reproduce knowledge successfully, students need to be motivated to learn (Hargis, 1999). Emily McIntosh (2017) explores how peer assisted study sessions engage and motivate students, and leads to improved retention. In addition, the development of problem solving and critical thinking skills improved student resilience (McIntosh, 2017). In respect to resilience and motivation, Claudia Muller and Caroll Dweck (1998) write that praising students' intelligence results in students being less likely to choose challenging work. (Dweck & Mueller, 1998) Furthermore, Lisa Blackwell (2007) supports Bandura's contention that students need to be motivated to learn (Bandura, 1977) Blackwell writes, 'Students with a Growth Mindset significantly outperform students with Fixed'(Blackwell, 2007). This is supported by the work of Barry Zimmerman, Albert Bandura and Manuel Martinez-Pons in their work on Self-Motivation for Academic Attainment (1992). Students that are self-regulated learners are able to attain their goals by setting challenging tasks for themselves (Zimmerman, Bandura & Manuel Martinez-Pons, 1992). Moving from the self-regulated learner to Peer Instruction, Catherine H. Crouch and Eric Mazura (2001) comment that students need to be actively involved and independent when involved in Peer Instruction. It is common that students ask why they are instructing each other when the instructor/expert is in the room. Accordingly, correct motivation and support from the instructor are required (Crouch & Mazura, 2001). Robert Beichner and Joseph Cevetello (2013) discuss how collaborative learning environments that support a hundred or more students can create a positive learning space that appears smaller and allows the opportunity for more intimate collaborative experiences. Accordingly, students are more engaged. The result is often greater student motivation and satisfaction in what they have been able to accomplish (Beichner & Cevetello, 2013). In 1992, Lawrence Carlson and Jacquelyn Sullivan created an integrated teaching and learning programs for undergraduate engineering students at the University of Colorado. The learning environment created allowed the integration of engineering theory with practice, which in turn supported active, collaborative learning to take place. The study notes that 80% of students who took this course during their first year have remained in engineering into their third year, a remarkably higher rate than the college's 55% average (Lawrence & Sullivan, 1999). Methods Setting This study was conducted at UC San Diego, a large public research University in the United States. Two sessions of the same course were offered in the first and second month of summer 2016. While the course offered in the first session was used as a control, the active learning techniques were implemented in the second course. Both courses had similar instruction schedule. They were offered at four days a week for 80 minutes per class. Participants The participants for this study included 51 students in summer session 1 (SS1), and 72 students in summer session 2 (SS2). Most participants were between the ages of 18-27 years. The detailed demographics of the participants in both courses is provided in Table1. Table 1. Number of students and their college year. Session Total Female Male 1st year 2nd year 3rd year 4th year Graduate Student 1 51 10 41 0 0 8 42 1 2 72 8 64 4 0 11 57 0 Data was collected during the summer session 1 and 2, 2016. The design is a pre/postassessment using achievement examinations as the assessment tool. The midterm and final grades of all SS1 and SS2 students were used in the study. In addition to the grades, the results of the mid-quarter and end-of-quarter surveys that 54 and 50 students from SS2, respectively participated in were used. About half of the summer session 1 was instructed by a professor who routinely teaches this course at this university and the other half was taught by the instructor of the second summer session. Both instructors in the summer session 1 (SS1), taught the course with the standard lecture-based teaching method. The lectures were delivered by explaining the concepts to the students in the classroom and writing the notes on the board. The students spent their time in the classroom listening to their instructor and writing their own notes from the material presented. In the other course, offered in summer session 2 (SS2), active learning techniques like daily guided reading assignments and their related quizzes, guided post-lecture quizzes and peer instruction in each class were implemented. All the course materials including the assignments and autograded quizzes were posted on the university's Learning Management System (LMS). The students were assigned a short reading section from their textbook with some guides that would draw students' attention to the points necessary to understand the topic. The content of these assignments were usually introductions to the topics discussed in each lecture. In addition, for some reading assignments, the students were asked to study the problem-solving process offered in the textbook for some examples with solutions. The students were provided with some extra guidance in addition to the instructions given in the textbook. Each reading assignment was followed by a pre-class reading quiz that tested their understanding of the topic covered in the reading assignment. The pre-class reading quizzes were directly related to the topic of the assignment and were intended to be easy questions to encourage the students to complete the reading assignment, further preparing them for the upcoming lecture and increase their confidence by letting them answer easier questions. The answers to the pre-class reading quizzes were shown to the students right after they submitted their response and briefly discussed by the instructor in the beginning of the following lecture. While the guided reading assignments continued throughout the course, the pre-class reading quizzes were posted only for the first half of the course and they were replaced by the guided post-lecture quizzes for the second half of the course. The guided post-lecture quizzes were assigned daily after each lecture. They were composed of problems related to the topics covered in the lecture of the day. Each problem had step by step instructions to guide the students in solving that problem. They were intended to provide a means for the students to review what they learned in the class and test their understanding of the material on their own. The answers for these quizzes were shown to the students after they submitted the test. To encourage peer instruction, several iClicker questions were designed for each lecture and were presented to the students at different times during the class. The iClicker questions were conceptually challenging problems. Answering these questions required the students to analyze the problem, find a way to use what they have learned from the lecturing part of the day, and for some questions, relating that day's topics to what they have learned in previous lectures. The choice of options for the multiple choice questions was based on the instructor's prior experience with conducting discussion classes and the points that students found confusing about the course topics during the past few years. The students were asked first to answer the multiple choice questions individually. Next, they were given some time to discuss their answers with their peers and explain to their partner why their answer was correct. Then, they all voted again. They were asked to select their original answer if their peers did not convince them that their answer was incorrect. This gave the instructor a tool to monitor how many of the students chose the wrong answer after the peer instruction section. Three surveys consisting of two end-of-quarter surveys and one mid-quarter survey taken by the SS2 students were used as an informal assessment tool. For the formal assessment, some conceptually challenging problems with the similar level of difficulty were included in the midterm and final exams in the SS1 and SS2 courses and the average grade that students in SS1 and SS2 received in these problems were compared. Results In this section, we present our findings and results. We include a presentation of our beta focus group, what we learned from it, and how it informed our official focus group. Sample This study provides a comparison between two offerings of the same five week summer engineering class, Course ECE 101, Linear Systems Fundamental. The researcher taught the iteration of the course offered in Summer Session 2 (SS2) and a different instructor taught the iteration offered in Summer Session 1 (SS1) Summer Session 1 (SS1) There were 51 students in SS1. The majority of enrolled students were engineering majors. Specifically 36 students (70.5%) were electrical engineering majors, eight were computer engineering majors (15.6%) through the engineering department and three were computer engineering (5.8%) through the computer science and engineering department. There were one (1.9%) of each of the following majors: engineering physics, aerospace engineering, mathematics, visual arts, and there was one graduate student in electronic circuits. 42 of the students were in their senior year of college (82.3%), 8 were juniors (15.6%), and there was one masters student. The average GPA (grade point average) of SS1 students was 2.86 out of 4. No students from SS1 retook the class in SS2. SS1 did not utilize the active learning techniques that the researcher used in SS2. Specifically, the instructor of SS1 did not use iClickers, pre-class reading quiz, nor post-lecture guided quizzes. Summer Session 2 (SS2) There were 72 students in SS2. As with SS1, the majority of students were engineering majors. Specifically 40 students were electrical engineering majors (55.5%), 15 were computer engineering majors through the engineering department [20.8%), eight were computer engineering majors through the computer science department (11.1%), four were international/4 year university students (5.5%), two students were engineering physics majors (2.7%), two were physics majors (2.7%), and one was a cognitive science majors (1.3%). 57 of the students were in their senior year of college (79.1%), 11 were juniors (15.2%), and four were freshman (5.5%). Two students dropped the class after the midterm. The average GPA of SS2 students was 2.96 out of 4. The researcher instructing SS2 used the following three active learning techniques: iClickers with peer instruction, pre-class pre-class reading quizzes, post-lecture guided quizzes. Midterm and Final Exams SS1 and SS2 offered a midterm and a final exam and gave students the same amount of time to complete them. Both classes gave the midterm at the same point in the term which was halfway through the third week of the five week course. The midterm for SS1 consisted of 16 questions and the midterm for SS2 had nine questions. The questions in the midterm exam of SS1 required shorter answers compared to the ones in the midterm exam of SS2. The final for SS1 had 12 questions and the final for SS2 was composed of 10 questions. In SS1 the midterm was worth 35% and the final exam was worth 55% of the students' final grades. In SS2 the midterm was worth 30% and the final was worth 50% of the students' class grades. Although both SS1 and SS2 were iterations of the same course, they did not offer the same exam. Instead, the instructors for both classes modeled the questions off of exams previous instructors used in ECE in previous years. Beta Focus Group To provide a brief summary, the purpose for the focus group was to determine whether the exams offered in SS1 and SS2 were similar in terms of difficulty. We decided on using a focus group of experts to address this question for two reasons: 1. We wanted to ensure that those participating in the focus group were familiar with the material, but were not current students due to the risk of varying degrees of competence. 2. We wanted to ensure that those participating had experience instructing engineering courses and had practice designing exam questions. To determine the similarity of the SS1 and SS2's exams in terms of difficulty, we first classified the questions from the four exams (two midterms and two finals) into five topical groups. These kinds of problems were tested on each exam in each iteration of the class. We then randomly sampled the questions in each group, selecting 23 questions in total. We asked several faculty members from the Engineering Department to use their expert opinions on the difficulty of each question without identifying which class or exam the questions originated. We would then ask the experts to discuss the questions to evaluate the difficulty of each question and come to a consensus, assigning each question a score of 1-5. 1 – Not at all difficult 2 – Not very difficult 3 – Somewhat difficult 4 – Moderately difficult 5 – Very difficult To develop the skill-set of researchers involved in this project who were new to qualitative methodology and to identify features of our presentation that facilitated or hindered the experts understanding the exam questions and coming to a consensus, we decided to conduct a beta version of the focus group. In the course of the beta focus group and upon reflection, we decided to make the following changes to our procedure for the official focus group: 1. Because the experts often ranked questions as being between two levels of difficulty, e.g. between a "2" and a "3," we determined that a scale of 1 10 would be more useful for representing the difficulty of the questions. 2. We presented the questions to the experts as they had been presented to the students on the exam without providing sample answers. Additionally, we verbally communicated to the experts what information the students had on-hand during the exams. Official Focus Group In the official focus group, we invited three experts who were engineering professors at the University, but who worked in subfields that differed from that represented in ECE 101 and had no prior experience in teaching the course. We invited faculty who had not taught this course because we wanted to avoid the possibility that the participating experts might have biases for how questions ought to be presented. Additionally, due to the small pool of faculty who teach this class, we wanted to ensure that none of the experts had neither designed, nor would be able to recognize, the questions we presented to them. The experts used a 1-10 scale to rank the difficulty of 16 questions that were randomly sampled from the midterms and finals from both SS1 and SS2. The experts came to a consensus on each of the 16 questions. Following the focus group, we averaged the difficulty scores the experts submitted by exam. Exam Average Score (out of 10.00) Midterm SS1 3.80 Midterm SS2 5.50 Final SS1 5.33 Final SS2 6.00 Average difficulty of exams in SS1 4.57 Average difficulty of exams in SS2 5.75 Based on the scores that the experts gave, the average difficulty of SS2's midterm and final was higher than that of the exams offered in SS1. Because of this, we determined that if students, on average, received higher grades in SS2, it was not due to the exams in SS2 being less difficult than those in SS1. On the contrary, the expert consensus on these questions supports the conclusion that the exams in SS2 were more difficult than those in SS1. The researcher employed three active learning techniques in SS2. Throughout the entire course, they used iClickers followed by peer instruction. Throughout the first half of the course, the instructor assigned the students pre-class reading quizzes. However, upon conducting a midquarter survey to receive feedback on the class, the researcher replaced the pre-class reading quizzes with guided post-lecture quizzes. The researcher gathered qualitative feedback through two surveys once in the middle of the term and once at the end. These surveys inquired into students' opinions regarding active learning components as well as their study habits relating to the course. The researcher also received course student evaluations through the university. In the mid-quarter survey, the researcher posed eleven questions through iClicker and six questions specifically inquired into the students' opinions about the active learning techniques the researcher implemented. The midquarter survey revealed that students did not find the pre-class reading quizzes to be quite useful. Because of this, the researcher no longer offered them. See Figure 1 for a graphical representation. Figure 1. Timeline of use of active learning techniques. From the anonymous mid-quarter survey, the researcher found that while 57% of the students did the reading assignment before taking the quiz, 38% did not complete the reading assignment before taking the quiz, they instead used resources other than what was provided in the reading assignment to answer the quiz question. Additionally, 5% of the students did not complete the reading assignment and randomly selected answers for the quiz. Figure 2. Representation of the percentage of student who either a) completed the reading assignment before taking the pre-class reading quiz; b) did not complete the reading assignment before taking the pre-class reading quiz and used resources other than what was provided in the reading assignment to answer the quiz question.; c) did not complete the reading assignment and randomly chose answers on the quiz. On the midterm for SS2, the average grade for the midterm exam was 71% and for the final, the average grade was 82%. For SS1, the average for the midterm was 60% and the average grade for the final was 45%. Figure 3. Comparison of the average grades from the midterms and finals from SS1 and SS2. In the post-course survey, the researcher posed eighteen questions and statements through Google Forms, seven which were multiple choice questions regarding the active learning techniques the researcher implemented. Three of the questions gave students the opportunity to provide general narrative feedback: In analyzing the narrative responses to the post-course survey the instructor requested their students complete as well as the standardized university student evaluations, we focused on responses concerning or referring to the active learning techniques employed in SS2. For the post-course survey the instructor offered, 50 students responded. We determined whether students were writing about one of the active learning techniques by coding for direct reference to the activities or phrases which described aspects of the active learning techniques. Due to the number of responses focused on particular active learning techniques, we also distinguished whether students were referring to iClickers, pre-class reading quizzes, or post-lecture guided quizzes. We counted the number of references to each active learning technique which allowed us to capture the fact that some students referenced multiple active learning elements. Next, we coded for whether students were reporting on having a positive, negative, or neutral experience with the active learning techniques. We interpreted the quality of their reported experience by referring to their word choices reflecting value, reaction, or emotion. For responses that did not contain words describing emotion, value, or a reaction, but merely described the presence of an active learning technique or a description of a course the student had taken previously, we coded these as neutral. Some students had a mixed reaction to the course, expressing both positive and negative reactions. Instead of coding these as neutral, we coded them as counting as both positive and negative. This allowed for a response to be counted as positive and negative. In our analysis, we focused on how the students responded to the question: "Compare and contrast your role as a student in *this* course's LECTURE with other "standard" course lectures." Figure 4. Percentage of students who referenced or did not reference active learning techniques in their narrative responses. Out of 50 responses, 32 (64%) students referenced active learning techniques in their response to the question. Figure 5. Percentage of references to specific active learning techniques. Out of the 32 students who referenced active learning techniques in their responses, there were 17 (43/6%) references to iClicker questions, 15 to peer instruction (38.5%), five references to pre-class reading quizzes (12.8%), and two to post-lecture quizzes (5.1%). Figure 6. Representation of students' overall perception of their quality of experience with the active learning techniques. Of the 32 students who mentioned active learning techniques in their response, 24 (75%) reported a positive experience associated with theses techniques; six (18.8%) students expressed a neutral experience; and two (6.3%) reported a negative experience. In the post-course survey, the students were asked to indicate how useful the post-lecture quizzes were in helping them learn the topics covered in the day's lecture. Out of the total of 50 students participating in this survey, 40% found the post-lecture quizzes very useful and 44% found them useful. Additionally, 12% of the participants found these quizzes a little useful and 4% found them not useful. Figure 7. The students' reported value of the post-lecture quizzes in helping them learn the topics covered in each lecture. Discussion Looking for a means to help the students communicate efficiently and actively with each other and with the instructor during the lecture class, the researcher decided to add multiple choice questions to every lecture and used iClickers for taking student responses. Using these questions the class time was divided into smaller lecture sections. After introducing a new concept, the researcher showed a multiple choice question related to that concept and asked the students to first answer the question individually and in the second step to explain their answers to their seatmates. The researcher found both of these two steps beneficial to the students' learning. Asking the students to solve the multiple choice problems individually, gave them the necessary time during the lecture to think about the topic and assess their learning of that concept. In the second step, the students got the chance to communicate their findings or the confusions with other students. Also, during the second step, the instructor joined the students' conversations, and it provided the opportunity for all of the students to interact with their professor in the format of a discussion rather than a formal Q and A practiced in the standard lecture classes. The students were asked to explain their solutions or their approach to solving the problems to each other so they would exchange ideas and get exposed to different ways of solving a problem. The instructor asked the students to find another student who came up with a different answer and challenge them and try to convince the other person that their answer was correct by reasoning through their solutions. During the first few classes, some of the students who were used to receiving all the answers explicitly from their professors in the classroom instead of attempting to find the answers themselves were surprised and occasionally asked the instructor why she didn't give them the final answer already. The instructor encouraged them to participate in the discussions and seek out how other students solved the problem. At the end of the second step, the discussion among students, she solved the problem herself and explained the solution to the entire class. Since the credit for answering multiple choice questions was based on participation, not choosing the correct answer, the students didn't need to worry about not understanding a concept or not knowing how to solve the problems. Instead, they embraced the opportunity to express their confusion and ask for extra clarification. Based on the researcher's observations of students behavior, which included noticeable mannerisms of relief, which the researcher interpreted as students were ready to translate theory into practice. Because of this ability to practice, students enjoyed this allocated time so much that they appeared to eagerly wait for the multiple choice questions. These events empowered students to pause and think about what they learned to use in solving a subsequent problem. In the mid-quarter and of quarter surveys the vast majority of them mentioned that they found Peer Instruction through multiple choice questions helpful for their learning (Figure 4). Another advantage of the multiple choice questions and implementing Peer Instruction using them was for the students to practice solving conceptually challenging problems in the class when they had a chance to hear out how their peers analyzed those problems and ask questions from their professor at the lecture class instead of solving them alone and debating on whether to go to the professor's office hours or not. Time constraints are one of the major challenges in integrating active learning techniques into a lecture class. To teach all the course topics in the allocated time for the summer session courses, the instructor did not use the chalk and board in the class and instead presented the partially filled PDF files of the lecture materials to the students and used her laptop to write down the problem solutions or to complete the text. This saved enough class time for the students to spend on the discussions. The PDF files were posted on the website at the beginning of the course, and the students were allowed to use electronic devices to pull up the notes during the lecture. A potential drawback of this method is that it does not fully facilitate note-taking by the students as they do not get to copy down all the lecture materials in the class. This matter was not brought up by the students in this study According to the survey results, about 60% of the students completed the reading assignments, and many of them mentioned that reading the assigned section from the textbook was not sufficient for them to answer the pre-class reading quizzes that were due before each class. In the future, the researcher would like to replace the reading assignments with short video assignments to test if the latter approach would increase the number of students who complete the pre-class preparation assignments. Majority of the students found the post-lecture quizzes helpful (Figure 7). Since these quizzes had step by step instructions on how to solve them, students used them to practice what they learned in each lecture after class. The researcher plans to post these quizzes from the beginning of the quarter. The focus group found that the average difficulty of the midterm and final of SS2 (5.50 and 6.00 respectively) were more difficult than those offered in SS1 (3.80 and 5.33 respectively). Because of this, we have concluded that SS2 offered more difficult assessments than SS1. However, the average score on the midterm in SS2 was 71%, compared to 60% in SS1. Additionally, whereas the average grade in the final exam of SS1 was 45%, it was 82% in SS2. The key difference between SS1 and SS2 was that SS2 used three different active learning techniques (iClickers followed by peer instruction, pre-class reading quizzes, and post-lecture guided quizzes). The results of the focus group combined with the higher average grade in SS2 lead us to conclude that it is likely that the active learning techniques played a crucial role in the students overall understanding and retention of the course material. Additionally, not only were the grades of SS2 higher than SS1, the adjustment that the instructor made part way through the course to implement post-lecture guided quizzes preceded an increase in the average grade for the final of the class, despite the final being more difficult than the midterm. This kind of midcourse correction in response to student feedback and the potential correlation for improved student performance is noteworthy and worthy of additional study. In the course of the focus groups discussion, what became clear was that the experts were readily able to implement their own system for determining the difficulty of a problem. Despite the fact that they had not engaged in this sort of activity before, the three experts were able to immediately discuss and come to a consensus on the difficulty of a question. In our facilitation of the group, we did not give them metrics for assessing the difficulty of the questions; however, the three experts were able to confidently agree on the difficulty of a problem and give lengthy explanations for why they gave it the numerical score they did. Although we did not pursue this in the course of this study, we became interested in how the experts were able to collaboratively develop standards for difficulty and deploy them in an area they did not teach. For future work, we intend to explore how faculty and students develop criteria for difficulty in the context of exam questions and how these criteria can be used in designing more effective assessments. The researcher found the Scholarly Teaching development she received from the Center for Teaching at the university vital for correctly implementing active learning techniques and recommends participating in similar workshops to other engineering professors. Acknowledgement: The authors acknowledge the assistance, mentoring and reflection on teaching sessions offered by the Teaching Center at University of California San Diego. References Atkinson, R. C., & Shiffrin, R. M. (1968). Chapter: Human memory: A proposed system and its control processes. In Spence, K. W., & Spence, J. T. The psychology of learning and motivation (Volume 2). New York: Academic Press. 90-93. Crouch, C. H., & Mazur, E. (2001). Peer Instruction: Ten years of experience and results. American Journal of Physics, 69(9), 970-977. doi:10.1119/1.1374249 Choudhury, A., & Hargis, J. (2017). THE confusion button: A formative assessment to identify real-time student misconceptions. Turkish Online Journal of Distance Education, 3(2), 116. Beichner, R., & Cevetello, J. (2013). Things you should know about collaborative learning spaced. Retrieved September 29, 2017, from https://library.educause.edu/resources/2013/1/7-things-you-should-know-aboutcollaborative-learning-spaces Blackwell, L., Trzesniewski, K., & Dweck, C.S. (2007). Implicit theories of intelligence predict achievement across an adolescent transition: A longitudinal study and an intervention. Child Development, 78. Davison, J., & Hargis, J. (2016). The challenges of determining student engage in a digital, mobile learning age.Special edition on Mobile learning as a scholarship for teaching and learning. Glokalde eJournal, 2(4). Freeman, S., Eddy, S.L., McDonough, M., Smith, M.K., Okorafor, N., Jordt, H., & Wenderoth, M.P., (2014). Active learning increases student performance in science, engineering, and mathematics. Proceedings of the National Academy of Sciences (PNAS), 111 (23), 841. Gunn, E. (2014). Using Clickers to collect formative feedback on teaching: A tool for faculty development. International Journal for the Scholarship of Teaching and Learning, 8(1). doi:10.20429/ijsotl.2014.080111 Marotta, S., & Hargis, J. (2011). Active learning strategies for mathematical classes. Problems, Resources, and Issues in Mathematics Undergraduate Studies, 21(4), 377-392. Hattie, J. (2007). The Power of Feedback. Retrieved October 16, 2017, from http://journals.sagepub.com/doi/abs/10.3102/003465430298487 Iwamoto, D., Hargis, J., Taitano, E., & Vuong, K. (2017). Analyzing the efficacy of the testing effect using Kahoot on student performance. The Turkish Online Journal of Distance Education, 18(2). Mueller, C. & Dweck, C. (1998). Praise for intelligence can undermine children's motivation and performance. Journal of Personality and Social Psychology, 75, 33-52. Puentedura, R. R. (2006). Transformation, technology, and education. Retrieved on September 2, 2016 from http://hippasus.com Zimmerman, B. J., Bandura, A., & Martinez-Pons, M. (1992). Self-Motivation for Academic Attainment: The Role of Self-Efficacy Beliefs and Personal Goal Setting. American Educational Research Journal, 29(3), 663-676. doi:10.3102/00028312029003663 Appendices Mid-Term Exam Analysis Average: 71 out of 100 Standard Deviation: 19 A few midterm problem averages: Problem 5. 71% Problem 8. 55% Problem 9. 70% Problem level analysis: Problem 1. Level of difficulty: Simple Application level. Students have to apply the condition of periodicity to this problem. Problem 2. Level of difficulty: Average-Simple Application level. Students have to apply what they have learned in class and choose the correct order of operations. Problem 3. Level of difficulty: Average Application level. Students have to relate what they have learned in another context to this problem to correctly plot the signal. Problem 4. Level of difficulty: Average-Simple Application level. Students have to apply what they have learned in class to this problem Problem 5. Level of difficulty: Difficult Evaluation level. Students need to evaluate the given system and draw results to answer the questions. Problem 6. Level of difficulty: Average Application level. Students have to apply what they have learned in class to this problem Problem 7. Level of difficulty: Average Application level. Students have to apply what they have learned in class to this problem Problem 8. Level of difficulty: Difficult-Average Synthesis level. Students have to analyze the given information and use them to construct a signal. Problem 9. Level of difficulty: Difficult Synthesis level. In order to solve this problem, students should have a conceptual understanding of the the process of using Fourier series to evaluate LTI systems. Final Exam Analysis Average: 82 out of 100 Standard deviation: 17 A few final problem averages: Problem 3. 67% Problem 5. 82% Problem 7. 91% Problem 6. 75% Problem 8. 86% Problem level analysis: Problem 1. Level of difficulty: Average to Simple Application level. Students saw the second term in examples of Fourier series problems and they have to apply what they learned in another context to this problem. Problem 2. Level of difficulty: Average Application level. Students have to apply the definitions of linearity and time invariance to the given system. Problem 3. Level of difficulty: Difficult Analysis level. Students have to break down the given information and analyze it to find the solution. Problem 4. Level of difficulty: Average Application and Analysis level. For part (a) students can apply what they learned in Fourier series to the given information and answer the problem. For part (b), they need a higher level of learning that is the analysis level. They have to find out what approach they need to take to solve it. Problem 5. Level of difficulty: Difficult Average Analysis and synthesis level. For part (b) and (c), students need to analyze their answer for part (a) and combine what they have learned so far to calculate the answer for part (c). Problem 6. Level of difficulty: Difficult Synthesis level. Students have to bring together ideas to find a correct approach to solve this problem. Problem 7. Level of difficulty: DifficultAverage Evaluation level. Students have to evaluate the given information and conceptually understand the filtering process to be able to answer this question. Problem 8. Level of difficulty: Difficult Evaluation level. Students have to evaluate the given information and related what they have learned in sampling theorem to the Fourier transformation of the signals. Problem 9. Level of difficulty: Average Application level. Students have to apply what they learned in Modulation topic to this problem. Problem 10. Level of difficulty: Average Application level. Students have to apply what they learned in Laplace transform topic to this problem.