The Value of Ethnography August, 2018 1 Transformative Dialogues: Teaching & Learning Journal Volume 11 Issue 2 August 2018 The Value of Ethnography: A Pilot Study on a Class on Pedagogical Instruction Rebecca Hardesty, Maxie Gluckman, Jace Hargis, University of California, San Diego Authors' Contact Information Rebecca Hardesty, PhD Candidate Maxie Gluckman, PhD Student Jace Hargis, PhD University of California, San Diego 9500 Gilman Dr., La Jolla, CA 92093 Phone: 415-595-9610 Email: rhardesty@ucsd.edu Abstract: Participant observation ethnography as a primary methodology, while common in other areas of social science, has been underrepresented in Scholarship of Teaching and Learning (SoTL) literature. In many studies, ethnography is used to supplement findings or address questions arrived at through other methodologies, whereas the present pilot study promotes its viability as a primary method. In the Fall of 2017, graduate student researchers (GSRs) and other staff at the "Teaching Center" (TC) used ethnographic methodology as a means of meeting the TC's mandate for the 20172018 year. The TC wanted to gather data on the efficacy of the classes as well as the staff members' own processes of curriculum design and implementation. By adopting participant observation ethnography, the GSRs provided data for the TC and also discovered the utility of ethnography as a tool for designing research questions. This paper has five core sections devoted to describing the pilot study. These sections cover the aims of the TC and the context in which this pilot study was developed; a review of SoTL literature and the presence of ethnography as a primary methodology within it; an outline of the development of the methods used in this study; and how ethnography served as a tool for designing subsequent research projects. It concludes by offering several recommendations and comments to SoTL practitioners who are considering using ethnographic methods in higher education. Key Words: research methodology; ethnography; scholarship of teaching and learning (SoTL); higher education. The Value of Ethnography August, 2018 2 Transformative Dialogues: Teaching & Learning Journal Volume 11 Issue 2 August 2018 Introduction The 2017-2018 academic year marked the "Teaching Center's"1 (TC) first year of providing service and support to instructors of all levels and experience. The TC is at a large public research intensive university which founded the center with the mandate to create broad opportunities for all educators to develop their teaching practices. As members of the TC, our task for our first year was to define and then execute the specifics of our mandate. One of the services we offered, in accordance with our mandate, was observing instructors in their classrooms, providing them with feedback on what we had observed, and then assisting the instructor in reflecting upon their pedagogical practices. Instructors had the opportunity to work with TC staff members to gather data on their own teaching, not only to explore how they might develop their pedagogical skills, but also to study teaching and learning processes through Boyer's (1990) Scholarship of Teaching and Learning (SoTL) model and other approaches within the SoTL literature. The TC not only provides support to instructors on campus through workshops, classes, observations and one-on-one meetings, but is also a research center. The center's unique position as an institution that both provides guidance on teaching as well as studies it has led its members to take a reflective stance on their own instruction on pedagogy. In the process of designing and teaching a course called "Teaching as New Instructors," which provided teaching support for new Teaching Assistants (TAs)2, the TC staff members gathered data on how well the course enabled the TAs to synthesize new techniques and on the course's overall effectiveness. This was a unique class because instead of it being comprised of undergraduate students, it was comprised entirely for graduate students who were going to be TAs for the first time. The new TAs had their own students in discussion sections who did not attend the "Teaching as New Instructors" class. We gathered short reflections from TAs at the end of class and asked the students to create multimedia artifacts to reflect on their development as new TAs. However, the team members noticed there was a gap in data collection with these collection methods. Specifically, these methods looked at the reflections TAs made before and after their their weekly "Teaching as New Instructors" meetings. These methods did not investigate what happened during the class periods. To supplement these methods of data collection and gain a window into the phenomena of in-class learning, reflection, and synthesis as they happen, TC members chose to place Graduate Student Researchers (GSRs) in the roles of ethnographic observers in the "Teaching as New Instructors" courses. Because classroom dynamics can vary despite being led by the same instructor and because this was a new course, it was difficult to predict in advance the specific research questions on which future work would focus. Therefore, the team members followed the tradition of contemporary ethnographic research of allowing the research questions to emerge based on the phenomena of the site (Becker, 1993). As the GSRs underwent development in participant observation ethnography and set foot in the classrooms, the GSRs and other TC staff members, began to reflect upon 1 We have given our center a pseudonym in an effort to anonymize the "Teaching Center" (TC) for the purposes of review and publication. The Value of Ethnography August, 2018 3 Transformative Dialogues: Teaching & Learning Journal Volume 11 Issue 2 August 2018 their own methods of conducting SoTL research. The GSRs became interested in how classroom ethnography can play a key role in successfully designing SoTL research questions. In collaboration with the novice ethnographers, members of the TC with more experience in ethnography studied the process by which the GSRs used ethnographic methods to design research questions. This study describes the process by which GSRs new to SoTL research and ethnography developed questions that would become the basis for future SoTL work. Although this case is limited in its scope and generalizability, we hope to show the utility of ethnographic methods for conducting SoTL research that is grounded in the emergent phenomena of classrooms. We also seek to illuminate how this particular methodology plays a role in the development of research trajectories. In pursuit of these aims, we will first outline the development of SoTL as a distinct field with attention to the methodologies its practitioners employ. Second, we will describe in detail the training TC staff members and GSRs received on ethnographic methods, the methods by which we gathered data on the GSRs, and how we structured their ongoing mentorship on ethnographic methods. Third, we will describe how the GSRs developed the questions which would motivate their future SoTL projects through using ethnographic methods and how those methods enabled them to do so. Fourth we will discuss how this pilot study reveals the value of ethnographic methods to SoTL research. Fifth, we will outline the limitations of this study and our recommendations for overcoming them based on what we have learned. Last, we will conclude by offering several suggestions and thoughts to those conducting SoTL studies who would like to use ethnography as a primary methodology. Literature Review With the foundation of the International Society for the Scholarship of Teaching and Learning in 2004, the Scholarship of Teaching and Learning (SoTL) emerged as a distinct field with its own agenda of integrating research, teaching, and learning (Hubball, Clarke, & Poole, 2010). This is very much in line with the overall objectives of the center. SoTL is a field with disciplinary diversity which has attracted historians, psychologists, mathematicians, as well as education scholars who aim to foster "deep understanding" in their classrooms as well as of their own teaching practices (Huber & Morreale, 2002, pp. 2). The multidisciplinary nature of the field presents opportunities and challenges to the practitioners. On the one hand, practitioners accept and have access to a multiplicity of methods, theoretical frameworks, and potential collaborators who share the same topical interests. On the other hand, it is difficult to gain fluency in the many disciplines contributing to SoTL research and this difficulty can occlude certain bodies of literature and scholarship from new researchers in the field (Huber & Morreale, 2002) Additionally, the foundational interdisciplinary of the field can simply be daunting to those considering entering into it (Tremonte & Colleen, 2011). Also, while it is no doubt rewarding to contribute to a field which embraces interdisciplinary and values teaching as a mode of doing research, reconciling the standards and epistemology of one's discipline with those of SoTL is an uncomfortable experience which challenges one's professional identity (Miller-Young, Yeo, & Manarin, 2018). SoTL is a field united by a shared set of topics and interests as opposed to a shared disciplinary background and prescribed methodologies. Traditionally, academics in the The Value of Ethnography August, 2018 4 Transformative Dialogues: Teaching & Learning Journal Volume 11 Issue 2 August 2018 United States are instructors and researchers and their teaching and research are regarded as two distinct activities. SoTL is unique because it gives academics an opportunity to treat their teaching as a site for research. Calder, Cutler, and Kelly (2002) also report their SoTL work has caused them to examine not only how history is taught, but how the discipline of history has not delineated the history of teaching and learning as a distinct subfield (pp. 52). By reflecting on how historians present and teach in their home field, these authors are able to gain insights into how historical methods could be used to contribute to SoTL research and how teaching historical methods can be an opportunity for historians to get a deeper understanding of their own field (Calder, Cutler, & Kelly 2002, p. 55-56). Other practitioners in the fields of Communication Studies (Morreale, et al., 2002), Sociology (Howery, 2002), and English Studies (Salvatori & Donahue, 2002), have also begun to view their classrooms and universities as extensions of their research with similar insights that shape their teaching and thoughts about their primary disciplines. The diverse collection of disciplines contributing to SoTL researchers give practitioners the opportunity to use a diverse set of methodologies to pursue questions on classroom assessment techniques, learning strategies, course design, etc. (McKinney, 2004). Some of these methods have included "experimental design grounded theory research, classroom ethnography3, and phenomenological study" (Hubball et al., 2010, p. 118; referencing Cohen, Manion, & Morrison, 1996). Other practitioners have sought to use historical methods in SoTL research and have deliberated the potential costs of turning their teaching into "on the record" historical material (Calder, Cutler, & Kelly, 2002, pp. 54). Although quantitative methodology is welcome, there is notable dominance of qualitative methods in SoTL research. This is a hurdle for members of the field who have backgrounds in disciplines that emphasize training in quantitative methods and have less appreciation for qualitative methods (Huber, 2002, pp. 36). For researchers from STEM fields, it is less clear how to employ the methods they have been trained in to their work on teaching and learning. For instance, a chemist is not able to directly apply their expertise in electro dialysis to understanding how students synthesize new information. Instead, these practitioners must often learn new methods for conducting SoTL research, such as selecting and analyzing case studies (Coppola & Jacobs, 2002). The key point of SoTL research is to understand how teaching and learning actually work, and how instructors can best facilitate effective learning. To pursue this research agenda, practitioners have a wealth of methods from which to choose. However, there are relatively few works of scholarship with classroom observation as their primary methodology. There are some notable instances of classroom ethnography in primary education, such as Tobin, Hsueh, & Karasawa's cross-cultural study of preschools in China, Japan, and the United States (2009); however, there is very little participant observation ethnographic work done on higher education. This is surprising because the classroom in higher education continues to be a crucial site of education where students and instructors interact, students encounter new information, attempt to implement new 3 "Classroom ethnography" is what sociology or anthropology would recognize as "participant-observation ethnography." As we elaborate later, the classroom ethnography conducted that is referred to here was not the primary method of the studies, conducted in higher educational settings, nor was specifically participant observation ethnography. The Value of Ethnography August, 2018 5 Transformative Dialogues: Teaching & Learning Journal Volume 11 Issue 2 August 2018 methods or ideas, and are assessed on what they have learned. It is notable that the records of the phenomena of classrooms-in-action are not often put forth as the primary evidence in SoTL studies. Instead, classroom observation through ethnography is either reported after the fact to provide support for a previously identified research interest (Cross & Steadman, 1996; Hargis & Soto, 2017), is the topic of discussion as opposed to a methodology that is deployed (Watson-Gegeo, 1997; Malin, 2003; Stebbins, 2011), or is positioned as being, at least in part, artistic work (Tobin & Hsueh, 2007). Other studies focus on classroom experiences, but arrive at it through interviews or focus groups where students and educators report on classroom dynamics after the fact (Booker, Merriweather, & Campbell-Whatley, 2016). The infrequent usage of ethnographic methods as primary methods is understandable for several reasons. First, the field is still young and it is logical that some methods have not yet gained prominence because of the idiosyncrasies of how a field develops. Indeed, the field has recently welcomed the use of auto ethnography which indicates that there is not a core resistance to the method in the field (Latz, 2012). Second, participant observation ethnography is a method in which it takes time to become proficient. Third, the method itself is labor intensive, taxing on the researcher, and does not produce data quickly. This study aimed to use participant-observation as its primary methodology, taking its cue from anthropology (Geertz, 1974; Gumperz, 1981) by focusing on social interaction and recording it through "thick description." In the context of this study, by providing "thick description," we mean "accurately describing and interpreting social actions within the appropriate context in which the social action took place" (Ponterotto, 2006, p. 542). However, we did not seek to interpret or figure out the mental states of the students or ascribe to them motivations or intentions. Remaining sensitive to issues education scholars have raised regarding the issues of making generalizable claims from ethnographic work (See Tobin, 1999, p. 116). Therefore this study was used to: 1) generate research questions grounded in the phenomena of the classroom; 2) illustrate particular and local processes of how teaching and learning occurred; and 3) advance the TC's objectives of gathering data on their own process of designing and implementing curriculum and teaching methods. Methods At the beginning of the Fall 2017 quarter, the GSRs and TC staff members attended a workshop on ethnography led by a member of the TC who had substantial experience conducting and training in ethnographic methods, who we will refer to as the senior ethnographer. The senior ethnographer was a doctoral student who was completing her dissertation comprised of a long-term ethnographic study, and had a record of both teaching ethnographic methods and publications using this method. Prior to this workshop, the attendees read selections of literature on ethnography by Merriam and Tisdell (2009) and Emerson (2011). The workshop was comprised of activities designed to give the participants practice in taking field notes. For the sake of instruction, the senior ethnographer played videos of people conducting everyday activities in public such as trying to hail a taxi or waiting for the bus. The workshop participants would take notes on the particularities of the activities in the videos and this would be followed by a discussion of what the members noticed (and what they did not notice), what they wrote The Value of Ethnography August, 2018 6 Transformative Dialogues: Teaching & Learning Journal Volume 11 Issue 2 August 2018 down, and their experience of taking notes. The senior ethnographer would provide feedback on the justifiability of the inferences the attendees had made in their notes, offer recommendations to overcome challenges the participants had reported, and she would offer tailored guidance for each member going into the next activity. The participants repeated this process for two subsequent videos. The leader did not expect any of the participants to have attained expertise or proficiency in the methodology after that workshop, but intended to have provided some foundational tools that they would carry into the "Teaching as New Instructors" courses which the GSRs would observe. The senior ethnographer attended the same courses as the GSRs and likewise took field notes for several weeks, and met with the GSRs individually after each class to discuss their experiences and address any challenges the GSRs reported. The GSRs took comprehensive field notes on what they observed, and developed particular foci that fit into the TC's mandate. The model of ethnography that GSRs used was "participant-observation" where they attempted to experience the classroom as one of the members of the class (i.e. students and instructors). Throughout the quarter, the GSR ethnographic observers attended the "Teaching as New Instructors" courses in their entirety. Each class was one hour long and there were two classes offered each week. In total, the GSRs and senior ethnographer each conducted 20 hours of in-class observation. There was at least one full observer and one participating observer for each section of the class. Despite the name, a full observer is a kind of participant due to their physical presence at the site and that this will have some effect on the phenomena of the classroom. In this case, we draw a distinction between the full observer and the participating observer because the GSRs commented upon the different kinds of skills each role required. The full observer would not be involved in the group activities or class discussions whereas the participating observer would participate in the group activities (but not the full class discussions) as much as they need to in order to understand the TAs' experience. The participating observers would take notes after their active participation in group activities or at convenient moments that would not disturb the early career instructors. The GSRs took on the roles of either full observers or participating observers to conduct participant-observation ethnography to experience the events and interactions in ways that approximated the members' own experiences (Wax, 1980, p. 272; Jorgensen, 1989; Spradley, 2016). In the field, researchers "jotted" notes by hand or on a laptop. Afterwards, the GSRs transferred notes to a shared online document and wrote narrative accounts of the day's events. Over the course of the study, and as the GSRs' skills improved, they and the TC staff members determined the extra phase of writing a narrative account was redundant if they were in a full observer role (Emerson, 2011). Once a week, after a class meeting of "Teaching as New Instructors", the GSRs, TC staff members involved in instructing the classes, and the senior ethnographer would write a reflection of their experience of their work that week. Team members also shared the photographs they took during the class of artifacts produced (e.g. whiteboard work, post-it notes, etc.) They uploaded these photographs to a shared document. After collecting the data, GSRs and TC staff members began to develop a qualitative coding scheme, and then coded the data, looking for trends or themes. The Value of Ethnography August, 2018 7 Transformative Dialogues: Teaching & Learning Journal Volume 11 Issue 2 August 2018 After each day of the course, the GSRs and the senior ethnographer would compare notes. The senior ethnographer would then offer tailored guidance and mentorship to the GSRs on how to develop their skills as observers and taking notes. She would also help the GSRs go through their data to find phenomena which interested them. She attended the courses with the GSRs for several weeks until they were confident in their ability to take field notes. After this, the senior ethnographer would review their notes, meet with the GSRs, and, in conjunction with the other TC staff members, give feedback on how they were developing their research interests and how the ethnographic data could be used to support them. Prior to beginning the pilot study at the start of Fall 2017 quarter, the TC staff members developed several foci of investigation they hoped would shed light on how the structural features of the class gave the attendees of the class, the TAs, the opportunity to reflect on their teaching experiences in future classrooms. These structural features included group discussions, think-pair shares, and small group discussions. The TC staff members also hoped to determine what features of the course best helped the TAs synthesize course content. They hoped that the GSRs would be able to provide observational data on how the questions the instructor asked, the questions students asked each other, group activities, and full class discussions contributed to the students' ability to synthesize the content of the class. While the GSRs investigated these initial elements of interest in "Teaching as New Instructors," they also developed questions that emerged through their observations, review of field notes, and early data analysis. It was these questions that ended up serving as the basis for the SoTL projects they pursued for the rest of the academic year instead of the topics that the TC staff members had thought a priori would be the most fruitful avenues of inquiry. These a priori foci were shared with the senior ethnographer, but not the GSRs. The TC staff members and GSRs naturally shifted to focusing on the research questions that organically developed from their observational work. Throughout the pilot study, the TC staff members and GSRs documented their process of designing and refining the avenues of future research. Developing Questions "Anne" On the basis of her ethnographic observations, one GSR, "Anne," developed an interest in how students reflect upon their actions and how they decide to proceed differently in light of their reflection. By the end of the quarter, she had settled on the research question: What does reflection reveal about Instructional Assistants' transformation as teachers, as well as challenges they face in higher education? This was not a question she had going into the classroom to conduct her observation. In fact, she was initially overwhelmed by how much was happening in the classroom: "Overall, interesting experience as a researcher and I found it good practice in making observations while also being a participant, as well as good practice in refining how I can take better field notes. I liked having a specific focus going into the class as a researcher, but I think it would be helpful to have fewer foci" (October 4, 2017) The Value of Ethnography August, 2018 8 Transformative Dialogues: Teaching & Learning Journal Volume 11 Issue 2 August 2018 In the subsequent weeks, Anne worked to limit the number of aspects of the class on which she was focusing. She reflected that she was gravitating towards the conversations students were having amongst themselves and with the instructor. "[Students] volunteered answers without being called on, and the quality of conversation was high and somewhat intimate especially when [the instructor] sat down and everyone was in a circle; students revealed personal experiences and reflection and self-criticism (October 11, 2017). Anne made note of the fact early on that the course provided many opportunities for the TAs to reflect. The TAs completed online surveys, they created media artifacts to represent their experiences as new TAs, and the instructors gave them opportunities to present their experiences orally in class to partners or to the larger group. She also observed that in addition to the timeslots the instructors allotted for students to provide oral reflections in class, the TAs would spontaneously volunteer their reflections on their recent teaching experiences. Spontaneous reflection seemed like a natural occurrence in a class designed to support new instructors and an apt target for investigation. However, it was so ordinary and mundane that the TC staff members and GSRs had overlooked it as a feature of the class to investigate. It is reasonable, though, that the TC members did not think to focus on spontaneous instances of reflection because it is an unpredictable event. Whether TAs would volunteer to share their experiences depended on a certain classroom dynamic and an element of trust between the TAs and the instructor. In the next week, Anne reported her initial framing of what would become her research question. "It came to me mid-class that what I was really interested in was the metacognition and reflection that was happening, the language that is used to promote this type of thinking, and wondering if the TAs were in turn promoting this kind of thinking in their classes. I think I will continue to focus on this for next week (the type of language [the instructor] uses to promote metacognition and reflection, seeing if the TAs are using that language, and at what times during class they are being metacognitive and reflective)" (October 18, 2017). In the meetings with the senior ethnographer and the weekly meetings with the instructors of the "Teaching as New Instructors" courses and the other GSRs, Anne began to refine her interest. When going through her field notes and those of the senior ethnographer, it became clear that there was little conversation amongst the students that could easily be recognized as metacognition. This realization made Anne realize how difficult it is to infer intention from spoken language. However, Anne was not discouraged by this and became more interested in the ways in which the TAs were reflecting. "It is wonderful to see in the short time of the course, the TAs were reflecting on their experience with the class and how it is impacting their own teaching and ways they see themselves growing (and for some, diverging with other TAs/professor) in what they see as 'quality' instruction" (October 25, 2017). As Anne settled into her research topic, she began to reflect on her experience conducting ethnography and the ethical issues this methodology poses. The Value of Ethnography August, 2018 9 Transformative Dialogues: Teaching & Learning Journal Volume 11 Issue 2 August 2018 "I feel like I am forming one-way friendships with the TAs being a complete observer; I feel like I know so much about everyone by taking notes on everything they say, yet have to remind myself that no one knows practically anything about me! I was thinking a lot about the positionality of the researcher and the researcher's tools (since I just had to examine a few examples of different ethnographic studies this past week). As this course winds down, how can I make sure this research is mutually beneficial, and I am not just the one doing all the taking?" (November 8, 2017). Towards the second half of the quarter, Anne and the other members of the TC began to review the data she had collected on the students' reflections. The preliminary data collection revealed that the TAs initially engaged in underdeveloped reflection-onaction based on Ken Bain's (2002) 5R framework (Hemans, Gluckman, & Hargis, 2018). This was demonstrated through the structure of their reflections which took the form of, first, reporting on their experiences and, second, saying how they responded to those experiences. As the course progressed, the early career instructors began to engage in higher forms of reflection-on-action. The TAs demonstrated this by volunteering how they would handle a situation differently in the future and, sometimes, would offer their reasoning for why they believed this change in their response would benefit their students. The data indicated there was a relationship between the TAs growing ability to reflect and their deliberate changes to their teaching practices. Anne was able to track this developmental process because her methodology allowed her to capture in detail the spontaneous reflections the TAs uttered and then create an emergent coding scheme specific to the phenomena she witnessed. "Beth" "Beth," the second GSR who conducted ethnography of the "Teaching as New Instructors" course developed an interest in how the instructors negotiated the timing of the class and its activities. She went on to pursue the question: How, in practice, do instructors negotiate the intended temporal structure and content of their lesson plans with students' enthusiasm/interest or lack thereof? Her interest in how instructors deal with timing can be traced back to the first day of the course. "The activities were a great way to break up the sections with a lot of information but there was a moment when two activities landed very close together and in my head I went "another one? already?." We think a lot about not having students sit and listen passively for too long, but I noticed that there was a rhythm to honor... In the next section, Anne and I chatted about recording a minute-by-minute timeline so we can better understand the balance of timing" (October 5, 2017). Beth reflected on her experience conducting ethnography throughout her fieldwork. Her experience finding a middle ground between recording everything with low accuracy and recording only select aspects of the class with high accuracy contributed to how she developed her research question. "As I started writing [field notes], I learned very quickly that I had to pick and choose what was important and make on the spot judgments about which information I thought was most valuable...I was also a little uncomfortable The Value of Ethnography August, 2018 10 Transformative Dialogues: Teaching & Learning Journal Volume 11 Issue 2 August 2018 observing because the class was very small. I kept wanting to say "I'll stop observing and participate" to help fill some of the space" (October 12, 2017). Towards the middle of the quarter, Beth began to explicitly focus on how the instructors managed the time the students spent on activities. She discovered instructors would take time to sit with students to either move their discussion along or encourage them to discuss something more deeply. "Then [the instructor] spent a good deal of time taking the fourth seat at the table with them. It was actually a little bit difficult to hear what was being said, because the tones were so quiet in the small group setting" (November 2, 2017). Beth noted that her ethnographic data collection and written reflections by the GSRs and TC members revealed that there was a significant variation in student interests across course topics. In response to her data, the TC members leading their respective iterations of the course made numerous modifications to their original lesson plans. In the course of taking field notes, Beth recorded when these modifications occurred and what form they took. During the preliminary data analysis, she noticed this happened frequently in both of the "Teaching as New Instructors" classes and had an effect on the content presented in the courses and how much time the instructors spent on a given concept. Beth wrote that she "had the unique vantage point as [an] ethnographic observer to capture not only where instructors made 'on the spot' adjustments, but also the stimuli and student reactions that cued the instructors to make such modifications" (Lee, Gluckman, & Hargis, 2018). In Beth's analysis, the ethnographic data revealed that the instructors had three general tactics for adjusting to student interests. Instructors would: a) change their examples or application in light of students' interests; b) change the agenda and give more time to a topic if students were interested in it; c) or incorporate student feedback in their future lesson plans. The Value of Ethnography The GSRs involved in this pilot study were able to develop substantive research questions by using ethnographic research to identify classroom interactions that spoke to topics of SoTL research. Anne discovered that the spontaneous reflections the TAs performed in class gave insight into their process of learning how to reflect upon their own teaching and make adjustments going forward. Students do not spontaneously reflect in every class on how what they are learning in that class is causing them to reflect upon their actions. Because of this, it took Anne seeing it happen in front of her to realize that this could be a feature of the course she could investigate further. Beth's experience as an ethnographer made her sensitive to how instructors managed time in the classroom. It is quite possible that a researcher would decide upon this topic without having first witnessed it as an ethnographer. However, because Beth developed this question out of her experience of dealing with a poorly timed class session she became interested in the instructors' strategies for overcoming obstacles posed by activities running too long or students wanting an activity to go on longer. Her experience conducting classroom ethnography resulted in her developing a question that was tailored specifically to the two classes which she observed. While this pilot study had local value to the TC because it resulted in conference presentations, opportunities for publication, and its members having a new method at The Value of Ethnography August, 2018 11 Transformative Dialogues: Teaching & Learning Journal Volume 11 Issue 2 August 2018 their disposal, we have also discovered the broader value of this methodology for SoTL research. It is important to note the ethnographic work the team members conducted is, like all ethnographic work, difficult to generalize due to its highly situated nature and the relatively few participants involved in the study (See "Limitations"), it is nonetheless possible to derive some generalizability from the pilot study. What is generalizable is the methodology itself, not necessarily its findings. Because the GSRs successfully used the methodology and arrived at generative research questions grounded in the phenomena they observed, this study demonstrates participant observation ethnography is a productive methodology to incorporate into SoTL as a tool for developing research questions. More importantly, given the intimate relation between research and teaching in SoTL, the method becomes a tool to do something about how we teach, not just a new method for research alone. It is a subject for future studies to investigate how ethnography enables researchers to observe the phenomena of a classroom in a way that generates research questions. However, the field notes and reflections of the GSRs in this study illuminate the process by which they did, in fact, develop research questions grounded in, and inspired by, their observations. We offer the structure of the process by which the GSRs and TC staff members worked together to develop and refine the research questions that emerged out of the ethnographic observations. We hope that this can serve as a model for other SoTL practitioners or centers that are engaged in research on teaching and learning. The Value of Ethnography August, 2018 12 Transformative Dialogues: Teaching & Learning Journal Volume 11 Issue 2 August 2018 Figure 1. Graphical depiction of the process by which GSRs developed research questions from their ethnographic observations. As Figure 1 depicts, the GSRs went into the field site of the classroom without a particular focus and their field notes reflected this. The GSRs and senior ethnographer took notes on all the features of the "Teaching as New Instructors" class, which included the details of the instructors' Google Slides, the timestamps for all interactions, the details of the activities including how they were presented and carried out, the body language of the members of the class, and the conversations that took place. Because they tried to capture all the phenomena of the classroom, their notes were cursory and often incomplete. In part due to the stress of using a new methodology for the first time and/or entering a new field site, there was no reflection on possible research questions or ethnographic methodology itself. Additionally, the large volume of notes each GSR took was physically and mentally taxing, which also precluded written reflections. The Value of Ethnography August, 2018 13 Transformative Dialogues: Teaching & Learning Journal Volume 11 Issue 2 August 2018 After the GSRs' weekly meeting where they discussed their experiences conducting ethnography and received guidance from the senior ethnographer, the researchers selected one or two features of the class to focus on for the next meeting. Two GSRs chose to focus on dialogue and one focused on how the instructor tried to execute their lesson plan in the allotted amount of time. During this phase of the process, the GSRs' notes were more complete, but were less voluminous. Their written reflections focused on the mundane challenges of conducting ethnographic work, such as remembering dialogue, staying focused, writing quickly, etc. They also began to reflect on issues they saw in the features of the classroom activity on which they were focusing. For example, Anne became interested in how the members of the class were becoming more metacognitive, as evidenced by their discussions. In their weekly discussions during this phase, the GSRs worked together to identify the specific aspects of the method which they found challenging. They received tailored guidance from the methodologically more senior GSR on how to address this challenge. Additionally, the researchers presented drafts of potential research questions that targeted their phenomenon of interest. The phase was iterated several times during which GSRs further specified their targets of inquiry and worked to overcome the technical challenges the method posed. When the GSRs felt confident in the research questions which emerged out of the previous phase, they transitioned into observing the "Teaching as New Instructors" classes for the purpose of gathering data to answer their questions. During this phase, their notes became thorough, often including ideas of how to pursue their questions in full studies. Notably, the GSRs also began to reflect on their relationship to the members of the class and the implications of ethnographic methods. As she did on November 8th, Anne reflected again on her position as an ethnographer a week later, "I mentioned to [Beth] after class that the feeling of "one-way friendship" is increasing because I am learning how wonderful all the TAs are and am so inspired by their growth, yet they still know little about me as the observer...I miss the relationship-building that happens in the classroom. It was funny: at the end of session, when I asked if I could interview each of them about their experience after the last class, [Student D] asked me if I had been taking notes on them the whole time, because he noticed me typing every time they talked. This goes to show that the observer, although normalized after some time, is never forgotten by the observed. I wonder if there is a way to make this less awkward, or if I could have done something to make this more of a reciprocally beneficial exchange?" November, 15, 2017). A clear value of ethnographic methodology for the purposes of TC is that it has led to two full studies that GSRs will pursue. One of these studies will be on the role reflection in the transformation of teaching practices. The other will focus on how instructors manage "on the spot" adjustments to their lesson plans in the face of student interest. Specifically, the data derived from observation, field notes, photographs, and reflections gave the TC a way of determining how the TAs reflected on their teaching experiences and whether the students were able to synthesize new pedagogical techniques in a way that made their own teaching more student-centered. Additionally, the data recorded on the instructors allowed the instructors to reflect upon the The Value of Ethnography August, 2018 14 Transformative Dialogues: Teaching & Learning Journal Volume 11 Issue 2 August 2018 effectiveness of their curriculum design and implementation by giving them a thirdperson perspective on the classes they were teaching. Moreover, the discussions the instructors had with the GSRs and the reflection process they engaged in each week functioned as a model for what the instructors were asking their students to do. Through this reflection and feedback process, the instructors saw opportunities to adjust their own practices and evaluate the success of the "Teaching as New Instructors" course. An additional value of this pilot study is that it demonstrates the utility of ethnography as a primary methodology in SoTL studies. Although the field is not opposed to ethnographic methods, there seems to be a sentiment that ethnography is a supplement to other kinds of qualitative research. In other words, many studies use ethnographic work to support claims and research questions arrived at through other established methodologies.4 However, as Anne's comment above reflects, a more significant value of this methodology was that it made those using it more aware of relationships in classrooms as well as their own love of teaching. The GSRs reported this experience conducting classroom ethnography has made them more conscientious researchers and instructors. The act of observing the TAs made the GSRs and senior ethnographer more aware of the power dynamics at play in the classroom already and also the vulnerability of those being analyzed through social scientific methodology. Limitations of the Study This pilot study has several limitations. As we discussed earlier, a limitation of ethnographic research is that it is highly contextualized and specific to the time, location, and participants of the study. The specificity and particularity of the method makes is difficult to produce general findings from the data. However, ethnography of classes in higher education is able to overcome this in some respects. This is because college classes, even ones that employ active learning techniques or have "flipped classroom" format, have shared structural features. These features are the people occupying the "instructor" role as well as people occupy the role of "students." Additionally, in the United States, there are some common ways an instructor leads a class. Instructors can deliver lectures, ask questions, lead group discussions, provide activities, and hold assessments. Although the content of these ways of conducting a class will differ widely, these points of commonality across college classrooms can allow ethnographers to arrive at some generalizable conclusions. For examples, Beth's concerns about timing are rooted in particular events that happened in the classroom; however, balancing timing is a concern shared by many (if not all) instructors) A limitation specific to this pilot study is that it was comprised of two GSRs who had no prior ethnographic experience and no one (including the senior ethnographer) had any experience conducting classroom ethnography. Another limitation was that we restricted ourselves to written field notes (first by hand and then using laptops). We did not use audio or video recordings to collect data. Additionally, the GSRs conducting this research knew the instructors personally and were subordinate staff to one of them in the TC. Finally, the "Teaching as New Instructors" classes had modest enrollment which can further limit the generalizability of this study's results. 4 See Cross & Steadman, 1996; Watson-Gegeo, 1997; Malin, 2003; Stebbins, 2011; Tobin & Hsueh, 2007 as examples. The Value of Ethnography August, 2018 15 Transformative Dialogues: Teaching & Learning Journal Volume 11 Issue 2 August 2018 Recommendations A key aim in presenting this pilot study was to show ethnographic methodology's relevance to SoTL researchers. Based on our experience conducting this study, we have the following recommendations and thoughts we would like to offer to SoTL practitioners who are interested in using this method to study teaching and learning in higher education: 1. Ethnographic methods afford unique insights, but often ones that do not obtain statistical significance. The method allows a researcher to explore and convey the richness of the classroom as it unfolds; however, the specificity of the data a researcher can record can pose difficulties for those seeking to present conclusions of sweeping generality. At the same time, using ethnography as a primary method for a study allows a researcher to take a new approach to analyzing data and developing questions. 2. As an ethnographer in a new classroom, it is rewarding to go in without a preset research agenda. It can be valuable to spend a couple of weeks attending a class without trying to decide on the specific topic of the study. Reviewing one's collection of notes with an open mind can enable a researcher to spot trends and topics of interest they would not have thought to focus on prior to beginning the fieldwork. 3. Ethnographic work on higher educational settings offers a window into how instructors are dealing with universal challenges within the context of teaching and learning. Examples of these include time management, student engagement, motivation, and curriculum design. By using this methodology, it is possible to gather detailed data on how individual instructors are attempting to address these challenges with varying degrees of success. Despite the difficulty in generalizing ethnographic findings, the data can be action-guiding for instructors and researchers who are confronting these shared features of higher education because it offers concrete examples of how other instructors dealt with similar challenges. As educators, it is valuable to see how others in our extended community approach our task of teaching and, in doing so, further develop our own abilities. The Value of Ethnography August, 2018 16 Transformative Dialogues: Teaching & Learning Journal Volume 11 Issue 2 August 2018 References Becker, H. S. (1993). How I learned what a crock was. Journal of Contemporary Ethnography, 22(1), 28-35. Booker, K. C., Merriweather, L., & Campbell-Whatley, G. (2016). The effects of diversity training on faculty and students' classroom experiences. International Journal for the Scholarship of Teaching and Learning, 10(1), 1-7. Calder, L., Cutler, W. W., Kelly, T. M. (2002). History Lessons: Historians and the Scholarship of Teaching and Learning. Disciplinary styles in the scholarship of teaching and learning: Exploring common ground. AAHE Publications Orders, PO Box 1932, Merrifield, VA 22116-1932. Cohen, L., Manion, L., & Morrison, K. (2000). Research Methods in Education. British Journal of Educational Studies 48 (4):446-446. Coppola, B. P., & Jacobs, D. C. (2002). Is the Scholarship of Teaching and Learning New to Chemistry? Disciplinary styles in the scholarship of teaching and learning: Exploring common ground. AAHE Publications Orders, PO Box 1932, Merrifield, VA 22116-1932. Cross, K. P., & Steadman, M. H. (1996). Classroom research: Implementing the scholarship of teaching (pp. 36-37). San Francisco: Jossey-Bass. Emerson, R. M., Fretz, R. I., & Shaw, L. L. (2011). Writing ethnographic field notes. University of Chicago Press. Geertz, C. (1974). "From the native's point of view": On the nature of anthropological understanding. Bulletin of the American Academy of Arts and Sciences, 26-45. Gumperz, J. J. (1981). Conversational inference and classroom learning. Ethnography and language in educational settings, 3-23. Hargis, J., & Soto, M. (2017). A proposal to use classroom observations as assessment data to measure and evaluate effective teaching. The Online Journal of New Horizons in Education, 7(2), 6-17. Hemans, P., Gluckman, M., & Hargis, J (2018), Reflective Teaching: What Instructional Assistant Reflection Can Inform Us About Transformation in Higher Education. Unpublished manuscript. Howery, C. B. (2002). The Culture of Teaching in Sociology. Disciplinary styles in the scholarship of teaching and learning: Exploring common ground. AAHE Publications Orders, PO Box 1932, Merrifield, VA 22116-1932. Hubball, H., Clarke, A., & Poole, G. (2010). Ten‐year reflections on mentoring SoTL research in a research‐intensive university. International Journal for Academic Development, 15(2), 117-129. Huber, M. T. (2002). Disciplinary Styles in the Scholarship of Teaching: Reflections on the Carnegie Academy for the Scholarship of Teaching and Learning. Disciplinary styles in the scholarship of teaching and learning: Exploring common ground. AAHE Publications Orders, PO Box 1932, Merrifield, VA 22116-1932. Huber, M. T., & Morreale, S. P. (2002). Situating the Scholarship of Teaching and Learning. Disciplinary styles in the scholarship of teaching and learning: Exploring common ground. AAHE Publications Orders, PO Box 1932, Merrifield, VA 22116-1932. Jorgensen, D. (1989). The methodology of participant observation. Sage publications. Thousand Oaks. Latz, A. O. (2012). Flow in the community college classroom?: an auto ethnographic exploration. Center for Teaching, Learning, & Scholarship at Georgia Southern University, Statesboro, Georgia, USA. The Value of Ethnography August, 2018 17 Transformative Dialogues: Teaching & Learning Journal Volume 11 Issue 2 August 2018 Lee, L., Gluckman, M., & Hargis, J. (2018). On the spot instructor adjustments to student interests. Proceedings from the Transformative Learning Conference, Oklahoma City, OK, March 8-9, 2018. Malin, M. (2003). Competing interests between researcher, teacher and student in the ethics of classroom ethnography. Westminster Studies in Education, 26(1), 21-31. McKinney, K. (2004). What is the Scholarship of Teaching and Learning (SoTL) and What Can SoTL Do for You and Your Department/School? Retrieved from http://www.issotl.com/issotl15/sites/default/files/chairDeanHandout.pdf Merriam, S., & Tisdell, E. T. (2009). Qualitative Research: A Guide to Design and Implementation. San Francisco: John Willey & Sons. Miller-Young, J. E., Yeo, M., & Manarin, K. (2018). Challenges to Disciplinary Knowing and Identity: Experiences of Scholars in a SoTL Development Program. International Journal for the Scholarship of Teaching and Learning, 12(1), 3. Morreale, S. P., Applegate, J. L., Wulff, D. H., & Sprague, J. (2002). The Scholarship of Teaching and Learning Communication Studies, and Communication Scholarship in the Process of Teaching and Learning. Disciplinary styles in the scholarship of teaching and learning: Exploring common ground. AAHE Publications Orders, PO Box 1932, Merrifield, VA 22116-1932. Salvatori, M. R., & Donahue, P. (2002). English Studies in the Scholarship of Teaching. Disciplinary styles in the scholarship of teaching and learning: Exploring common ground. AAHE Publications Orders, PO Box 1932, Merrifield, VA 22116-1932. Spradley, J. P. (2016). Participant observation. Waveland Press. Stebbins, R. A. (2011). Classroom Ethnography and the Definition of the Situation. Schools, Teachers and Teaching, 218, 243. Ponterotto, J. G. (2006). Brief note on the origins, evolution, and meaning of the qualitative research concept thick description. The Qualitative Report, 11(3), 538-549. Tobin, J. (1999). Comparative Classroom Ethnography. Learning from Comparing: new directions in comparative education research: Volume 1: contexts, classrooms and outcomes, 1, 113. Tobin, J., & Hsueh, Y. (2007). The poetics and pleasures of video ethnography of education. Video research in the learning sciences, 77-92. Tobin, J., Hsueh, Y., & Karasawa, M. (2009). Preschool in three cultures revisited: China, Japan, and the United States. University of Chicago Press. Tremonte, C. M. (2011). Window shopping: Fashioning a scholarship of interdisciplinary teaching and learning. International Journal for the Scholarship of Teaching and Learning, 5(1), 26. Watson-Gegeo, K. A. (1997). Classroom ethnography. In Encyclopedia of language and education (pp. 135-144). Springer Netherlands. Wax, M. L. (1980). Paradoxes of" consent" to the practice of fieldwork. Social Problems, 27(3), 272-283.