Edward Prather
http://www.aacu.org/peerreview/pr-su11/UsingResearch.cfm
The college-level general education (GE) curriculum in the United States can have many goals: exposing students to the breadth of human thoughts and ideas; elevating their reading comprehension, writing abilities, evaluation of information and complex systems, critical reasoning skills; and providing an understanding of and appreciation for subjects outside of their chosen field of study. Unfortunately, the majority of this learning takes place in large enrollment courses. Therefore, as educators and researchers from many fields have documented, students often emerge from our courses without a deeper understanding of, or appreciation for, our disciplines. Further, they fail to acquire the skills and abilities we have worked so hard to help them develop. How, then, can we expect these students to go out into society and successfully engage with, and help solve, the complex and critical problems that face our nation?
In the wake of the recent US financial crisis, many institutions of higher learning have faced extreme budget cuts. Due to these cuts, faculty are being asked to teach in substantially larger classes with increasingly fewer resources. At the University of Arizona this issue has manifested itself in offering mega-classes, in the performing arts center, with enrollments from 700?1,400 students.
To address the challenges of teaching large enrollment courses, we have been engaging in a series of research studies into student learning in introductory astronomy GE college courses, commonly called Astro 101. This is a very popular introductory science course. Nationally, over 250,000 students take Astro 101 courses each year, and 10 percent of all college students take an Astro 101 course during their college education. For many of these students, this GE science class will be the final science course they take for the rest of their lives. This population of students is incredibly important to our nation?s well-being as they represent our future business leaders, politicians, journalists, historians, artists, and most importantly, societal leaders, parents, tax payers, voters, and teachers. The quality of education these students receive in these courses may, therefore, have a lasting impact on their scientific literacy, their attitudes toward science, and their decision whether or not to pursue a career in a STEM field. With so much at stake, it is clearly in our nation?s best interests to improve the teaching and learning happening in Astro 101 classrooms.
The Astro 101 course differs from other introductory college science courses in that it is intended for students from all majors; it is not the prerequisite for any other course and there is no commonly agreed upon set of topics for the course?essentially the charge is to teach the universe in a semester! On a practical level, the development of curricular materials for Astro 101 is constrained by the lack of recitation sessions, labs, and teaching assistants. The lecture portion of an Astro 101 course is commonly the only time instructors meet with their students. Therefore, instructional strategies must help students resolve conceptual and reasoning difficulties without significant help from the instructor, and they must be designed for use in large lecture halls with fixed seats. Furthermore, because new strategies can require instructors to give up precious class time normally spent lecturing, teaching innovations must be relatively brief.
The Center for Astronomy Education (CAE) at the University of Arizona has three programs dedicated to improving teaching and learning in Astro 101 classrooms. Foremost has been the development of research-validated instructional strategies shown to improve student understanding and evidenced-based reasoning abilities. With funding from NASA and the NSF, CAE has developed a national Teaching Excellence workshop series to increase instructors? understanding of collaborative learning activities, improve their pedagogical content knowledge, and elevate their implementation abilities. Over the past six years, these multiday, participation-based professional development workshops have been attended by more than two thousand instructors from all types of institutions (from research university to community college). Finally, with funding from NSF, CAE has developed a program to expand the number of instructors, postdocs, and grad students engaging in astronomy education research in their own classrooms. This program, called the Collaboration of Astronomy Teaching Scholars (CATS), has led to several nationwide research initiatives to improve the teaching and learning of Astro 101.
These CAE/CATS efforts have culminated in a multi-institutional study to assess the effectiveness of instruction in a wide range of different Astro 101 classrooms across the country. It is worth noting that these courses ranged from the extremely traditional, lecture-only format to the highly transformed classroom environment where students are engaged for approximately half of class time in sense-making discussions in collaborative learning groups involving guided-inquiry pencil-and-paper based Lecture-Tutorials and Ranking Tasks; student discourse intensive Think-Pair-Share questioning techniques using colored voting cards; and interactive-lecture methodologies focused on demonstrations and computer simulations. We want to emphasize that the instructional methodologies and research findings discussed are not discipline specific, and have been successfully used to transform learning in classrooms across many disciplines in the physical and life sciences. Following the presentation of the results of this study, we will discuss the programmatic and pedagogical factors involved in trying to successfully implement the same collaborative learning strategies into a mega-class of nearly 800 students at the University of Arizona.
The recently completed Astro 101 national study of nearly four thousand students at thirty-one colleges and universities involved sixty-nine class sections, taught by thirty-nine different instructors. Phase I of this research focused on the relationships between class size, type of institution, amount of course time spent using interactive learning strategies, and course-averaged student learning gains (Prather et al. 2009; Prather, Rudolph, and Brissenden 2009). Phase II of this work focused on an analysis of student responses to a fifteen-question demographic survey. A multivariate regression analysis was conducted to determine how ascribed characteristics (personal, demographic and family characteristics), achieved characteristics (academic achievement and student major), and the use of interactive learning strategies are related to student learning gains (Rudolph et al. 2010).
These studies show dramatic improvement in student learning with the increased use of interactive learning strategies even after controlling for individual and ascribed population characteristics. Classes that spent 25 percent of their class time (or more) using interactive learning strategies averaged more than twice the normalized gain scores as compared to classes that spent less than 25 percent of class time teaching interactively. Furthermore, we found no correlation between student learning gain and type of institution or class size (even in a class of almost 800 students, as we discuss in detail below). The wide range in learning gains observed for the high-interactivity classes suggests that the quality of an instructor?s implementation of interactive learning strategies may well be the most important factor in determining the learning gain of a class. These research findings help to bolster the argument that faculty professional development efforts focused on how to effectively implement active learning strategies are in great need within the college teaching community. Additionally, institutional and departmental support must be provided to faculty who work to transform their courses so that their careers are not penalized for bringing proven instructional strategies into the classroom.
Perhaps most important of all our findings was that the positive effects of interactive learning strategies apply equally to men and women, across ethnicities, for students with all levels of prior mathematical preparation and physical science course experience, independent of GPA, and regardless of primary language. These results powerfully illustrate that all categories of students enrolled in GE science courses can benefit from the effective implementation of interactive learning strategies.
Logistical Matters
Centennial Hall (CH), where Astro 101 mega-classes are taught, is a venue designed for theater, dance, ballet, and orchestral performances. There are no desktops for the more than two thousand seats. The seats are fixed and don?t swivel. The rows are packed very closely together, making getting to students with questions quite difficult. The lighting is much dimmer than a normal classroom. The podium, and all media and lecture controls, are located in the corner of a raised stage well above the first row of seating, far away from where you would like to stand while addressing the students. But the acoustics are fantastic! Members of CAE looked at the mega-course as a good opportunity to investigate whether findings from courses with approximately 150 students could still be achieved in this significantly larger classroom.
There were some logistical concerns we had to work through just to create a functioning classroom. Every fifth row of the class was initially blocked with caution taped to prevent students from sitting in these rows, allowing us to move easily throughout the class to assist students during collaborative group work. We had to formulate a complex flowchart, detailing where to go and what to do in order to make handing out and picking up paperwork (participation forms, homework, surveys, etc.) possible in only a few minutes.
For in-term examinations, we had to schedule Centennial Hall outside of normal class times in order to accommodate all the subtle issues of maintaining exam security and checking student IDs in a reasonable amount of time. We had to schedule a different large lecture hall (our former classroom held for 150 students) multiple times a week in order to accommodate office hours, as it is common for between 10 to 30 percent of the class to attend office hours. A zero tolerance cell phone and laptop policy was established from the start, and strictly enforced, to prevent hundreds of students from texting and using Facebook, Twitter, or YouTube during class. In addition, we chose to have multiple-choice exams, and make use of an online and auto-graded homework system, so as to reduce the number of hours needed for grading.
From a curriculum implementation standpoint, what was most challenging about this course was determining how to emulate the same vibrant and productive collaborative learning environment we had been able to foster in our 150?300 student courses. The university provided three graduate student teaching assistants (TAs). The astronomy department provided an additional graduate TA and one undergraduate astronomy major to help with grading. We knew we would need much more help to facilitate the in-class student discourse-intensive activities and to provide sufficient support in office hours. Our solution to this problem has come through what we call the Ambassador Program, a program that employs former students of the class to provide instructional help in the classroom. This program is modeled after key elements of both the Supplemental Instruction (SI) Program developed at the University of Missouri?Kansas City, and the NSF-funded Learning Assistant (LA) Model developed at the
University of Colorado at Boulder. Through the Ambassador Program, former students who have demonstrated a high level of content understanding (having received an ?A? in the course), and demonstrated strong communication skills, are recruited to become TAs.
Impacts of the Program
From interviews with, and evaluations by, other students in the class we have learned that these Ambassador TAs are often preferred by students over the graduate students or even the instructor when they find themselves in need of help in class or office hours. The popularity of this program is evidenced by the increased number of students who state that they are attending office hours with the sole purpose of trying to get an ?A? in the course so that they can become the next semester?s Ambassadors. With the Ambassador Program we have found a pedagogically sound solution to an important instructional resource issue, and elevated the conceptual understanding and science literacy of a group of nonscience majors who have become skilled and eager to share their knowledge with others, and who will carry that ability and desire into their roles as members of our society.
The broader impacts of this program are now being felt in our second year as we see the role of returning Ambassadors elevated to astronomy education researcher. A cadre of these Ambassadors have engaged in a self-directed research program to investigate the relationship between the level of correctness and coherence in students? written responses to in-class and ungraded collaborative learning activities (Lecture?Tutorials) with the students? performance on corresponding questions on exams and concept inventories. The goal is for this work to lead to a published peer-reviewed science education journal article. With this work we see the progression of participants in the Ambassador Program from high-achieving nonscience majors taking a GE course, to peer-teaching assistants within the course, to astronomy education researchers evaluating the success of the course.
To expand the opportunities for highly motivated GE students beyond the Astro 101 classroom, the college of sciences at University of Arizona (UA) has recently created a liberal arts minor in astronomy. Through this pathway, nonscience majors will be able to graduate with a degree in astronomy/science and participate in a program that directly supports STEM growth. In a recent survey approximately 10 percent of students in UA Astro 101 classes stated they are very interested in pursuing this minor. At UA alone this could result in two hundred nonscience students coming into the astronomy liberal arts minor program every year. This will represent a significant increase over the handful of students who have changed their major to astronomy over the years.
While the Ambassador Program and liberal arts minor will enhance the opportunities for a subset of the nonscience majors, the question still remains: Does the 800-student mega-course achieve its goals for the majority of students? By a number of measures, the answer is yes. Exam averages are comparable to those of the prior seven years of Astro 101 courses with enrollments of 150+ students. More important, class-averaged normalized gain scores on two different research-validated concept inventories, the Light and Spectroscopy Concept Inventory and the Stellar Properties Concept Inventory, are among the highest in the nation.
Caution in the Face of Success
Evidence of successful student learning in a mega-course of 800 students is cause for excitement but also concern. Given the realities of current and future college budgets, especially at state universities, it is exciting to think that we can truly educate these students in such a setting. However, the danger comes when such a finding is misused. University leadership will undoubtedly use these positive research results to defend the teaching of mega-courses to their many stakeholders. We are concerned, however, that they will also promote the creation of these courses without providing the instructional resources, and advocating for the pedagogical practices, that are necessary if one is to create an active learning environment that leads to student success.
First and foremost our results illustrate the need for instructors to move from a professor-centered to a learner-centered teaching approach that involves effective implementation of interactive learning strategies. Without this shift in the framing of the classroom, one could ask why it wouldn?t be financially sounder to simply produce high-end lecture videos of the class for students to download and watch on their own time. We fear this could well be advocated as the next step.
While our research results document that students develop improved conceptual understanding and reasoning abilities related to key astronomy topics, for those of us who have broader goals for our courses, there are still many unanswered questions. For instance, does their increased understanding of astronomy last? To what extent has their understanding of the critical role science plays in our society improved? Have we helped to create citizens capable of intellectually engaging in the issues we face as a nation? The research to answer these and other questions is being pursued in many fields, including astronomy education research, but this work is in its infancy in comparison to the research results on students? conceptual understanding of the discipline. One of our goals with this work is to motivate a national conversation among GE instructors from all topics about the importance of such goals, and of the need to conduct research about how gen. ed. science courses can best accomplish these goals.
In conclusion, we wish to thank the thousands of students for all their hard work and contributions to helping us better understand how to teach and better meet their needs as learners.
References
Prather, E. E., A. L. Rudolph, G. Brissenden, and W. M. Schlingman. 2009. ?A National Study Assessing the Teaching and Learning of Introductory Astronomy. Part I. The Effect of Interactive Instruction.? American Journal of Physics 77 (4): 320?330.
Prather, E. E., A. L. Rudolph, and G. Brissenden. 2009. ?Teaching and Learning Astronomy in the 21st Century.? Physics Today 62 (10): 41.
Rudolph, A. L., E. E. Prather, G. Brissenden, D. Consiglio, and V. Gonzaga. 2010. ?A National Study Assessing the Teaching and Learning of Introductory Astronomy. Part II. The Connection between Student Demographics and Learning.? Astronomy Education Review 9, 1.
The University of Arizona Ambassador Program
The Ambassador teaching assistants go through a rigorous screening process before being hired. Students are paid $8/hour for their work, including in-class teaching, holding two office hours a week, and for attending training sessions. Every week students receive three hours of intense pedagogical training to improve their understanding of key implementation and conceptual issues regarding the following week?s instructional strategies and discipline content. Our Ambassador Program is quite different from many other peer instruction programs since it does not involve majors from within our discipline (astronomy), but rather students who are almost exclusively nonscience majors.
These Ambassador TAs have proven to be exceptionally talented at facilitating Socratic dialogue with students struggling through the conceptually challenging collaborative learning activities used in the ?lecture? portion of our mega-classes. We believe this ability comes from the fact that the Ambassadors were recently students in the class and, therefore, have a first-hand student perspective of how the course really works. Unlike many of our astronomy and astrophysics graduate TAs, these Ambassadors share a common understanding of the metaphors and analogies one might use to engage in a discussion that would help general education students overcome learning difficulties.
