Kristin Bonnie, assistant professor of psychology at Beloit College (Wisconsin), was curious about her students? performance on exams in the introductory psychology course. As she explained to a packed room at the 2011 annual meeting of the Association of American Colleges and Universities (AAC&U) in San Francisco, she had been giving her students, mostly first-years, the choice of deciding not to answer two or three of the approximately twenty-five multiple-choice questions that appeared on each exam. In some cases, students answered every question anyway, before indicating which ones they did not want graded.
Bonnie?s curiosity was partly just the response of a good teacher and researcher wanting to know more about student learning in her classes. But her questions were considerably deepened and developed?and connected to ongoing discussions about metacognition?by her participation in the Teagle Foundation?funded Collegium on Student Learning through the Associated Colleges of the Midwest (ACM).
In November 2008, participants in the ACM collegium embarked upon a thirty-month project to examine recent work in the cognitive sciences, to test out the theories through classroom interventions and experiments, and, ideally, to improve student learning through the process. They focused especially on the importance of metacognition, which might be summarized as knowledge of one?s own thoughts and the factors that influence one?s thinking. Other researchers emphasize the ability to plan, monitor, and evaluate the learning process as key elements of metacognition.
As Bonnie and her colleagues reported at the annual meeting, that focus not only had good effects on student learning but often a profound effect on the teachers. In creating scholarship of teaching and learning (SoTL) projects that documented their questions and interventions, and working as part of a group of scholars, collegium members reported becoming significantly more thoughtful about their teaching practice.
Bonnie was joined in the AAC&U session by David Thompson, associate professor of Spanish at Luther College (Iowa); Holly Swyers, assistant professor of anthropology at Lake Forest College (Illinois); Karl Wirth, associate professor of geology at Macalester College (Minnesota); and John Ottenhoff, vice president of the ACM. They worked with a dozen colleagues from other ACM colleges in the collegium, which began with an opening conference featuring a keynote address from Patricia M. King, whose work on reflective thinking and self-authorship helped shape the thinking of the participants. Classroom interventions were carried out over the 2009?10 academic year, and a final conference about the group?s work was held in October 2010.
Throughout the process, as reported in San Francisco, the group found that metacognition was by no means a ?silver bullet? for improving student learning, but nonetheless was an effective tool for focusing students? attention more consciously on their learning and, ultimately, providing a means to encourage students to think about the larger purpose of their education. Perhaps as important, the collegium group found that by asking metacognitive questions of students, they became both more aware of their students? learning and increasingly self-reflective about their own teaching practices and effectiveness.
David Thompson?s work exemplified well the point about faculty learning. He reported that his early questions about the effects of cumulative testing on increasing students? ability to monitor their learning in Spanish classes prompted him to set up separate ?control? and ?intervention? sections of intermediate Spanish at Luther. Finding no correlation between cumulative testing and increased self-monitoring, he began introducing students to more explicit self-monitoring processes, including post-assignment and post-exam ?wrappers,? brief writing exercises that asked students to reflect on their learning process both before and after seeing their graded tests.
Again, Thompson?s results were mixed; increased metacognitive skills, as measured by the Metacognitive Self-Regulation subscale of the Motivated Strategies for Learning Questionnaire (Pintrich 1991), didn?t necessarily lead to better learning of Spanish. But in comparing the gaps between student predictions and performance on exams, he came to see that instruction in metacognitive skills may be particularly important for first-year students as they adjust to the expectations of college-level work and learn to evaluate and monitor their own understanding relative to those expectations. Tracking metacognitive growth in first-year students and in the weakest test-performers gave Thompson a new focus in his teaching, which he will carry forward in a recursive cycle of new interventions and modifications.
As is true of most good SoTL projects, inquiries into student learning begat further questions and more reflection about the practice of teaching. Thompson learned that as a humanities scholar engaging in this kind of research, he needed to find more suitable methods; humanities approaches such as discourse analysis might serve better than control groups for shedding light on his questions. This realization was made possible in large part by the mix of peer and expert support provided by the collegium, which offered the encouragement and framework for Thompson?s initial foray into the literature of metacognition and the scholarship of teaching and learning.
Both Thompson and Bonnie were influenced by Karl Wirth?s work on ?knowledge surveys? as a central strategy for helping students think about their thinking. Knowledge surveys involve simple self-reports from students about their knowledge of course concepts, content, and skills, Wirth explained at the AAC&U session in San Francisco. In knowledge surveys, students are presented with detailed content and skill objectives for each topic and are asked to indicate their perceived mastery of each. Faculty can use these pre- and post-reports to gauge how confident students feel in their understanding of course material at the beginning or end of a course, before exams or papers, or even as graduating seniors or alumni.
Wirth noted that the surveys need not take much class time and can be administered via paper or the web. The surveys can be significant for clarifying course objectives, structure, and design. For students, knowledge surveys achieve several purposes: they help make clear course objectives and expectations, are useful as study guides, can serve as a formative assessment tool, and, perhaps most critically, aid in their development of self-assessment and metacognitive skills. For instructors, the surveys help them assess learning gains, instructional practices, and course design.
Wirth?s San Francisco presentation featured several charts showing how knowledge surveys matched up with student performance on exams. Perhaps most strikingly, Wirth found that students in the lower quartile of performance on exams were least able to predict their performance; students who performed best were often likely to underestimate how well they understood the material.
Wirth?s collegium work also involved a collaborative project with Fahima Aziz of Hamline University (Minnesota) on the use of ?reading reflections,? another attempt to help students monitor their learning through brief online writing about their reading assignments. According to Wirth, expert readers are skilled at using a wide range of strategies during all phases of reading (e.g., setting goals for learning, monitoring comprehension during reading, checking comprehension, and self-reflection), but most college instruction simply assumes the mastery of such metacognitive skills.
In making the reading reflections a regular part of their courses, Wirth and Asiz concluded, ?there is no longer any question in our minds whether reading reflections are a powerful intervention for improving learning. With reading reflections, students read more regularly before coming to class, they read more deeply, and they use a wider range of reading strategies. They are better prepared to participate in, and learn from, classroom activities resulting in deeper content learning. There is also evidence that this intervention might hold even greater potential for underperforming students.?
Other members of the ACM collegium group experimented with various forms of knowledge surveys, exam wrappers, and reflective writing. The common theme was that metacognitive awareness didn?t lead directly to greater mastery of course content, but helped improve the focus of instruction and especially seemed to benefit students who tended to perform poorly. For example, Tim Tibbetts (Monmouth College, Illinois) found in his introductory biology classes that ?reading reflections give me a tool to hear where students are struggling and respond, knowledge surveys help students see what topics are important and what types of questions they should anticipate on exams.? He also found clear improvements in learning outcomes for the students who did exam wrappers.
Diane Angell (St. Olaf College, Minnesota), also a biologist, found that metacognitive assignments in the form of ?exam preparation assignments? and ?wrappers? produced a consistent, if small, effect on improving student learning. She speculated that even more explicit metacognitive instruction, especially for underprepared students, would be helpful. Clara Hardy (Carleton College, Minnesota) concentrated on making learning strategies more explicit in her introductory Latin classes. Her conclusions, based on a small sample, were that meta?cognitive activities were especially helpful for the very lowest-achieving students, who in other years did not complete the course successfully. She also found that exam wrappers for the higher-achieving students were much fuller in their accounts of what they had tried and how they had thought about what to try than were the lower-achieving ones.
The collegium also produced some interesting collocations. At the final conference for the collegium, held at Macalester College in October 2010, Joy Jordan (Lawrence University, Wisconsin) reported about some meta?cognitive interventions in her intermediate statistics course in the same session in which Kent McWilliams (St. Olaf College, Minnesota) reported on his piano performance class. Jordan concentrated on helping students learn the essential but difficult concept of sampling distribution, focusing especially on reflections about ?confidence judgments.? She reported being most surprised by the ?repeated overconfidence in the lower-performing half of the class? and became interested in what happens when these students, working in groups, become more confident even though their understanding may not have changed. McWilliams also became intrigued by the issue of confidence and by what changed as he asked his piano students to be consistently self-reflective while they learned a new piece of music. He reported that students found the metacognitive framework helped them pose a wider range of questions, from well-structured questions to higher-level ill-structured questions, and that they seem to have been successful in transferring these same learning processes to other repertoire they studied.
Holly Swyers, on the other hand, used meta?cognition as a common language in a ?pod? of three first-year seminar classes, focused in the diverse disciplines of anthropology, chemistry, and education, at Lake Forest College. The ?pod? came together throughout the year with a team of colleagues, including not only the faculty teaching the courses but also a coach, a public safety officer, student-life professionals, and learning specialists. As she described in San Francisco, Swyers saw the real breakthrough in the project in the value of a shared vocabulary for talking about what is happening in student learning. ?The metacognition frame proved adaptable by all members of the pod, so students would hear the same ideas in the classroom, in the dorm, in study sessions, and on the playing field,? Swyers reported. ?Most members found the principles very similar to ideas they already had in practice, so the real value was in helping students see that the overriding premise of all their college activities was consistent.?
Metacognition is a topic that has attracted increasing attention nationally, starting with the groundbreaking How People Learn (Bransford, Brown, and Cocking 2000), and the ACM collegium offered some intriguing insights into how abilities in self-monitoring and awareness of the learning process can improve learning. The ACM-Teagle collaboration also delivered significant insights into how successful faculty development work can occur.
Quite simply, successful faculty development takes time, particularly if it involves work (like the scholarship of teaching and learning) with which faculty are not familiar. Successful projects are long-term, blending support and accountability. The collegium project, initially funded for thirty months, has been extended, as members of the group continue to collaborate on projects and discussions. Throughout the project, requests for project proposals, updates, and final reports kept participants focused and engaged?and provided opportunities for discussion and suggestions.
Faculty projects such as the Collegium on Student Learning also require collegial conversations nurtured carefully with appropriate support. All the faculty involved in this project engaged in common work, even though they came from disciplines ranging from classics to statistics and geology to music. In order to support this collaborative faculty work, the group reaffirmed the need for an egalitarian ethic, substantive collegial interaction, and a supportive intellectual community that inspired and transformed teaching practices. Working together with agreed-upon goals around a common issue (in this case, student learning and metacognition) led to a natural progression of discussions concerning common practices inspired by issues raised by classroom practices. This collegial support was enhanced by the creation of small cohort groups of faculty members who worked closely together across disciplines and institutions.
Colleagues in institutions with a history of close collaboration through the consortium found a network of support and challenge, as well as a way to reduce the isolation that commonly characterizes faculty research. This isolation was further reduced through the use of a project website that allowed for sharing of relevant materials and resources and for continuing online conversations about work in progress. As several members of the group observed, metacognition itself became a fruitful concept for increasing the cohesiveness of the group.
Finally, the work initiated in the ACM-Teagle Collegium project signals a shifting attitude about who learns in college. As Gerald Graff noted in speaking about his Clueless in Academe, ?We?ve gotten accustomed to a system in which the very few excel in school (and reap the rewards in the vocational world beyond) and the many stumble along and more or less get by, or get through, or fail. In some ways such a system suits us academics?it?s not our fault if the majority stumble or fail, we can easily say, that?s just the way it is; only an elite in any society is going to ?get? the intellectual club? (Warner 2003). Metacognitive interventions, this project suggested, may be an especially powerful tool in helping the ?academically adrift? student find a way to get into the game, to become more aware of the kind of thinking that supports strong academic performance. And while that?s not the whole answer to the problems of ?limited learning? on our campuses, it?s certainly something worth thinking about.
Bransford, J. D., A. L. Brown, and R. R. Cocking. 2000. How People Learn: Brain, Mind, Experience, and School. Washington, DC: National Research Council.
Pintrich, P. R., D. A. F. Smith, T. Garcia, and W. J. McKeachie. 1991. A Manual for the Use of the Motivated Strategies for Learning Questionnaire (MSLQ). Ann Arbor: University of Michigan.
Warner, J. 2003. ?Clueless in Academe: An Interview with Gerald Graff.? The Morning News, September 16, http://www.themorningnews.org/article/clueless-in-academe-an-interview-with-gerald-graff.
John Ottenhoff is vice president of the Associated Colleges of the Midwest (ACM). On behalf of the ACM, the author expresses deep gratitude to the Teagle Foundation for funding the project described in this article and for its record of extraordinary support for liberal education. The author also thanks Kristin Bonnie, Rachel Ragland, David Schodt, Holly Swyers, and David Thompson for their contributions to this article.