January 26, 2011
What should a college graduate know and be able to do? There are as many views on that as there are colleges (thousands), if not individual professors and students (many more).
The diversity of opinions about what a college education means has long been seen as a strength of American higher education. But in recent years, many employers and policy makers have argued that the lack of a common definition of what students should know and be able to do -- and a dearth of adequate methods of gauging whether they know it and can do it -- has contributed to a decline in the quality of higher education and to the awarding of more degrees, but of lesser value.
The push to set such standards at a national (and even federal) level has, in turn, led many college and faculty leaders to complain that a one-size-fits-all approach to defining student learning would result in greatly oversimplified definitions of learning.
To try to provide a shared understanding of what degrees mean -- but without, its designers insist, turning that into a government or other mandate -- the Lumina Foundation for Education is today releasing a draft of its Degree Qualifications Profile, created by four leading higher education researchers and policy experts (more on them later).
The profile, around which Lumina officials plan to begin a several-year discussion in which colleges, accreditors and other groups will test and refine it, is intended to establish, in more specific ways than has historically been the case, what the recipients of associate, bachelor's and master's degrees (regardless of discipline) should know and be able to do.
"There is no generally accepted understanding of what quality represents in American higher education," says Jamie P. Merisotis, Lumina's president and chief executive officer. "Our view is that the absence of that shared understanding of what quality means has resulted in employers being dissatisfied with what higher education produces; policy makers being unsure of their investment; and students and families having a lack of clarity about choice of institutions and the relative value of their degrees."
The degree framework, he says, is designed to help develop that shared understanding across majors, programs and institutions. Paired with the foundation's other major student learning project, known as "tuning" -- which focuses on what degree holders should know within specific disciplines -- the qualifications profile could help create a common definition of what a college credential should represent, he said, so that better tools (portfolios, projects, and tests) can be developed to measure how well students are actually accumulating the requisite knowledge and skills.
Lumina officials describe the qualifications profile -- loosely modeled on the degree frameworks that the European Union and other countries have adopted -- as potentially "transformational" for American higher education, but they acknowledge that a great deal would need to happen for it to have anywhere near that effect. Several outside experts that Inside Higher Ed asked to assess the standards saw significant hurdles to its adoption by colleges -- especially overcoming the faculty skepticism about any effort that seems to be dropped from on high and that can be seen as substituting national standards for institutionally based ones.
"A learning framework that really promotes student success has to be developed at the local level and has to be led by front-line faculty and staff," Larry Gold, higher education director for the American Federation of Teachers, said via e-mail. "Nothing is less likely to help students succeed than an overly standardized curriculum and assessment regime imposed from the outside. So the Degree Profile can make a contribution if the efforts coming out of it are developed and driven by front-line faculty and staff at the institution and if these efforts receive the kind of continuing support that makes them sustainable -- particularly investment in the faculty and staff charged with making the program succeed."
The Genesis
Lumina first began floating the idea of developing a qualifications framework about 18 months ago, in a series of behind-the-scenes meetings with higher education leaders and policy makers. Foundation officials portrayed a commonly agreed-upon understanding of what a college degree means as an important tool to knocking down some of the barriers that inhibit college completion, which has been at the center of Lumina's agenda for the last several years. Those barriers include the difficulty of transferring academic credit from one institution to another, and the failure to recognize the value of learning that takes place outside traditional classroom settings, which often goes unrecognized in a system in which the "credit hour" is the coin of the realm.
But Lumina's leaders also saw the profile as key to ensuring that not only are more degrees awarded, "but that those must be of high quality," said Holiday Hart McKiernan, senior vice president and general counsel at Lumina.
The foundation assembled four longtime experts on assessment and student learning -- Clifford Adelman of the Institute for Higher Education Policy, Peter Ewell of the National Center for Higher Education Management Systems, Paul Gaston of Kent State University, and Carol Geary Schneider of the American Association of Colleges & Universities -- to "reduce to writing, without worrying about implementation, what would be the core competencies" for recipients of various degrees. The four researchers spent months poring over the comments of outside reviewers and, not surprisingly to those who know the four of them, debating intensely among themselves.
Their final product divides the desired skills and knowledge into five basic areas -- specialized knowledge; broad, integrative knowledge; intellectual skills; applied learning; and civic learning -- and lists within each the key learning outcomes in which a student should be competent, with the expectations for students increasing as they move up the three degree levels. For specialized knowledge -- knowledge acquired in a specialized field of study, for instance -- the profile lists the following outcomes at different levels:
The associate degree holder:
* Describes the scope and principal features of the field of study, citing core theories and practices, and offers a similar explication of a related field.
* Illustrates the field’s current terminology.
* Generates substantially error-free products, exhibits, or performances in the field.
The bachelor's degree holder:
* Defines and explains the boundaries, divisions, styles and practices of the field.
* Defines and properly uses the principal terms in the field, both historical and contemporaneous.
* Demonstrates fluency in the use of tools, technologies and methods in the field.
* Evaluates, clarifies and frames a complex question or challenge using perspectives and scholarship from the student’s major field and at least one other.
* Constructs a project related to a familiar but complex problem in the field of study by assembling, arranging and reformulating ideas, concepts, designs or techniques.
* Constructs a summative project, paper or practice-based performance that draws on current research, scholarship and/or techniques in the field.
The master's degree holder:
* Elucidates the major theories, research methods and approaches to inquiry, and/or schools of practice in the field; articulates relevant sources; and illustrates their relationship to allied fields.
* Assesses the contributions of major figures and organizations in the field; describes its major methodologies and practices; and implements at least two such methodologies and practices through projects, papers, exhibits or performances.
* Articulates major challenges involved in practicing the field, elucidates its leading edges, and delineates its current limits with respect to theory, knowledge and practice.
* Initiates, assembles, arranges and reformulates ideas, concepts, designs and techniques in carrying out a project directed at a challenge in the field beyond conventional boundaries.
Lumina officials and the authors say they went out of their way to fill the degree profile with "concrete, illustrative student learning outcome statements" that use "active verbs that tell all parties -- students, faculty, employers, policymakers and the general public -- what students actually should do to demonstrate their mastery."
That may be true compared to other documents of its kind, but even some of the outside reviewers -- many of whom were higher education researchers -- described the profile as "abstract." Lumina officials admit that the document is complex, and that the full impact of the degree profile won't be felt unless and until college leaders and their faculties begin discussing whether and how the qualifications laid out in the profile apply to their own students, and how they might go about documenting that the students to whom they award degrees have accumulated the knowledge and skills in question.
Most of the outside experts with whom Inside Higher Ed shared the degree profile said they found little to argue with in the qualifications and outcomes put forward by the four authors. "This is a very good document that shows where higher education should end up," said Michael Poliakoff, policy director at the American Council of Trustees and Alumni. "It is very strong in stressing the fact that this needs to be an institutionwide conversation, and ensures that there is thorough integration of these learning elements" across academic programs.
But like others interviewed, Poliakoff stressed that without a clear sense of how (and whether) it can be implemented on campuses, the profile will make little difference. "Without getting it down to the level of requirements and assessment, it will fall short of its full impact," he said.
Richard Arum, whose new book, Academically Adrift, gives observers of higher education new ammunition with which to question how much learning is taking place on campuses, said that he and his co-author, Josipa Roksa, were impressed on first review of the Lumina document. "It draws attention to what students know, rather than to the general 'let's count course credits and assume they've gotten something out of it' approach to students," he said. "If colleges and universities did this, it would shift things in a positive way."
Arum described himself as "skeptical," however, about "where the incentives are to get the colleges and universities to do this, to decide 'this is a useful framework to us.' "
Like Gold of the AFT, Arum argued that "the only way [that higher education can successfully] focus on improving student learning is with the faculty at the center of it," but added that "learning has been pushed to the periphery of higher education, just the way faculty have been pushed to the margins."
If the degree profile provides a "common vocabulary about what students can do" around which trustees, administrators and instructors can ultimately have a discussion about what they want students on their individual campuses to know and be able to do, it could have the sort of "transformative impact" that Lumina envisions, Arum said. "You could imagine trustees or regents, using this framework or something like it, saying to administrators, 'What are you doing to ensure that our students can do these things? Show me the plan you have for making sure they do, and show me the evidence that you're accomplishing this.' If that leads campuses to use what's out there and identify better ways to measure higher-order skills, this could be a useful tool."
The Lumina Strategy
Lumina officials, who have in the past been accused of pushing initiatives on colleges (and especially the foot-draggers on campuses) rather than collaborating with them, say they know that the degree qualifications profile project will work only if institutions (and their faculties) buy in. "We do not think this is something that should be imposed on [faculty members], ... but they do bear the brunt of failure if [higher education] does not deliver on defining quality," said Merisotis. "We had a really good faculty-level conversation about this, and we think we have a good plan for involving them. Are we expecting universal harmony about this? Heck no. But I think we’ll be very transparent about what it is, and help people digest it."
The foundation plans to enlist several groups to help it test and refine the degree profile, including, to start, two accrediting agencies (the Western Association of Schools and Colleges' senior college commission and the Higher Learning Commission of the North Central Association of Colleges and Schools) and the Council of Independent Colleges.
Ralph Wolff, executive director of the Western accreditor, said in an interview Monday that the association had created a panel that will decide whether and how the agency might incorporate the qualifications profile into its standard aimed at ensuring that institutions have clearly defined graduation requirements and levels of achievement for degrees. "Whether it becomes just a suggested framework or is actually incorporated into the standards is very premature," Wolff said.
"For some people, this could raise a series of concerns about homogenizing curriculums and setting outcomes for institutions, and all those issues will be fair game in our discussions," Wolff said. "But the authors seem to have struck a balance between writing [the learning outcomes] with enough specificity and clarity that they have meaning, yet enough diversity and flexibility so that different institutions can adapt it to fit their kind of education. We see the idea having enough potential to explore further."
Richard Ekman, president of the Council of Independent Colleges, said he and his group have come around after some early skepticism about the Lumina project, which he said he found "really unappealing" from early discussions with Lumina. In those planning meetings, he said, Lumina officials described the project as designed to "ensure comparability" of American degrees with those in the European Union (which has embraced its own qualifications framework through the Bologna accords). And Lumina was also, he said, "talking about working hand in hand with the [Obama] administration," foreboding the kind of federal involvement in curricular and other matters that private colleges like Ekman's members tend to loathe.
But Lumina's language has changed, Ekman said. "They're now talking about these frameworks as a way of working toward higher quality across a variety of American colleges and universities in a voluntary way, and contributing toward this ubiquitous discussion without putting it in the context of standards that are in the control of the government. This is now a concept that I believe is worth our exploring."
Ekman said that teams of presidents, faculty leaders and assessment experts from roughly 25 campuses will work on the CIC project, with the ultimate goal of "finding language that might work for a broad group of our institutions." That could be a challenge, he acknowledged, because the group's members include small institutions that do nothing but liberal arts and other institutions with more professional orientations.
"I think the general categories are fine, but the test will be making them fit what colleges actually do," Ekman said. "I made it clear I wasn't buying into it 100 percent, but that I did think it was concept worth exploring. I'm pleased that Lumina is willing to take a flyer on us."
Is Commonality the Goal?
In releasing the profile, Merisotis and others at Lumina have emphasized that one of its goals is in creating a "shared understanding of what a degree represents in terms of learning," as he put it. But how common must a common framework be to be valuable?
Perhaps the biggest tension in the last few years' debates over higher education accountability has been the pressure on the part of some policy makers to insist on outcomes and measures that are comparable across programs and institutions, on the theory that to make decisions among colleges, consumers need the same information from all of them. But that, critics argue, leads to reductionism that helps no one.
Adelman, the former Education Department researcher who is among the profile's authors, says he envisions colleges engaging in an "iterative process" that leaves lots of room for individual institutions, or groups of them, to go their own way in crafting degree profiles that work for them.
Imagine the degree profile, he says, as the outline of Alfred Hitchcock that came to be an iconic representation of the filmmaker. Then imagine provosts, deans and professors at a lot of individual colleges, or associations of them, discussing which learning outcomes in the profile work for them just as-is, which they'd jettison, a few they'd add.
"College X's portrait might look like a Gauguin, another's might be a Dürer," he said. "Three years from now, you might find 24 different versions, but they'd have the same reference point. They'd have the same form, and be based on the same palette -- the Hitchcock outline."
That result isn't neat and clean, perhaps, but it stands independently of government, and "it's a heck of a lot better than what we have now," Adelman said -- a situation in which politicians perceive colleges to pay too little attention to learning and constantly threaten to wade in and fix the problem.
And they will, Adelman warned, if colleges (and accreditors, whom he sees playing a key role) don't confront their perceived failings themselves. That may be difficult, he acknowledged, but the Lumina project offers a pathway toward doing so. "If higher education runs away from this challenge, it will lose all its claims to sanctity" on questions of student learning.
—Doug Lederman
Inside Higher Education: What Degrees Should Mean
