Feb. 14, 2005 Reality Check

Missing the Mark: Graduation Rates and University Performance

By John V. Lombardi

In the quest to measure university performance, both for valid or other reasons, we often latch on to statistics in ways that do more harm than good.

SAT scores, GPA’s and graduation rates are all examples. Each of these data points provides some information about some characteristics of students and their universities, but none of them can serve as the ultimate test of institutional or individual worth. Universities and colleges are complicated places that serve widely varying individual students, each with a particular set of abilities, resources, preparation, background, attitudes, and goals. No single number captures this complexity, and when we extract simple numbers from this complicated mix and use them to compare colleges, we usually generate cynicism, misinformation, and, on occasion, misplaced remedies for inaccurately identified problems.

Let us take the graduation rate. What could be simpler or easier to understand? This rate measures the percentage of those students starting college in year one who graduate by the end of year four (or for some purposes year six). The four-year rate reflects the ideal model of the small liberal arts college where students attend full time for four years to complete a program that requires exactly eight semesters of academic work. Although we do not say so, we also have in the back of our minds that perhaps the very best college of all graduates 100 percent of its students within the four years. When a university graduates only 70 percent or 60 percent within the four-year period, we wonder why it fails to educate all of its students.

University people know better, but the answer, being complicated, fails the Lombardi 25 word test: “If you can’t explain it in 25 words, most people won’t listen.”

Universities and colleges do want their students to graduate, and to do so having taken a rigorous course of study that challenges them. The institutions want to accept some students whose academic promise may exceed their academic preparation for college, and to accommodate students whose financial circumstances require them to work and attend part time. So, they try to explain graduation rates with a few more than 25 words.

Most colleges and universities institutions do not fit the full-time, small residential liberal arts college model. Some have programs like engineering that require five years of full time study, or pharmacy programs that also require more than the standard four years. Many institutions specialize in high-risk students, and by definition, many of these students will not succeed. Many students register for only 12 credits a semester because that is how the federal government defines full time for financial aid, yet to graduate in four years, students must take 15 credits. The graduation rate includes students who register for 12 credits, so by definition many students will fail to meet the four-year goal.

In large complex university systems, students frequently start at one campus, then transfer and finish at another. These students may transfer because they choose to pursue a program the first institution doesn’t have, or because they couldn’t gain admission to a highly selective major in their first institution, or found they preferred a larger or a smaller institution, or wanted to be closer to home, or had financial difficulties at the first campus. None of these reasons necessarily provides cause for alarm, but students leaving for these reasons will lower the graduation rate.

Even so, some reasons for low graduation rates do require attention. Sometimes students drop out because they cannot find seats in required classes or because they fail classes. Students can become isolated and discouraged and drop out; they can find themselves distracted by social and other non-academic activities and neglect their class work. Sometimes these failures reflect weaknesses in campus support services, and with careful attention to academic advising along with effective student support programs, campuses can reduce the dropout rate and improve the graduation rate.

When we use graduation rates to compare campuses and infer from these comparisons that one campus is doing a better job of educating students than another, we exceed the value of the data point. It may be that the campus with a high grad rate is simply giving away its degrees (after all, a diploma mill has a 100 percent graduation rate). It may be that the campus with a low rate is working with high risk students. Or, it may be that the high rate campus is doing a superb job of advising and the low rate campus is ignoring its students.

The grad rate doesn’t tell which answer is right. The right answer requires a detailed analysis of what happened to the students who dropped out, and that takes more than one data point. If the campus improves its own graduation rate, that is probably a good thing, but comparisons of grad rates among campuses usually produce more distortion and misunderstanding than progress.

State colleges with lower tuition usually have lower graduation rates than private colleges with high costs because they accept a wider variety of often less affluent students. Private college parents, who pay a lot of money for their children’s education, expect the college to find a way to graduate those students, and paying parents encourage their students to finish in the minimum four years. Sometimes this produces grade inflation as eager institutions seek to meet the expectations of paying parents.

As most university administrators know, graduation rates are very different for different subsets of students within the same campus. Some groups will graduate at rates approaching 100 percent, others at rates well below 40 percent. Both sets of students attend the same institution, have access to the same classes and faculty and advisers, and experience the same intellectual, cultural, and social environment. They have remarkably different graduation rates because they are remarkably different students.

Some of those who succeed easily are smart and academically well prepared, others who fail are also smart and academically well prepared but perhaps not yet ready to live away from home or experience a large university. Some of those who fail are poorly prepared and economically challenged, but others in similar circumstances meet the challenge and succeed. Improving graduation rates requires attention to the specific needs of significantly different groups of students.

If we overemphasize graduation rates, colleges may suffer the temptation to do whatever it takes to get the high percentages celebrated by rating agencies and trustees. One way to help improve the rate, after all, is to inflate grades so that all students are above average, and water down the curriculum so everyone can get a degree — not outcomes of much interest to the policy makers, board members, or politicians who tout graduation rates as the touchstone of college performance.

Comments