Breaking the College Rankings System

Higher Voltage is brought to you by eCity Interactive. For over 20 years, eCity Interactive has created websites and digital marketing strategies and solutions for colleges and universities that deliver results and exceed expectations. Their latest offerings to higher ed clients include enrollment funneled by diagnostics and enrollment support services that efficiently attract and engage potential applicants with results you have to see for yourself. To learn more, visit ecityinteractive.com.

Hello and welcome to Higher Voltage, a podcast about higher education that explores what’s working, what’s not, and what needs to change in higher ed marketing and administration. I’m your host, Kevin Tyler. Welcome back to Higher Voltage. It’s been a while since we’ve been together, but I’m so excited for today’s guest. Today we have Colin Diver, professor emeritus at the University of Pennsylvania Law School and former president of Reed College, one of the trailblazers in the rankings conversation. He has a new book out called Breaking Ranks: How the Rankings Industry Rules Higher Education and What to Do About It. This is an exciting read. I loved it.

I’m glad to have this conversation with you, Colin. I love your bio, which reads, “I am active on LinkedIn and very inactive on Facebook.” I think that is the most powerful way to say it. It’s a great line. I’d love for folks to hear a little bit about your background outside of the items that I’ve just rattled off about where you are now and where you’ve been, but just kind of what got you into this conversation around rankings before we dive into some of the more meaty parts of this conversation.

Sure. Kevin, I’d be happy to. My academic career started in professional schools. I was teaching in a business school in a law school, and eventually, like many of my colleagues, blundered my way into administration and I became a dean. That was at the University of Pennsylvania Law School, as you mentioned. During the time, the very first year, in fact that I was dean at Penn, U.S. News & World Report started to rank law schools. Prior to that, of course, they had ranked colleges, but they decided they would expand their empire and get into ranking of professional schools, so they ranked law schools for essentially the rest of my 10 years as dean at the law school. That taught me to hate the rankings.

But then I had the odd experience, unusual experience of becoming the president of a college that had renounced the rankings, Reed College. Prior to my becoming its president, had decided that they would not cooperate with the U.S. News rankings. I of course had to decide whether to continue that policy, which I happily did, but I also had to decide how we could be honest and not boast about how we’re doing in other rankings because after all, there are a lot of different rankings, not just the U.S. News rankings, so pretty soon we had pulled out of virtually all the rankings and we made a policy of not boasting about how we were doing in any of the rankings.

I think this is exciting. Anyone who knows me knows that I am very vocal about my disdain about the rankings industry and my disdain and disappointment about their impact on higher education. But seeing your book and really very quickly understanding some of the perspectives that you have on rankings were pretty exciting. I just want to read real quick one line that opens very early in the book, page 49, a quote from Leon Botstein who in a 2001 New York Times interview stated that, “The rankings system is the most successful journalistic scam I have seen in my entire adult lifetime, a catastrophic fraud, corrupt, intellectually bankrupt, and revolting.” I am so curious, even now today, we hear some of the leading minds around higher education, thought leadership, administration, innovation, saying things like, “I know that the rankings are bad, but they are here to stay.” I’m curious from your perspective that even though we’ve seen so much evidence proving that all of the ways that rankings hurt higher education, why have they remained so potent in this industry?

Well, I think the short answer is because prospective applicants to colleges and graduate schools take them seriously. And when applicants take rankings seriously, the colleges and universities are going to take them seriously as well, even if they don’t like them. It’s interesting that you quoted that pithy, spicy comment by Leon Botstein. He made that comment over 20 years ago, and yet it wasn’t until last month that he decided that his college, Bard College would pull out of the rankings, so there’s a example of somebody who was harshly critical of the rankings and yet continued to comply with them, continued to cooperate with them. I think it’s because he obviously felt, as most educators do, that applicants take them seriously.

That begs the question that you might be poised to ask me, Kevin, which is, why do applicants take them seriously? I think there are lots of reasons, but there are two main reasons. One is the promise of simplicity, simplification. Choosing a college, or even a graduate school, is a really complicated and very consequential decision for 17, 18-year-old high schoolers, it’s probably the biggest, most complicated decision they’ve made yet in their life, and they want somebody to help them simplify it. Along comes somebody like U.S. News and says, “Don’t worry, we will simplify it. We will take all of the many, many different criteria and factors that you would consider and we will distill it into a single number for you,” so that’s one thing.

Then the second thing is status. I think a lot of people, not by any means all, but a lot of people choose to go to college and even graduate school so as to enhance their social status. They want a credential, they want a pedigree, and as I say in the book, “In England, the crown confers pedigree, but in America it’s U.S. News that confers pedigree,” so if you’re interested in pedigree, you want to go to a school that an outfit like U.S. News says is really good and you can be proud of the fact that you’re going to hang its diploma on your wall for the rest of your life.

It’s an interesting conversation here. I think the point you make around the Bard College example, that’s exactly what we see in the work that we do every day, “We don’t like this behind closed doors, but here’s our application or here’s our qualifications for rankings,” and I think that’s the ongoing conversation that I’m trying to kind of unpack here. I think in the last several months we have seen law schools and medical schools finally leaving the ranking system. We saw RISD not too long ago pull out of a ranking system as well. Why do you think this is happening now and what will it take to get more undergraduate institutions to follow suit? You talked about this briefly in your book, but I’d love for our listeners to get a high-level view of what you’re thinking there.

Yeah. Well, actually the boycott of U.S. News by the law schools and some of the medical schools occurred after my book came out. I guess I’ll have to talk about it in the second edition, but I think that I’ve talked to some of the law school deans whom I know and my sense is they got exhausted with the hypocrisy of cooperating with a system that really fundamentally conflicts with their values.

Now, to be sure, they’ve been doing it for 30 years, so what’s changed? Well, in the last year or so, there’s been a mounting degree of criticism of the rankings. My book is one example of that, but there have been news stories about universities and individuals who have lied to the U.S. News rankers and got caught doing it. Columbia University was an example and that was given international publicity because of Columbia’s prominence. Then there’s been a great deal of criticism of the way in which the rankings privilege the privileged as I say and give essentially advantage to the people who are already academically and economically advantaged.

I think especially a lot of the law schools whose mission after all is something to do with justice, it’s really something to do with promoting justice and making the world a better juster place just found it increasingly difficult to defend cooperating with the rankings given their mission. I think the medical schools, the top-tier medical schools have felt the same way. Their job is to serve the public and to serve the community and not simply to promote prestige and wealth. The undergraduate schools, well, it’s an interesting question. Why haven’t more of them followed their law school and medical school cousins? To date, I’m only aware of three. You mentioned the Rhode Island School of Design and I mentioned the Bard College and then Colorado College, a very well-regarded, relatively highly-ranked liberal arts college recently pulled out. There are special reasons in each of those three cases.

But what’s interesting is that they are not the industry leaders the way the first group to of the law schools and medical schools to pull out. I’ve said often, if there’s going to be a significant withdrawal from rankings by undergraduate institutions, it has to start with the industry leaders, not the quirky, individualized, distinctive schools like Reed and the others I mentioned, but Princeton and Harvard and Yale and Williams and Amherst and Pomona. Those are the ones who are going to have to pull out and they’re obviously well aware of the controversy. Amherst College being my alma mater, I know the new president there quite well, and I published an article in the winter 2023 edition of their alumni magazine about rankings and about my book, so they’re obviously thinking about it. But at the moment, I think they’re waiting to see what happens to the law schools and the medical schools to see maybe if U.S. News is going to punish them.

Yeah. You do a really great job. Well, first of all, you do a great job of breaking down the different components of the ranking system and explaining why they’re somewhat broken. Then you do a really great job of explaining some of the ramifications or the consequences some institutions have faced, including Reed in rebuking, some of the ranking structures. The other point is really a great one as well is around the schools like Colorado College who just exist differently in the higher ed space and around how they deliver education, et cetera. Those are the ones that I would expect to be a bit more vocal in their refusal or rejection of the ranking system and I’m glad that some of those schools are doing that. I’m curious about your perspective on which seems or feels like the better solution. Is it about changing the rankings methodology altogether to be more equitable, or is it having more institutions walk away from the practice entirely, or is it something else?

Well, my idea of the ideal world of rankings is that there not be a single dominant one-size-fits-all kind of ranking as exemplified, of course, by U.S. News. But instead, there would be six or seven or eight different rankings that all measure different qualities, different aspects of higher education, and that they would each have comparable credibility. There would be no one dominant ranking, but there would be a lot of different ones. For example, there’d be a ranking of quality of instruction, which is hard to rank, but so is everything. There’d be a ranking of social mobility, there’d be a ranking of graduation rates, there’d be a ranking of return on investment, maybe a ranking of affordability, maybe a ranking of ethnic and racial diversity. All of these are possible, in fact, most of them already can be found in various places. Even U.S. News provides some specialized rankings from time to time.

The question is, to come back to your question, how do we get there? I think the first thing that we have to do is we have to dismantle the monopoly. It’s not a real monopoly, but it’s certainly market power. We have to dismantle the market power of U.S. News. The only way I can think of to do that is to have a lot of schools pull out and do so in a very public way, very noisy way. And what that does is it sends very powerful messages to the applicant population, the potential applicant population that these rankings aren’t to be trusted. If all the Ivy League universities say they’re garbage, then gee, maybe they are garbage.

That’s what has to happen. Then in place of this foolish, idiotic one-size-fits-all, we can reduce the complexity of higher education to one number kind of ranking, would grow up a respectable group of different kinds of rankings. People as they think about applying to college would start not with the question, “What is the best college?”, but start with the question, “Who am I and what do I need and what do I want from higher education”? Then they could look for the kind of ranking that would answer their questions. That’s what I hope will happen.

I think the ancillary benefit there is that that would then encourage institutions to talk about themselves more authentically and you would find a bit more distinction or differentiation in all the offers that exist in the higher ed space. One of the issues that we find so often in the marketing of higher education is that at the end of the day, the product you are getting in return for your investment is basically the same, the qualification, the credential that you receive, but all of the things that make a college experience or an institutional experience what it is are some of the things that you don’t find in ranking structures around student life and diversity, all these other things, and so making sure that there is space for the really important and critical parts of a student experience to be found and talked about authentically and openly is going to be, I think, an ancillary benefit to walking away from this ranking structure because right now everything looks the same because everything’s graded on the same rubric.

Oh, yeah. I mean, my good friend Crystal Williams, who’s the new president at Rhode Island School of Design pointed this out when she explained why they pulled out. RISD is a unique school. It is a very specialized, distinctive school, and now, it is now being just thrown together in the U.S. News rankings along with hundreds of general mainstream regional universities. I think she recognized that obliterates their distinction and that’s what the rankings do, they obliterate distinctiveness. I mean, think about the minority-serving universities. There are some miraculously wonderful historically Black colleges and universities, they don’t get any credit for doing what they do, which is unique in our culture. They just get thrown into the rankings, and of course, because they’re not terribly well-endowed, they tend to fall far down in the lists.

That’s the next space I was actually going to move to. I think about all the schools doing really great work who happen to be under-resourced and underfunded, who work real hard to get the outcomes that they’re trying to get for their students and fall very low on the ranking system or ranking structure. That has always been such a challenge for me to understand, especially when it comes to the endowment piece. I know many of people have heard the Malcolm Gladwell series on higher education and rankings, and there was really a lot of great arguments made and that those episodes around Dillard University, I’m sorry, and these are the kinds of things that I think about when young people or any learner is trying to make a decision on where to go to college and you go to a list that essentially is lifting up the same institutions that were not made for people who didn’t look like Europeans, for white people.

Who look like the founders of those universities, right?

Exactly. Exactly. Those are the kinds of conversations that I think are important to have. I’m glad we’re having this one. I want to shift gears a little bit because obviously rankings is a proxy for assessing quality, and if we move away from this method of assessing quality, what then takes its place, and does it redefine what quality looks like in the higher ed space?

Yeah. Well, I think my previous answer speaks to that question as well. I think that quality in higher education cannot be unidimensional. It just has to be multidimensional because different people need different things from higher education and different schools offer different things. There are so-called “access schools,” which are primarily trying to reach out to underserved populations, whether they be underserved because of racial or ethnic discrimination or because of economic disadvantage or whatever. There needs to be a recognition of the work that they do and that can only be done by a different kind of ranking. But it’s not the only ranking, it’s just one of many rankings that I think needs to be used.

I would rather get away from ranking altogether and just list schools by various characteristics that they have. Perhaps the best example is the College Scorecard, which is produced by the U.S. Department of Education and is published on their website and is free and easy to access. Everybody who says, “I have to use U.S. News because it’s one-stop shopping,” my answer is, “No. The College Scorecard is one-stop shopping and it’s every bit is good and it doesn’t have those beguiling distracting numbers, the ranking numbers.”

It has plenty of numbers, and you can easily compare schools by whatever you’re interested in, so if you’re really interested in graduation rate, you can compare schools by graduation rate if you’re really interested in affordability, return on investment. They’re very big on return on investment. I understand, people want to know, “I’m going to spend a lot of money going to school for four years. Am I going to get a return on that investment or not?” Those are options that I think measure quality. I mentioned academic quality, which is extremely hard, and being a lifelong academic, of course, I think that’s the one and only primary purpose of higher education.

Sure, sure.

I’m always upset that the rankings basically ignore it or virtually ignore it, but there are some proxies. I think, for example, the percentage of courses that are taught by full-time faculty would be a proxy. Measures of the percentage of courses that are taught in small enrollment classes, but not just in the fall because that just encourages colleges to shift the big enrollment courses to the spring, but measure it across the entire academic year.

There are student surveys. I mean, they’re not terribly reliable, but they tell you something about what students do. I would like to see schools surveyed by the average number of hours that students study in a week. It turns out that number varies enormously from school to school. It also turns out, unfortunately, that that number has declined everywhere in the last 50 years, but I think it’s a measure of how seriously the academic program is taken. There are things like that that could be done to measure academic quality. I would love to see somebody come along and say, “We’re going to do a pure academic quality ranking. None of this other stuff, nothing about the SAT student scores of entering students, nothing about institutional wealth, endowments, spending per student, blah, blah, blah. We’re going to just try to measure academic quality the best we can.”

One of the things that I learned reading your book, among many other things, was the consumer reports model of ranking products and how if you’re ranked in this magazine or reference, you’re not able to market that. I think about all of the ways in which institutions and the work that I do, marketing institutions put their stamp right on every homepage, landing page, piece of literature, “We are ranked this.” Your argument in that section of the book was about the value of the ranking, or maybe the way that the ranking is communicated would shift if there was that kind of rule around higher education rankings. Can you speak a little bit more about that?

Yeah. I mean, I would love to see the rankings that people pay attention to all be produced by not-for-profit or perhaps governmental bodies that do not take advertising because the business model of U.S. News and most of the other rankings is to basically sell eyeballs to advertisers. That’s what they’re doing and they want to attract eyeballs. The way you attract eyeballs is by having these magical numbers that tell you where a particular school belongs in a hierarchy.

But they also accept advertising from many of the schools that they rank. Many of these rankings, if you look at their webpage, you’ll see schools popping up that obviously don’t deserve to be ranked where they are, but they’re popping up where they do because they’re advertising, they’re paying for advertising. Right between the ranking for Stanford and Princeton, you’ll see a ranking for Podunk University, which has bought the right to advertise on that webpage. Stuff like that to me undermines the credibility of the whole enterprise. The people who do the rankings should be professionals who know something about higher education, not a bunch of amateurs, U.S. News people call themselves journalists. They’re not journalists. They are amateur rankings producers. That’s what they are. I’d like to see professionals who are professional academics, professional educators curating the data and presenting it, and that I think would give the rankings, however they’re produced or presented a lot more credibility.

I love it. Colin Diver came with a smoke this morning. I love it. I love it. I only have two more questions for you, and one is around higher education administration. In this book and in several other spaces, I hear, and I’ve read about presidents behind closed doors saying, listen, I understand that this is messed up, but this is just something that we have to do. Is there something that presidents need to know or hear that will help them shift away from this ranking structure that we haven’t talked about already in this conversation?

Well, I wrote about this in an article in 2005 in the Atlantic Monthly, and I said, Reed College, which pulled out of the rankings in 1995, has not only survived, but it has thrived. The reason is because it made the fact that it rejected the rankings part of its signature, part of its message to its audience. It basically said, “We are looking for people who take academic pursuits really seriously, who really care about research and investigation and the use of evidence.” Those are the people who would recognize that rankings are bogus, they’re phony, and therefore the fact that we’ve rejected cooperating with the rankings would actually appeal to those people. Part of my message to schools that really do care about the academic integrity of their product and the intellectual rigor of their instruction is to make that part of your message, “This is why we’ve pulled out.”

Likewise, if you really care about public service, whether it be social mobility on the way in the door, or helping students prepare for lives of public service on the way out the door, reject the rankings and say to your public, “We reject the rankings because they celebrate wealth and privilege and they undermine the credibility of public service.” That would be my advice to these presidents. I think they know this. In effect, they’ve heard this before, but they’re afraid. They’re afraid that their constituency is so gullible and so easily distracted and misled by these rankings and they’re afraid, of course, also that U.S. News is going to dump them if they stop providing data to them just the way it did initially dump Reed College.

I understand that’s a legitimate concern. They have to worry about their product, after all, and their brand. But they also, I think, should be thinking about what kind of a legacy they’re going to leave after a lifetime in higher education and what are they going to be proud of to tell their grandchildren or their great-grandchildren, and I don’t think the fact that, “Well, we hated the rankings, but we knuckled under them because we thought that our constituency was too dumb to take them for what they’re worth,” I don’t think that’s something you’re going to want to tell your grandchildren.

Right, right. Before we get to this last question, I think about, and this is a question I’ve asked other guests on this show, if ranking and prestige can exist in the same space as diversity, equity, and inclusion initiatives on college campuses, and I for one, think that a ranking system is counterintuitive to diversity, equity, and inclusion initiatives on campuses among many other consequences and impacts it can have on a higher ed experience and I think presidents would do well with understanding the kind of ramifications of these rankings on their student experience in those kinds of other student lifeways. Last question for you, either rankings aside or in the middle, I’m curious what your thoughts are on the future of higher education in America. I think about all the politicization that’s happening, all the challenges we have around trust and return on investment. What do you think this is going to look like in 5, 10, or 20 years, or more, whichever?

Oh, well, it’s very hard to predict the future in any domain, but I start from the proposition that institutions of higher education are engineered for stability moreso than the businesses in almost any other industry that I can think of. As we’ve seen banks change a lot and they sometimes get themselves into huge trouble as a result of that and high-tech firms, they buy other firms and they change dramatically, but higher education is designed to remain the same, so I guess what I would say is that the elite sector of higher education is going to look pretty much the way it does today in 5, 10, or 25 years because they’re engineered to do that and because they have the cushion, they have the brand value, the reputation, and God knows they have the endowment. They produce alumni who make a lot of money and they keep them very happy, so they in effect have engineered a very, very reliable income stream from donations.

Where I think you’re going to see a big change, however, is in the lower tiers, the less selective tiers, if you will, of higher education. These are the ones, unfortunately, who do most of the really important work in higher education because they’re the ones who try to educate people of color and lower-income people and immigrants and people who are older and have families and have disabilities, you name it. What’s going to happen, I’m afraid, is that there’s going to be a lot of closures and there’s going to be a lot of consolidation in that slice of the market and it’s going to happen for, I guess, two primary reasons.

First, there is of course, just plain demographic change. The population of high schoolers is declining, particularly of course in the Northeast and in the West, much less so in the South. But the other reason is because of this growing movement to provide alternative pathways to white-collar jobs without requiring a college degree. We’ve heard about the governor of Pennsylvania saying, “We’re not going to require a college degree in order to hire people into white-collar jobs.” We’ve heard about lots of the biggest private employers in the country doing the same thing. What’s going to happen is that there’s going to be a growth in an industry of skills assessment, which enables these employers to determine whether people that they might hire have the skills that they want and that they need and they’re not going to insist that people have a bachelor’s degree.

When that happens, I think there’s going to be a big retreat from higher education, and I think it’s going to really hammer the less selective institutions because a lot of the people who go to those institutions now will say, “Why should I spend the money and the time and the effort for getting a bachelor’s degree when I can get the skills on the job or some other way?” I think that is what the future looks like. We’ve seen in Pennsylvania, for example, a big consolidation of their public sector. A lot of their campuses are being consolidated into a few of the flagship-type schools. I think we’re going to see a lot of that going forward.

The other things that are going on in higher education. Well, there is this shift away from the liberal arts and away from the disciplines that the liberal arts teaches like literature and philosophy and the humanities toward much more vocationally-oriented, trendy, current kinds of content, computer programming and various kinds of engineering and so forth. I suppose that will continue, but the irony there is that a lot of those kinds of skills are the skills that I just talked about, which can be learned on the job or in other methods. My son studied art. He got a MFA, but like most artists decided he was never going to make any money painting, so he taught himself to be a computer program, and he is now a very skilled senior engineer at one of the big tech firms in the country. People will be increasingly doing that. All these schools that are getting rid of their philosophy and humanities departments and investing like crazy in computer science, I think they’re going to have a comeuppance.

I agree with you 100% on all of the things that you just said. Colin Diver, thank you so much for joining us today. Again, his book Breaking Ranks: How the Rankings Industry Rules Higher Education and What to Do About It, available on Amazon and other bookstores, is a great read. Incredible. Thank you for joining us today. Really appreciated the conversation and you sharing your thoughts. It was great.

It was my pleasure. I really enjoyed it. Thanks, Kevin.

That’s it for this week’s episode of Higher Voltage. We’ll be back soon with a new episode, and until then, you can find us on Twitter, @BoltHigherEd, and you can find me Kevin Tyler on Twitter, @KevinCTyler2.