10 Reasons to Disregard U.S. News and World Report College Rankings

Admissions season at colleges and universities across the United States is quickly approaching. Soon every higher education institution will be mailing decision letters to applicants, and campuses will be overrun with prospective students and their ambitious parents for the ubiquitous spring visit days.  

It is not uncommon this time of year to hear mention of college rankings. Admissions brochures and other promotional literature at colleges and universities will play up favorable performance in rankings. Students and their parents will use rankings, along with a host of other factors, to determine institutional quality and, perhaps more importantly, prestige. In the end, that’s what rankings are all about for institutions and prospective students: the pursuit of prestige.

College rankings emerged in the 1980s as a result of the accountability movement and have grown in popularity since then. Today, a cottage industry has developed around rankings. The annual rankings produced by U.S. News and World Report (USNWR) are the most popular, influential, authoritative, and—perhaps because of the preceding attributes—critiqued rankings in the business. USNWR’s annual “Best Colleges” issue sells 2.2 million copies, reaching an estimated 11 million readers. Together with the more detailed guidebook, USNWR publications account for almost half of the college rankings market.

Many students and parents see the USNWR rankings as a way to sort through the massive number of higher education institutions in the United States, as well as a way to compare colleges and universities. The problem is that USNWR rankings actually provide remarkably little in the way of useful information to students and parents. Here’s a list of 10 research-informed reasons why anyone interested in learning about higher education institutions should disregard the USNWR rankings. I include at the end of the list references and resources for further reading.

1.      They falsely claim to be objective.

USNWR uses numerical information and statistics to convey objectivity. However, their methodology of ranking colleges and universities is based upon weights subjectively assigned to seven criteria the editors believe to be measures of institutional quality: academic reputation, student selectivity, faculty resources, graduation and retention rates, financial resources, alumni giving, and graduation rate performance. Bob Morse, who is in charge of USNWR’s rankings, has explained that what they do is not based upon social science research. Rather, it is part of USNWR’s “consumer journalism.” However, at no point do they acknowledge the inherent subjectivity of their methodology—the legitimacy of the publication rests upon its perceived authority.

2.      They base 25% of the rankings on academic reputation.

At first blush, this seems fairly straightforward. An institution’s academic reputation should be taken into consideration. The issue is that USNWR captures academic reputation through a survey mailed to presidents, provosts, and deans of admissions. They are asked in this survey to vote using a Likert scale on the colleges and universities with the best academic reputation. In some instances, this has the effect of rubber stamping into the top of the hierarchy those institutions with a historic and cultural association with prestige. When academic reputation surveys are sent out to deans of particular departments for USNWR’s more program-specific rankings, this undoubtedly affects their voting patterns. One commonly cited study showed, for example, that law school leaders ranked Princeton’s law school as one of the best in the country—even though it closed in the 19^th century. In other instances, campus leaders try to vote out the competition. One administrator at my university admitted to giving our institution a 1 on the academic reputation survey, and all other schools a 3.

3.      They place too much emphasis on the profile of applicants.

The criterion that probably receives the most attention from students deals with selectivity. Receiving 15% of the total in weighting, student selectivity includes the percentage of applicants admitted, the yield rate, the number of in-coming students in the top 10% of their class, and the average SAT or ACT of entering freshmen. Aside from the possible positive benefits of being in class with other talented students, this criterion conveys no information about the institution or the quality of education offered there. It essentially tells students how much they will pay to sit next to others who are of a similar academic caliber. Additionally, many institutions have difficulty accurately reporting student selectivity data. For example, most high schools in the state of Maryland do not rank students. In order to report how many students were in the top 10% of their class, many colleges and universities must extrapolate based upon a formula. 

4.      They are of questionable validity.

Webster (2001) concluded that average SAT/ACT score of in-coming students is the criterion that most affects an institution’s rank. Kuh and Pascarella (2004) repeated and confirmed Webster’s test, demonstrating that “for all practical purposes, U.S. News rankings of best colleges can largely be reproduced simply by knowing the average SAT/ACT scores of their students” (p. 53). This means that, despite USNWR using student selectivity measures as proxy indicators of quality, the two are largely unrelated. Dichev (2001) sought to discern the validity of USNWR rankings by looking at the predictability of changes over time. Her logic was that a “good” ranking should not change in predictable ways. Yet she discovered that between 70 and 80 percent of variation in rankings is transitory, changes are likely to reverse within two cycles, and most of the changes are due to aggregated noise in underlying components.

5.      They encourage prestige-seeking behaviors.

The prestige that accompanies performing well in the USNWR rankings also helps colleges and universities attract gifted students and resources in the form of grants and donations. Given that state appropriations have reached a 10-year low, these resources are increasingly important. Thus, institutions striving for prestige often emphasize graduate over undergraduate education. They also prioritize research, particularly in areas where the knowledge can be commercialized. Faculty are expected to win competitive grants and publish research, meaning adjuncts and graduate assistants shoulder much of the teaching. In other words, the pursuit of prestige has the potential to undermine the teaching mission of colleges and universities.

6.      They promote institutional isomorphism.

Institutional diversity has been a defining feature of American higher education. We have many different types of people seeking higher learning, so it makes sense to make many different types of institutions. However, rankings are causing competitive comparison, which results in institutions, over time, coming to resemble one another. Consider a liberal arts college that falls in the middle of the USNWR rankings. They look to those schools within their field of play and develop a list of “aspirational peers” from which to borrow ideas and practices. Scholars have thus likened the U.S. higher education system to a snake. The head represents prestigious schools, the body represents the prestige-seeking schools, and the tail represents those schools attempting to minimally increase their reputation. The body, then, follows the head, and the tail follows the body. Not all institutions can or should be research one institutions. But that hasn’t stopped them from trying.

7.      They create a positional arms race.

The essence of an arms race is that there is no absolute goal, only the relative goal of staying ahead of other competitors. This makes it difficult to leave the race, and it means there is no finish line. Colleges and universities simply continue competing against one other to secure a relatively better position in the USNWR rankings. Repositioning oneself usually requires that institutions spend more or charge less, and both options necessitate additional non-tuition resources. Most competition takes the form of increased spending, especially on amenities like dining halls, gymnasiums, and residence halls. Positional competition caused by rankings, therefore, increases the cost of higher education.

8.      They make institutions less accessible.

Monks and Ehrenberg (1999) were interested in how changes in USNWR rankings among elite schools changed their admissions practices and pricing policies. According to their study, an improvement in rank means an institution is more selective, offers less grant money to students, and experiences an increase in the average SAT score of applicants. Meredith (2004) corroborated these findings. Several observers have also noted that institutions have turned to early-decision applications as a means of improving yield rate. Early-decision applicants tend to be those students from families that can easily say “yes” if offered admission to a university without having to shop around for the best financial aid package. As follows, the early-decision application process advantages upper and middle-income students.

9.      They are unrelated to activities that contribute to learning.

Questions surround the relationship between USNWR rankings and the extent to which an institution promotes activities that contribute to learning. Pike (2004) measured the strength of the relationship between the USNWR criteria and five benchmarks of the National Survey of Student Engagement. In short, Pike’s analysis revealed that, with the exception of students at selective institutions reporting enriching educational experiences, USNWR criteria and NSSE benchmarks are unrelated.

10.  They have caused unethical behaviors.

There is a great deal of pressure placed on administrators to improve, or at least maintain, USNWR rankings. This pressure has caused some administrators to report false information or manipulate the numbers for personal and/or institutional gain. There have been repeated stories in the media about such cases, the most recent of which involved George Washington University and Tulane University’s business school.

These 10 reasons provide sufficient justification to shrug off the U.S. News and World Report college rankings. Next time you see the newest “Best Colleges” issue at your grocery store, walk by it as you would a tabloid. If a friend posts the rankings to Facebook, give the link a confident “dislike.” Most importantly, encourage students not to make their college choice based upon these rankings. Instead, push them to visit campuses and seek out alternative rankings, such as the one created by Washington Monthly, which accounts for how an institutions contribute to the public good. The reality is that the only people benefiting from the USNWR rankings is the corporation profiting from them.

For Reference and Reading

Dichev, I. (2001). News or noise? Estimating the noise in the U.S. News university rankings.
Research in Higher Education, 42, 237-266.

Ehrenberg, R. G. (2003). Reaching for the brass ring: The U.S. News and World Report rankings and competition. The Review of Higher Education, 26(2), 145-162.

Kuh, G. D. & Pascarella, E. T. (2004). What does institutional selectivity tell us about educational quality? Change, 36(5), 52-58.

Meredith, M. (2004). Why do universities compete in the ratings game? An empirical analysis of the effects of the U.S. News and World Report college rankings. Research in Higher Education, 45(5), 443-461.

Monks, J. & Ehrenberg, R. G. (1999). U.S. News & World Report rankings: Why they do matter.
Change, 31(6), 43-51.

O’Meara, K. (2007). Striving for what? Exploring the pursuit of prestige. J.C. Smart (ed.). Higher Education: Handbook of Theory and Research, Vol. XXII, 121-179.

Pike, G. R. (2004). Measuring quality: A comparison of U.S. News rankings and NSSE
benchmarks. Research in Higher Education, 45(2), 193-208.

Webster, D. S. (1992). Reputational rankings of colleges, universities, and individual disciplines and fields of study, from their beginnings to the present. Higher Education Handbook of Theory and Research: Vol. VIII, 234-304.

Webster, T. J. (2001). A principal component analysis of the U.S. News & World Report tier
rankings of colleges and universities. Economics of Education Review 20, 235-244.

The Endangered Liberal Arts College

In the spirit of a few blogs I follow that make a habit of re-posting, or re-presenting, well-articulated, thought-provoking writing, I offer this selection of excerpts from an article by Eva T. H. Brann. Brann is the longest-serving tutor at St. John’s College in Annapolis, Maryland. St. John’s is famous in higher education for its use of the Great Books curriculum.

This article originally appeared as “Straight Talk: About the Small Independent Liberal Arts Colleges” in the fall 1995 volume of “Liberal Education,” produced by the Association of American Colleges and Universities. (AACU: Please don’t sue me for not requesting permission.)

Despite being nearly twenty years old, the article is a gem and still offers rich ideas for understanding present circumstances. I don’t agree with all of Brann’s points, particularly those that smack of private elitism, but I believe her perspective is valuable and worth disseminating amidst continued debate surrounding what knowledge is most valuable and what purposes should constitute the work of higher education institutions in American society.

As colleges and universities sacrifice diversity to pursue prestige and compete for scarce resources by catering to private interests with narrow notions of “relevance,” we must at some point reflect on what we stand to lose and how happy we will be with our ultimate destination.

Eva T. H. Brann

Straight Talk: About the Small Independent Liberal Arts Colleges

CAN ANYONE DOUBT that the small independent liberal arts colleges ought now to be put on the endangered species list? They are threatened materially by an environment inhospitable to smallness as well as institutional diversity, and they are harassed intellectually by a climate unfriendly to the tradition of liberal learning, that is, to learning done for its own sake.

As we cast about for the means of salvation for our schools, we should, I think, consider that not the least of our devices might be a new way of speaking for ourselves: more self-assured, brisk, candid, distinctive. The assumption is that we have a lot to be confident about, and that, current theory notwithstanding, rhetoric becomes persuasive by telling truths in the language of conviction.

[…]

Hence the question is, what might we say in our common behalf to the different constituencies of our world? Let me have a shot at recalling the plain things we might want to say in unison.

To prospective students: We offer you tough freedom. We will constrain your choices to widen your world. We will listen to you until you make sense. We will not teach you to think (an absurdity), but we will give you occasions for thinking. We will not prepare you for careers, but we will give you the chance to discover a vocation. Your business with us is the activation of your intellect and the shaping of your sensibility, and we won’t let other agendas you might have—“experimenting” or partying—interfere with it for long; although it is costly to us in money and reputation, we will ask you to leave if in our opinion you have ceased to learn. But if you stay with us and work hard, your future life will be both more complex and more coherent, more interesting, and more manageable.

To prospective parents: A liberal education, the final good you can give your child, is not a public entitlement. The day of conception is not too soon to begin saving for your child’s college education. It cannot help but be terribly expensive, because only human beings, themselves expensively education and still in the process of learning, can educate other human beings. But if it is expensive, it is also invaluable, for the happiness of your child’s life is at stake: It is education that gives life its significance; these four years of leisure you are giving your child are a necessary luxury. We promise no quantifiable outcome, since the benefits of a good education take decades to show up. Nor can we guarantee that your child will make a good living right away, since education is distinct from training, which comes later in professional school or on the job. Nothing is more expensive, materially and spiritually, than an early but bad career choice.

[…]

To ourselves, the faculties: …Let us acquaint ourselves thoroughly with our situation and be ready to act speedily and flexibly to protect our colleges, but let us also do, in the cause of preserving a human scale in education, what we do best: some inspired footdragging—though with a lively and well-informed appreciation of the current condition. And let us not be ashamed of a little last-ditch fervor…We are entitled to believe that this country depends on its small colleges to ensure their own survival toward the recovery of universal liberal learning, the kind of education that helps people to make not only a living but a life.

To legislators and education officials: The small liberal arts college is a national treasure. The small private liberal arts college is a phenomenon unique to this country. No other nation can boast of so lively a nonsystem. It is the educational analogue to the small business—the fruit of local individual initiative, diverse and hardy. Colleges differ significantly, however, from businesses in being not-for-profit public service institutions. They serve their students in providing a humane education. They serve their communities in bringing educational appointments and jobs. Colleges serve their states and the nation in saving public education dollars and in educating the sort of citizens most need right now: people who have learned how to learn and who reflect on their moral obligations.

We are not ashamed to ask for public support—for work-study, loan, and grant money—to help our dedicated and hard-working students, as well as for direct support to our hard-pressed institutions. We understand that in taking public money we become publicly accountable, and we are determined to be responsive. It is wasteful to strangle our independent educational judgment with bureaucratic regulation and to overwhelm our small resources with pointless reporting mandates.

If you are tempted to think of us as “elitist” because we are small, consider that smallness is no sin, least of all an elitist sin, and that mere largeness is often quite inert. Most of our schools are not, in fact, terribly selective; most of our teachers do the kind of teaching that welcomes and cherishes students of quite various gifts. If you are tempted to think that “small” might mean “ineffective,” consider that it is a small lump of leaven that makes the loaf rise, and the small local initiatives often have great consequences.

To public interest groups: Your manifestos, reports, and position papers help to guide the public policy that influences our fate. These publications, we are convinced, are most helpful when the administrators and educationists who write them seek the advice of teachers in the trenches. Without this advice these papers often advocate directions simply deleterious to education.

For example, these reports often underwrite a fierce vocationalism. They translate education into market terms. Students become clients, schools delivery systems, knowledge a product. The writers have forgotten the student explosions of the sixties, which were largely the result of such impersonal approaches. They forget that these are young human beings and that the business of colleges is the shaping of their lives, of their vocational and moral future.

Educational writers too often think of communities of learning in managerial and quantitative terms. They fail to consider that well-working schools are collegial in their governance and qualitative in the intended effects. Consequently the efficiencies of the marketplace—more products for less cost—can be catastrophes of education—hordes of alienated and undereducated students. These writers address themselves to the projected demands of the future, setting aside the past as if it were not the ground in which our humanity is rooted. They present the perceived demands of the twenty-first century as a fixed fate, rather than as a set of choices t be free accepted and sometimes, when their educated judgment so directs them, to be firmly rejected by our young. They demand that colleges and universities be all things at once: flexible in their approaches and also subject to national standards, inviting to expressions of diversity and also rigorous in technological preparation, tolerant of all views and also supportive of common values. While these are separately excellent purposes, we remind our public that institutional diversity, the best educational “system” for a diverse country, requires a firm and coherent curricular vision within each institution; good colleges do not attempt to be and do all things.

[…]

This teachers’ wisdom is what the small independent liberal arts colleges live by, and it is confirmed in the lives of our alumni. We think that the time will come when the public will be glad that the models of teaching which the small liberal arts college offer have survived. 

Returning to the Farm

I rewarded myself two weeks ago with an end-of-semester victory hike up Old Rag Mountain in Shenandoah National Park. After the hike, I was meandering down one of several state routes back to the highway when I came across this barn. I pulled over and did something I almost never do—snapped a photo of it.

The barn looked beautiful. I suppose, in the moment, this was all the reason I needed to stop and capture it. But I couldn’t stop thinking about the barn on the drive home. There was something about it that struck me. Perhaps it was its dilapidated exterior; its sheer isolation against a chilling backdrop of grey clouds and leafless trees; its unfamiliarity after years living in urban areas; or the contemplation only possible when you’re alone in the car for several hours.

Growing up in Ohio, I was surrounded by operational farms. When my parents relocated to “The Heart of it All!” state in the 1980s, my mom fell in love with our neighborhood specifically because it reminded her of her small hometown in Iowa—right down to the smell of freshly spread manure. I used to sneak through the nearby woods, rush by the “Trespassers Will Be Prosecuted Sign,” and play games in the fields of a Christmas tree farm—one of several family farms dotting the landscape as I went to school each day. 

Of late, I have virtually no contact with operational farms. I know, of course, that they exist. They lie just outside my orbit. Most of my friends and I chose after college to settle in urban areas, where “there is stuff to do,” by which we generally mean areas where we can find easily accessible concentrations of restaurants, bars, and at least one Target. This doesn’t always mean we live in the heart of a city. More often, we find less expensive housing in city outskirts: the urban sprawl districts close enough to public transportation that we can claim to live in Washington, D. C., Philadelphia, or New York City.

The sprawl has widened enough in these areas to subsume a few farms, meaning it is still possible to encounter a barn or two. However, these barns have been repurposed as homes or antique stores. As cultural symbols, the barn no longer carries just the connotations of backbreaking toil on behalf of feeding the nation or providing sustenance and livelihoods to generations of hardworking American families. Barns have also come to symbolize a bygone era. They are sought after for their age, rust, and “country kitsch” effects. In other words, it sometimes seems that barns are appreciated not for their functionality, but rather because of their aesthetic value. Two industries, in particular, seem to have capitalized upon this aesthetic valorization: the wedding and dining industries.

I've arrived at the stretch of years in which many of my friends are getting married. My Facebook newsfeed basically resembles an online wedding album.  As I passively click through wedding pictures, it is impossible to escape images like this:

There are now entire websites, blogs, and catalogues dedicated to creating the perfect “country kitsch” wedding, providing instructions for producing mason jar center pieces and horseshoe favors. I commend the crafty people who are able to follow these instructions and, in some cases, reduce their wedding costs. Additionally, a good number of the wedding photos I have seen that use barns as props are gorgeous. The contrast of two impeccably dressed people of the present with an old, beat-up structure can be visually stunning. Yet I would bet that the barns in these photos were never destined to be wedding venues. And I know for a fact that my Facebook friends posing for the camera are not at all involved with crop cultivation. They want their photos to communicate a message about them as a couple.

Restaurants have also tapped into the cultural allure of the barn, but for somewhat different reasons. I’ll use as an example Woodberry Kitchen in Baltimore, Maryland. Woodberry is not actually housed in a barn, but rather the old Clipper Mill complex that has since been redeveloped into artists’ studios and upscale condominiums. Upon approaching the restaurant, diners are greeted by old farm equipment and corn stalks. Inside, wooden beams are embellished to give a barn-like appearance, and shelves are stocked with canned fruits and vegetables. Waiters scurry around in plaid shirts and blue jeans, while waitresses wear simple dresses and aprons.

Part of the reason that Woodberry has purposefully tried to give diners the feeling that they are eating in a barn or farmhouse is that the menu is designed to showcase fresh ingredients from Chesapeake area farms. The restaurant’s website explains: “At our table, you join us in supporting sustainable agriculture that respects the abundance and traditions of the region while helping to ensure its future.” Woodberry’s agrarian décor speaks to the “country kitsch” aesthetic demand as well as a business ethic based upon sustainability and reconnecting guests to the origins of the food they eat.

At the same time that barn-side wedding photos and “down on the farm” dining options have become popular, so, too, have sustainability and eating local. It is trendy to purchase produce from farmers’ markets and, in general, to give thought to where our food is coming from. This is especially true among a growing number of young people, even those in urban areas. Many cities have started to create urban farms to increase the amount of fresh, locally sourced produce available to residents. Many of these farms exist thanks to dedicated volunteers. Community Supported Agriculture (CSA), in which a farm delivers a share of its harvest to members or subscribers, is also growing in popularity among a select subset of urbanites.

A recent article even announced that Millennials are suiting up to be the next generation of farmers. Citing the fact that America’s farmers are, on average, almost 60 years old, the article describes a program in Maine to link young people who want to learn about agriculture with veteran farmers to eventually take over untended acres. The initiative was featured in the documentary film, GROW!

It would be inaccurate to say that eating local has become widespread practice. Similarly, the Maine Millennials who are taking over farms are few in number, with most young people still opting for a career path that carries more prestige. However, The New York Times reported a few months ago that the number of farms in the United States increased by 4%—the first increase after decades of decline. Optimistically, the resurgent interest in working on farms, eating locally sourced food, and even “country kitsch” is bringing renewed vitality to agriculture in America. Barns as symbols of livelihood and sustenance may eclipse any connotations of them as objects of a bygone era.

Perhaps if I return to Old Rag Mountain in the near future, the barn in my photo will speak to this shift. Instead of being a striking scene of abandonment, it may be the workplace of a new college graduate in search of a promising career, providing fresh produce to Americans through farmers’ markets and upscale restaurants.