Berkeleyan

What greater wisdom do the U.S. News graduate-school rankings provide?
Berkeley's disciplines - 11 this year - all finished in the top 10. The magazine offers no blanket institutional ranking to inspire cries of "We're No.1!" - but where there are data to crunch, there's always a way

By Jonathan King, Public Affairs | 13 April 2006

The latest set of U.S. News & World Report rankings, released on March 31, provide the grad-school bound with a rough assessment of which universities might best serve their intellectual needs. That is, after all, the traditional function of the U.S. News & World Report rankings. Another tradition is the twice-yearly opportunity they provide for energetic box-score analyzing and methodological head-scratching on the part of academics and their administrators.

(Source: U.S. News & World Report; table prepared by UC Berkeley Office of Planning & Analysis)

This year's grad-school rankings were in five professional-school categories (engineering, medicine, law, business, and education) and six academic groupings (biological sciences, chemistry, computer science, earth sciences, mathematics, and physics). A further breakdown of those master categories into their major programs and specialties yielded further rankings, and at least the promise of further nuance for those analyzing the available data.

But first, the overall rankings in those areas in which Berkeley maintains a program (which is to say, everywhere but medicine this year): business (7); law (8); engineering (3); education (6); biological sciences (2); chemistry (1); computer science (1); earth sciences (4); mathematics (2); and physics (4).

So: Go Bears? It's notable that Berkeley places in the top 10 in all four professional-school disciplines. But because U.S.News offers no overall ranking, bragging rights at this level reside primarily within disciplines - though there are ways to extrapolate from these atomic findings to a more global institutional message.

Still, the nuance thing is not yet apparent. The rankings of constituent programs and specialties within the 11 larger subject areas yield a few more insights (see table at right) - but not all that many: Few spectacular variances or glaring weak spots emerge even at that level of detail. And while checking the ranking of one Berkeley program against a rival's, or one's alma mater's, can be highly gratifying (or, on occasion, irritating), as an individual data point it means very little.

That said, let's indulge a bit: Of six UC campus ranked among the top 50 engineering schools, Berkeley, at No. 3, outpaced San Diego (11), UCLA (15), Santa Barbara (21), Davis (35), and Irvine (41). In biological sciences, Berkeley (tied for No. 2) not only kept pace with Harvard and MIT but ranked higher than its own fraternal campuses in San Francisco (9), San Diego (12), Davis, (17), Los Angeles, (22), Irvine (32), and Santa Barbara. (40). You wanna piece of chemistry? Berkeley (1, tied with MIT) trounced UCLA (12), Irvine and San Diego (18), UCSF and Santa Barbara (31) . . . .

Modesty demands we stop. But as noted above, at a certain level gratification can be reliably distilled from these tea leaves, even if wisdom can't.

Complicating any campus's attempt to see its true reflection in this much-consulted mirror is a lack of clarity about the methodolog(ies) U.S. News employs to produce its calculations . never mind its final rankings. "Our rankings combine expert opinion and statistical indicators" reads the large-type highlight from the magazine's very general front note on the topic. What one learns from the brief text itself is that rankings for the sciences, social sciences, and humanities "are based solely on the ratings of academic experts," in contrast with the professional-school assessments, which are based on "surveys of more than 1,200 programs and some 9,600 academics and professionals.."

Clearly, relying overmuch on the data from 1,200 self-reporting surveys could lead to, if not the outright skewing of results, at least a sense that something less than total accuracy has been achieved. That risk is compounded by the belief of many observers that changes in U.S. News methodologies can lead to apparent gains or drops in the rankings that have little if anything to do with program quality. (Indeed, rankings aficionados can generally be counted on to understate the accuracy of all U.S. News results, right up to the point where - if they are university employees - they acknowledge the utility of those results as a recruitment and development talking point, and change the subject to recent trends in the display of tabular data.)

Those caveats established, there remain ways to look at the U.S. News data that yield useful institutional results. The Graduate Division has crunched the numbers available for crunching from this spring's rankings, yielding a measurement of "total rank" by looking at ranked institutions with nine or more measured academic (not professional) programs and then taking the mean of the "reputation scores" used by the magazine in both its methodological processes. By that metric, Berkeley (2.4) comes out No. 1 in total rank, a tenth of a mean rating ahead of Stanford (2.5). We are, in other words, by this standard numero uno.

On the other hand, in calculating total rank for professional-school categories (business, education, engineering, and law) the Grad Division number-crunchers found that Berkeley this year drops to second place in total rank, with a mean rank of 5.0, far behind Stanford (1.5). But then, no other university paced the Cardinal either in this area: Clustered with Berkeley in the two-through-four spots are Harvard (6.6) and Michigan (6.8), with the field weakening markedly beyond that (fourth-placed Michigan, with a mean rank of 11, had no professional school ranked higher than fifth).

Will total rank be cited henceforth by campus boosters in search of a memorable metric to impart about our excellence? Not likely: It's hardly common parlance, and is in any event a difficult metric to interpret, particularly when the sample used to calculate it is large. It may be useful, though - substantively so - as a measurement of campus "excellence." This most elusive (yet most cherished) of all institutional metrics is one a prospective grad student might well prize if he or she were in a field that creatively interacts with many others - or in a field, the student believes, that ought to.