Berkeleyan
Faculty's NRC input is ready for grading
But will their high response rate translate into coveted high rankings?
| 02 May 2007
We'll just have to live with the suspense.
It's going to take until December for the National Research Council to sort, check, and analyze the mountain of data they've received from universities across the nation and emerge with an update of their massive and influential report, "Research-Doctoral Programs in the United States."
The NRC's has been one of the most-respected assessments in the small cluster of published academic rankings -- in part because it emerges from the NRC, the working arm of the National Academy of Sciences and the National Academy of Engineering, not from a general- interest magazine or newspaper. Of course, Berkeley has done well in several recent, more informal rankings. The current NRC survey is an attempt to create a more defensible set of rankings, with a new methodology based on quantitative measures.
When the last NRC report came out, back in 1995, so many of Berkeley's doctoral programs placed in or at the top of the top 10 in their field that this campus was ranked as the nation's best graduate institution overall. Whether that history will repeat itself is, at this point, anybody's guess.
The NRC's initial work was done by questionnaire. The drop-dead deadline was April 1. By early March, nearly 109,000 faculty from 5,400 programs at 216 universities had responded - roughly a 70 percent response rate from eligible faculty, which is not bad. Berkeley responders, characteristically high-achieving, had filed 89 percent of their questionnaires by the Feb. 22 deadline for full inclusion of their data. That's a shared accomplishment of the staff of the Graduate Division, the NRC facilitators in departments and graduate groups, and especially the faculty members who completed their questionnaires.
 |
 Andrew Szeri (Deborah Stalford photo) |
Some responses made the merely prompt look sluggish. Graduate Division Associate Dean Andrew Szeri, who made a point of visiting as many programs as he could to answer questions and improve response rates, recalls that one department "almost immediately had a high response rate from its faculty - even though I hadn't visited them. I called the department chair to ask what their secret was, and I was told this: He had said to the faculty that if the response rate wasn't high enough by a certain date, he was going to invite some associate dean in to talk to them for an hour about why and how to fill out the questionnaire! That was enough to get them to do their duty. Never before has an hour with me been used as a weapon."
The NRC's boffins are actually everyone's boffins. Over half of the faculty surveyed were willing to serve as raters (including here at Berkeley), and are being drafted as such. This is a peer-review process: In their respective fields, they will each tackle data from about 15 graduate programs, visit websites, and make judgments about the quality of the programs, submitting their reviews online to the NRC.
The information collected from all sources will pour into the NRC's Research-Doctoral Database, the rock on which the analysis will be built, and from the analytic work will come the long-anticipated report.
Meanwhile, the NRC is highly aware, and appreciative, of the labor invested in the project thus far. Charlotte Kuh, who is directing the study for the NRC, says, "We are tremendously impressed with the Herculean efforts of the institutional coordinators, the program respondents, and the institutional research staff at the universities in the study." The survey's sheer numbers, and the fact that it was organized differently than its predecessors, challenged university IT systems. Overall, says Kuh, "it wasn't easy, but most schools were very responsive."
The plan is for analysis and data to be released by the NRC in December 2007. In many cultures there's a tradition of exchanging gifts that time of year. In some, the gifts can be warnings, too - like a lump of coal in the stocking. We'll just have to wait and see.