Berkeleyan

[an error occurred while processing this directive]

Survey Research Center marks half-century of data-based insight

Random sampling may sound bloodless, but SRC director Henry Brady seeks illumination — and artistry — in numbers

By Barry Bergman, Public Affairs | 05 February 2009

BERKELEY — "My wife claims that she once saw a note I'd written to my Aunt Lorraine, I think it was, after I'd been to New York City," confides Berkeley political scientist Henry Brady. "And the note apparently begins something like this: 'We went to New York City. We passed 22 gas stations, we passed seven restaurants, we passed three of this, four of that…' And then at the end it said, 'Love, Henry, XXXX — each X means 10 kisses.' "

Brady's pre-adolescent relish for numbers would find more sophisticated outlets in adulthood, but his fascination with data has never left him. Today, as director of the campus's Survey Research Center — established, coincidentally, in 1958, around the time of young Henry's Big Apple excursion — he presides, in what appears to be a labor of love, over one of the nation's premier institutions for collecting, processing, and disseminating research data.

As the Survey Research Center prepares to celebrate its 50th anniversary this month, Brady sees its mission in terms that might seem to belong more to the realms of art and archaeology than to math and statistics. Much as he once set out to describe New York in numbers, the SRC has spent a half-century illuminating American life through the innovative — and often pioneering — use of survey, administrative, and ethnographic information.

"Part of this is people who see patterns in data," explains Brady, who maintains a document-laden office at the center's rambling, brown-shingle headquarters on Channing Way, a century-old structure that once was home to the Anna Head School for Girls and now is listed on the National Register of Historic Places. "They can take a data set and, it's an art, sort of paint a picture with that data set. And the pictures are about, how does the world really work?"

Along with other such institutions at places like Columbia and the University of Michigan, Berkeley's Survey Research Center was built on the random sample, an outgrowth of the realization by 20th-century social scientists that existing survey methodologies were not, in fact, scientific. The magnitude of the problem became clear in 1936, when the Literary Digest, using an opt-in poll it mailed to its Republican-leaning subscribers, fearlessly projected that President Franklin Roosevelt would be unseated by Kansas Gov. Alf Landon. Roosevelt, of course, cruised to re-election in a 523-8 electoral-vote landslide, winning every state but Maine and Vermont. And the Digest was soon out of business.

That disaster led to the improved, albeit still biased, quota sample, employed by Gallup in 1948 to call the White House for Thomas Dewey over Harry Truman. From that notorious blunder emerged the technique called random sampling — in which a relatively small, representative group is selected, completely by chance, from a larger population — and, with it, the modern survey-research institution.

"It was the realization that we had found a technique whereby we could actually do soundings on the American public, and that with a well-designed random sample you could actually figure out what the American public thinks at a given moment in time," Brady says. "And it became clear that it was good not just for finding out their opinions, but for studying welfare and policy issues, all kinds of things."

Jon Stiles, a data archivist who leads one of the Berkeley center's four specialized divisions, also cites the "growing recognition" in the 1950s of the importance of keeping data available for other researchers to explore.

"It's a tremendously expensive thing to do survey research, but it's a much cheaper proposition to analyze someone else's data," he observes. "Maintaining consistent archives of data — of other people's research, of your own research — is really important, so the data gets reused."

Stiles, whose association with the SRC began while he was a grad student in 1991, manages several centuries' worth of data — a quantity measured in petabytes, each of which is equal to some 1,000 terabytes, a million gigabytes, or a quadrillion just-plain bytes. He devotes much of his energy to helping academic researchers, journalists, and the general public access and navigate the almost-literally endless treasure trove of computerized information, the better to discover patterns of their own.

"The first thing we want to do," he explains, "is teach people how to fish."

How the West was won

To Brady, Stiles, and their SRC colleagues — including field researchers in the Center for Urban Ethnography, one of its research units — numbers and statistics are anything but bloodless. Over the past five decades, the center's data has been deployed to study corners of life in the United States ranging from anti-Semitism and racial discrimination to water quality and welfare. In the early 1980s, the SRC did groundbreaking research on AIDS that involved drawing not just information from interview subjects, but actual blood as well.

"We collect blood samples, dust samples, all sorts of bizarre things," Brady says. "If it can be collected, we've probably done it."

The AIDS survey, conducted under the auspices of the center's Survey Services Facility, illustrates what makes the center a vital resource for social scientists and policy makers alike.

"We started doing AIDS studies before anybody knew what the virus was that caused AIDS," Brady recalls. The work required that SRC staff win the confidence of gay leaders in San Francisco in order to interview at-risk men — "there was an element of hysteria then," notes Brady, and potential subjects were "terrified" their confidentiality would be breached — design questionnaires, take blood samples (phlebotomists were on staff), and finally analyze the data and specimens they had gathered.

"We found out there were behavioral correlates to AIDS," says Brady. "Before anyone knew about the virus or anything else, we found out that sex had a lot to do with it, sharing needles had a lot to do with it — it turned out the type of sex you had mattered a lot."

Working with clients in academia and government, the unit has also conducted studies on such topics as barriers to prenatal care for pregnant women in California, differences in ethnic perceptions of law enforcement in Oakland and Los Angeles, and the incidence of gastrointestinal illness in homes with and without water-treatment devices. For the latter study, the center installed filters in half the houses in a random sample, while the other half got "placebo" filters — and then, after six months, switched the types of devices for each half.

"Lo and behold, we found out in the end you don't need these water-filter things on your tap if you have a good municipal water supply," Brady says. The project, he adds, was "a very important public-health kind of study that tells you something about, actually, how the West was won — it's about how Western civilization was built upon good public health."

Among the center's ongoing studies is PACES — the Public Agendas and Citizen Engagement Survey — an election-year look at voters' concerns overseen by political scientist J. Merrill Shanks. Brady calls it "the most extensive survey of people's issues and concerns that is done in America," and deems it an antidote of sorts to the quadrennial obsession with horse-race politics.

Pollsters and journalists "focus too much on the horse race, who's ahead and who's behind, and not enough on what's behind who's ahead and who's behind," he says. "If you really want to understand American public opinion, you can't just ask a liberal/conservative question, a party-identification question, and something about abortion. You have to go beyond that. And when you do, you find out there are some real complexities out there. People are more nuanced in their perspectives than you might imagine."

Painting by numbers

Reflecting on the center's history — an exercise in which all of its previous directors are expected to participate on Feb. 12, when they gather on campus for its anniversary celebration — Brady calls it "a place that has innovated in how we've measured things — how do we measure anti-Semitism, how do we measure or understand sexual behavior, how do we measure or understand racism."

The center pioneered the use of what's known as "experiments embedded in surveys," a technique whereby respondents are questioned about, for example, their views on welfare — except that half might be asked about white welfare recipients, while the other half are asked about African Americans. ("The people who get the stimulus of 'black welfare recipient,' " it turns out, "are much less positive about welfare.")

The Berkeley center has been a nationwide leader in linking administrative data to information gathered from surveys, as well as to Census data — including non-public data to which the center has special access — as a means of learning more about long-term trends in phenomena like welfare and immigration. It also serves as the distributor of 50 years' worth of Field (California) polls, whose potential for discovery makes Brady positively giddy.

"You've got all these resources, and you say, OK, how can we make them more than the sum of their parts?" he wonders. "How, by putting together 270 or so Field polls" — that is, 50 times five per typical year, somewhat more in election years — "can you have a run of data that now allows us to tell a historical story about how California evolved from 50 years ago, with Pat Brown as governor, through all the twists and turns?"

That, finally, is Brady's kind of math, a sort of data-based alchemy in which seemingly ordinary survey information — carefully compiled, meticulously archived, and analyzed in new and creative ways — yields unexpected insights into politics, sociology, religion, and other facets of American life. When he speaks of his pride in the center's role in training grad students over the years, in fact, he sounds more like an art instructor than an authority on voting systems, one of his areas of poli-sci expertise.

"You don't really learn about how to do a survey, how to put together a set of surveys, how to really analyze data unless you're at a place where people are doing that on an everyday basis. This is a craft knowledge — you have to sit and do it with people who know the craft," he says.

"You don't become a great painter by just going to classes. You have to sit there and paint. And the more you paint, the more you work with people who really know about painting, the more you say, 'no, that's not quite right' — you become better at painting.

"We teach people how to paint," he says, "in ways they couldn't have learned just from classes."

The Survey Research Center's public celebration begins at 1 p.m. on Thursday, Feb. 12, in Barrows Hall's Lipman Room, with a reception to follow at 5:30 p.m.