Research explores policy research and impressions of bias
| 03 February 2009
BERKELEY — A University of California, Berkeley, study shows that when people learn about research findings that conflict with their own beliefs about politically controversial topics, they not only doubt the conclusions, but also question the researcher's objectivity.
Robert MacCounThis "attitude attribution" effect means that unexpected study findings are seen as characteristics of the researcher, rather than characteristics of the issue being examined, said MacCoun, and turns upside down the notion of social scientists as impartial arbiters of the truth.
MacCoun's study will be published in the February issue of the journal Political Psychology and already is online.
"Our findings raise concerns about how social science researchers are seen by the public," said MacCoun. "Because researchers' ideological views are supposed to be irrelevant to their empirical results, even partial support for the attitude attribution effect is impressive and troubling," he wrote in the journal.
Other researchers have examined whether people see what they want to see in policy reports, but MacCoun said his study breaks new ground by extending the research to the views of the general public (rather than students, as done in most surveys) and by examining people's assumptions about the politics of the researcher.
MacCoun's experiment was embedded in a telephone survey conducted in 2003 with 1,050 randomly selected California adults. It tested for several psychological biases, including naïve realism (where people believe others with different beliefs are not reality-based) and attitude attribution (where study findings are attributed to motives, traits and politics rather than facts). MacCoun was assisted with the research design and data collection by co-author Susannah Paletz, while she was a UC Berkeley doctoral student in psychology.
Survey participants were told about a hypothetical study on one of four politicized topics. Two of the topics - gun control and medical marijuana,- involved policy generally more favored by liberals and Democrats. Two other topics - school vouchers and the death penalty - included study outcomes more likely to be supported by conservatives and Republicans. Capital punishment and medical marijuana are part of California's legal status quo, while gun control and school voucher proposals have met with less law-making success.
A fifth hypothetical study involving nutrition advertising was used to establish a relatively apolitical baseline.
Participants were asked how surprised they were by the study findings, how believable they found the conclusions, and about the political views and motives of the study's author. They also were queried about their own attitudes about the study topics and about their personal politics and affiliations.
More than half (56 percent) of the participants in MacCoun's study speculated about the imaginary researcher's politics and were almost twice as likely (21 percent) to assume the author to be liberal as they were to infer conservatism (12 percent). Social scientists are, in fact, more likely to report being liberal than conservative, but MacCoun noted that study participants invoked this fact selectively and that conservatives were most likely to cite the author's liberalism when they didn't like a finding.
Besides being more skeptical about findings that contradicted prior beliefs, survey participants -
especially those with conservative beliefs - tended to attribute studies with liberal findings to liberal researchers. They were less likely to conclude that conservative findings were due to a researcher's conservatism.
While a majority of participants surveyed found the study's purported research results to be completely or somewhat believable and the suspected biases assessed as fairly modest, MacCoun said researchers are "not philosopher kings" and should accept that disagreeable research findings are often held to higher standards.
"Our findings raise concerns about how social science researchers are seen by the public," said MacCoun. "Because researchers' ideological views are supposed to be irrelevant to their empirical results, even partial support for the attitude attribution effect is impressive and troubling," he wrote in the journal.
In a 24-hour news cycle world where the general public - and even politicians and policymakers - rely on quick headlines, snapshots and commentators' "punch lines," policy researchers writing up their findings need to be sensitive to this suspicion and work harder to develop trust, said MacCoun.
Whether research is biased is a valid question, MacCoun said, acknowledging that he has encountered the raised eyebrow and knowing winks even from colleagues within academia.
When he published research about decriminalizing marijuana and co-authored the book "Drug War Heresies: Learning from Other Vices, Times and Places" (2001), he said he interpreted chuckles and knowing looks as signals that others thought he smoked pot. When MacCoun's investigations showed there would be negligible adverse impact from accepting openly gay members into the U.S. military, he said he was met with not-that-subtle comments and questions that failed to disguise assumptions about his sexuality.
And when he joined UC Berkeley in 1993 after working as a behavioral scientist and policy analyst at the non-profit, non-partisan RAND think tank, he said many of his new associates automatically assumed he was a conservative because RAND is active in national security research.
"If we really want to inform citizens and affect public policy, American social scientists need to learn more about how conservatives view our research," MacCoun said, "in order to root out hidden assumptions and communicate our research more effectively."