Note: Special bonus! Gordon Gauchat, the author of PSPS, wrote a reflective response that I've posted in a "followup" below. I can't think or write as fast he does (in fact, I'm sort of freaked out by his speed & coherence), but after I think for a bit, I'll likely add something, too, since it is the case, as he says, that we "largely agree" & I think it might be useful for me to be even clearer about that, & also to engage some of the other really good interesting points he makes.
This is a longish post, & I apologize for that to this blog’s 14 billion regular readers. Honestly, I know you are all very busy.
To make it a little easier, I’m willing to start with a really compact summary.
But I’ll do that only if you promise to read the whole thing. Deal?
Okay, then.
This post examines Gordon Gauchat’s Politicization of Science in the Public Sphere, Am. Sociological Rev., 77, 167-187 (2012).
PSPS is widely cited to support the proposition that controversy over climate change reflects the “increasingly skeptical and distrustful” attitude of “conservative” members of the general public (Lewandowsky et al. 2013).
Is that supposed to be an elephant? Looks more like a snuffleupagus--everyone knows they don't believe in science (it's reciprocal)This contention merits empirical investigation, certainly.
But the data analyzed in PSPS, an admittedly interesting study!, don’t even remotely support it.
PSPS’s analysis rests entirely on variance in one response level for a single part of a multiple-part survey item. The reported changes in the proportion of survey takers who selected that particular response level for that particular part of the single item in question cannot be understood to measure “trust” in science generally or in any group of “scientists.”
Undeniably, indisputably cannot.
Actually—what am I saying?
Sure, go ahead and treat nonselection of that particular response level to that one part of the single survey item analyzed in PSPS as evincing a “decline” in “trust of scientists” for “several decades among U.S. conservatives” (Hmielowski et al. 2013).
But if you do, then you will be obliged to conclude that a majority of those who identify themselves as “liberals” are deeply "skeptical" and “distrustful” of scientists too. The whole nation, on this reading of the data featured in PSPS, would have to be regarded as having “lost faith” in science—indeed, as never having had any to begin with.
That would be absurd.
It would be absurd because the very GSS survey item in question has consistently found—for decades—that members of the US general public are more “confident” in those who “run” the “scientific community” than they are in those who “run” “major companies,” the “education” system, “banks and financial institutions,” “organized religion,” the “Supreme Court,” and the “press.”
For the entire period under investigation, conservatives rated the “scientific community” second among the 13 major U.S. institutions respondents were instructed to evaluate.
If one accepts that it is valid to measure public "trust” in institutions by focusing so selectively on this portion of the data from the GSS "confidence in institutions" item, then we’d also have to conclude that conservatives were twice as likely to “distrust” those who “run . . . major companies” in the US as they were to “distrust” scientists .
That’s an absurd conclusion, too.
PSPS’s analysis for sure adds to the stock of knowledge that scholars who study public attitudes toward science can usefully reflect on.
But the trend the study shows cannot plausibly be viewed as supporting inferences about the level of trust that anyone, much less conservatives, have in science.
That’s the summary. Now keep your promise and continue reading.
A. Let’s get some things out of the way
Okay, first some introductory provisos
1. I think PSPS is a decent study. The study notes a real trend & it’s interesting to try to figure out what is driving it. In addition, PSPS is also by no means the only study by Gordon Gauchat that has taught me things and profitably guided the path of my own research. Maybe he'll want to say something about how I'm addressing the data he presented (I'd be delighted if he posted a response here!). But I suspect he cringes when he hears some of the extravagant claims that people make--the playground-like prattle people engage in--based on the interesting but very limited and tightly focused data he reported in PSPS.
2. There’s no question (in my mind at least) that various “conservative” politicians and conflict entrepreneurs have behaved despicably in misinforming the public about climate change. No question that they have adopted a stance that is contrary to the best available evidence, & have done so for well over a decade.
3. There are plenty of legitimate and interesting issues to examine relating to cognitive reasoning dispositions and characteristics such as political ideology, cutural outlooks, and religiosity. Lots of intriguing and important issues, too, about the connection between these indicators of identity and attitudes toward science. Many scholars (including Gauchat) and reflective commentators are reporting interesting data and making important arguments relating to these matters. Nevertheless, I don’t think “who is more anti-science—liberals or conservatives” is an intrinsically interesting question—or even a coherent one. There are many many more things I’d rather spend my time addressing.
But sadly, it is the case that many scholars and commentators and ordinary citizens insist there is a growing “anti-science” sensibility among a meaningful segment of the US population. The “anti-science” chorus doesn’t confine itself to one score but “conservatives” and “religious” citizens are typically the population segments they characterize in this manner.
Advocates and commentators incessantly invoke this “anti-science” sentiment as the source of political conflict over climate change, among other issues.
Those who make this point also constantly invoke one or another “peer reviewed empirical study” as “proving” their position.
And one of the studies they point to is PSPS.
Because I think the anti-science trope is wrong; because I think it actually aggravates the real dynamics of cultural status competition that drive conflict over climate science and various other science-informed issues; because I think many reasonable people are nevertheless drawn to this account as a kind of a palliative for the frustration they feel over the persistence of cultural conflict over climate change; because I think empirical evidence shouldn’t be mischaracterized or treated as a kind of strategic adornment for arguments being advanced on other grounds; because I have absolutely no worries that another scholar would resent my engaging his or her work in the critical manner characteristic of the process of conjecture and refutation that advances scientific understanding; and because only a zealot or a moron would make the mistake of thinking that questioning what conclusions can appropriately be drawn from another scholar’s empirical research, criticizing counterproductive advocacy, or correcting widespread misimpressions is equivalent to “taking the side of” political actors who are misinforming the public on climate change, I’m going to explain why PSPS does not support claims like these:
;)
Have they actually read the study? click to see what they say ...
B. Have you actually read PSPS?
It only takes about 5 seconds of conversation to make it clear that 99% of the people who cite PSPS have never read it.
They don’t know it consists of an analysis of one response level to a single multi-part public opinion item contained in the General Social Survey, a public opinion survey that has been conducted repeatedly for over four decades (28 times between 1974 and 2012).
Despite how it is characterized by those citing PSPS, the item does not purport to measure “trust” in science.
It is an awkwardly worded question, formulated by commercial pollsters in the 1960s, that is supposed to gauge “public confidence” in a diverse variety of (ill-defined, overlapping) institutions (Smith 2012):
I am going to name some institutions in this country. As far as the people running these institutions are concerned, would you say you have a great deal of confidence, only some confidence, or hardly any confidence at all in them?
a. Banks and Financial Institutions [added in 1975]
b. Major Companies
c. Organized Religion
d. Education
e. Executive Branch of the Federal Government
f. Organized Labor
g. Press
h. Medicine
i. TV
j. U.S. Supreme Court
k. Scientific Community
l. Congress
m. Military
For the period from 1974 to 2010, PSPS examines what proportion of respondents selected the response “a great deal of confidence” in those “running” the “Science community.”
As should be clear, the PSPS figure above plots changes only in the “great deal of confidence” response.
I’m sure everyone knows how easy it is to make invalid inferences when one examines only a portion rather than all of the response data associated with a survey item.
Thus, I’ve constructed Figures that make it possible to observe changes in all three levels of response for both liberals and conservatives over the relevant time period:
;)
As can be seen in these Figures, the proportion selecting “great deal” has held pretty constant at just under 50% for individuals who identified themselves as “liberals” of some degree (“slight,” “extreme,” or in between) on a seven-point ideology measure (one that was added to the GSS in 1974).
Among persons who described themselves as “conservatives” of some degree, the proportion declined from about 50% to just under 40%. (In the 2012 GSS—the most recent edition—the figures for liberals and conservatives were 48% and 40%, respectively. I also plotted pcts for "great deal" in relation to the relevant GSS surveys "yesterday" in this post.)
The decline in the proportion of conservatives selecting “great deal” looks pretty continuous to the naked eye, but using a multi-level multivariate analysis (more on that below), PSPS reported finding that the decline was steeper after the election of Ronald Reagan in 1980 and George W. Bush in 2006.
That’s it.
Do you think that these data justify conclusions like "conservatives' trust in science has declined sharply," "conservatives have turned on science," "Republicans really don't like science," "conservatives have lost their faith in science," "fewer conservatives than ever believe in science," etc?
If so, let me explain why you are wrong.
C. Critically engaging the data
1. Is everyone anti-science?
To begin, why should we regard the “great deal of confidence” response level as the only one that evinces “trust”?
“Hardly any” confidence would seem distrustful, I agree.
But note that the proportion of survey respondents selecting “hardly any at all” held constant at under 10% over the entire period for both conservatives and liberals.
Imagine I said that I regarded that as inconsistent with the inference that either conservatives or liberals “distrust” scientists.
Could you argue against that?
Sure.
But if you did, you’d necessarily have to be saying that selecting “some confidence” evinces “distrust” in scientists.
If you accept that, then you’ll have to conclude that a majority of “liberals” distrust scientists today, too, and have for over 40 years.
For sure, that would be a conclusion worthy of headlines, blog posts, and repeated statements of deep concern among the supporters of enlightened self-government.
But such a reading of this item would also make the decision to characterize only conservatives as racked with “distrust” pathetically selective.
2. Wow--conservative Republicans sure “distrust” business!
You’d also still be basing your conclusion on only a small portion of the data associated with the survey item.
Take a look, for example, at the responses for Major companies”:
;)
It’s not a surprise, to me at least, that conservatives have had more confidence than liberals in “major companies over the entire period.
I’m also not that surprised that even conservatives have less confidence in major companies today than they did before the financial meltdown.
But if you are of the view that any response level other than “a great deal of confidence” evinces “distrust,” then you’d have to conclude that 80% of conservatives today “distrust” our nation’s business leaders.
You’d also have to conclude that conservatives are twice as likely to trust those “running . . . the scientific community” as they are to trust those “running . . . major companies.”
I’d find those conclusions surprising, wouldn’t you?
But of course we should be willing to update our priors when shown valid evidence that contradicts them.
The prior under examination here is that PSPS supports the claim that conservatives “don’t believe in science,” "have turned on science," “reject it," have "lost their faith in it," have been becoming "increasingly skeptical" of it "for decades," etc.
The absurdity of the conclusions that would follow from this reading of PSPS---that liberals and conservatives alike "really don't like science," that conservatives have so little trust in major companies that they'd no doubt vote to nationalize the healthcare industry, etc. -- is super strong evidence that it's unjustifiable to treat the single response level of the GSS "confidence" item featured in PSPS as a litmus test of anyone's "trust" in science.
3. Everyone is pro-science according to the data presented in PSPS
What exactly do response to the GSS “confidence” item signify about how conservatives and liberals feel about those “running” the “Scientific community”?
Again, it’s always a mistake to draw inferences from a portion of the response to a multi-part survey item. So let’s look at all of the data for the GSS confidence item.
;)
The mean scores are plotted separately for “liberals” and “conservatives. The 13 institutions are listed in descending order as rated by conservatives-- i.e., from the institution in which conservatives expressed the greatest level of confidence to one in which they expressed the least in each period.
The variance in selection of the "great deal" response level analyzed in PSPS is reflected in the growing difference between liberals' and conservatives' respective overall "confidence" scores for "the Scientific Community."
Various other things change, too.
But as can be seen, during every time period—including the ones in which Ronald Reagan and G.W. Bush were presidents—conservatives awarded “Science community” the second highest confidence score among the 13 rated institutions. Before 1990, conservatives ranked the “science community” just a smidgen below “medicine”; since then, conservatives have vested more confidence in the “military.”
Conservatives rated the “science community” ahead of “major companies,” “organized religion,” “banks and financial institutions,” and “education,” not to mention “organized labor,” the “Executive Branch of the Federal Government” (during the Reagan and G.W. Bush administrations!), Congress, and “TV” throughout the entire period!
Basically the same story with liberals. They rated the “science community” second behind “medicine” before 1990, and first in the periods thereafter.
So what inference can be drawn?
Certainly not that conservatives distrust science or any group of scientists.
Much more plausible is that conservatives, along with everyone else, hold science in extremely high regard.
That’s obvious, actually, given that the “Confidence” item sets up a beauty-contest by having respondents evaluate all 13 institutions.
;)
click on me for thanksgiving treat! mmmmmm!But this reading—that conservatives, liberals, and everyone else has a high regard for science—also fits the results plainly indicated by a variety of other science-attitude items that appear in the GSS and in other studies.
It’s really really really not a good idea to draw a contentious/tendentious conclusion from one survey item (much less one response level to one part of a multi-part one) when that conclusion is contrary to the import of numerous other pertinent measures of public opinion.
4. Multivariate analysis
The analyses I’'ve offered are very simple summary ones based on “raw data” and group means.
There really is nothing to model statistically here, if we are trying to figure out whether these data could support claims like "conservatives have lost their faith in science" or have become “increasingly skeptical and distrustful” toward it. If that were so, the raw data wouldn't look the way it does.
Nevertheless, PSPS contains a multivariate regression model that puts liberal-conservative ideology on the right-hand side with numerous other individual characteristics. How does that cut?
As much as I admire the article, I'm not a fan of the style of model PSPS uses here.
E.g., what exactly are we supposed to learn from a parameter that reflects how much being a "conservative" rather than a "liberal" affects the probability of selecting the "great deal" response "controlling for" respondents' political party affiliation?
Overspecified regressions like these treat characteristics like being “Republican,” “conservative,” a regular church goer, white, male, etc. as if they were all independently operating modules that could be screwed together to create whatever sort of person one likes.
In fact, real people have identities associated with particular, recognizable collections of these characteristics. Because we want to know how real people vary, the statistical model should be specified in a way that reflects differences in the combinations of characteristics that indicate these identities--something that can’t be validly done when the covariance of these characteristics is partialed out in a multivariate regression (Lieberson 1985; Berry & Feldman 1985).
But none of this changes anything. The raw data tell the story. The misspecified model doesn’t tell a different one—it just generates a questionable estimate of the
difference in likelihood that a liberal as opposed to a conservative will select “great deal” as the response on "Confidence" when assessing those who "run ... the Scientific Community” (although in fact PSPS reports a regression-model estimate of 10%--which is perfectly reasonable given that that's exactly what one observes in the raw data).
5. Someone should do a study on this!
There’s one last question worth considering, of course.
If I’m right that PSPS doesn’t support the conclusion that conservatives have “lost faith” in science, why do so many commentators keep insisting that that’s what the study says? Don’t we need an explanation for that?
Yes. It is the same explanation we need for how a liberal democracy whose citizens are as dedicated to pluralism and science as ours are could be so plagued by unreasoning sectarian discourse about the enormous stock of knowledge at its disposal.
Refs
Berry, W.D. & Feldman, S. Multiple Regression in Practice (Sage Publications, Beverly Hills, 1985).
Hmielowski, J.D., Feldman, L., Myers, T.A., Leiserowitz, A. & Maibach, E. An attack on science? Media use, trust in scientists, and perceptions of global warming. Public Understanding of Science (2013).
Lewandowsky, S., Gignac, G.E. & Oberauer, K. The role of conspiracist ideation and worldviews in predicting rejection of science. PloS one 8, e75637 (2013).
Lieberson, S. Making it count : the improvement of social research and theory (University of California Press, Berkeley, 1985).
Smith, T.W. Trends in Confidence in Institutions, 1973-2006. in Social Trends in American Life: Findings from the General Social Survey Since 1972 (ed. P.V. Marsden) (Princeton University Press, 2012).
Dan Kahan
