Don't make free, reasoning people choose between learning posterior predictive model checking & *being who they are*!
For crying out loud, if he can pull that off, then surely science communicators can overcome cultural polarization on climate change.
Note: Special bonus! Gordon Gauchat, the author of PSPS, wrote a reflective response that I've posted in a "followup" below. I can't think or write as fast he does (in fact, I'm sort of freaked out by his speed & coherence), but after I think for a bit, I'll likely add something, too, since it is the case, as he says, that we "largely agree" & I think it might be useful for me to be even clearer about that, & also to engage some of the other really good interesting points he makes.
This is a longish post, & I apologize for that to this blog’s 14 billion regular readers. Honestly, I know you are all very busy.
To make it a little easier, I’m willing to start with a really compact summary.
But I’ll do that only if you promise to read the whole thing. Deal?
PSPS is widely cited to support the proposition that controversy over climate change reflects the “increasingly skeptical and distrustful” attitude of “conservative” members of the general public (Lewandowsky et al. 2013).
This contention merits empirical investigation, certainly.
But the data analyzed in PSPS, an admittedly interesting study!, don’t even remotely support it.
PSPS’s analysis rests entirely on variance in one response level for a single part of a multiple-part survey item. The reported changes in the proportion of survey takers who selected that particular response level for that particular part of the single item in question cannot be understood to measure “trust” in science generally or in any group of “scientists.”
Undeniably, indisputably cannot.
Actually—what am I saying?
Sure, go ahead and treat nonselection of that particular response level to that one part of the single survey item analyzed in PSPS as evincing a “decline” in “trust of scientists” for “several decades among U.S. conservatives” (Hmielowski et al. 2013).
But if you do, then you will be obliged to conclude that a majority of those who identify themselves as “liberals” are deeply "skeptical" and “distrustful” of scientists too. The whole nation, on this reading of the data featured in PSPS, would have to be regarded as having “lost faith” in science—indeed, as never having had any to begin with.
That would be absurd.
It would be absurd because the very GSS survey item in question has consistently found—for decades—that members of the US general public are more “confident” in those who “run” the “scientific community” than they are in those who “run” “major companies,” the “education” system, “banks and financial institutions,” “organized religion,” the “Supreme Court,” and the “press.”
For the entire period under investigation, conservatives rated the “scientific community” second among the 13 major U.S. institutions respondents were instructed to evaluate.
If one accepts that it is valid to measure public "trust” in institutions by focusing so selectively on this portion of the data from the GSS "confidence in institutions" item, then we’d also have to conclude that conservatives were twice as likely to “distrust” those who “run . . . major companies” in the US as they were to “distrust” scientists .
That’s an absurd conclusion, too.
PSPS’s analysis for sure adds to the stock of knowledge that scholars who study public attitudes toward science can usefully reflect on.
But the trend the study shows cannot plausibly be viewed as supporting inferences about the level of trust that anyone, much less conservatives, have in science.
That’s the summary. Now keep your promise and continue reading.
A. Let’s get some things out of the way
Okay, first some introductory provisos
1. I think PSPS is a decent study. The study notes a real trend & it’s interesting to try to figure out what is driving it. In addition, PSPS is also by no means the only study by Gordon Gauchat that has taught me things and profitably guided the path of my own research. Maybe he'll want to say something about how I'm addressing the data he presented (I'd be delighted if he posted a response here!). But I suspect he cringes when he hears some of the extravagant claims that people make--the playground-like prattle people engage in--based on the interesting but very limited and tightly focused data he reported in PSPS.
2. There’s no question (in my mind at least) that various “conservative” politicians and conflict entrepreneurs have behaved despicably in misinforming the public about climate change. No question that they have adopted a stance that is contrary to the best available evidence, & have done so for well over a decade.
3. There are plenty of legitimate and interesting issues to examine relating to cognitive reasoning dispositions and characteristics such as political ideology, cutural outlooks, and religiosity. Lots of intriguing and important issues, too, about the connection between these indicators of identity and attitudes toward science. Many scholars (including Gauchat) and reflective commentators are reporting interesting data and making important arguments relating to these matters. Nevertheless, I don’t think “who is more anti-science—liberals or conservatives” is an intrinsically interesting question—or even a coherent one. There are many many more things I’d rather spend my time addressing.
But sadly, it is the case that many scholars and commentators and ordinary citizens insist there is a growing “anti-science” sensibility among a meaningful segment of the US population. The “anti-science” chorus doesn’t confine itself to one score but “conservatives” and “religious” citizens are typically the population segments they characterize in this manner.
Advocates and commentators incessantly invoke this “anti-science” sentiment as the source of political conflict over climate change, among other issues.
Those who make this point also constantly invoke one or another “peer reviewed empirical study” as “proving” their position.
And one of the studies they point to is PSPS.
Because I think the anti-science trope is wrong; because I think it actually aggravates the real dynamics of cultural status competition that drive conflict over climate science and various other science-informed issues; because I think many reasonable people are nevertheless drawn to this account as a kind of a palliative for the frustration they feel over the persistence of cultural conflict over climate change; because I think empirical evidence shouldn’t be mischaracterized or treated as a kind of strategic adornment for arguments being advanced on other grounds; because I have absolutely no worries that another scholar would resent my engaging his or her work in the critical manner characteristic of the process of conjecture and refutation that advances scientific understanding; and because only a zealot or a moron would make the mistake of thinking that questioning what conclusions can appropriately be drawn from another scholar’s empirical research, criticizing counterproductive advocacy, or correcting widespread misimpressions is equivalent to “taking the side of” political actors who are misinforming the public on climate change, I’m going to explain why PSPS does not support claims like these:
B. Have you actually read PSPS?
It only takes about 5 seconds of conversation to make it clear that 99% of the people who cite PSPS have never read it.
They don’t know it consists of an analysis of one response level to a single multi-part public opinion item contained in the General Social Survey, a public opinion survey that has been conducted repeatedly for over four decades (28 times between 1974 and 2012).
Despite how it is characterized by those citing PSPS, the item does not purport to measure “trust” in science.
It is an awkwardly worded question, formulated by commercial pollsters in the 1960s, that is supposed to gauge “public confidence” in a diverse variety of (ill-defined, overlapping) institutions (Smith 2012):
I am going to name some institutions in this country. As far as the people running these institutions are concerned, would you say you have a great deal of confidence, only some confidence, or hardly any confidence at all in them?
a. Banks and Financial Institutions [added in 1975]
b. Major Companies
c. Organized Religion
e. Executive Branch of the Federal Government
f. Organized Labor
j. U.S. Supreme Court
k. Scientific Community
For the period from 1974 to 2010, PSPS examines what proportion of respondents selected the response “a great deal of confidence” in those “running” the “Science community.”
As should be clear, the PSPS figure above plots changes only in the “great deal of confidence” response.
I’m sure everyone knows how easy it is to make invalid inferences when one examines only a portion rather than all of the response data associated with a survey item.
Thus, I’ve constructed Figures that make it possible to observe changes in all three levels of response for both liberals and conservatives over the relevant time period:
As can be seen in these Figures, the proportion selecting “great deal” has held pretty constant at just under 50% for individuals who identified themselves as “liberals” of some degree (“slight,” “extreme,” or in between) on a seven-point ideology measure (one that was added to the GSS in 1974).
Among persons who described themselves as “conservatives” of some degree, the proportion declined from about 50% to just under 40%. (In the 2012 GSS—the most recent edition—the figures for liberals and conservatives were 48% and 40%, respectively. I also plotted pcts for "great deal" in relation to the relevant GSS surveys "yesterday" in this post.)
The decline in the proportion of conservatives selecting “great deal” looks pretty continuous to the naked eye, but using a multi-level multivariate analysis (more on that below), PSPS reported finding that the decline was steeper after the election of Ronald Reagan in 1980 and George W. Bush in 2006.
Do you think that these data justify conclusions like "conservatives' trust in science has declined sharply," "conservatives have turned on science," "Republicans really don't like science," "conservatives have lost their faith in science," "fewer conservatives than ever believe in science," etc?
If so, let me explain why you are wrong.
C. Critically engaging the data
1. Is everyone anti-science?
To begin, why should we regard the “great deal of confidence” response level as the only one that evinces “trust”?
“Hardly any” confidence would seem distrustful, I agree.
But note that the proportion of survey respondents selecting “hardly any at all” held constant at under 10% over the entire period for both conservatives and liberals.
Imagine I said that I regarded that as inconsistent with the inference that either conservatives or liberals “distrust” scientists.
Could you argue against that?
But if you did, you’d necessarily have to be saying that selecting “some confidence” evinces “distrust” in scientists.
If you accept that, then you’ll have to conclude that a majority of “liberals” distrust scientists today, too, and have for over 40 years.
For sure, that would be a conclusion worthy of headlines, blog posts, and repeated statements of deep concern among the supporters of enlightened self-government.
But such a reading of this item would also make the decision to characterize only conservatives as racked with “distrust” pathetically selective.
2. Wow--conservative Republicans sure “distrust “business!
You’d also still be basing your conclusion on only a small portion of the data associated with the survey item.
Take a look, for example, at the responses for Major companies”:
It’s not a surprise, to me at least, that conservatives have had more confidence than liberals in “major companies over the entire period.
I’m also not that surprised that even conservatives have less confidence in major companies today than they did before the financial meltdown.
But if you are of the view that any response level other than “a great deal of confidence” evinces “distrust,” then you’d have to conclude that 80% of conservatives today “distrust” our nation’s business leaders.
You’d also have to conclude that conservatives are twice as likely to trust those “running . . . the scientific community” as they are to trust those “running . . . major companies.”
I’d find those conclusions surprising, wouldn’t you?
But of course we should be willing to update our priors when shown valid evidence that contradicts them.
The prior under examination here is that PSPS supports the claim that conservatives “don’t believe in science,” "have turned on science," “reject it," have "lost their faith in it," have been becoming "increasingly skeptical" of it "for decades," etc.
The absurdity of the conclusions that would follow from this reading of PSPS---that liberals and conservatives alike "really don't like science," that conservatives have so little trust in major companies that they'd no doubt vote to nationalize the healthcare industry, etc. -- is super strong evidence that it's unjustifiable to treat the single response level of the GSS "confidence" item featured in PSPS as a litmus test of anyone's "trust" in science.
3. Everyone is pro-science according to the data presented in PSPS
What exactly do response to the GSS “confidence” item signify about how conservatives and liberals feel about those “running” the “Scientific community”?
Again, it’s always a mistake to draw inferences from a portion of the response to a multi-part survey item. So let’s look at all of the data for the GSS confidence item.
Below I’ve plotted the mean scores for item, treating it as a 3-point measure (Smith 2012), for four time periods.
The mean scores are plotted separately for “liberals” and “conservatives. The 13 institutions are listed in descending order as rated by conservatives-- i.e., from the institution in which conservatives expressed the greatest level of confidence to one in which they expressed the least in each period.
The variance in selection of the "great deal" response level analyzed in PSPS is reflected in the growing difference between liberals' and conservatives' respective overall "confidence" scores for "the Scientific Community."
Various other things change, too.
But as can be seen, during every time period—including the ones in which Ronald Reagan and G.W. Bush were presidents—conservatives awarded “Science community” the second highest confidence score among the 13 rated institutions. Before 1990, conservatives ranked the “science community” just a smidgen below “medicine”; since then, conservatives have vested more confidence in the “military.”
Conservatives rated the “science community” ahead of “major companies,” “organized religion,” “banks and financial institutions,” and “education,” not to mention “organized labor,” the “Executive Branch of the Federal Government” (during the Reagan and G.W. Bush administrations!), Congress, and “TV” throughout the entire period!
Basically the same story with liberals. They rated the “science community” second behind “medicine” before 1990, and first in the periods thereafter.
So what inference can be drawn?
Certainly not that conservatives distrust science or any group of scientists.
Much more plausible is that conservatives, along with everyone else, hold science in extremely high regard.
That’s obvious, actually, given that the “Confidence” item sets up a beauty-contest by having respondents evaluate all 13 institutions.
But this reading—that conservatives, liberals, and everyone else has a high regard for science—also fits the results plainly indicated by a variety of other science-attitude items that appear in the GSS and in other studies.
It’s really really really not a good idea to draw a contentious/tendentious conclusion from one survey item (much less one response level to one part of a multi-part one) when that conclusion is contrary to the import of numerous other pertinent measures of public opinion.
4. Multivariate analysis
The analyses I’'ve offered are very simple summary ones based on “raw data” and group means.
There really is nothing to model statistically here, if we are trying to figure out whether these data could support claims like "conservatives have lost their faith in science" or have become “increasingly skeptical and distrustful” toward it. If that were so, the raw data wouldn't look the way it does.
Nevertheless, PSPS contains a multivariate regression model that puts liberal-conservative ideology on the right-hand side with numerous other individual characteristics. How does that cut?
As much as I admire the article, I'm not a fan of the style of model PSPS uses here.
E.g., what exactly are we supposed to learn from a parameter that reflects how much being a "conservative" rather than a "liberal" affects the probability of selecting the "great deal" response "controlling for" respondents' political party affiliation?
Overspecified regressions like these treat characteristics like being “Republican,” “conservative,” a regular church goer, white, male, etc. as if they were all independently operating modules that could be screwed together to create whatever sort of person one likes.
In fact, real people have identities associated with particular, recognizable collections of these characteristics. Because we want to know how real people vary, the statistical model should be specified in a way that reflects differences in the combinations of characteristics that indicate these identities--something that can’t be validly done when the covariance of these characteristics is partialed out in a multivariate regression (Lieberson 1985; Berry & Feldman 1985).
But none of this changes anything. The raw data tell the story. The misspecified model doesn’t tell a different one—it just generates a questionable estimate of the difference in likelihood that a liberal as opposed to a conservative will select “great deal” as the response on "Confidence" when assessing those who "run ... the Scientific Community” (although in fact PSPS reports a regression-model estimate of 10%--which is perfectly reasonable given that that's exactly what one observes in the raw data).
5. Someone should do a study on this!
There’s one last question worth considering, of course.
If I’m right that PSPS doesn’t support the conclusion that conservatives have “lost faith” in science, why do so many commentators keep insisting that that’s what the study says? Don’t we need an explanation for that?
Yes. It is the same explanation we need for how a liberal democracy whose citizens are as dedicated to pluralism and science as ours are could be so plagued by unreasoning sectarian discourse about the enormous stock of knowledge at its disposal.
Berry, W.D. & Feldman, S. Multiple Regression in Practice (Sage Publications, Beverly Hills, 1985).
Hmielowski, J.D., Feldman, L., Myers, T.A., Leiserowitz, A. & Maibach, E. An attack on science? Media use, trust in scientists, and perceptions of global warming. Public Understanding of Science (2013).
Lewandowsky, S., Gignac, G.E. & Oberauer, K. The role of conspiracist ideation and worldviews in predicting rejection of science. PloS one 8, e75637 (2013).
Lieberson, S. Making it count : the improvement of social research and theory (University of California Press, Berkeley, 1985).
Smith, T.W. Trends in Confidence in Institutions, 1973-2006. in Social Trends in American Life: Findings from the General Social Survey Since 1972 (ed. P.V. Marsden) (Princeton University Press, 2012).
Response from Gordon Gauchat
Thanks for the heads up and I read your thoughtful comments. I think I largely agree with the substance of what you are saying here, with some minor esoteric points of disagreement. For example, your argument about over-specification seems overblown given that the point estimate the model produces seems very accurate.
So, I will start with the particulars about the PSPS paper, and then move onto broader themes. I do agree about some measurement/over-specification issues you bring up, especially about how to measure left-right orientation after controlling forDemocrat-Republican. Unfortunately, one has to accommodate reviewers in the process of publishing articles. I no longer use this technique (and wasn't a big fan in the first place), and in my current research I combine left-right ideology and Democrat-Republican party identity into a single continuous scale. I also agree that the dependent variable is limiting, with only three response categories and the need "for simplicity" to split up the data into a "great deal of confidence" and "only some" and "hardly any." I think the most important point here is not about the point estimates of particular categories, but change over time.
I think there has been too much emphasis on the "limits" of the question and not enough attention paid to identifying the mechanism behind the change in public perceptions among conservatives. That is, what is the mechanism(s) that produces the observed change among conservatives? I also think historical and social context is important, but often forgotten here.
As you point out, the devil is in the details, but I would elaborate on other complications, some of which you only touch on. In an earlier post on the PSPS paper, you suggested that the mechanism behind the change is likely climate change: conservatives perceptions of science are "biased" or I would argue related to the highly politicized discourse about climate change (which they oppose). The reasonable question is: does a change in conservative perceptions, driven by their perceptions toward climate science and environmental activism, really say anything about the cultural authority of science. I think the answer is YES. Here are a couple of very important points that need clarifying.
1) Science enjoys a tremendous amount of cultural authority across all segments of society. All of the public opinion data support this story. But, this DOES NOT mean that science's cultural authority is limitless. And, it maybe useful to understand where these limitations generally lie and to understand group-specific trends. That is, social science researchers (you and I) want to know how society is culturally divided in its acceptance/skepticism of the various aspects of science-in-society, including its relationship to the state or regulatory apparatus, or some specific-issue like climate change.
2) In fact, observing that most people say science is trustworthy or "beneficial" has a limited meaning, because we are really concerned about the connection between science and public policy. And, even if a small but well-funded and mobilized group opposes some key policy (climate change; OSHA regulation; vaccines), this can have an impact on how or whether a policy is enacted. Simply, in democratic societies like our own, some minorities have significant power. Also, because there are no "anti-science activists" who oppose all scientific activity and believe it must be stopped at all costs, it might be difficult to identify the limitations of science's authority. I would argue that the limits of scientific authority relate not to its "epistemic authority," (its ability to make claims about the world), but to its ability to influence policy or regulate human behavior. However, the use of science and scientific knowledge toward these ends seems to be exactly the direction advanced democracies are moving (or at least some parties in these societies). So, I would argue that public perceptions about the intersection of science and the state are really at issue here. Also at issue are attitudes about specific policies where scientific evidence are manifestly important to identifying and managing a social problem. In this sense, I wonder if climate science is a particular example of something larger about "regulatory science" in late modern society. How can we use the scientific knowledge that the public is paying for effectively? What are the limits here? Is the large scale funding and use of scientific knowledge for public policy feasible and politically supported?
3) The cultural authority of science is multidimensional and public perceptions are relational. This is an important point, because public perceptions involve a field of relationships, people don't view science NET OF other objects in society (groups and institutions). I think we need to be reminded of this continuously, because conservatives could become more "skeptical of science" if science and scientists are seen as more liberal and if Democrats are seen as strong supporters of science (these perceptions about other groups need not be true either). Multiple regression models do not encourage this sort of relational thinking, but are not entirely useless in this regard either. However, the researcher has to look at a variety of data and, in many cases, speculate about the mechanisms driving them. Thus, what started as a concern about government regulation and environmental issues could turn into a general disposition toward science among conservatives: if people perceive science in relation to their perceptions of other groups and their perceptions, then this is would produce a "more general" skepticism of scientific authority across the issues. In fact, recent public opinion data from CBS and PEW research point in this direction: climate change, vaccines and views on evolution are becoming "more polarized." Thus, attitudes toward science must account for public perceptions about science IN RELATION TO government policy, technological advances, public funding, economic innovation, their perceptions of other social groups' perceptions, and not search endlessly for the single dimension of science's cultural authority to measure. Given that researchers have yet to fully embrace this multidimensional approach, I think the jury is still out on whether or how science is politicized in the U.S., but I think the data encourage further exploration. Certainly, no single study of public opinion data can be definitive.
4) Finally, I think that conservative challenges to science (if real) are significant, because these dispositions COULD represent a rethinking of the way science relates to the regulatory state. If there is a politicized skepticism about how science is funded, and how scientific knowledge is incorporated into state-policy, then the way science is organized may well change, possibly in the form of severe retrenchment of federal funding in the U.S. Whether this is good or bad, I can't say for certain. But, I think there is currently much stronger support for science austerity on the political right in the U.S., not just attempts to "defund" the EPA. This proposition might be tested in a few years.
My response to Gordon's response
Thank you, Gordon.
I've been thinking all this time since you posted your reflections & still haven't come up with a set of my own that fully reciprocates the contribution your response makes to thinking about the issues we are discussing. If we add to the value of your response the insights reflected in the comments of @L.Hamilton & others, then I'm really coming up short-handed.
But I am moved to make these observations:
1. What's the point?
I just want to underscore your observation that we "largely agree."
Indeed, your response doesn't take issue with my analysis of why the results in PSPS don't support claims (asserted incessantly by tribal contempt mongers and even by some scholars) that members of the public who identify as "conservative" "distrust" science or scientists, and that this "anti-science" sensibility explains conflict over climate change & other issues that turn on policy-relevant science.
I'm pointing this out not in the spirit of a high-school debater who tries to convince the judges that she "wins" because her opponent has implicitly "conceded" her point by "ignoring" it, etc. (you didn't ignore anything, certainly).
Rather, I feel impelled to note that you aren't taking issue with my analysis so that I can remedy what might well have been a failure on my part to emphasize even more strongly than I did that the sort of critique I was offering -- one aimed at the inferences being drawn from your study -- is not a criticism of the soundness or value of the research reflected in PSPS!
As I've mentioned in the post and have stated before, I think it's a really cool paper & calls our attention to a real trend in one measure of public opinion that we ought to try to understand as we puzzle through the larger question of why issues that admit of scientific inquiry so often (but less often than people think) generate political polarization.
It doesn't surprise me at all, then, that you aren't joining issue directly with my analysis. It heartens me by corroborating my surmise that you that you yourself see nothing worth defending in the cartoonish characterizations of your study featured in sectarian and pseudo-scholarly discourse.
2. What's the question?
Your comments are directed at the much more interesting, substantive issue of what we are to make of divisive, cultural conflict over policy-relevant science within a society that itself features very broad, very deep commitment to science, both as an institution and as a way of knowing. How do such conflicts happen? What impact might they have on how science, as an institution and as a way of knowing, figures in political life & our society generally?
These are the issues that fascinate you & me and others who are in our scholarly conversation. Unlike the pathetic and universally demeaning "who's really ant-science-- you or us?" question, they are worth serious attention. They really matter.
In relation to them, I think you & I & others agree on many points but likely do disagree on some important ones too.
In a nutshell (a nanotube even), I think political polarization over risks and other policy-relevant facts is a consequence of the latent distrust citizens with opposing cultural identities have of one another, & their suspicion that "science" is being invoked opportunistically, disingenuously to disguise as claims about "how the world works" what are in fact contested understandings of "how we should live."
I don't think this dynamic reflects any sort of lurking, imminent ambivalence toward science on anyone's part.
I share your worries that the consequence of this sort of conflict could damage the credibility of science, but to be honest I don't worry that much about that.
I am more worried about the barrier this sort of "cultural conflict of fact" poses to our getting the value of our common commitment to science-informed policies, and about the degrading effect it has on liberal public reason in our political culture generally.
But much more important to me than "proving" I'm "right" and you or anyone else "wrong" about these things is that we address our competing conjectures in the manner that we agree is the only one that can warrant a basis for crediting any of them: by using the disciplined method of collecting observations, making reliable measurements, and drawing valid inferences that is the signature of science.
My engagement with PSPS here has to do with whether the evidence that it validly assembles and measures supports particular inferences relating to claims that matter to you and to me, and also relating to some that I think likely don't matter to either of us except to the extent that they create an ugly and unfortunate distraction.
You might be right in the broader claims you advance in your response-- ones that also inform the interesting comments of @LHamilton & others below.
But as I know you know (I make this point b/c so many others don't seem to get this), the question is whether and how much the PSPS evidence supports your position--not whether the position is right, much less how, if one already accepts that conservative members of the public "distrust" science, that position can be used to support an interpretation of the evidence in PSPS consistent with that very position.
3. What's the upshot?
Implicit in your response & I now want to make explicit in mine is the most important agreement we share: that exchanges like this -- earnest, open ones, focused on challenging one another's best understandings of what we are able to observe using the tools of valid empirical inquiry at our disposal -- are essential to the process by which shared knowledge grows.
I was moved to offer this set of reflections on your paper at this particular time in part to answer people who clearly don't understand it, and who mistakenly view an empirical study as "settling" or "proving" a proposition rather than as simply adding another increment of evidence to be balanced along with all the rest as reasoning people weigh the considerations on both sides of a difficult question; who mistakenly view "peer review" as ending with publication, when in fact, that is precisely when the most important form of peer review starts!
But the truth is, I've been reflecting on and engaging with PSPS since the day it was published. It has unquestionably advanced my understanding of matters at the center of my research.
This exchange, including your response & the comments of @LHamilton & others I have received off-line, have helped crystallize for me too that what is very much needed to advance our shared interest in the questions we are addressing are much better measures of trust in science.
Actually, I know you already agree with this. It is one of the conclusions of your excellent study,The cultural authority of science: Public trust and acceptance of organized science, Public Understanding of Science 20, 751-770 (2011), which is filled with interesting analyses of new "science attitude" measures from the 2006 GSS but which you yourself help to show really don't help us to sort out anything -- because it really isn't clear what they are measuring (and they certainly weren't measuring the same thing; they didn't form a reliable scale).
When people trot out the tiresome, empirically uniformed "growing public distrust in science" claim, I like to point out how NSF Science Indicator items (themselves part of the GSS) and other items included in the 2009 Pew public science attitude study uniformly indicate public reverence for science. I like to point out too how universal that sentiment is across diverse cultural, political, and religious groups, etc.
I'd say, in fact, that the GSS "Confidence" in the "science community" item tells the same story, once you consider how responses to it relate to the public's confidence in the full set of 13 public institutions featured in the GSS "Confidence" item.
You in your response and others at various times in their own comments suggest that these "I love science!" measures (let's call them) might be "missing" some more basic kind of ambivalence toward at least some forms of science, a disaffection that might be growing and that might be consequential for certain sorts of policy disputes.
I agree that it would be a mistake to assume the "I love science!" items are supplying a valid measure of what you have in mind.
I'm not sure what they are measuring, frankly, because no one, despite all the scholarly writings about what one or another measure of public opinion suggests about "trust" in science, has ever even defined the construct of "trust in science" they have in mind, much less validated the measures they are using to test their various hypotheses about it!
But then it's clear what's holding us up-- what we need not necessarily to assure convergence of our views but rather to continue the conversation that we know will make us both smarter whether or not we end up agreeing!
We need to articulate clearly what the relevant constructs are as we continue to assess competing conjectures that feature opposing claims about "trust in science," and we need reliable validated measures of those constructs.
(I would have thought it would be obvious that if we are trying to figure out whether in fact "distrust" in science among the public generally or among some particular subpopulation accounts for disputes over issues like climate change or even evolution, the positions of members of the public on those issues can't be used as measures of "trust in science"; those who correlate one or another group characteristic with climate skepticism or disbelief in evolution or any other issue to "prove" that "diminishing trust in science" can be attributed to one or another group's "science skepticism" are plainly arguing in a circle. But they keep chasing their tails in this way, as do those who ignore evidence that the "trust" measures they use to examine particular risk controversies can in fact be shown to be measuring the very attitudes they are trying to explain. Bizarre!)
I'm betting that we'll eventually see valid "trust in science" measures that can be used to examine the interesting, important, and certainly very plausible claims you discuss in your response. And I'm betting that your future work will make a major contribution to their development.
Weekend update: Does marching around in costumes help overcome cultural polarization? Comparative data might help answer this Q
Someone should do some research on the effectiveness of this "communication strategy" as a means of extinguishing the cultural conflagration surrounding the issue of badger infestation in UK. Who knows-- what's learned might help us to learn to defuse other hyper-charged risk issues in US, like GM foods and raw milk.
Those close to the effort have a strong sense-- the kind that certainly is unlikely to lead them astray-- that costumed marchers banging symbols, blowing whistles & waving signs will spur the government to act within days! But I suppose we should wait for data before drawing any firm conclusions.
Two additional things to note here.
One is the novel idea to use of ideologically diverse vouchers. Seeing these two well-known representatives of opposite sides of the UK political spectrum (it doesn't exactly run from left-to-right; it's more up & downish) is definitely something that communicators on other complex science issues like climate change should consider trying--it's really quite astonishing that they haven't! I really don't see how this could fail to work.
The other is to tie the divisive issue of badger infestation to a "solution" on which there is already widespread consensus in the UK -- vaccination!
We all know, e.g., that conservatives as well as liberals love making money! Why not "frame" climate-change mitigation as an opportunity to profit from innovative "green marketing solutions"?! I bet that would work! Why has no one even thought to try it? Weird!
This worked pretty well before, so why not try again?
Recently I posted some data on right-left political outlooks, religion, and positions on disputed science issues and asked you, the 14.33 billion readers of this blog, to say what inferences the data support. The responses, including one submitted by Chris Mooney in his Washington Post Wonkblog, were really interesting, and led to an informative set of exchanges, which continued when I finally added my own assessment. I'm pretty confident the discussion wouldn't have been as enlightening had I offered my own views of the data in the original post.
Well, here's some more data!
These are all from the General Social Survey data set, 1973-2012. The question are ...
Do these data support inferences on the cause of public controversy over science issues such as climate change? In particular, how do they bear on the commonplace claim that such controversy originates in a growing "distrust"of or hostility toward science associated with right-of-center political outlooks?
Well, you tell me! I'll say nothing, nothing for at least 48 hours (±6 hrs).
Okay, I said I would hold back for a bit on my own conclusions, but here are a few reactions to these very data from others, just to prime the pump of reasoned engagement with evidence:
Hey--once again: prize to commentator who offers "best" interpretation (as determined by expert consensus survey)!
ICT eats RAT & CAT for breakfast: More (and more data on) religiosity, political predispositions, and "anti-science"
The answer, in my view, is that both CAT & RAT are barking up the wrong tree!
Neither conservative ideology nor religiosity has been shown to predict a greater anti-science disposition than the other by the evidence presented. And indeed, that evidence, plus some more, suggests that it's a mistake to think either of them is connected to such a disposition at all.
For those of you just tuning in (site traffic suggests only 10 billion readers for the original post; apparently there was a climate-change induced net outage in the Netherlands Antilles, where there is a very strong CCP following), the question was, “What ‘explains’ public science conflict—political predispositions or religiosity?”
The inspiration for posing the question was an intriguing study that pinned the blame on religion. CCP blog readers viewed the study as methodologically dubious.
But the question was interesting so I decided to try to help us think about it by gathering data and presenting models that seemed responsive to commentators’ concerns.
I characterized the two positions that the original study seemed to be pitting against each other as the “Conservative-science Antipathy Thesis,” or CAT, which identifies antagonism between conservative or right-leaning ideology toward science as the source of public conflicts over climate change and various other science-informed policy issues; and the “Religion-science Antipathy Thesis,” or RAT, which states that religious animosity toward science is the cause.
I used appropriately modeled data from CCP and from the Pew Research Center studies to try to remedy shortcomings in the study that inspired the question—and then asked you, the loyal, perfectly rational 14 billion readers of this blog (or at least the 10 billion who managed to get through and submit response) to say what you made of the evidence.
I’d say Chris Mooney offered the best response, a conclusion I validated by doing an “expert consensus survey." He has been awarded the prize that was offered (he chose the Synbio Ipad—the very last one in stock).
The three issues that were featured in the original study (the one I tried to remedy the methodological defects of)—were climate change skepticism, disbelief in evolution, and opposition to federal support for stem-cell research. Political predispositions and religiosity both seemed to predict these attitudes but in ways that varied in degree and that interacted with one another in diverse patterns. CM thus concluded:
So what's the upshot? Obviously, both politics and religious beliefs contribute to science resistance, and the relative influence of one over the other varies on an issue-by-issue basis. The role of religion is very strong on the evolution issue, far weaker on the climate issue, and somewhere in between on the stem cell issue. And if you picked other issues to examine, you would assuredly find different results yet again.
What this exercise underscores, most of all, is that when people deny science, they do it because they think it conflicts with their personal identity. But many elements go into each of our identities, with both politics and religion constituting vital components for many people.
In light of this, it really doesn't make much sense to assert the power of one over the other.
Yup, for sure I agree with that.
But I’d go further: the evidence presented helped to reveal that neither CAT nor RAT is a very well supported.
In a mistake that is pervasive in the study of public attitudes toward science, the original study constructed its sample of observations in a manner that presupposed a generalized anti-science sensibility is the explanation for conflicts over evolution, climate change, evolution, etc.
But that’s a seriously contested issue too!
ICT—the “identity-protective cognition thesis”—is a major alternative to both RAT & CAT (Sherman & Cohen 2006). On this account, when policy relevant facts become entangled in antagonistic cultural meanings, people start to see individuals’ positions on them as badges of membership in & loyalty to opposing cultural groups. As a kind of identity self-defense, then, they begin to process information relating to these facts in a manner that conforms their beliefs to the positions that are dominant in their group (Kahan 2010, 2012).
RAT & CAT predict, respectively, that “religiosity” and “conservatism” can be shown to offer the best "explanation" for science-hostile positions generally.
It’s not clear that ICT will take any view on the relative influence of religion & conservatism in science disputes. Indeed, for the reasons CM stated, I think it’s strange to imagine that one could meaningfully specify cultural identities in the US in a way that split religiosity and political commitments apart.
But ICT (or at least the variant I find most compelling) does join issue with both RAT and CAT on whether disputes over science can should be attributed to any particular cultural group's “anti-science” dispositions.
Being “liberal” and even “nonreligious” are both integral to self-defining commitments of certain people. Accordingly, where positions on some science-informed policy or other matter becomes entangled in antagonistic cultural meanings, we should expect liberal and nonreligious individuals, as well as conservative and religious ones, to display the signature forms of motivated reasoning that distort their perception of the best available science (Kahan, Peters, Dawson & Slovic 2013).
ICT is a combatant in the “great asymmetry thesis debate,” which asks whether motivated reasoning on science-informed policy issues is concentrated in one end of the political spectrum or instead spread evenly across it is the(Crawford 2013; Brandt & Crawford 2013; Kahan 2013; Mooney 2012).
It was a serious defect in the study that inspired this exercise that it didn’t include in its observations any disputed science issues that might show neither conservatism nor religiosity is distinctively “anti-science.”
By doing so, it assumed particular answers to the question it purported to be investigating.
To remedy this defect, I added another disputed science issue: nuclear power.
That’s one where individuals whose cultural identities are more secular and left-leaning are typically understood to be the ones disposed to adopt “science-skeptical” or “science hostile” positions.
That's more consistent with ICT--and its position on the ideological symmetry of motivated reasoning (Kahan 2013)-- than with either RAT or CAT.
A related point: if a researcher wants to do a valid test of whether disputed science issues are a consequence of one or another group's supposed "anti-science" disposition, then he or she definitely should not rely on simple correlations between the disputed issues and group identities.
Yes, conservatism and religion are associated with hostility to stem-cell research, climate skepticism, and disbelief in evolution.
But to treat that as evidence that conservatism and religion are anti-science and that that is what causes disputes on these issues presupposes that these positions are all explained by some sort of anti-science sensibility rather than something else.
To avoid this obvious error (an instance of selecting on the dependent variable), the researcher has to have a way of measuring whether groups are “pro-“ or “anti-science” independently of their positions on climate skepticism, disbelief in evolution, and opposition to stem-cell research.
Here are some data that help to do that. They come from the GSS, the source used by the intriguing but questionable study that initiated this exercise, and the Pew Public Attitudes Toward Science data base that I identified as much more suited for their study:
What do they show? You tell me! (Click on either for more detail.)
But I will tell you in the meantime what inference I draw from them: (a) that the US public is overwhelmingly pro-science; and (b) that any differences associated with politics and religiosity both are ambiguous and, more importantly, trivial in magnitude.
These sorts of responses—and there are many more items in these data sets that support the dame conclusions (one should in fact look at all, not just one, if one is trying to figure out what they signify!)—are inconsistent with the inference that either conservatism or religiosity is antithetical to science, and hence inconsistent with the assumption that the correlation of these characteristics with climate skepticism, disbelief in evolution, or opposition to stem cell research evinces an anti-science orientation. Accordingly, it is even less sensible to think that one could look at these issues to say which one is “more” anti-science than the other.
Those who attribute disagreement with their views on science disputes to their opponents’ “anti-science” dispositions don’t come off looking especially “pro-science” themselves when they fail to use evidence in a scientifically valid way.
Brandt, M. J., & Crawford, J. (2013). Replication-Extension of 'Not for All the Tea in China!' Political Ideology and the Avoidance of Dissonance-Arousing Situations' (Nam, Jost, & Van Bavel, 2013, Plos One). Retrieved from http://ssrn.com/abstract=2365281 or http://dx.doi.org/10.2139/ssrn.2365281
Crawford, J. T. (2012). The ideologically objectionable premise model: Predicting biased political judgments on the left and right. Journal of Experimental Social Psychology, 48(1), 138-151.
Mooney, C. (2012). The Republican Brain: The Science of Why They Deny Science--And Reality. Hoboken, NJ: John Wiley & Sons.
Sherman, D. K., & Cohen, G. L. (2006). The Psychology of Self-defense: Self-Affirmation Theory Advances in Experimental Social Psychology (Vol. 38, pp. 183-242): Academic Press.
What accounts for public conflict over science--religiosity or political predispositions? Here are some data: you declare the winner in this RAT vs. CAT fight!
Outsourcing my critical reading (i.e., just plain reading!) of this article worked really well. Given the great points that came out in the comments, I don’t think there’s any value added in my offering a full assessment of the paper, which reported the results of a study that analyzed less than ideal data with a questionably specified statistical model from which the authors seemed to draw very debatable inferences.
But I do think it might be interesting to explore, at least to a degree, what one might learn about the authors’ research questions if one applied a valid statistical model to data that could support some reasonable inferences.
Basically, the authors purport to find that “religious variables,” but not “partisan identification,” predict a general hostility to science as manifested by attitudes toward climate change, evolution, and stem-cell research. They treat this finding as suggesting reason to doubt claims that partisan political predispositions (e.g., Gauchat 2012), shaped by elite discourse (e.g., Mooney 2005; Brulle et al. 2012), account for public conflict over science issues.
Because the authors believe that religiosity bears a greater share of the responsibility for such conflicts than is normally appreciated, let’s call this the “Religion-science Antipathy thesis” (RAT).
The main “political” competitor the authors advert to is one that attributes such conflict to antipathy—perhaps psychologically grounded (Mooney 2012), perhaps economically (Brulle et al., 2012), or maybe both—between a conservative political orientation and science. Let’s call this the Conservativism-science Antipathy Thesis” (CAT).
So here’s what I’m going to do. I’ve compiled a bunch of observational (i.e., survey!) data and modeled them in a way that I think arguably bears on the relative plausibility of RAT and CAT.
But exercising a credulity-defying degree self-restraint, I am going to refrain, for at least 24 hrs, from telling you what sorts of conclusions I think these data support.
In that period, you, the 14 billion readers of this blog, will be afforded the exclusive opportunity to specify and defend your own inferences!
How many of the myriad other “cultural cognition blogs” out there do you think would have the necessary levels of self-confidence and respect for their readers to surrender their “first word” prerogative? That’s right—not a one!
Let’s start with global warming.
As you can see, this figure plots the probability of agreeing that there is “solid evidence” for human-caused global warming in relation to right-left political outlooks conditional on one’s level of “religiosity.” Political outlooks and religiosity are measured with separate multi-item scales. (Because partisan self-identification and liberal-conservative ideology are both indicators of the same thing—an unobserved or latent political disposition—it is really not a good idea to treat them as “independent” right-hand side variables in a multivariate regression.) The colored hashmarks are the 0.95 confidence intervals for the predicted probability at the indicated point on the left-right political outlook scale.
If you want to “see” the regression model or the “raw data,” then click on the specified thumbnails in the margin.
As you can tell, there’s an interaction between religiosity and political outlooks: the contribution that moving left in outlook makes to acceptance of climate change is bigger the less religious one is.
Next, let’s look at belief in evolution. Same model, used now to examine the impact of political outlooks on belief in evolution conditional on religiosity.
Next, support for stem-cell research.
Actually, I’ve never collected data on this topic. So I popped open a canned dataset that has such data: the super great 2009 Pew public attitudes toward science survey.
I again constructed a political outlook scale by aggregating response to partisan self-identification and liberal-conservative ideology items.
Pew didn’t have all the same items from which I constructed the religiosity scale in the previous models. So I constructed one using self-reported church attendance (one of the items that I did have in my CCP data set), self-identified “born again” evangelical status, and a “non-religious” self-identification variable that separated out persons who self-identified as agnostics or atheists from those who reported affiliation with any religious denomination.
How good a measure is this? I wasn’t sure, so I came up with a method to externally validate it.
It turns out the Pew survey also has measures for global-warming acceptance and belief in evolution (the authors of the study that inspired this exercise should have used the Pew dataset rather than the 2006 GSS dataset, which lacked a genuine measure of global-warming acceptance). When I used the Pew religiosity scale and the right-left political outlook measures as predictors of these beliefs, the Pew religiosity scale behaved very comparably to the CCP-dataset religiosity scale in the modelsreported above. That struck me as pretty good evidence that the Pew scale is tapping into pretty much the same unobserved or latent disposition being tapped into by the CCP religiosity scale.
Here’s the result for stem-cell funding:
What do you think?
These are the three issues—global-warming acceptance, belief in evolution, and support for stem-cell research—that the article we read used to test RAT vs. CAT.
But some of you pointed out that disbelief in evolution and opposition to stem-cell research are arguably the sorts of positions one might expect highly religious individuals to form independent of any sort of general hostility to science. For that reason, one might conclude they don’t supply as clean a test of RAT vs. CAT as, say, climate-change acceptance, where generalized "science hostility" is less likely to be confounded with issue-specific religious concerns.
So, I decided to add one more issue to try to make the fight more fair: nuclear power!
The great Pew study had two nuclear-power items, positions on which I also modeled in relation to political outlooks conditional on religiosity:
So there you go.
Brulle, R.J., Carmichael, J. & Jenkins, J.C. Shifting public opinion on climate change: an empirical assessment of factors influencing concern over climate change in the US, 2002–2010. Climatic Change 114, 169-188 (2012).
Gauchat, G. Politicization of Science in the Public Sphere. American Sociological Review 77, 167-187 (2012).
Mooney, C. The Republican Brain: The Science of Why They Deny Science--And Reality (John Wiley & Sons, Hoboken, NJ, 2012).
Mooney, C. The Republican war on science (Fine Communications/MJF Books, New York, NY, 2009).
So now one of the 14 billion readers of this blog, Chris Mooney, has offered his views on RAT vs. CAT in his own writeup on wonkblog.
Others might want to consult what he has to say as they continue to formulate their own interpretations.
You know, it's not too late for me to turn this into one of the cool contests that we regularly run here on CCP blog.
Submitter of "best interpretation" (as determined by "scientific consensus" survey) will get his or her choice of any of these prizes (subject to availability):
Weekend update: cognitive illiberalism--what is it? & what does it have to do with the Constitution?
Now & again people ask me what I mean by the term "cognitive illiberalism." That's reasonable; I often use that term w/o stopping to spell it out. That's because I & my collaborators have already done so in various places. But of course people join conversations in progress all the time, & their participation is impeded by unfamiliar, specialized terms that those who've been participating for a longer period have constructed to condense information of recurring significance. It wouldn't make much sense for those who are parties to an ongoing conversation to deny themselves the efficiency of this device just so new entrants could follow along without confusion; indeed, the conversations that would result would be so burdened by throat-clearing re-elaboration of all that has come before that no one would have the time or patience to take part in them. But what does make sense is for the parties to such conservations to help reduce the cost of joining by contributing to the stock of reasonably accessible materials that a curious person can consult to recover the content that is being compacted into these terms of art. So here is an excerpt from Kahan, D. M., Hoffman, D. A., Braman, D., Evans, D., & Rachlinski, J. J. (2012). They Saw a Protest : Cognitive Illiberalism and the Speech-Conduct Distinction. Stan. L. Rev., 64, 851-906, that helps explicate the concept of "cognitive illiberalism" and its significance to enforcement of the individual liberty provisions of the Constitution. In the future, I'll hyperlink to it, refer to it, etc., when I think will help out someone who might be interested in what I'm saying but who (reasonably enough!) has never heard of "cognitive illiberalism."
In a 1950s social psychology experiment, students from two Ivy League colleges were instructed to evaluate a series of controversial officiating calls made during a football game between their respective schools. Researchers found that the students, from both institutions, were much more likely to perceive error in the penalty assessments imposed on their school’s team than in those imposed on their rival’s. The students’ emotional stake in affirming their loyalty to their institutions, researchers concluded, had unconsciously shaped what they had seen when viewing events captured on film. This study is now recognized as a classic demonstration of “motivated cognition,” the ubiquitous tendency of people to form perceptions, and to process factual information generally, in a manner congenial to their values and desires.
Motivated cognition poses an obvious hazard for law. Sports fans are permitted—even expected—to be partisan. But legal decisionmakers must be neutral. Just as the integrity of a sporting contest would be undermined by unconscious favoritism on the part of the referee, so the legitimacy of the law would likewise be compromised if legal decisionmakers, as a result of motivated cognition, unwittingly formed perceptions of facts that promoted the interests and values of groups with whom they had an affinity.
This effect could be particularly subversive of constitutional law. The Free Speech, Equal Protection, and Due Process Clauses all mandate governmental evenhandedness. Within their respective domains, each forecloses the state from privileging particular affiliations, ways of life, or points of view and mandates that law be justified by its contribution to secular interests—physical security, public health, economic prosperity—valued by all citizens. But if decisionmakers (particularly adjudicators) unconsciously apply these provisions to favor outcomes congenial to favored ways of life, citizens who adhere to disfavored ones will suffer the same array of disadvantages for failing to conform that they would in a regime expressly dedicated to propagation of a sectarian orthodoxy. This distinctively psychological threat to constitutional ideals, which we will refer to as “cognitive illiberalism,” has received relatively little attention from commentators or jurists.
We performed an experimental study designed to help assess just how much of a threat cognitive illiberalism poses to constitutional ideals. The study focused on a discrete and recurring task in constitutional law: discernment of the line between “speech” and “conduct” for purposes of the First Amendment. Embodied in a variety of doctrines, the speech-conduct distinction aims to assure that coercive regulation is justified on grounds unrelated to governmental or public hostility to disfavored ideas. Most importantly, the speech-conduct distinction has historically played, and continues to play, a vital function in preventing the government from invoking its responsibility for maintaining “public order” to disguise suppression of impassioned political dissent. Our study furnishes strong evidence that this function is indeed highly vulnerable to the power of motivated cognition to shape decisionmakers’ perceptions of the facts that mark the speech-conduct boundary.
The features of the speech-conduct distinction that make it susceptible to this influence, moreover, are shared by a host of other constitutional doctrines. The study results thus highlight the need to fortify constitutional theorizing with psychological realism. Normatively ideal standards for enforcing the Constitution are of little value if applying them defies the capacities of constitutional decisionmakers. * * *
[III.]B. Cognitive Illiberalism and the Constitution
The practical motivation for this study was to focus attention on the danger that cognitive illiberalism can pose to constitutional law. We use this term to refer to the vulnerability of political and legal decisionmakers to betray their commitment to liberal neutrality by unconsciously fitting their perceptions of risk and related facts to their sectarian understandings of the good life. This is the dynamic, we believe, that transforms seemingly empirical debates over how to protect the environment, promote public health, and secure the nation from external threats into occasions for divisive group-based status competition. Our study results show how readily constitutional decisionmaking can become infected by this pathology.
In our subjects, cognitive illiberalism eviscerated the line between “speech” and “conduct.” The speech-conduct distinction can be seen as one doctrinal device courts employ to test whether a regulation conforms to liberal prohibitions on governmental promotion of a moral or political orthodoxy: by requiring that a regulation be shown to promote a governmental interest independent of hostility to any particular idea, the teleological conception of the speech-conduct divide assures that law is used to pursue secular goods of value to all citizens regardless of their cultural outlooks.
Enforcing this test, however, necessarily requires decisionmakers to make critical determinations of fact: in the case of a mass demonstration, for example, did the protestors intend to intimidate or only persuade? Were the protestors simply expressing impassioned dissent, or did they impose themselves on members of the public in anassaultive or invasive manner (e.g., “screaming in their faces”)? Were onlookers genuinely frightened of physical assault, or merely angry, offended, or possibly even ashamed by exposure to the protestors’ message? Did law enforcement actors intervene to preempt incitement to violence or only to quell a public backlash propelled by animosity toward the demonstrators’ point of view?
For our subjects, the answers were decisively shaped by the congruence between the protestors’ message and the subjects’ own cultural worldviews. As a result, in the course of certifying that the law was free of culturally partisan influence, they ended up infusing it with exactly that.
Other First Amendment doctrines also seem vulnerable to this type of subversion. * * *
Indeed, we suspect this point can be generalized to constitutional theory as a whole. As discussed, the First Amendment can be integrated into a general theory that reads the Constitution as implementing the liberal prohibition on state endorsement of partisan conceptions of the good life. Like the First Amendment, the Equal Protection and Due Process Clauses require courts to “strictly scrutinize” proffered secular rationales—public health, deterrence of criminal violence, national security, and the like—to “flush out” the impact, conscious or unconscious, of regulators’ animosity toward those whose identity or values defy dominant norrms. But if legal decisionmakers, like everyone for else, are unconsciously motivated by their cultural affiliations, then they—like everyone else—are more or less likely to see challenged laws as contributing to the attainment of secular ends depending on whether those laws affirm or denigrate their own cultural commitments. Angry denunciations of judges who have thrown their lot in with one or the another of the belligerents in the American “culture wars” is itself a form of status conflict characteristic of cognitive illiberalism.
Some legal commentators (and historically certain jurists) have criticized constitutional standards that “balance” constitutional liberties against “compelling interests,” such as national security, public order, and diversity. The phenomenon of culturally motivated cognition vindicates their anxiety that such “tests inevitably become intertwined with the ideological predispositions of those doing the balancing.” But our study results suggest that these commentators are too quick to assume that their preferred alternative to balancing—such as the “teleological conception” of the speech-conduct distinction, the “anticaste” principle, the liberal “harm” criterion, and the like—will necessarily avoid such entanglement. The primary implication of our study— the main message we are trying to get across—is that constitutional theorists have paid too much attention to explicating the normative content of various free speech standards and too little to the psychology of enforcing them.
"Religion, not political predispositions or political elite discourse, generates conflict over science" Seriously?!!!
Okay, I'll get to this but not for a bit. Maybe one or more of our 14 billion readers can read it in meantime and report in comments field?
As you can see from the abstract, the basic claim is that neither political predispositions nor the positions of political elites contribute much to conflicts over science relative to the contribuiton that religion makes.
I'll admit that I have priors very strongly opposed to this thesis. But I'll do my best not to let those infect my likelihood ratio as I examine the authors' evidence.
In the meantime, I'd be grateful as I'm sure billions of others would be for anyone else's assessment.
From correspondence with a friend:
You mentioned you were eager to learn what I had in mind about how to use the science of science communication to improve science journalism. I'm sure you can guess what I'd say: you tell me -- & I'll measure!
We've talked about this philosophy, of course. I think all the professions that traffic in the dissemination of what's known by science can benefit from the use of science's signature methods to improve their craft. Not b/c those methods furnish a substitute for the exercise of professional judgment or craft sense; but b/c they are suited for generating information -- and inspiring informative action -- that those with professional judgment would recognize as valuable.
These methods are uniquely suited for doing that, I think, on questions that experienced professionals themselves recognize as having competing plausible answers. In that situation, there will be no progress through more & more talk, in the form, say, of perennial panels that rehash the opposing positions yr after yr at professional conferences, as predictably entertaining as those are!
What's needed are appropriate tests -- ones designed to generate observations the nature of which will give the professionals at least some more reason than they had previously for crediting one or another of their competing surmises.
Those tests are unlikely to definitively resolve any particular disputed issue!
But they can be expected to infuse new information into science journalists' own continuous process of professional self-assessment. They can be expected, too, to inspire particular practitioners to try something new in their work, generating outcomes that can themselves supply a basis for additional reflective assessment.
As a result, the ongoing critical engagement of science journalists with their own craft norms will unfold in a manner that these professionals will themselves find more satisfying.
But if you ask me what to do, then you are not fully grasping what I'm saying!
I am not of your profession; I don't have your craft sense, your professional judgment.
There are some things that I can do, using my own craft judgment and skill to the best of my ability, that will give you relevant information. I can do an experiment, for example, designed to pit two of your plausible conjectures against each other & generate the information that would give you more reason to view one or the other as more likely true than you previously had.
But you must tell me what the plausible conjectures are.
You must tell me whether the design I have crafted is such that it really will generate a result that those w/ professional judgment would regard as supporting the sort of inference I'm describing.
And most importantly of all, once we are done with that experiment, you must tell me what you think can be done in the real world, the particulars of which were stripped away in our study so we could be confident we knew what was happening and why, to reproduce the effect observed in the lab. At which point -- I will again help by measuring: that is, by applying my knowledge to figure out how to fit to your real-world activity some apparatus for collecting observations on the basis of which you will have more reason than you otherwise would have had to think that what you are doing is or isn't working.
After that--or better still over the course of the process, at the various stages at which there are observations to share--you will go to the professional conference & describe what you have been up to. And everyone will talk about what can be learned. Professional judgment will continue to evolve in the way that it always has --in response to members’ reflective engagement with their shared experiences--but now with the benefit of this additional input on a disputed issue that had been resisting resolution with the information previously at hand.
This is what it is like to have a genuine evidence-based culture within a profession.
To have people from outside your profession do stylized studies & then purport to tell you what do is not. Not seeing that is, I think, is one reason that science journalists report getting little value from events like the Sackler Science of Science Communication symposia. You actually should be dissatisfied if researchers who do what I do--conduct studies designed to explore the relative significance of alternative mechanisms thought to be of consequence for one or another aspect of science communication -- tell you "here's what to do"; b/c they don't know how to connect that relevant research to practice & shouldn't pretend to (really really really shouldn't; I think it is in fact unethical for them to peddle “how to” advice manuals and the like to science communicators—rather than being clear on the need for evidence-based practice “all the way down”).
But in turn, you shouldn't expect that sort of counsel from them! You have the situation sense that is essential to figuring out how to translate the relevant lab studies into practices that might plausibly link up with what the studies have identified to be the relevant mechanisms; at which point, there is again a role to be played by those who measure.
So -- don't ask what the science of science communication can do for your; instead ask, "What can I do with the science of science communication for myself."
I am saying only that making this sort of evidence-based practice a part of the professional culture of science journalism -- along with all the other professions that traffic in disseminating what's known by science -- will make the evolution of its members' professional judgment better by their members' own lights.
That's a hypothesis! I'm happy to help anyone in these professions test it.
p.s. I’ve addressed this before; there is a groundhog-day quality to discussing the need for & character of the “science of science communication." But that’s okay, b/c you actually can sometimes make the same day a bit better or more complete than it was last time -- & can hyperlink to things that still seem to make sense.
Anyway, here are some relevant posts:
- "A science of science journalism & filmmaking" vs. "Throw strikes & keep 'em off the bases" (lecture synopsis & slides)
- Science and the craft norms of science journalism, Part 2: Making craft norms evidence based
- "So what?" vs. "You tell me!"
- Science journalists: Ask not what the science of science communication can do for you . . .
I've been corresponding with a friend whom I -- & many others -- regard as an extraordinary climate-science communicator (& whose skills in this regard are matched by the depth of her civic virtue). In addition to how she manages to communicate so successfully, we have been discussing my view of how big an impact her efforts, if they could be enlarged in scale, could be expected to have in reducing public conflict over climate change. My position is complicated; and I'm not nearly as good a science communicator as she is! But I thought I'd share my best efforts so that the 14 billion readers of this blog might also tell me how to improve my communication of this point, and of course whether it is a point that merits wider communication.
I don't disagree w/ anything you are saying about how to engage people whose cultural identities are threatened by information on global warming. On the contrary, I think there is a tremendous amount to be learned from your example about how to counteract this dynamic in settings in which educators and others are helping people understand the basic mechanisms of climate change and the weight of the scientific evidence on its causes & consequences.
What I'm less sure about is the connection between promoting comprehension in those settings & promoting more constructive engagement with climate science in our national political life.
Essentially, the science communication environment has become polluted with antagonistic cultural meanings that transform "positions" on global warming into badges of membership in & loyalty to competing cultural groups. Those meanings effectively disable the faculties that diverse citizens use, very successfully most of the time, to align their own decisionmaking (personal & collective) with the best available evidence.
I see you as implementing, as it were, a kind of cognitive adaptation strategy. By proving to suspicious listeners that you harbor no hostility to their identities, you create conditions in which people do what they normally do with their reason--use it to make sense of complex things &, even more important, to reliably recognize what’s known by science.
Again, I view that as enormously important-- & will say more in a moment about why.
But the fact remains that what you are doing doesn't actually repair the polluted science communication environment.
You are making it possible for people to reason within that environment when you yourself are presenting information to them. But when you finish, the antagonistic meanings that make global-warming positions into symbols of membership in opposing cultural groups persist in the world in which those people live.
Those meanings will certainly continue to shape the perceptions of those you didn’t get a chance to talk to-- who outnumber those you did by orders of magnitude.
But even more important, those meanings are also likely to continue shaping how the people you did talk to engage the climate issue in democratic political life. That's because the positions people adopt on climate change in that realm aren’t caused by any deficit in their understanding of how climate works; on the contrary, they are a consequence of just how keenly perceptive they are of what stances on global warming express about people's group identities.
Observational studies support this: individuals who display the highest degree of science comprehension generally & climate science comprehension in particular are the most culturally polarized over whether human-caused climate change is even occurring. That question, on a survey & in our political life, measures who they are--whose side they are on--not what they know.
My surmise, then, is that even though the people you have communicated with have genuinely learned something--& likely now enjoy a comprehension level that puts them at the very top of the scale for public "climate literacy”-- they'll continue to orient themselves toward the issue of global warming in a way that evinces their group identity.
That means, among other things, that someone who otherwise possesses the requisite sorts of values & the formidable degree of intelligence that these citizens appropriately look for in selecting congressional representatives is unlikely to enjoy a very long career in national politics if that person decides to make addressing the risks of climate change one of his or her priorities.
That would be the equivalent of the obtuse Presidential candidate who decides to highlight his support for gun control in advertisements in West Virginia because a pollster has just handed him a survey showing that a majority of its residents favor it. He doesn't get that what stances on gun control say is much more important to citizens than what restrictions on guns would do. For West Virginianians, the decision of a candidate to make gun control one of his key positions "says a lot about who you are and who you aren't"--and the only message his ads will succeed in communicating is that "he's not one of us.'' (If only someone had told him!)
As long as the positions that people--ordinary citizens or politicians--adopt on climate change convey "who they are," the issue will continue to polarize culturally diverse groups, no matter how much their members know about the dangers of failing to address global warming.
That means we need a science communication environment mitigation strategy. We need to staunch the "us-vs.-them" sources of pollution, which emanate from both sides. We need to detoxify that environment, so that reasoning people & their representatives don't face the sorts of conditions that in fact make it perfectly rational for them to form climate-change positions that express who they are instead of what they know.
As I said, I don't think creating conditions in which people can learn & comprehend is sufficient to do that.
Nevertheless, creating such conditions is vitally important.
It is vitally important, first, because it has intrinsic value: people who want to be able to know what is known by science should be enabled to know that. Science communicators who virtuously respond to this need should learn how to do what you do.
Second, doing what you do is vital because even amidst the toxic conditions that stifle constructive national policymaking on climate change, many people will need to make consequential decisions that should be informed by the best available evidence.
No matter what happens at the national level, e.g., people will have to make collective decisions about how to adapt to climate change--not cognitively but physically.
They'll have to decide individually, too, how to make all manner of adjustments in their private affairs to reflect a changing climate. Think, e.g., of farmers in the midwest who, despite "not believing" in human-caused climate change, are in fact very interested to hear about the latest climate modeling forecasts of the USDA & EPA (also the work that firms are doing to create genetically modified crops that will fare better in changed climate conditions).
Communicators need to know how to convey this information to these actors, too -- and can learn something from you about how to do it.
Third, what you are doing is vital because it is making us smarter about how science communication works.
Again, I don't think that the success you are achieving in helping individuals to learn about climate science by itself offsets the dynamics that make climate change polarizing, and that stifle exploitation of our scientific knowledge in national policymaking.
But you are showing in one very important setting how to disentangle the question "who are you, whose side are you on?" from "what do we know about how the world works?"
By systematically studying how you & other communicators (particularly educators) are able to achieve this effect, we can learn a tremendous amount about the dynamics of "disentanglement" generally & thus figure out more quickly what sorts of things we should do to reproduce that effect in our politics.
In sum, the work you do not only fills me with admiration & awe. It also fills me with hope & excitement, and with a sense of motivation & direction in my own research, my highest aspiration for which is that it will contribute to formation of a science-communication culture that embodies your skill & knowledge.
But I do think that the benefit we can get from learning how to do what you do will depend on getting a lot of other people to recognize that improving popular comprehension of climate science won't in itself do much to resolve the cultural conflict over global warming.
On the contrary, we need to decontaminate our science communication environment of antagonistic cultural meanings so that we can get the benefit of what you & others are doing to help citizens comprehend what science knows.
"A science of science journalism & filmmaking" vs. "Throw strikes & keep 'em off the bases" (lecture synopsis & slides)
I haven't been faithfully reporting on recent talks, workshops etc. But fortunately, James Bell, who attended one recently, did a great writeup!
In addition, the moderator for the panel was Katie Carpenter, my collaborator in the CCP's ongoing project to supply evidence-based science communication support to the Southeast Florida Climate Compact.
Just the day before, the Festival had awarded Katie's documentary Battle for the Elephants the prize for "Best Environmental & Conservation Science Program"! So of course she did a great job directing our panel while also contributing her own insights to the discussion.
Read Bell's excellent writeup if you want a blow-by-blow.
The only thing that it occurs to me to add concerns the relationship between Carl Zimmer's talk & mine. There was a bit of point-counterpoint to it.
My basic message -- surprise surprise-- was that science filmmakers & journalists could benefit by using empirical methods to refine and extend their craft norms (slides here).
The gist of Carl's talk, to paraphrase Stanley Fish, was that a science journalist doesn't use a science of science of communication; he or she is a science of science communication. By training and experience, science journalists acquire a form of professional judgment -- one not amenable to quantitative specification-- distinctively suited to discerning how to make what's known by science accessible to curious members of the public.
Or at least I agree with him that no set of methods, empirical or otherwise, can be a substitute for the facility Carl described, which for the most part operates tacitly and automatically as science journalists do what they do.
But the situation sense of science filmmakers, like that of other professionals, is neither static nor impervious to the conscious reflection of those who exercise it.
On the contrary, professional judgment evolves through the interactions of a profession's members, as they accumulate, observe, and share their experiences--formally via training, informally by conversation, and semi-formally through cool events like the Jackson Hole Film Festival!
My claim would be that that process would be enriched by access to empirical information generated with the input of science journalists and filmmakers for the specific purpose of addressing important questions of craft that they themselves recognize as admitting of multiple, competing plausible answers.
The studies would be unlikely to definitely resolve such issues.
But the results would give those participating in professional exchange an additional source of evidence they all agreed was relevant. Such studies could also be expected to spark insight in individual science journalists and filmmakers, whose use of the study results to inform their actions would thereafter furnish even more material for collective assessment.
So for sure empirical methods are no substitute for professional judgment. But they can supply professionals with information that they themselves will value for the contribution it makes to the exercise of their professional judgment and to the accumulation of shared experiences through which such judgment is formed and transmitted.
Or at least that is my hypothesis! I'd bet (say, $10,000) that even Carl would agree it's worth testing.
& if he and other professional science journalists or filmmakers decided to try such an experiment, I and other scientists of science communication would be honored to help them design and carry out studies in service of their continuing mission to perfect their craft.
At West Point yesterday & today, where I'm giving talks & today co-teaching a criminal law class.
The military, it seems to me, is an institution that is ruthlessly self-evaluative & remarkably unambivalent -- to point of lacking any self-consciousness of the attitude it has adopted -- about use of empirical methods of self-assessment.
The questions & discussions are great & there are tons of really smart people here thinking about how to teach critical thinking & cultivate professional judgment.
The educational enviornment here is, I think, a token of how successfully the US military has adapted its practices and outlooks to the political culture of the Liberal Republic of Science.
I'm not an historian, of course, but it does seem to me that unpardonable damage has been done to our military by a civilian leadership that lacked these very commitments to empirical self-evaluation & liberal principles of self-government.
Some lecture slides:
If above photo is nature of "before," here is "after" -- a picture on 2d day of me & some of the cadets & also Frank Wattenberg, a faculty member in the math dept, which sponsored my visit. Obviously I'm now fully assimilated.
Plus an audience member shook my hand, after which I found this in mine:
After trying unsuccessfully to use it at the commisary to buy a gatoraid, I learned that in fact it isn't currency but a "challenge coin," different versions of which are awarded by one or another unit to recognize special achieve by an individual service member or, as in my case, presented by one or another division of the military to a visitor as a token of gratitude or honor. I couldn't have received a more valueable "tip"!
Chris Mooney has written an interesting story in the Washington Post about the SPBMC paper on climate-science literacy and cultural cognition of global warming. There is also interesting discussion appearing in the comments section after my own post on the paper--including an important and informative response by S[evenson].
So today I'd rather see what others think about SPBMC & my response to it (including whether I'm missing something; wouldn't be first time!) than divert anyone to a new topic (like whether ebola dog should be released from quarantine etc).
Unlike our myriad competitors, the CCP blog now & again gets genuine experts to come in & address complicated stuff that these commentators actually know something about. We've been criticized for this, but sometimes I'm too busy to write myself & have no choice. Anyway, the following is an expert guest post from a commentator making his second "guest" appearance. Kevin Arceneaux's last essay, Partisan Media Are Not Destroying America (while subsequently disproven by events), was the most popular post ever on this blog, being read by an estimated 19.3 billion readers. Now he's back to address related issues on study design and causal inference in assessments of the impact of partisan news coverage on public opinion. Arceneaux is the author, with Martin Johnson, of the acclaimed Changing Minds, Changing Channels (Univ. Chicago Press 2013).
News and entertainment media have the dubious distinction of serving as both a whipping boy and a potential savior. They are often treated as the source of many social ills. Beauty magazines perpetuate unhealthy body images; political advertisements inveigle; partisan news programs mislead and confuse (especially if we happen to disagree with them). We also imagine that their power can be put to good use. Media can serve as a catalyst for positive change, however defined.
As seductive as these narratives are, the problem is that they are difficult to evaluate empirically. How could this be? In modern advanced democracies, like the United States, we are surrounded by media. Traditional forms of mass media – newspapers, magazines, radio, television – operate along side newer forms of interactive media on the Internet. Can’t we just observe how people respond to all these forms of news media?
We can certainly observe what people consume and what they do, but we can’t always infer the effects of media consumption on their behavior. Observational research is inherently beset by many threats to causal inference, and the current media environment only makes it worse. The study of media effects could easily be the poster child for the dictum that correlation does not necessarily imply causation.
The biggest hurdle to divining the effects of media from observational research is the fact that people, by and large, choose what to consume. For instance, we know that conservatives say that they consume conservative media at higher rates than other Americans. But because conservatives are consciously choosing to view conservative media and construct conservative networks on social media, it is difficult to sort out how much of their conservatism come from their personal predispositions and how much of it comes from the messages that they encounter.
To muddy the waters further, the ability to select among news and entertainment alternatives creates incentives for media producers to fashion content that will appeal to particular segments of the population. To take a current day example, Fox News has received its fair share of criticism for how it has covered the threats posed by the Islamic State of Iraq and Syria (ISIS) and the Ebola epidemic. It is easy to accuse Fox News and other media outlets of whipping up hysteria, but we must also entertain the possibility that they are just giving their viewers what they think they want. People who are chronically worried about threats need a place to turn to for answers and outlets like Fox News are happy to oblige.
From the standpoint of causal inference, it is difficult to pinpoint the effects of Fox News, because people who aren’t predisposed to be worried about Ebola are happily consuming different media content and if, for some reason, they happened across Fox News coverage of the Ebola epidemic, they may find in more amusing than worrying.
The problem here is so bad that statisticians refer to it as the Fundamental Problem of Causal Inference. In a nutshell, the only way we can really now the effect of media content to observe two states of the world: one where the person consumes it and one where the person does not. Of course, that’s impossible. The only way forward for intrepid researchers is to figure out how to construct a comparable group of people. For example, people who are just like the ones who watch Fox News but who do not. That is easier said than done.
Fancy statistical models that try to address the problem by accounting for people’s viewing preferences (i.e., “control variables”) can actually cause more harm than good. At the very least, this approach rests on the strong assumption that one has accounted for everything, and we can never know if we have.
Another approach that fares a little better is observing the same people overtime. In doing so, we can get a before and after take on their behavior. Yet this approach also makes strong assumptions, too, and as Tobias Konitzer points out in a recent conference paper even if we can make those assumptions, we need lots of observations across time. Panel surveys are rare and long-running panel surveys, even rarer.
Many scholars, including myself, have pointed to randomized experiments as a way forward. Experiments use random assignment to construct comparable groups of individuals. Some people are exposed to media content while others are not. Because people were assigned to groups at random, we know that they should have similar tastes and similar responses. So, if we see one group behaving differently than another, we can more credibly infer that the difference was caused by the treatment that we administered.
While randomized experiments do allow us to say with more confidence that exposure to, say, partisan news content causes people to do X, Y, or Z, it is also not a panacea. For one, experimentalists generally construct comparable groups and then ask people to do things that they would not always do or expose them to things that they may not have encountered but for the intervention of the researcher. Consequently, we cannot be certain that they would not behave differently if the treatment had unfolded through natural means. Field experiments and natural “experiments” (i.e., observational designs that have plausibly exogenous treatments) do better on this score, but they are often difficult to employ.
Another limitation is that experiments are not particularly good at measuring the cumulative effect of media exposure, but rather at pinpointing the effect of a particular intervention. So, the upshot here should be familiar: nothing is perfect and there is no silver bullet. It may be trite, but it is true. We learn the most through the triangulation of methods.
Unconfounding knowledge from cultural identity--as big a challenge for measuring the climate-science literacy of middle schoolers as grown ups
A friend (of the best sort—one who has “got your back” to protect you from entropy’s diabolical plan to deprive you of the benefits of advances in collective knowledge) sent me a very interesting new study:
Stevenson, K. T., Peterson, M. N., Bondell, H. D., Moore, S. E., & Carrier, S. J., Overcoming skepticism with education: interacting influences of worldview and climate change knowledge on perceived climate change risk among adolescents, Climatic Change, 126(3-4), 293-304 (2014).
I very much like the SPBMC paper.
One cool thing about it is that it tests the influence of cultural predispositions on the global-warming beliefs of middle schoolers. It’s not the only study that has adapted the cultural cognition worldview measures to students, but it’s one of only a few and the only one I know of that is applying the measures to kids this young.
Consistent with research involving adult subjects, SPBMC find that cultural outlooks—in particular “individualism”—predicts skepticism about climate change.
SPBMC decided not to use (or at least not to report results involving) the hierarchy-egalitarianism worldview measure (maybe they figured some of the items weren’t suited for minors; I could understand that).
Instead they used a “social dominance” one and found that it didn’t predict anything relating to climate change attitudes—also interesting.
But of course the most important & interesting thing is what SPBMC have to say about the relationship between climate-literacy & acceptance/belief in human-caused global warming, & the influence of cultural individualism on the same.
I found this part of the paper extremely valuable & informative. I have a strong feeling that they have mined only a portion of the rich deposits of knowledge that their data contain.
Nevertheless, I found myself unconvinced (at least at this point) that the results they reported had the significance that they attached to them.
SPBMC present two principal findings. One is that acceptance of human-caused climate change in their student sample was associated with higher climate-science literacy.
The other is that climate-science literacy had a bigger impact on kids who were relatively individualistic. That is, as those kids display higher levels of climate science literacy, the change in the probability that they will believe in human-caused climate change increases even more than it does in kids who are relatively “communitarian” as their science-literacy levels increase.
SPBMC infer from these findings that “[c]climate literacy efforts designed for adolescents may represent a critical strategy to overcoming climate change related challenges, given stable or declining concern among adults that is driven in part by entrenched worldviews.”
For adults, worldviews are well entrenched and exert considerable influence over climate change risk perception. During the teenage years, however, worldviews are still forming, and this plasticity may explain why climate change knowledge overcomes skepticism among individualist adolescents . . . .
I myself strongly agree with SPBMC that climate-science education can make a big contribution to overcoming cultural polarization on climate change—although for reasons that I think differ from those of SPBMC. But put that aside for a second.
The problem, in my view, is that the measure of climate-science literacy that SPBMC constructed fails to address what existing research teaches us is the biggest challenge in measuring public understanding of climate science.
That challenge is how to unconfound or disentangle genuine knowledge from the positions people take by virtue of their cultural identity. An assessment instrument must overcome this challenge in order to be a valid measure of climate-science literacy.
In general, people’s perceptions of risk reflect affective appraisals—positive or negative—of the putative risk source (nuclear power, guns, vaccines, etc.).
For most people most of the time, these feelings don’t reflect their comprehension of scientific data or the like. On the contrary, how people feel is more likely to shape their assessments of all manner of information, which they can be expected to conform to their pro- or con-attitude toward the putative risk source.
In this circumstance, survey items that elicit people’s understandings of the risks and benfits associated with some activity or state of affairs are best understood as simply indicators of the unobserved or latent affective orientation that people have toward that activity or state of affairs. That attitude is all they are genuinely measuring (Loewenstein et al. 2001; Slovic et al. 2004).
This is a huge issue for measuring climate-science literacy.
Sadly, propositions of fact on climate change—like whether it is happening & whether humans are causing it—have become entangled in antagonistic cultural meanings, transforming them into badges of membership & loyalty to affinity groups of immense significance in people’s everyday lives.
Study respondents can thus be expected to answer questions relating to climate change in a manner that reflects the pro- or con- affective stance that corresponds to their cultural identities.
If they are the sort of persons who are culturally predisposed to believe in human-caused global warming (or “accept” it; let’s be sure to avoid the confused & confusing idea that there’s an important distinction between “believing” something & “accepting” or “knowing” it), they will affirm pretty much any proposition that to them sounds like the sort of thing one who “believes in” climate change would say.
As a result, they’ll incorrectly agree that human-caused global warming will increase the incidence of skin cancer, that industrial sulfur pollutions are causing climate change, that water vapor traps more heat than any other greenhouse gas etc.
Their “acceptance” of human-caused global warming, in other words, doesn’t reflect knowledge of the basic mechanisms that drive climate change or of the scientific evidence for how they work.
Study after study after study after study has demonstrated this (Bostrom et al 1994 Reynolds et al. 2010; Tobler, Visschers & Siegrist 2012; Guy et al. 2014).
To be valid, then, a climate-science literacy scale must successfully distinguish between respondents whose correct answers reflect only their identity-based affective orientation toward global warming from those whose correct answers show genuine climate-science comprehension.
The only way to design such a scale is to include a sufficiently large number of appropriately weighted items for which the incorrect answers are likely to seem correct to someone who is culturally predisposed to believe in climate change but who lacks understanding of the scientific basis for that position.
Now here’s the most interesting thing: if one includes a mix of items that successfully distinguishes those who “accept” human-caused climate change based on their predispositions from those who genuinely get the mechanisms of climate change, then one will discover that those who don’t “accept” or “believe in” human-caused climate change know just as much about the mechanisms of climate change as those who say they do accept it.
For sure, most “skeptics” are painfully ignorant about climate change science.
But that’s true for most “believers” too!
Only a very small portion of the general public—consisting of individuals who score very high on a general science comprehension test—can consistently distinguish propositions that most expert climate scientists accept from propositions that sound like ones such experts might accept but that in fact are wholly out of keeping with the basic mechanisms and dynamics of global warming.
Yet even among these very climate-science literate members of the public, there is no consensus on whether global warming is occurring. Just like their climate-science illiterate counterparts, their “beliefs” about human-caused global warming are predicted by their cultural identities (Kahan in press).
In sum, “acceptance” or “belief in” human caused global warming is not a valid indicator in them either. It is an indicator of who one is, culturally speaking—nothing more and nothing less.
Judging from the results they reported in their paper, at least, SPBMC did not construct a climate-science literacy measure geared to avoiding the “identity-knowledge” confound.
In fact, they actually selected from a larger battery of items (Tobler, Visschers & Siegrist 2012) a subset skewed toward ones that a test-taker who is culturally predisposed to “believe in” human-caused global warming could be expected to answer correctly regardless of how much or little that person actually knows about the mechanisms of climate change (e.g., “For the next few decades, the majority of climate scientists expect a warmer climate to increase the melting of polar ice, which will lead to an overall rise of the sea level ”; “... an increase in extreme events, such as droughts, floods, and storms”; “... a cooling down of the climate”; “The decade from 2000 to 2009 was warmer than any other decade since 1850.”).
SPBMC left out of their battery items that Tobler et al. (2012) and other studies have found believers in climate change are highly likely to get wrong (e.g., “For the next few decades, the majority of climate scientists expect an increasing amount of CO2 risks will cause more UV radiation and therefore a larger risk for skin cancer”; “Water vapor is a greenhouse gas”; “In a nuclear power plant, CO2 is emitted during the electricity production”; “On short-haul flights (e.g., within Europe) the average CO2 emission per person and kilometer is lower than on long-haul flights (e.g., Europe to America).”)
By my count, only 3 of the 17-19 items SPBMC identify as ones included in their scale (there is a discrepancy in the number that they report using in the text and number that appear in the on-line supplementary information, where the item-wording appears) are ones that existing studies have shown were likely to elicit wrong answers from low climate-science comprehending respondents who are nonetheless culturally predisposed to believe in climate change (“the ozone hole is the main cause of the greenhouse effect [true-false]”; “For the next few decades, the majority of climate scientists expect a precipitation increase in every region worldwide”; “Carbon dioxide (CO2) is harmful to plants”).
If one constructs a “climate science literacy” scale like this, it is bound to correlate with “acceptance” of global warming because the scale will itself be measuring the same cultural predisposition that inclines people to accept human-caused global warming.
Indeed, included in the SPBMC scale were true-false items that measured acceptance of human-caused climate change:
- The increase of greenhouse gasses is mainly caused by human activities.
- With a high probability, the increase of carbon dioxide (CO2) is the main cause of climate change.
- Climate change is mainly caused by natural variations (such as changes in solar radiation and volcanic eruptions).
Obviously, if one is testing the hypothesis that acceptance/belief in human-caused global warming is caused by understanding of climate science, then the former must be defined independently of the latter.
Because SPBMC put "acceptance" items in their climate literacy scale, their finding that global-warming acceptance is associated with climate-science literacy is circular.
The same problem, in my view, characterizes SPBMC’s finding on the relative impact of climate-science literacy on students who are relatively individualistic.
Again, SPBMC’s climate-science measure is itself measuring acceptance of human-caused climate change.
So for them to say (based on a correlational model) that “climate science literacy” has a bigger impact on individualists' willingness to “accept climate change” than it does on communitarians’ is equivalent (mathematically/logically) to saying: “Reducing climate-skepticism in cultural individualists who don't believe in climate change would have a bigger impact on their willingness to accept human-caused climate change than would reducing the skepticism of cultural communitarians who already believe in climate change. . . .”
Can’t argue with that—but only because it’s essentially a tautology.
The practical question has always been why individualists are so strongly predisposed to skepticism (and communitarians to belief—same thing).
There is already evidence that the cultural individualists who score highest on a valid climate-science literacy scale are not more likely than low-scoring cultural individualists to say they accept/believe in human-caused global warming.
Because it is unclear that SPBMC constructed a scale that measures knowledge & not just a pro-belief affective orientation—indeed, because their climate-science comprehension scale includes acceptance of human-caused climate change—it doesn’t support any inference that greater climate-science comprehension would have such an effect in culturally individualistic middle schoolers.
As I mentioned, I do believe that improving climate-science education would make a very big contribution to dissipating political polarization on global warming.
The reason isn’t that understanding climate-science in itself can be expected to induce people to say they “believe in” climate change. Again, what people say about what they believe in climate change isn’t a measure of what they know; it is a measure of why they are.
But precisely for that reason, learning to teach kids climate science will require teachers to learn how to dispel from the classroom the toxic affiliation between climate change positions and identities that now divides adults in the political realm. When teachers learn how to do that—as I’m confident they will—then we can apply those lessons more broadly to the political domain so that there too we can use what we know rather than fight over whose side the state is going to take in a mean, illiberal status competition.
Indeed, that SPBMC performed a study like this in an educational context fills me with deep admiration. This is the sort of research we desperately need more of, in my view.
And notwithstanding the critique I’m offering, I’m convinced there is a lot that can be learned from this paper.
In particular, I really really hope SPBMC will report more of their data—including the psychometric properties of their climate-science literacy scale and summary data on how scores actually are distributed in their sample.
They’d certainly be welcome to do so in this blog!
Still, as a scholar grappling with the central psychometric issues involved in measuring climate science literacy, I just don’t think the particular results SPBMC have reported support the conclusions that they purport to draw.
I’m sure they’d agree with me, too, that scholars investigating these issues are obliged to speak up when they see a study that they think hasn’t fully addressed them. If scholars don't do this out of some misplaced sense of politeness (or any other sensibility, for that matter, that constrains open and candid scholarly exchange), then science communicators and educators who are relying on empirical work to make informed judgments will end up making serious and costly errors.
It should also go without saying that it is a mistake to think peer review happens only before a paper is published. If anything, that’s precisely when meaningful peer review begins.
Bostrom, A., Morgan, M. G., Fischhoff, B., & Read, D. (1994). What Do People Know About Global Climate Change? 1. Mental Models. Risk Analysis, 14(6), 959-970. doi: 10.1111/j.1539-6924.1994.tb00065.x
Guy, S., Kashima, Y., Walker, I., & O'Neill, S. (2014). Investigating the effects of knowledge and ideology on climate change beliefs. European Journal of Social Psychology, 44(5), 421-429.
Loewenstein, G.F., Weber, E.U., Hsee, C.K. & Welch, N. Risk as Feelings. Psychological Bulletin 127, 267-287 (2001).
Reynolds, T. W., Bostrom, A., Read, D., & Morgan, M. G. (2010). Now What Do People Know About Global Climate Change? Survey Studies of Educated Laypeople. Risk Analysis, 30(10), 1520-1538. doi: 10.1111/j.1539-6924.2010.01448.x
Slovic, P., Finucane, M.L., Peters, E. & MacGregor, D.G. Risk as Analysis and Risk as Feelings: Some Thoughts About Affect, Reason, Risk, and Rationality. Risk Analysis 24, 311-322 (2004).
Stevenson, K. T., Peterson, M. N., Bondell, H. D., Moore, S. E., & Carrier, S. J. (2014). Overcoming skepticism with education: interacting influences of worldview and climate change knowledge on perceived climate change risk among adolescents. Climatic Change, 126(3-4), 293-304.
Tobler, C., Visschers, V. H. M., & Siegrist, M. (2012). Addressing climate change: Determinants of consumers' willingness to act and to support policy measures. Journal of Environmental Psychology, 32(3), 197-207. doi: http://dx.doi.org/10.1016/j.jenvp.2012.02.001
B/c this post is linked in an insightful Wonkblog writeup on the SPBMC paper, & b/c "S" has now written a very informative response to my post in the comments section, it occurs to me that I should be clear about my own attitude toward the study.
I think I would bet against the claim that SPBMC are making.
But I'd be hedged: if I lost the bet, the gain in knowledge would more than compensate me for whatever amount I lost on the bet!
My current best understanding is that for grownup Americans belief in or acceptance of global warming has nothing to do with comprehension of either science generally or climate science in particular. It is simply an expression of cultural identity.
I wouldn't be surprised to learn that in middle schoolers there is no or only a very weak relationship between belief in human-caused global warming & cultural outlooks. It's plausible to me that these sorts of connections get worked out in people's minds at a later age.
But SPBMC report that cultural individualism is correlated with "disbelieving" human-caused global warming. Also very believable (& sad).
Yet they also report that this effect abates as the level of climate-science knowledge in middle schoolers increases. That's the part I find surprising. It's not what one sees in adults.
& it's puzzling, at least to me: if middle school kids are already "mature" enough to start to figure out what sorts of "positions" on global warming "go with" their cultural identities -- are already socialized enough to start to form stances that protect their status within groups important to their well being-- then I'd expect their positions to be unaffected by their science comprehension, b/c the sort of symbolic stance-taking involved in climate change, evolution & other culturally contested "science" issues in fact has nothing to do w/ what people (grownups at least) know about science.
But what SPBMC report finding could well be true! At which point, I'd certainly like to know what the best explanation is for such an important result.
But before accepting an explanation for it, I'd like to be sure (or be assured) that it is true that the impact of cultural individualism on middle schoolers' acceptance of human-caused climate change abates as their climate science comprehension increases.
To be assured, I'd need more information about the performance of the scale & the relationship of its items to the study subjects' cultural outlooks. I want to see enough to be sure the scale is valid & doesn't involve the identity-knowledge confound or knowledge-acceptance circularity problems I advert to.
SPBMC have whetted my appetite; I am hungry for even more insight more from their study!
This teeny weeny paper is for a special issue of the journal Cognition. The little diagrams illustrating how one or another cognitive dynamic can be understood in relation to a simple Bayesian information-processing model are best part, I think; I am almost as obsessed with constructing these as I am with generating the multi-colored "Industrial Strength Risk Perception Measure" scatterplots.
I was corresponding with friend, someone who has done really great science education research, about the related challenges of teaching evolution & climate science to high school students.
Defending what I've called the "disentanglement principle"-- the obligation of those who are responsible for promoting comprehension of science to create an environment in which free, reasoning people don’t have to choose between knowing what’s known and being who they are-- I stated that I viewed "the whole concept of 'believing' [as] so absurd . . . ."
He smartly challenged me on this:
I must admit, however, that I do not find the concept of believing to be absurd. I for example, believe that I have been married to the same women since I was XX years old. I also believe that I have XX children. I also believe that the best theory to explain modern day species diversity is Darwin's evolution theory. I do not believe the alternative theory called creationism. Lastly, I believe that the Earth is warming due largely to human caused CO2 emissions. These beliefs are the product of my experience and a careful consideration of the alternatives, their predictions, and a comparison of those prediction and the evidence. This is not a matter of who I am ( for example it matters not whether I am a man or a women, straight or gay, black or white) as much as it is a matter of my understanding of how one comes to a belief in a rational way, and my willingness to not make up my mind, not to form a belief, until all steps of that rational way have been completed to the extent that no reasonable doubt remains regarding the validity of the alternative explanations that have been advanced.
His response made me realize that I've been doing a poor job in recent attempts to explain why it seems to me that "belief in" evolution & global warming is the wrong focus for imparting and assessing knowledge of those subjects.
I don't think the following reply completely fixes the problem, but here is what I wrote back:
I believe you are right!
In fact, I generally believe it is very confused and confusing for people to say "X is not a matter of belief; it's a fact ....," something that for some reason seems to strike people as an important point to make in debates about politically controversial matters of science.
Scientists "believe" things based on evidence, as you say, and presumably view "facts" as merely propositions that happen to be worthy of belief at the moment based on the best available evidence.
I expressed myself imprecisely, although it might be the case that even when I clarify you'll disagree. That would be interesting to me & certainly something I'd want to hear and reflect on.
What I meant to refer to as "absurd" was the position that treats as an object of science education students' affirmation of "belief in" a fact that has been transformed by cultural status competition into nothing more than an emblem of affiliation.
That's so in the case of affirmation of "belief in" evolution. To my surprise, actually, I am close to concluding that exactly the same is true at this point of affirmation of "belief in" global warming.
Saying one "disbelieves" those things, in contrast, is an indicator (not a perfect one, of course) of having a certain cultural identity or style-- one that turns out to be unconnected to a person's capacity to learn anything.
So those who say that one can gauge anything about the quality of science instruction in the US from the %'s of people who say that they "believe in" evolution or climate change is, in my view, seriously mistaken.
Or so I believe--very strongly-- based on my current assessment of the best evidence, which includes [a set of extremely important studies] of the the effective teaching of evolution to kids who "don't believe" it. I'd be hard pressed to identify a book or an article much less a paragraph that conveyed as much to me about the communication of scientific knowledge as this one:
[E]very teacher who has addressed the issue of special creation and evolution in the classroom already knows that highly religious students are not likely to change their belief in special creation as a consequence of relative brief lessons on evolution. Our suggestion is that it is best not to try to [change students’ beliefs], not directly at least. Rather, our experience and results suggest to us that a more prudent plan would be to utilize instruction time, much as we did, to explore the alternatives, their predicted consequences, and the evidence in a hypothetico-deductive way in an effort to provoke argumentation and the use of reflective thought. Thus, the primary aims of the lesson should not be to convince students of one belief or another, but, instead, to help students (a) gain a better understanding of how scientists compare alternative hypotheses, their predicated consequences, and the evidence to arrive at belief and (b) acquire skill in the use of this important reasoning pattern—a pattern that appears to be necessary for independent learning and critical thought.
Maybe you now have a better sense of what I meant to call "absurd," but now it occurs to me too that "absurd" really doesn't capture the sentiment I meant to express.
It makes me sad to think that some curious student might not get the benefit of knowing what is known to science about the natural history of our (and other) species because his or her teacher made the understandable mistake of tying that benefit to a gesture the only meaning of which for that student in that setting would be a renunciation of his or her identity.
It makes me angry to think that some curious person might be denied the benefit of knowing what's known by science precisely because an "educator" or "science communicator" who does recognize that affirmation of "belief in" evolution signifies identity & not knowledge nevertheless feels that he or she is entitled to exactract this gesture of self-denigration as an appropriate fee for assisting someone else to learn.
Such a stance is itself a form of sectarianism that is both illiberal and inimical to dissemination of scientific knowledge.
I have seen that there are teachers who know the importance of disentangling the opportunity to learn from the the necessity to choose sides in a mean cultural status struggle, but who don't know how to do that yet for climate science education. They want to figure out how to do it; and they of course know that the way to figure it out is to resort to the very forms of disciplined observation, measurement, and inference that are the signatures of science.
I know they will succeed. And I hope other science communication professionals will pay attention and learn something from them.
Weekend update: New paper on why "affirmative consent" (in addition to being old news) does not mean "only 'yes' means yes"
As I explained in a recent post, the media/blogosphere shit storm over the "affirmative consent" standard Calif just mandated for campus behavioral codes displays massive unfamiliarity with existing law & with tons of evidence on how law & norms interact.
First, the "affirmative consent" standard isn't a radical "redefinition" of the offense of rape. It's been around for three decades.
Second, contrary to what the stock characters who are today reprising the roles from the 1990s "sexual correctness" debate are saying, an "affirmative consent" standard certainly doesn't require a verbal "yes" to sexual intercourse. It simply requires communication of consent by acts or words.
Third, for exactly that reason it hasn't changed outcomes in cases in which decision makers--jurors, judges, university disciplinary board members, etc. -- assess date rape cases.
Because members (male & female) of certain cultural subcommunities subscribe to norms in which a woman can "consent" to sex despite saying "no," decisionmakers who interpret facts against the background of those norms will sill treat various forms of behavior -- including suggestive dress, consensual sexual behavior short of intercourse, etc.-- as "communicating" that a woman who says "no" really meant yes.
When those individuals apply the "affirmative consent" standard, they reach the same result that they would have reached under the traditional common-law definition -- or indeed that they would have reached if they were furnished no definition of rape at all.
Today I happened to come across an intersting new paper that presents a review of the literature on these dynamics & that adds a relevant analysis on how cultural norms influence testimony of the parties.
In Honest False Testimony in Allegations of Sexual Offenses, J. Villalobos, Deborah Davis, & Richard Leo explain why the same norms that influence decisionmakers' perceptions of "consent" in date rape cases--including ones in which a woman says no--are likely to shape the perceptions of the parties, whose conflicting "honest" testimony will create doubt on the part of decisionmakers. This dynamic, they conclude, helps explain why "cultural predispositions often outweigh legal definitions of sexual consent when individuals make assessments of whether consent has been granted."
People genuinely interested in this issue might want to read it.
Those playing the stock characters in the media remake of the 1990s (and earlier) reform debate probably won't-- if they had any interest in what the law actually is and how social norms have constrained enforcement of reform formulations of rape, they'd have already been familiar with much of this literature & would have recognized that their positions are actually divorced from reality.
Again, changing behavior on campuses requires changing norms. Moreover, rather than being an effective instrument for norm change, legal reforms--including affirmative consent standards-- have in the past been rendered impotent b/c of the impact that norms have in shaping decisionmakers' understanding of what those standards mean.
This is a hard issue.
Maybe a reform like Estrich's "no means no" standard-- an irrebuttable presumption that uttering "no" constitutes lack of consent-- would actually change results by blocking decisionmakers' reliance on contrary social norms. There's some experimental evidence that this is so.
Or maybe (as some argue) it would produce a backlash that would further entrench existing norms.
Accordingly, maybe the emphasis should be on trying to promote forms of behavior that, through one or another mechanism of social influence, will change norms on campuses.
That sort of thinking is likely the motivation for the Obama Administration's new "It's on Us" social marketing campaign.
But one thing is clear: nothing will change if people ignore evidence -- on what the law is, on social norms, and on what real-world experience shows about how the two interact -- and instead opt to engage this issue through platitudinous claims the only function of which is to signify whose "team" people are on in a culture conflict only tangentially connected to the problem at hand.
ooops-- loyal listener @Gaytia alerted me that the linke to the Villalobos, Davis & Leo paper was wrong (embarrassingly, was linke to one of my papers-- argh!).
Download their cool paper Honest False Testimony in Allegations of Sexual Offenses now, before it sells out!!!
What happens to Pat's perceptions of climate change risks as his/her science literacy score goes up?
A curious and thoughtful correspondent asks:
A while ago, I had read your chart with two lines in red and blue, showing the association between scientific literacy and opinion on climate change separately for liberals and conservatives. [A colleague] gave it favorable mention again in her excellent presentation at the * * * seminar today.The subsequent conversation reminded me that I had always wanted to see in addition the simple line chart showing the association between scientific literacy and opinion on climate change for all respondents (without breakdown for liberals and conservatives). Have you ever published or shared that? Please share chart, or, if you haven't ever run that one, please share the data?Much thanks!
The line that plots the relationship for the sample as a whole will be exactly in between the other 2 lines. The "right/left" measure is a composite Likert scale formed by summing the (standardized) responses to 5-point left-right ideology & 7-point party-identification measure. In the figures you are referring to, the relationship between science literacy and partisan identity is plotted separately for subjects based on their score in relation to the mean on that scale.
I've added a line plotting "sample mean" relationship between global warming risk perceptions (measured on the "Industrial Strength Risk Perception Measure") to figures for two data sets, one in which subjects' science comprehension was measured with "Ordinary Science Intelligence 1.0" (used in the CCP Nature Climate Change study) & other in which the same was measured with OSI_2.0.
I'm sure you can see the significance (practical, as well as "statistical") of this display for the question you proposed, viz., "What's the impact of science literacy in general, for the population as a whole, controlling for partisanship, etc?"
It's that the question has no meaningful answer.
The main effect is just a simple average of the opposing effects that science comprehension has on climate change risk perceptions (beliefs, etc) conditional on one's cultural identity (for which right-left political outlooks are only 1 measure of many).
If the effect is "positive" or "negative," that just tells you something about the distribution of cultural-affinities, the relative impact of such affinities on risk perceptions, &/or differences in the correlation between science comprehension and cultural outlooks (which turn out to be trivially small, too) in that particular sample.
Maybe this scatterplot can get this point across visually:
In sum, because science comprehension interacts with cultural identity and b/c everyone identifies more or less with one or another cultural group, talking about the "main" effect is not a meaningful thing to do. All one can say is, "the effect of science comprehension on perceptions of climate change risk depends on who one is."
Or put it this way: the question, "What's the effect of science comprehension in general, for the population as a whole?" amounts to asking what happens to Pat as he/she becomes more science comprehending. Pssssst . . . Pat doesn’t exist!
Again, I'm sure you get this now that you've seen the data, but it's quite remarkable how many people don't. How many want to seize on the (trivially small) "main effect" & if it happens to be sloped toward their group's position, say "See! Smart people agree with our group! Ha ha! Nah, nah, boo, boo!"
They end up looking stupid.
Not just because anyone who thinks about this can figure out what I've explained about the meaninglessness of "main effect" when the data display this relationship.
But also because when we see his relationship and when the "main effect" is this small, that effect is likely to shift direction the next time someone collects data, something that could happen for any of myriad non-consequential reasons (proportion of cultural types in the sample, random variation in the size of the interaction effect, slight modifications in the measure of science literacy). At that point, those who proclaimed themselves the "winners" of the last round of the "whose is bigger" game look like fools (they are, aren't they?).
But how about some more information about Pat? And about his/her cultural worldview & ideology & their effect on his/her beliefs about climate change? Why not-- we all love Pat!