follow CCP

Recent blog entries
popular papers

Science Curiosity and Political Information Processing

What Is the "Science of Science Communication"?

Climate-Science Communication and the Measurement Problem

Ideology, Motivated Cognition, and Cognitive Reflection: An Experimental Study

'Ideology' or 'Situation Sense'? An Experimental Investigation of Motivated Reasoning and Professional Judgment

A Risky Science Communication Environment for Vaccines

Motivated Numeracy and Enlightened Self-Government

Ideology, Motivated Cognition, and Cognitive Reflection: An Experimental Study

Making Climate Science Communication Evidence-based—All the Way Down 

Neutral Principles, Motivated Cognition, and Some Problems for Constitutional Law 

Cultural Cognition of Scientific Consensus
 

The Tragedy of the Risk-Perception Commons: Science Literacy and Climate Change

"They Saw a Protest": Cognitive Illiberalism and the Speech-Conduct Distinction 

Geoengineering and the Science Communication Environment: a Cross-Cultural Experiment

Fixing the Communications Failure

Why We Are Poles Apart on Climate Change

The Cognitively Illiberal State 

Who Fears the HPV Vaccine, Who Doesn't, and Why? An Experimental Study

Cultural Cognition of the Risks and Benefits of Nanotechnology

Whose Eyes Are You Going to Believe? An Empirical Examination of Scott v. Harris

Cultural Cognition and Public Policy

Culture, Cognition, and Consent: Who Perceives What, and Why, in "Acquaintance Rape" Cases

Culture and Identity-Protective Cognition: Explaining the White Male Effect

Fear of Democracy: A Cultural Evaluation of Sunstein on Risk

Cultural Cognition as a Conception of the Cultural Theory of Risk

« Don't select on the dependent variable in studying the science communication problem | Main | Evidence based science communication ... a fragment »
Monday
Nov182013

WSMD? JA! How different are tea party members' views from those of other Republicans on climate change?

This is either the 53rd or 734th--it's hard to keep track!--episode in the insanely popular CCP series, "Wanna see more data? Just ask!," the game in which commentators compete for world-wide recognition and fame by proposing amazingly clever hypotheses that can be tested by re-analyzing data collected in one or another CCP study. For "WSMD?, JA!" rules and conditions (including the mandatory release from defamation claims), click here.

If there weren't already more than enough reason to question my sanity, I've decided to return to my data on tea party members.

Actually, I was moved to poke at them again by a question posed to me by Joe Witte, a well-known Washington, D.C., meteorologist who also does science communication, after a "webinar" talk I gave last Friday for NOAA.

Joe Witte, perversely taking delight after correctly predicting dreary rainy day for 4th of July

Joe asked whether the data I was discussing in my talk on climate change polarization contained anything on tea party and non- tea party Republicans.

This is an interesting question and was explored in a very interesting  report issued by the Pew Center for the People & the Press, definitely one of the top survey research outfits around.  Distinguishing the positions of tea party & non- tea party Republicans, Pew characterized its findings as suggesting that the "GOP is Deeply Divided Over Climate Change" (the title of the report).

I'm conjecturing that Joe was conjecturing that maybe the divisions aren't as meaningful as Pew suggests -- or in any case, Joe's question made me curious about this & so I thought this was close enough to a conjecture on his part to qualify for a "WSMD? JA!" episode.

Why did I suspect that maybe Joe was suspecting that Pew was overstating divisions among Republicans?

Well, basically because I assumed that Joe, like me, would regard identifying with the "tea party" as simply an indicator of a latent ideological or cultural disposition.  Same thing for identifying with the Republican or Democratic Parties, and for characterizing oneself as a "liberal" or a "conservative."  Ditto for responding in particular ways to the items that make up the cultural cognition worldview scales.

The disposition in question likely originates in membership in one or another of the affinity groups that shape -- through one or another psychological mechanism -- perceptions of risk including those of climate change. We'd measure that disposition directly if we could. But since we can't actually see it, we settle for observable correlates of it -- like what people will say in response to survey items that have been validated as indicators of that disposition.

Indeed, the simple statement "I'm a Republican/Democrat" is itself a relatively weak indicator of such a disposition.  Again, self-descriptions of this sort are just observable proxies for a disposition that we can't actually measure directly--and proxies are always noisy. Moreover, dispositions of this sort vary in intensity across persons.  Accordingly, a single binary question such as "are you a Republican or a Democrat?" will elicit a response that measures the disposition in a very crude, wobbly manner.

It's much better to ask multiple questions that are valid indicators of such a disposition (and even better if they themselves permit responses that vary in degree) and then aggregate them into a scale (by just adding them, or by assigning differential weights to them based on some model like factor analysis). Assuming the indicators are valid--that is, that they do indeed correlate with the unobserved disposition--they will reinforce one another's contribution to measuring the disposition and cancel out each other's noise when combined in this way.

I figured that identifying as a Republican and saying "yes" when asked "hey, do you consider yourself part of that tea party movement thing" (I don't think there is an agreed-upon item yet for assessing tp membership) indicates a stronger form of the "same" disposition as as identifying as a "Republican" but saying "no."  

So, yeah, sure, tea party member are more skeptical than non-tea party Republicans--which is about as edifying as saying that "strong" Republicans are more skeptical than "weak" ones (or than individuals who describe themselves as "independents" who "lean" Republican).  Hey, "socialist" members of the Democratic party are probably even more convinced that climate change is happening than non-socialist ones too.

Well, this know-it-all  hypothesis is easily testable!  All one has to do is form a more discerning, continuous measure of the disposition that simply identifying as "Republican"indicates and then see how saying "yes" to the tea party question influences the probability of being skeptical about climate change. 

My "disposition intensity" hypothesis--that saying one belongs to the tea party merely indicates a stronger version of that disposition than identifying as a Republican--implies that belonging to the tea party will have relatively little impact on the degree of climate skepticism of individuals who identify as Republican and who score relatively high on the dispositional scale.  If we see that, we have more reason to believe my hypothesis is correct.

If we see, in contrast, that identifying as a tea party member has an appreciable effect even among those who score relatively high in the disposition scale, then we have reason to doubt my hypothesis and reason to believe some alternative-- such as those who have the disposition that Republican party self-identification indicates are "divided" on climate change (there probably are other hypotheses too, but the likelihood of this one would deserve to be revised upward at least to some extent, I'd say, if my test "fails").

Okay.  One way to form a valid measure of the disposition indicated by saying "Howdy, I'm a Republican!" is to form a scale from their responses to a multi-point item that registers how strongly they identify with the Democrat or Republican party with a multi-point measure of how "liberal" or "conservative" they would say they are.

I did that-- simply adding responses on a 7-point version of the former and a 5-point version of the latter administered to a nationally representative sample of about 2,000 respondents who were added to the CCP subject pool last June.

Actually, I normalized responses to each item -- a procedure that helps to prevent one from having a bigger impact on the scale just because it has a higher mean or a larger degree of variance -- and then normalized the sum so that the units of the scale would itself reflect "standard deviations," which have at least a bit more meaning than some other arbitrary metric.

The resulting measure had a "Cronbach's alpha"--a scale reliability measure that ranges from 0 to 1.0--of 0.87, indicating (unsurprisingly) that the items had the high degree of intercorrelation that treating them as a scale requires.

Because the score on this scale increases as either a respondent's identification with the Republican party or his or her degree of "conservativism" does, it's handy to call the scale "Conserv_repub." It turns out that that someone who identifies as both a "strong Republican" on the 7-point party self-identification scale and as "extremely conservative" on the 5-point "liberal-conservative" ideology item will get about 1.65 on Conserv_Repub, whereas someone who identifies as a "strong Democrat" and as "extremely liberal" will get a score of about -1.65.

Next, I looked at the positions of the study respondents to a standard "do you believe in climate change" item.  It has two parts: first, respondents indicate whether they believe "there is solid evidence that the average temperature on earth has been getting warmer over the past few decades"; second, those who say "yes" are then asked whether they believe that this trend is attributable to "human activity such as burning fossil fuels" or instead "mostly to natural patterns in the earth’s environment."

The Pew survey used this item, and my results and theirs are pretty comparable.

To start, roughly the same proportion of my sample—45%—indicated a belief in human-caused global warming (Pew: 44%).

In addition, the sampe relative proportions of my sample and Pew’s (33% to 22% vs. 26% to 18%, respectively) indicated that they either saw “no solid evidence” go global warming at all or attributed such evidence to “natural patterns” rather than “human activity” (the actual percents varied because only 1% of my sample, as opposed to 7% for Pew, selected “don’t know”).

The partisan divide in my sample--reflected in the Figure above-- was also comparable to Pew's.

Pew found that 64% of “Democrats” (including “independents” who “lean Democrat”; political scientists have found that independent “leaners” are more accurately classified as partisans, if one insists on limiting oneself to categorical measures) but only 23% of “Republicans” believe in human-caused global warming.

Splitting my sample at the mean on Conserv_Repub, I found that 69% of relatively “liberal, Democratic” respondents (ones scoring below the mean) but only 21% of relative “conservative, Republican” ones do.

Like Pew, I also found that tea party Republicans are decidedly more skeptical than non­­­–tea party ones. In my sample, only 5% of tea party members identifying as Republicans indicated belief in human-caused global warming, whereas 28% of non–tp ones did. In Pew’s survey, 9% of tp Republicans and 32% of non–tp ones indicated such belief.

Now, I’m just warming up here! Nothing yet that goes to the validity of my “partisan intensity” hypothesis for the tp/non–tp disparity, but the comparability of the CCP results and Pew’s does suggest that the test I proposed for my conjecture about Pew’s conclusion can be fairly tested with my data.

Not to prolong the excruciating suspense, but I will say one more thing before getting to the test.

In both the CCP data and the Pew survey—not to mention scores of other studies conducted over the last decade, during which time the numbers really haven’t budged—the partisan divide on belief in human-caused climate change is immense.  I’ve heard from professional political pollsters (ones who make their living advising political candidates) that there is no issue at this point—not even abortion or gun control—that polarizes Americans to this extent.

Climate-change advocacy groups & those who perform surveys for them sometimes try to put a smiling face on these numbers by noting that around two-thirds of Americans “believe in climate change.”

But this formulation merges those who “believe” that global warming is caused “mostly” by “natural patterns” with those who attribute global warming to “human activity.”  Consistent with margins reported in dozens and dozens of nationally represtenative studies, the Pew survey and the CCP study both found  that only around 50% (less actually) of the respondents--Democrat or Republican--indicated that they believed that there is “solid evidence” that “burning fossil fuels” is a significant contributor to climate change.

That’s the key issue, one would think, both scientifically and politically.  On the latter, people presumably aren’t going to support a carbon tax or other measures for regulating CO2 emissions if they don't believe human activity is really the source of the problem. (Maybe there will consensus for geoengineering?!)

Being realistic (and one really should be if one wants to get anything accomplished), there’s a long way to go still if one is banking on a groundswell of public support to change U.S. climate policy.

And if one is realistic, one should also try to figure out whether focusing on "public opinion" of the sort measured by polls like these is a meaningful way to make policymaking more responsive to the best available evidence on climate.

I’ve asked many many times and still not heard from those who focus obsessively on responses to survey items a cogent explanation of how “moving the public opinion needle” (is anyone else tired of this simplistic metaphor?) will “advance the ball” from a policymaking perspective.  As the consistent rebuffs to “background checks” for gun purchases and for campaign finance reforms—measures that genuinely enjoy popular opinion poll support—attest, the currency of survey majorities won’t buy one very much in a political economy that features small, well-organized, intensely interested and well-financed interest-group opponents.

Along with many others, I can think of some political strategies that might penetrate the political-economy barrier to science-informed climate policy in the U.S., but none of them involves any of the various kinds of diffuse public “messaging” campaigns that climate advocacy groups have been obsessessed with for over a decade.

But I digress! Back to the issue at hand: is the TP/non–tp divide really evidence that Republicans are split on climate change?

Applying my test—in which tp-Republicans are compared to non–tp ones whose partisan disposition can be shown to be comparably strong by an independent measure—I’d have to say . . . gee, it really does look like the tp-identifying Republicans are a distinctive group!

To begin with, the disposition measured by Conserv_repub does predict being a tea party member but less strongly than I would have guessed.  As can be seen from this figure, even those who score highest on this scale are only about 50% likely to identify with being in the tea party.

Moreover, if one examines the impact on belief in climate change as a function of the strength of the disposition measured by Conserv_repub, one can see that there really is a pretty significant discrepancy between tp-members and non–tp members even as one approaches the highest or strongest levels of the partisan outlook reflected in the Conserv_repub measure. I thought the gap would be narrow and diminishing to nearly nothing as scores reached the upper limit of the scale.

I've used a lowess regression smoother here because I think it makes the size and character of the tp effect readily apparent--and without misleadingly constraining it to appear uniform or linear across the range of Conserv_repub as even a logistic regression might. But for those of you who'd like to see a conventional regression model, and confirm the "statistical significance" of these effects, here you go.

Now one thing that still leaves me a bit unsatisfied is the outcome measure here.  

The standard "do you believe" item is crude.  Like the ideological or cultural disposition that motivates it, the perception of climate change risks is also best viewed as a latent or unobserved attitude or disposition.  Single-item indicators will measure it imperfectly; and ones that are nominal and categorical are less precise, more quirky than ones that try to elicit degree or intensity of the attitude in question.

Ideally, we'd combine this measure with a bunch of others. But I don't have a bunch in my data set. 

I do, however, mave the trusty "industrial grade" risk perception measure.  As I've explained before, this simple "how serious would you say the risk is on a scale of 0 to n" scale has been shown to be an exceptionally discerning measure because of its high degree of correlation with pretty much any and all more specific things that one can ask a survey respondent about climate change.  This makes it a psychometrically attractive single-item measure for assessing variance in climate-change or other risk perceptions.

Here's what we see when we use it to assess the difference between tp & non–tp members:

Well, the gap between tp and non–tp seems to be narrowing, but not by much! (Again, here's the regression--this time a linear, OLS one, if you prefer that to lowess; notice how easily misled one could be by the positive sign of the tp_x_Conservrepub interaction, which reflects the narrowing of the gap but which doesn't allow one to see as the figure above does that convergence of tp and non–tp would occur somewhere way off the end of the Conser_repub scale in some "disposition twilight zone" that doesn't exist in our universe.)

So what to say? 

For sure, this evidence is more consistent with the "Republicans are divided" hypothesis than with my rival "dispositional intensity" one as an explanation for the gap between tp and non–tp Republican Party members.

Maybe the "tp movement"-- which I had been viewing as kind of a sport, a kind of made-for-tv product jointly produced by MSNBC and Fox to add spice to their coverage of the team sport of partisan politics--is a real and profound thing that really should be probed more intensely and ultimately accommodated in some theoretically defensible way into measures of the dispositions that motivate perceptions and like facts.

Of course, this could turn out to be premature if tp, which is obviously an evolving, volatile form of identification, changes in some way.  We'll just have to stay tuned -- but I'll at least being paying more serious attention! (Go ahead, Rush Limbaugh & Glenn Beck for calling me a bigoted moron etc for simply testing my beliefs with evidence and acknowledging that I'm able to adjust my beliefs from what I learn in doing so.)

Oh, one more thing: An alternative way to test my "partisan intensity" hypothesis would be to measure respodents' motivating dispositions with the cultural worldview scales. Then one could see, as I did here, whether being a "tea party member" generates a strong influence on risk perception over and above intensity of the hierarchical-individualistic worldview that shapes climate skepticism.

Indeed, that would be a pretty good thing to do next, since the culture measures are, as I've explained before, more discerning measures of the underlying risk-perception dispositions here than conventional political outlook measures, which tend to exaggerate the degree to which polarization occurs only among highly partisan citizens.  

But I'll leave that for another day -- and leave it to you to make predictions about whether tp would still emerge as a meaningful distinguishing indicator under such a test!

 

PrintView Printer Friendly Version

EmailEmail Article to Friend

Reader Comments (11)

Dan,

In a lot of ways, the numbers on how many people “believe in climate change”, without any mention of human attribution etc, are “vanity metrics”, numbers that make things look good on paper, but don’t actually tell you what is important in achieving the changes you want. See here:

http://blogs.hbr.org/2010/02/entrepreneurs-beware-of-vanity-metrics/

So what would be a non-vanity metric? What is the “gold standard” of data we should be looking for?
I would say that it is:

Evidence from a

1) field experiment of a
2) causal impact of
3) realistic interventions on
4) individuals’ likelihood of participating in
5) organized, political efforts to change climate policy.

I’m not saying that every study needs to meet this standard to contribute in a meaningful way to the debate.

But until we get that kind of data, we can’t really say we know with certainty what kinds of communication can produce meaningful action on climate change.

November 18, 2013 | Unregistered CommenterNeil Stenhouse

@Neil:

I share your basic assessment. But I'd use a different strategy for responding to the problem.

I think one has to be able to answer the questions "what is being measured?" and "why?" ...

I started to write somethig and it became clear that the point you raise merits is own blog entry. "Tomorrow" I shall post it!

November 19, 2013 | Unregistered Commenterdmk38

Dan -

Indeed, the simple statement "I'm a Republican/Democrat" is itself a relatively weak indicator of such a disposition.

I assume that you're familiar with the stuff that Nate Silver (or was it Sam Wang?) has done w/r/t how variable (meaning inconsistent and unreliable) is party identification?

November 21, 2013 | Unregistered CommenterJoshua

It would be interesting to know which other beliefs tracked as dp beliefs about climate change in TP vs. Non-TP (but strongly disposed) Republicans.

Some other candidates:

Belief that Obama is the Anti-Christ
Belief that Obama isn't a US citizen
Belief that Obama is a Muslim
Belief that there is a "war on Christmas."

and on the other side

Opposition to gay marriage
Opposition to the "war on drugs."

It would be very interesting to find some beliefs that don't track in the same manner.

November 21, 2013 | Unregistered CommenterJoshua

@Dan - were there any questions which would select out members or sympathizers of the occupy movement?

FrankL

(individualist/heirarchist white male sincerely dedicated to the transcendence of identity-protective cognition in myself and others)

November 21, 2013 | Unregistered CommenterFrankL

@Frank L:

Sadly, no.

I'm pretty much a blank slate on this one, too. My guess is that less than 1% would identify w/ this "were the 99%" movement -- down from 2% several yrs ago.

Seen any data?

November 22, 2013 | Unregistered Commenterdmk38

Not really any solid data, but I was thinking about measures of "interest" or "importance". As you say, the number of advocates would be one, maybe Lexis/Nexis search on "occupy movement" would be another, and compare to similar tea party numbers. Another one would be asking the question as part of the data gathering.

Also, I was wondering what you thought about the idea of plotting each data point (person) in the heirarchical/egalitarian individualist/communitarian space (is there an acronym for that?) in which each data point has a particular color representing perceived risk due to global warming. There would be increased density and spatial resolution information, but less refined risk information.

November 22, 2013 | Unregistered CommenterFrankL

@Frank L: wow. that's an interesting picture! I'll give it some thought. (I fooled w/ graphics for a few hrs last night to figure out a better way than ughly gray "confidence bands" for indicating 0.95 CI for estimated/predicted values plotted over range of x)

November 23, 2013 | Registered CommenterDan Kahan

@Dan - I would like to see that plot, for sure, or I could do it if your data is not proprietary. I was thinking that spectral colors (blue-green-yellow-orange-red) with, I don't know, blue for low risk, red for high, or maybe vice versa (democrats associated with high risk perception are from "blue states", low risk republicans are red).

November 23, 2013 | Unregistered CommenterFrankL

@Frank-- is this what you had in mind? I like it!
Check out this... Sort of like Andy Warhol meets Mary Douglas...

November 29, 2013 | Registered CommenterDan Kahan

Maybe it is my lack of knowledge of American politics, but the result that tea party identification matters does not surprise me that much. Or am I wrong to equate the Tea Party with libertarians? A libertarian is something else as just a strong conservative. Sometimes I wonder how they are able to combine these two ideologies into one party.

What I am wondering is whether some people might like the consequences of climate change, thinking that it will hurt the poor more than them (privileged/rich). Such people may say they do not "believe" in climate change, but actually only mean that they do not want to do something about it. I would say you notice that people hardly talk about whether climate change exist when it comes to adaptation (something which helps the local community) and that the climate "sceptics" are mainly up in arms about mitigation (something which also helps others). Consequently, I am wondering whether people that say they do not believe in climate change also really do not believe in climate change. I could imagine that that could be tested with fMRI. Has that been done?

That question probably was not a WSMD. Does your data include questions to make a scale for disposition to racism? I would be curious if that would be an every better predictor as Tea Party Identification.

June 6, 2014 | Unregistered CommenterVictor Venema

PostPost a New Comment

Enter your information below to add a new comment.

My response is on my own website »
Author Email (optional):
Author URL (optional):
Post:
 
Some HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>