follow CCP

Recent blog entries
popular papers

What Is the "Science of Science Communication"?

Climate-Science Communication and the Measurement Problem

Ideology, Motivated Cognition, and Cognitive Reflection: An Experimental Study

'Ideology' or 'Situation Sense'? An Experimental Investigation of Motivated Reasoning and Professional Judgment

A Risky Science Communication Environment for Vaccines

Motivated Numeracy and Enlightened Self-Government

Ideology, Motivated Cognition, and Cognitive Reflection: An Experimental Study

Making Climate Science Communication Evidence-based—All the Way Down 

Neutral Principles, Motivated Cognition, and Some Problems for Constitutional Law 

Cultural Cognition of Scientific Consensus
 

The Tragedy of the Risk-Perception Commons: Science Literacy and Climate Change

"They Saw a Protest": Cognitive Illiberalism and the Speech-Conduct Distinction 

Geoengineering and the Science Communication Environment: a Cross-Cultural Experiment

Fixing the Communications Failure

Why We Are Poles Apart on Climate Change

The Cognitively Illiberal State 

Who Fears the HPV Vaccine, Who Doesn't, and Why? An Experimental Study

Cultural Cognition of the Risks and Benefits of Nanotechnology

Whose Eyes Are You Going to Believe? An Empirical Examination of Scott v. Harris

Cultural Cognition and Public Policy

Culture, Cognition, and Consent: Who Perceives What, and Why, in "Acquaintance Rape" Cases

Culture and Identity-Protective Cognition: Explaining the White Male Effect

Fear of Democracy: A Cultural Evaluation of Sunstein on Risk

Cultural Cognition as a Conception of the Cultural Theory of Risk

« Communicating the normality/banality of climate science (lecture slides) | Main | Congratulations, tea party members: You are just as vulnerable to politically biased misinterpretation of science as everyone else! Is fixing this threat to our Republic part of your program? »
Tuesday
Oct222013

Are "moderates" less affected by politically motivated reasoning? Either "yes by definition," or "maybe, depending on what you mean exactly"

click me ... click me ...A thoughtful person wrote to me about our Motivated Numeracy experiment, posing a variation of a question that I'm frequently asked. That question, essentially, relates to the impact of identity-protective cognition -- the species of motivated reasoning that cultural cognition & politically motivated reasoning are concerned with -- in individuals of a "moderate" political ideology or "Independent" partisan identification.

Here is her question:

I finally got around to looking at your interesting research working paper (that I learned about from http://theincidentaleconomist.com/wordpress/rational-group-think/).

 One thing that bothers me about the design is the creation and labeling of the two political orientation groups. The description in the paper says: "Responses to these two items formed a reliable aggregate Likert scale (α = 0.83), which was labeled "Conserv_Repub" and transformed into a z-score to facilitate interpretation (Smith 2000)." 

In the study this relatively continuous scale was split in the middle into two groups. While I agree that people at each end of the political spectrum would generally subscribe to opposing positions on the utility of gun bans, I do not think that applies to people in the middle third or half of the political spectrum.  I think it is inappropriate to ascribe MOTIVATED numeracy on the gun ban problem to people in the middle of the political spectrum. 

How would your results look if your political orientation groups were restricted to only those at the outer third or quartile of the distribution?

My response:

As you note, the scale for political outlooks is a continuous one -- or at least is treated as such when we test for the hypothesized effects. We split the sample only for purposes of exploratory or preliminary analysis -- when we are trying to show a "picture," essentially of the raw data, as in Fig 6.  In the regression model (Table 1), we estimate the impact of "Conservrepub," including its interaction w/ Numeracy in the various experimental conditions, as a continuous variable; Fig. 7 reflects predicted probabilities derived from the model -- not the responses for different subsamples ("high numeracy" & "low numeracy" "conservative republicans" &. "liberal democrats" etc.) determined w/ reference to the means on Conservrepub or Numeracy.

Necessarily, then, were we to model the performance of subjects in the "middle" of Conservrepub, we'd see no (or, if not literally at the "middle" but at intervals relatively close to either side of the mean, "less") motivated reasoning. But that is what we are constrained to see if we choose to measure the hypothesized motivating disposition with a continuous measure, the effect of which is assumed to be uniform or linear across its range of values.  

If one wanted to test the hypothesis that "moderates" or "Independents" are less subject to motivated reasoning, one would have to have a way to model the data that made this claim something other than a tautology.

One way to do it would be to measure the motivating disposition independently of how people identify themselves on the party-id and liberal-conservative ideology measures.  Then we could construct a model that estimates the motivated reasoning effect w/ respect to variance in that disposition & see if *that* effect interacts with being an "Independent" (on the party id scale) or a "moderate" (on the ideology scale).  

I did that with the data from an experiment that had a similar design -- one that tested whether identity-protective cognition, the kind of motivated reasoning we are concerned with, varies with respect to "cognitive reflection' as opposed to Numeracy.  I substituted "cultural worldviews" for political outlooks as the mesure of  the motivating disposition -- and then did as I said by looking at whether the influence the motivating disposition interacted with being a political "Independent." See this blog post (title is hyperlinked) for details:

WSMD? JA!, episode 3: It turns out that Independents are as just partisan in cognition as Democrats & Republicans after all!

I could do the same for the data in Motivated Numeracy.  Likely I will at some point!

You ask what our "results look if your political orientation groups were restricted to only those at the outer third or quartile of the distribution." 

We could, in fact, split the continuous predictor Conservrepub into thirds or quarters and measure the impact of "motivated reasoning" separately in each --  by, say, comparing the means for the different groups at different levels of numeracy within them or by fitting a separate regression model to each subgroup. But I'd not trust the results of any such analysis.

For one thing, because the subsamples would be relatively small, such a testing strategy would be underpowered, so we'd not be able to draw any inferences from "null" findings w/ respect to the middling groups, if that is what you hypothesize.  Also, splitting continuous predictors like conservrepub & numeracy risks generating spurious differences among subgrups as a result of the random or lumpy distribution of (or really the imprecision of our measurement of) a "true" linear effect. See Maxwell, S.E. & Delaney, H.D. Bivariate Median Splits and Spurious Statistical Significance. Psychological Bulletin 113, 181-190 (1993).
 
Accordingly, sample splitting of this sort  s not a valid strategy, in my view, for testing hypotheses relating to variation in motivated reasoning across the left-right spectrum (whether the hypothesis is that the effect grows more extreme toward both ends, as you surmise, or is that it grows more extreme as one goes toward one end but not the other-- the so-called "asymmetry thesis"...).

But I'm sure there are other valid strategies, too, for testing the hypothesis that motivated reasoning increases as a function of the intensity of political partisanship, a proposition that is, as indicated, *assumed* rather than tested in the model we use in the paper.  Be happy to hear of any you come up with.  Might make for a fun episode of WSMD? JA! 

I am also curious, though, why this would be a surprising or interesting thing? Measurement issues aside, why wouldn't it be just a matter of logic to say that the higher the level of partisan motivation, the higher the impact of politically or culturally motivated reasoning?  Or what is the motivation for asserting such a claim?

Is it the sense that the effect we are demonstrating experimentally is confined to "hard core" partisans?  For that, one needs to have some practical way of assessing the experimental impact-- one that rests on assumptions outside the experiment (e.g.,  aboutwho a "hardcore" partisan is w/ respect to the values reflected in Conservrepub & the relative impact that people of varying levels of partisanship make to the overall shape of public opinion & overall character of deliberations, etc.).

In that regard, one more thing you might find interesting is:

Politically nonpartisan folks are culturally polarized on climate change

 

PrintView Printer Friendly Version

EmailEmail Article to Friend

Reader Comments (7)

"But I'm sure there are other valid strategies, too, for testing the hypothesis that motivated reasoning increases as a function of the intensity of political partisanship,"

I'm not sure what that would really show. Intensity of political partisanship can be a function of something else, and my opinion is that it is largely a function of intensity of identification with one group and intensity in viewing other groups as "the other."

Consider the possibility that a "moderate" or "independent" could be just as committed to their viewpoint as as someone at either end of the political spectrum. They might be just as heavily or intensely committed to their viewpoint, or not, as anyone else.

So then we get to this:

"I am also curious, though, why this would be a surprising or interesting thing? "

I think it would be surprising and interesting if political orientation were associated with motivated reasoning more strongly than intensity of identification.

I think it would be useful to know that it isn't intensity of political orientation that correlates with "motivated reasoning" (with location on the scale from one end to the other being the measure of "intensity," i.e, with the assumption that just because someone is located at a more extreme end of the spectrum they are necessarily more strongly identified with their political ideology), but the degree to which one creates their identity through their political orientation (and through differentiating themselves from others with a differing political orientation).

My guess is that there is some overlap. I'd guess it is probably true that people at more extreme ends of the political spectrum are more identified by their political orientation - but certainly we can point to many people who are very active and engaged politically but who define themselves as moderates. Not all moderates are equal (just as not all libz and conz are equal). I will again bang on my personal perspective that it is intensity of identification that drives motivated reasoning, and that is the reason why intensity of identification would drive something like scientific literacy - not the other way around.

October 22, 2013 | Unregistered CommenterJoshua

Though I suspect that susceptibility to motivated reasoning does differ, at least to some extent, between individuals across contexts (i.e. in a domain-general, personality trait kind of way), I agree that extremity of professed political ideology isn't a reliable measure of this trait. It is a plausible conjecture but you have provided some evidence that it isn't the case that I'm relatively satisfied with! Nevertheless, it would be interesting to see this trait (if it exists) measured reliably and what correlations it might have with different beliefs.

As an expert in the field, do you expect that individual differences in susceptibility to domain-general motivated reasoning exist? And if so, do you think that those scoring low on trait motivated reasoning susceptibility are likely, however slightly, to hold views more consistent with the available evidence?

What I've often felt like when reading cultural cognition research papers is an outsider looking in. I'm very much an Egalitarian-Communitarian but my beliefs about the culturally-contentious issues you frequently explore tend to cohere far more with the scientific evidence than with what my cultural world-view would predict. For example, I "should" be wary about nuclear power and reading more about it "should" have polarised this view even further, but after exposure to the evidence I do not consider it to pose a significant risk. The same applies to Nanotechnology. Furthermore, though my opinion on the relationship between gun control and gun-related crime is weakly skewed towards what it "should" be, I remain relatively ambivalent because I'm led to believe that the evidence is equivocal.

I can think of 4 explanations for why I buck the trend:

1) I've lived all of my life in the UK and the sorts of issues that are culturally contentious in the US are less so (if at all) here.
- While I think the this is true and may partly account for the phenomena I'm seeking to explain, I get the distinct impression that you buck the trend also, as I'm sure many other people in the US do. I think there's more to it...

2) I am less susceptible to motivated reasoning than the average person in general (in the trait sense):
- While I think this could be possible I also find it hard to believe that my susceptibility to motivated reasoning would be sufficiently lower than average to account for the very large discrepancy between some of my actual views and the views my cultural world-view would predict. I'm also aware I might be falling into the Naive Realism trap!

3) The habits of mind I've developed during my science education lead me to appraise evidence more even-handedly than is typical, and, crucially, in a way not necessarily typical of people with higher than average numeracy skills or propensity to engage in effortful cognitive activities, who you have cleverly shown to be generally *more* susceptible to the polarising effects of motivated reasoning.
- I'd like to think that this accounts for a fairly large portion of the variance. Though, once again, naive realism is a consideration.

4) Due at least partly to my education, I've incorporated "empiricist" into my self-concept and its my motivation to form beliefs congruent with this identity that has lead me to form beliefs about culturally-contentious issues more consistent with the evidence than my cultural world-view would predict.
- I suspect that this explanation accounts for the largest portion of the variance, particularly because it's congruent with many of your findings. Nevertheless, I think it's very plausible that explanations 2 and 3 also account for at least some of the variance. Quite possibly, there is an interaction between 2 and 3 (and 4, for that matter) such that a lower than average susceptibility to motivated reasoning made a science education more attractive to me which in turn reinforced my predisposition. I also think that 2 and 3 combined lend themselves nicely to 4 (formation of an "empiricist" identity with it's own consequences).

What I'm trying to get at with this poorly structured comment is whether you think that individuals differ significantly in the extent to which they are prone to engage in motivated reasoning *in general*, and if so, what you think the factors underlying these differences might be (with reference to my own musings, if you find them relevant). I think that understanding the reasons that individuals differ in the extent to which they engage in motivated reasoning - if there is a significant one - would be very useful to anyone interesting in mitigating its effects.

October 22, 2013 | Unregistered CommenterJosh Lord

To be clear, when I talk about a "trait" for motivated reasoning susceptibility, I don't mean to say that such a susceptibility can be reduced to a single trait or disposition. I understand the phenomena emerges from the interactions of many simpler process that themselves probably map onto various traits. What I mean to convey is the idea that a conglomeration of simpler traits, possibly correlated, may differ across individuals such that they're reliably less likely to engage in motivated reasoning across contexts than are others.

Also, Joshua: You highlight an incredibly important factor that I lamentably failed to consider: The extent to which an individual identifies with a self defining group influences the extent to which they will adopt and maintain beliefs that predominate in that group (or might even serve as "badge" of allegiance to it). This has probably played a greater role in my deviation from the predictions of my cultural world-view than anything else. I'm would say that I'm less groupish than most people but that the one group I do identify with to some degree is "scientists". Ever likely I form beliefs in line with the predominant scientific view...

Is there a validated scale that measures "groupish-ness", as it were? Something like a group centricity scale? The closest that comes to it that I can think of off hand is The Individualism and Collectivism Scale but I aren't sure to what extent that would measure what I'm getting at. It'd be interesting to see whether people's self-reported tendency to derive identity from group affiliation is correlated with their tendency to engage in motivation reasoning in response to conflicting culturally-contentious information.

October 22, 2013 | Unregistered CommenterJosh Lord

Josh -

Very interesting comments.

'till the expert shows up....

Seems to me that your "suspicion" makes sense. Everyone has diverse and multiple group identifications. Sometimes different identifications would "motivate" you in different directions. So I would imagine that the relative strength of your different identifications that could help explain the variance of which you speak.

I, personally, belong to the group that would not accept someone like me as a member. That means that I find it hard to reconcile an enthusiastic membership in any particular group. When I am tempted to affirm my identification in a group by locating my views on an issue, I start looking for an exit route.

October 22, 2013 | Unregistered CommenterJoshua

It is a conceit that one can be immune to the effects of groupishness. It is itself a form of groupishness to say one is not like any of those groupish people.

October 27, 2013 | Unregistered CommenterIsabel Penraeth

@Isabel

Perhaps it is naive to think that one is entirely immune to groupish but I don't think it's unreasonable to speculate that one may be less groupish than average. It's quite clear that there are individual differences in the tendency to join social groups. Some people choose to live lives of isolation whereas others practically live for a particular group (e.g. fanatical sports fans). I also suspect that all group members aren't uniformly likely to align their views with other group members. Some people do appear to be more independently minded than others; group leaders being a prime example.

"It is itself a form of groupishness to say one is not like any of those groupish people."

I think that this can be the case, such as in non-conformist groups where group members (somewhat ironically) conform to a norm of non-conformism, but I really don't see why it must be the case. Would you like to elaborate?

October 28, 2013 | Unregistered CommenterJosh Lord

@Josh & @Isabel:

I suspect there are likely individual differences in susceptibility to identity-protective cognition but that they are unlikely to be related to membership in the sorts of groups that generate this phenomenon (e.g., I don't buy the idea that "conservatives" are more dogmatic in assessment of policy-relevant evidence than "liberals"). I'd also guess that such differences are likely to be ones that can't reliably be inculcated through education & the like -- if that were so, I don't think so many critical reasoning dispositions could be shown to *magnify* identity-protective cognition. Obviously, these are matters that deserve more empirical study!

But just assume I'm right.

I *still* think that it might be the case that there are devices one might employ in presenting informatoin that offset the defensive or reason-enervating tendencies associated with identity protective-cognition.

That's the topic of today's post, which distinguishes between "mitigation" strategies aimed at removing from the science communication environment the "contaminants" that trigger identity-proective reasoning & "adaptation" strategies aimed at rehabilitating, strengthening, or supplementing with alternative processes the cognitive operations that are enfeebled by a polluted science communication environment.

I think social scientists are investigating both "mitigation" & "adaptation" responses. I'm interested in studying both.

I think that research investigating those types of interventions are more useful than research that tries to measure differences in the degree of vulnerability to motivated cogniton across groups or among people who vary in one or another cognitive disposition -- for the reasons I stated at the outset.

But I'm glad that someone disagrees w/ me & is taking another approach -- since no one actually knows the answer to the important questions here, a "hedged" research strategy is in order.

October 28, 2013 | Registered CommenterDan Kahan

PostPost a New Comment

Enter your information below to add a new comment.

My response is on my own website »
Author Email (optional):
Author URL (optional):
Post:
 
Some HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>