Are "moderates" less affected by politically motivated reasoning? Either "yes by definition," or "maybe, depending on what you mean exactly"
A thoughtful person wrote to me about our Motivated Numeracy experiment, posing a variation of a question that I'm frequently asked. That question, essentially, relates to the impact of identity-protective cognition -- the species of motivated reasoning that cultural cognition & politically motivated reasoning are concerned with -- in individuals of a "moderate" political ideology or "Independent" partisan identification.
Here is her question:
I finally got around to looking at your interesting research working paper (that I learned about from http://theincidentaleconomist.com/wordpress/rational-group-think/).
One thing that bothers me about the design is the creation and labeling of the two political orientation groups. The description in the paper says: "Responses to these two items formed a reliable aggregate Likert scale (α = 0.83), which was labeled "Conserv_Repub" and transformed into a z-score to facilitate interpretation (Smith 2000)."
In the study this relatively continuous scale was split in the middle into two groups. While I agree that people at each end of the political spectrum would generally subscribe to opposing positions on the utility of gun bans, I do not think that applies to people in the middle third or half of the political spectrum. I think it is inappropriate to ascribe MOTIVATED numeracy on the gun ban problem to people in the middle of the political spectrum.
How would your results look if your political orientation groups were restricted to only those at the outer third or quartile of the distribution?
As you note, the scale for political outlooks is a continuous one -- or at least is treated as such when we test for the hypothesized effects. We split the sample only for purposes of exploratory or preliminary analysis -- when we are trying to show a "picture," essentially of the raw data, as in Fig 6. In the regression model (Table 1), we estimate the impact of "Conservrepub," including its interaction w/ Numeracy in the various experimental conditions, as a continuous variable; Fig. 7 reflects predicted probabilities derived from the model -- not the responses for different subsamples ("high numeracy" & "low numeracy" "conservative republicans" &. "liberal democrats" etc.) determined w/ reference to the means on Conservrepub or Numeracy.
Necessarily, then, were we to model the performance of subjects in the "middle" of Conservrepub, we'd see no (or, if not literally at the "middle" but at intervals relatively close to either side of the mean, "less") motivated reasoning. But that is what we are constrained to see if we choose to measure the hypothesized motivating disposition with a continuous measure, the effect of which is assumed to be uniform or linear across its range of values.
If one wanted to test the hypothesis that "moderates" or "Independents" are less subject to motivated reasoning, one would have to have a way to model the data that made this claim something other than a tautology.
One way to do it would be to measure the motivating disposition independently of how people identify themselves on the party-id and liberal-conservative ideology measures. Then we could construct a model that estimates the motivated reasoning effect w/ respect to variance in that disposition & see if *that* effect interacts with being an "Independent" (on the party id scale) or a "moderate" (on the ideology scale).
I did that with the data from an experiment that had a similar design -- one that tested whether identity-protective cognition, the kind of motivated reasoning we are concerned with, varies with respect to "cognitive reflection' as opposed to Numeracy. I substituted "cultural worldviews" for political outlooks as the mesure of the motivating disposition -- and then did as I said by looking at whether the influence the motivating disposition interacted with being a political "Independent." See this blog post (title is hyperlinked) for details:
I could do the same for the data in Motivated Numeracy. Likely I will at some point!
You ask what our "results look if your political orientation groups were restricted to only those at the outer third or quartile of the distribution."
We could, in fact, split the continuous predictor Conservrepub into thirds or quarters and measure the impact of "motivated reasoning" separately in each -- by, say, comparing the means for the different groups at different levels of numeracy within them or by fitting a separate regression model to each subgroup. But I'd not trust the results of any such analysis.
For one thing, because the subsamples would be relatively small, such a testing strategy would be underpowered, so we'd not be able to draw any inferences from "null" findings w/ respect to the middling groups, if that is what you hypothesize. Also, splitting continuous predictors like conservrepub & numeracy risks generating spurious differences among subgrups as a result of the random or lumpy distribution of (or really the imprecision of our measurement of) a "true" linear effect. See Maxwell, S.E. & Delaney, H.D. Bivariate Median Splits and Spurious Statistical Significance. Psychological Bulletin 113, 181-190 (1993).
Accordingly, sample splitting of this sort s not a valid strategy, in my view, for testing hypotheses relating to variation in motivated reasoning across the left-right spectrum (whether the hypothesis is that the effect grows more extreme toward both ends, as you surmise, or is that it grows more extreme as one goes toward one end but not the other-- the so-called "asymmetry thesis"...).
But I'm sure there are other valid strategies, too, for testing the hypothesis that motivated reasoning increases as a function of the intensity of political partisanship, a proposition that is, as indicated, *assumed* rather than tested in the model we use in the paper. Be happy to hear of any you come up with. Might make for a fun episode of WSMD? JA!
I am also curious, though, why this would be a surprising or interesting thing? Measurement issues aside, why wouldn't it be just a matter of logic to say that the higher the level of partisan motivation, the higher the impact of politically or culturally motivated reasoning? Or what is the motivation for asserting such a claim?
Is it the sense that the effect we are demonstrating experimentally is confined to "hard core" partisans? For that, one needs to have some practical way of assessing the experimental impact-- one that rests on assumptions outside the experiment (e.g., aboutwho a "hardcore" partisan is w/ respect to the values reflected in Conservrepub & the relative impact that people of varying levels of partisanship make to the overall shape of public opinion & overall character of deliberations, etc.).
In that regard, one more thing you might find interesting is: