follow CCP

Recent blog entries
popular papers

Science Curiosity and Political Information Processing

What Is the "Science of Science Communication"?

Climate-Science Communication and the Measurement Problem

Ideology, Motivated Cognition, and Cognitive Reflection: An Experimental Study

'Ideology' or 'Situation Sense'? An Experimental Investigation of Motivated Reasoning and Professional Judgment

A Risky Science Communication Environment for Vaccines

Motivated Numeracy and Enlightened Self-Government

Making Climate Science Communication Evidence-based—All the Way Down 

Neutral Principles, Motivated Cognition, and Some Problems for Constitutional Law 

Cultural Cognition of Scientific Consensus

The Tragedy of the Risk-Perception Commons: Science Literacy and Climate Change

"They Saw a Protest": Cognitive Illiberalism and the Speech-Conduct Distinction 

Geoengineering and the Science Communication Environment: a Cross-Cultural Experiment

Fixing the Communications Failure

Why We Are Poles Apart on Climate Change

The Cognitively Illiberal State 

Who Fears the HPV Vaccine, Who Doesn't, and Why? An Experimental Study

Cultural Cognition of the Risks and Benefits of Nanotechnology

Whose Eyes Are You Going to Believe? An Empirical Examination of Scott v. Harris

Cultural Cognition and Public Policy

Culture, Cognition, and Consent: Who Perceives What, and Why, in "Acquaintance Rape" Cases

Culture and Identity-Protective Cognition: Explaining the White Male Effect

Fear of Democracy: A Cultural Evaluation of Sunstein on Risk

Cultural Cognition as a Conception of the Cultural Theory of Risk


Catch up on commentary day

Lots of interesting comments on the "Less drug laws, less gun homicide" & "Mass opinion coherence & do psychologists tell more tall tales than political scientists?" posts, including some on latter from Stats Legend Andrew Gelman (who also responds w/ post at his own site).

Much more interesting than anything I had planned to say today -- plus I'd like to devote my own "blog time" today to re-reading & responding to them.


Do mass political opinions cohere? And do psychologists "generalize without evidence" more often than political scientists?

Stats Legend Andrew Gelman (whose blog everyone who enjoys being surprised and who values high-quality analytical thinking should read daily) has an interesting post on Steven Pinker.

Pinker asks “[w]hy, if you know a person’s position on gay marriage, can you predict that he or she will want to increase the military budget and decrease the tax rate,” a question he answers by observing that “[p]olitical philosophers have long known that the ideologies are rooted in different conceptions of human nature — a conflict of visions so fundamental as to align opinions on dozens of issues that would seem to have nothing in common.”

Gelman responds by (1) doing some quick GSS correlations, on the basis of which he concludes that “attitudes on such diverse issues are not so highly correlated”; and then (2) attributing Pinker’s error to Pinker’s being a psychologist rather than a political scientist and thus prone to “present[ing] ideas that are thought-provoking but . . . too general to quite work,” in contrast to political scientists who “take such ideas and try to adapt them more closely to particular circumstances.”

Some thoughts:

1. Pinker is clearly right to note that mass political opinions on seemingly diverse issues cohere, and Andrew, I think, is way too quick to challenge this.

I could cite to billions of interesting papers, but I’ll just show you what I mean instead. A recent CCP data collection involving a nationally representative on-line sample of 1750 subjects included a module that asked the subjects to indicate on a six-point scale “how strongly . . . you support or oppose” a collection of policies:  

  1. policy_gun  Stricter gun control laws in the United States.
  2. policy_healthcare  Universal health care.
  3. policy_taxcut  Raising income taxes for persons in the highest-income tax bracket.
  4. policy_affirmative action  Affirmative action for minorities.
  5. policy_warming  Stricter carbon emission standards to reduce global warming.


 Positions clustered on these “diverse” items big time. The average inter-item correlation was 0.66. The Cronbach’s alpha—a scale reliability measure based on item covariance and the number of items—was 0.91.

This is a degree of coherence that would  make any social scientist – psychologist or political scientist – beam. The highest possible alpha is 1.0, and anything above 0.70 is usually regarded as signifying a high degree of reliability.  Low reliability, measured in this way, is it’s own punishment, since it constrains the power of any sort of explanatory or predictive model involving the scale. With a score of 0.91 you can be confident that the power of your model won’t be dissipated by the noise associated with the imprecision of the observable "indicators" you are using to measure the latent variable. 

The latent variable being picked up by these policy items is obviously something akin to right-left political preferences, so let’s call the resulting measure “Liberal_policy.” (Additional items cohered better with each other than with these, forming a second "libertarian policy prefernce" scale; but let's keep things simple.)

Being able to form a scale like this with a general population sample is pretty good evidence in itself (and better than just picking two items out of GSS and seeing if they correlate) that people’s opinions on such matters cohere.

But just to make the case even stronger, let’s consider how much of the variance in liberal policy preferences can be explained by ideology. 

In the same data set, there was a five-point measure for self-described “liberal-conservative ideology” and a  seven-point one for identification with the two major political parties. Those two items were also highly correlated (r = 0.70), so I combined them into a scale (α = 0.82) coded to represent a right-wing ideological disposition, which I labeled “Conserv_repub.”

Regressing Liberal_policy on Conserv_repub, I discovered that the percentage of variance explained (R2) was 0.60. That’s high, as any competent psychologist or political scientist would tell you, and as I’m sure Andrew would agree!

Now Andrew noted that the degree of coherence in political preferences tends to be conditional on other characteristics, such as wealth, education, and political interest. Typically, political scientists use a “political knowledge” measure to assess how coherence in ideological positions vary.

I had a measure of that (a 9-item civics-test sort of thing) in the data set too. So I added it and a cross-product interaction term to my regression model. It bumped up the R2 – variance explained – by 4%, an increment that was statistically significant.  

Seems small, but how practically important is that? A commenter on Andrew’s blog noted that I tend to criticize fixating on R2 as an effect-size measure; my point, which is one that good social scientists—political scientists and psychologists! Andrew too!--have been making for decades is that R2 is not a good measure of the practical significance of an effect size, a matter that has to be determined by use of judgment with relation to the phenomenon at issue.

Well, to help us figure that out, I ran a Monte Carlo simulation to generate the predicted probability that a typical “Liberal Democrat” (-1 SD on Conserv_Repub) and a typical “Conservative Republican” (+1 SD) would support “stricter gun control laws” (seems topical; this is pre-Newtown, so it would be interesting to collect some data now to follow up), conditional on being “low” (-1 SD) or “high” (+1 SD) in political knowledge.

Seems (a) like variance in political knowledge (whatever its contribution to R2) can matter a lot – the probability that a high–political-knowledge Republican will oppose gun control is a lot lower than that for a low–political-knowledge one—but (b) there is still plenty of disagreement even among low–political-knowledge subjects. 

I’d say, then, that Andrew is being a bit too harsh on Pinker’s premise about political preference coherence.

2. Pinker is clearly wrong—not just in his answer but in his style of reasoning—to connect this sort of coherence to “different conceptions of human nature" among people of opposing ideologies

Pinker, however, is indeed doing something very objectionable: he is engaged in rank story-telling

He notes that political philosophers identify ideologies with different conceptions of “human nature,” a “conflict of visions so fundamental as to align opinions on dozens of issues.” Well, maybe political philosophers do do that. But the idea that “different conceptions of ‘human nature’ ” explain coherence and variance in mass political opinion is an empirical claim, and as far as I know there’s not any support for it. 

I think it’s almost certainly false. Measures of ideology of the sort that I have used here have not – as far as I know; please do tell me if I’m wrong: the pleasure of learning something new will more than compensate me for the embarrassment of being shown to be ignorant -- been validated as predictors of “different conceptions of human nature.” Indeed, I think the idea that ordinary members of the public have “conceptions of human nature” is extravagant—the sort of thing only someone who has never ventured outside a university campus would likely believe.

There are myriad theories about the puzzling question of how ordinary people, who really aren’t philosophers, aren’t that interested in politics, and who are very consumed with other things can manage to form coherent ideological preferences. And they’ve been tested empirically.

It’s irritating for anyone who is familiar with all that work to see Pinker advance the sort of claim he does—which he presents not even as a conjecture but as a simple, unqualified, fact-of-the matter report.

3. Pinker’s mistake is one psychologists would resent as much as political scientists.

The sort of thing Pinker is doing here generalizes.  Popular commentators love to reach into the grab bag of decision science mechanisms and construct just-so stories that purport to “explain” complicated phenomena (e.g., political controversy over climate change). 

Good social scientists hate this.  Indeed, Pinker generally doesn’t like it, in fact; he complains about this practice in his excellent book, The Better Angels of Our Nature: Why Violence Has Declined, which admirably tries to connect trends in violence over history to mechanisms that themselves have support in evidence. I find it sort of deflating to see that he seems to adopt a different approach in the writing he does for the New York Times.

But the point is, resentment of story-telling is something that psychologists and political scientists would both experience. It’s not a consequence of Pinker being a psychologist!

4. Ironically, Andrew is making the sort of mistake he says Pinker made.

This last point follows from all the others. Andrew sees Pinker doing something irritating, and then treats a conjecture (I think a pretty uninteresting, implausible one; but all conjectures are created equal – test away!) as a general law that explains this particular instance, etc.

But now I will offer a conjecture, based on an observation-grounded theory.

The observation-grounded theory is that Andrew Gelman has a virtuous Bayesian disposition. That is, he is the sort of person who very happily updates and revises his views, which he always regards as just provisional estimates anyway.

The conjecture: that Andrew, on reflection, will agree that he offered a poor diagnosis (“psychologists generalize without evidence, unlike political scientists, who look for concrete evidence in particulars!”) of Pinker’s objectionable style of argumentation here (which, again, strikes me as uncharacteristic of Pinker himself!).

And now, let’s collect some evidence.

(One more prediction, or hope: Andrew will like my graphic!)

p.s. Ideological coherence in policy prefernces isn't nearly as interesting -- nearly as surprising, puzzling --as ideological or cultural coherence in factual beliefs (e.g., "earth is/is not heating up" & "children of gay & lesbians do worse/no worse in life than ones raised by heterosexual parents." That's what CCP research is all about. Perhaps I'll do another post on that.


Abelson, R.P. A Variance Explanation Paradox: When a Little is a Lot. Psychological Bulletin 97, 129-133 (1985).

Delli Carpini, M.X. & Keeter, S. What Americans Know About Politics and Why It Matters. (Yale University Press, New Haven; 1996).

Gelman, A. & Hill, J. Data Analysis Using Regression and Multilevel/Hierarchical Models. (Cambridge University Press, Cambridge ; New York; 2007).

John, O.P. & Benet-Martínez, V. in Handbook of research methods in social and personality psychology. (eds. H.T. Reis & C.M. Judd) 339-369 (Cambridge University Press, New York; 2000).

King, G. How Not to Lie with Statistics. Am. J. Pol. Sci. 30, 666-687 (1986). 

King, G., Tomz, M. & Wittenberg., J. Making the Most of Statistical Analyses: Improving Interpretation and Presentation. Am. J. Pol. Sci 44, 347-361 (2000).

Pinker, S. The better angels of our nature : why violence has declined. (Viking, New York; 2011).


Actually, empirical evidence suggests a sure fire way to dramatically lower gun homicides: repeal drug laws

Sticking my umbrella up to try to deflect the tsunami of specious, beside-the-point, hiding-the-cultural-imperialism-ball, insult-to-reason uses of empirical evidence on violence & gun control (emanating from those on both sides of the debate), I offered a couple of feeble blog posts (here and here) urging more thoughtful, grounded arguments that are mindful both of the limits of our knowledge and our duty, as citizens of a liberal democracy, to justify our policy positions on grounds free of the impulse to use law as an expressive weapon for denigrating cultural styles that differ from our own.

I'm not satisfied by my performance.

The problem isn't that what I wrote will make no difference, have no effect, etc. If that were the decisive issue, why would I ever say anything? I'm not trying to change the world; I'm just trying to engage other curious and concerned citizens who also want to think about issues reflectively and form positions that they think are factually supportable and morally defensible. I'm trying to help them so they'll help me back, since I'm by no means certain I'm right either.

No, the problem wasn't the futility of what I said but the incompleteness. "Don't look to statistics" -- could be read as denigrating the utility of empirical inquiry in assessing public policy and as expressing a sort of nihilistic "who cares, nothing we can do!" attitude, both of which would deeply misrepresent how I feel about evidence-based policymaking generally & about using evidence to think about guns.

So to amend I will emend.  

I now want to point out that in fact, while the empirical evidence on the relationship between gun control and homicide is (at this time at least) utterly inconclusive, there certainly are policies out there that we have very solid evidence to believe would reduce gun-related homicides very substantially.

The one at the top of the list, in my view, is to legalize recreational drugs such as marijuana and cocaine.

The theory behind this policy prescription is that illegal markets breed competition-driven violence among suppliers by offering the prospect of monopoly profits and by denying them lawful means for enforcing commercial obligations.

The evidence is ample. In addition to empirical studies of drug-law enforcement and crime rates, it includes the marked increase in homicide rates that attended alcohol prohibition and the subsequent, dramatic deline of it after repeal of the 18th Amendment.

from Claude Fischer, Berkeley Blog Actually, it's pretty interesting to look at homicide rates over a broader historical time frame than typically is brought into view by those who opportunistically crop the picture in one way or another to support their position for or against gun control.  What you see is that there is a pretty steady historical trend toward decline in the US punctuated by expected noisy interludes but also by what appear to be some genuine, and genuinely dramatic, jumps & declines.

One of the jumps appears to have occurred with the onset of prohibition and one of the declines with repeal of prohibition.  Social scientists doing their best to understand the evidence generally have concluded that that those are real shifts, and that they really were caused by prohibition and repeal.  

from Claude Fischer, Berkeley Blog

Criminologists looking at the impact of drug prohibition can use the models developed in connection with alcohol prohibition and other modeling strategies to try to assess the impact of drug prohibition on crime. Obviously the evidence needs to be interpreted, supports reasonable competing interpretations, and can never do more than justify provisional conclusions, ones  that are necessarily subject to revision in light of new evidence, new analyses, and so forth.

But I'd say the weight of the evidence pretty convincingly shows that drug-related homicides generated as a consequence of drug prohibition are tremendously high and account for much of the difference in the homicide rates in the U.S. and those in comparable liberal market societies (the non-liberal, non-market societies all are burdened with homicide rates orders of magnitude higher; guns don't explain that--  the pacifying influence of doux commerce does).  By all means decide for yourself, though; I've cited some reading material at the end of this post & urge others to call our attention to more in the comments section.

There are obviously other spikes & dips in the "secular" (as econometricians would say; here that's a very nice adjective to use) downward trend in homicide in the U.S. E.g., the upsurge of homicides in the 1960s and the decline in the 1990s. Most of the gun-control combatants mine this period for support for their claims -- they should, since the data are rich with support for specious inference.  Scholarly discussion here recognizes that the evidence on the contribution of guns to these jumps is utterly, hopelessly inconclusive.

There is a very interesting empirical study, though, by economist Jeffrey Miron, who concludes that the available evidence is consistent with the hypothesis that the difference in homicide rates in the US and in other liberal market societies is attributable to our drug prohibition policies. Gun availability in the US, according to this hypothesis, doesn't directly account for the difference in homicide rates between the US and these countries; rather, gun availability mediates the impact between drug prohibition and homicide rates in the US, because the criminogenic properties of drug prohibition create both a demand for murder of one's competitors and a demand for guns to use for that purpose.  

One of the very nice things about Miron's analysis, too, is that he is appropriately provisional about his conclusions:  

The empirical results presented above provide a possible explanation for the large differences in violence rates across countries, and they suggest that previous analyses might have spuriously attributed these differences to gun control or availability. According to the analysis here, differences in drug prohibition enforcement explain differences in violence, which in turn explain differences in gun ownership that correlate positively with violence but do not cause that violence. Further, the results provide a hint that restrictive gun control regimes can themselves increase violence. As noted above, these results should be considered suggestive rather than conclusive. Future research on these issues will need to exploit time‐series rather than cross‐sectional data.

 That's what a real scholar sounds like, you see. In my view, it's what an open-minded citizen sounds like too.

Now one thing to note: Obviously, decriminalizing marijuana and cocaine couldn't be expected to prevent mass shootings like the one in Newtown, or Aurora, or Phoenix, or Columbine, etc. (Maybe there'd be fewer guns around, actually, if we didn't have the demand for them associated with their contribution to the illegal drug trade, but there are already so many around -- the gun & people populations are neck-and-neck -- that I think disturbed people would still have no trouble getting their hands on them.)

But here's another thing to note: these very sad incidents "represent only a sliver of America's overall gun violence." Those who are appropriately interested in reducing gun homicides generally and who are (also appropriately) making this tragedy the occasion to discuss how we as a society can and must do more to make our citizens safe, and who are, in the course of making their arguments invoking (appropraitely!) the overall gun homicide rate should be focusing on what we can be done most directly and feasibly to save the most lives.

Repealing drug laws would do more --  much, much, much more -- than banning assault rifles (a measure I would agree is quite appropriate); barring carrying of concealed handguns in public  (I'd vote for that in my state, if after hearing from people who felt differently from me, I could give an account of my position that fairly meets their points and doesn't trade on tacit hostility toward or mere incomprehension of  whatever contribution owning a gun makes to their experience of a meaningful free life); closing the "gun show" loophole; extending waiting periods etc.  Or at least there is evidence for believing that, and we are entitled to make policy on the best understanding we can form of how the world works so long as we are open to new evidence and aren't otherwise interfering with liberties that we ought, in a liberal society, to respect.

...Now, what other policies might help? And in particular, if we are concerned about deaths of children?   Well, there's swimming pools .... But I've said enough for now.


Barnett, R. & Trip, B. Drug Prohibition and the Weakness of Public Policy’(1994). Yale LJ 103, 2593. 

Husak, D. Legalize This! The Case for Decriminalizing Drugs. (Verso, New York; 2002).

Jensen, G.F. Prohibition, Alcohol, and Murder Untangling Countervailing Mechanisms. Homicide Studies 4, 18-36 (2000).

Kleiman, M. Marijuana : costs of abuse, costs of control. (Greenwood Press, New York; 1989).

Kleiman, M., Caulkins, J.P. & Hawken, A. Drugs and drug policy : what everyone needs to know. (Oxford University Press, Oxford ; New York; 2011).

MacCoun, R.J. & Reuter, P. Drug war heresies : learning from other vices, times, and places. (Cambridge University Press, Cambridge, U.K. ; New York; 2001).

Miron, J.A. & Zwiebel, J. The Economic Case Against Drug Prohibition. The Journal of Economic Perspectives 9, 175-192 (1995).

Miron, J.A. Violence, Guns, and Drugs: A Cross‐Country Analysis*. Journal of Law and Economics 44, 615-633 (2001).

 Pinker, S. The better angels of our nature : why violence has declined. (Viking, New York; 2011).



Statistics: the monopoly money of the gun-debate marketplace of ideas

This is what makes people cynical about empirical arguments in policy debate. Recognize when the data are inconclusive, or else no one will be able to recognize what counts as sound evidence.

If you are contributing to this devaluation of the currency of reason, just stop. In particular, stop insisting that everyone who disagrees with you on facts is either an "idiot" or a "liar."

Look: There are lots of good, thoughtful arguments to be made here, ones based on value and ones based on the best factual surmises we can make based on experience and sense.  

These are arguments that citizens in a liberal society can advance openly, and should, to confirm, for themselves and for others, that their positions aren't motivated by the illiberal ambition to denigrate those whose cultural identities differ from their own. Reliance on one-sided, cherry-picked empirical arguments doesn't furnish that assurance; on the contrary, it's the "smoking gun" of cognitive illiberalism.




"More statistics, less persuasion": the gun control debate continues, and continues to miss the point...

I was planning to write something about Moore v. Madigan, No. 12-1269 (7th Cir. Dec. 11, 2012),  the U.S. Court of Appeals decision earlier in the week that overturned an Illinois law that more or less bans ordinary citizens from carrying concealed weapons in public.

Judge Richard Posner, our century's Oliver Wendell Holmes Jr., wrote the opinion.  Not surprisingly, it's a good read.

Posner irreverently mocks any attempt to wring meaning from the "original intent" of the Second Amendment (Posner is now engaged in a very unseemly but entertaining pissing match with Justice Scalia, in which both have basically said "liar, liar pants on fire!" about Posner's New Republic review of Scalia's grammar-school quality text on  interpretation).

He also ruthlessly --nihilistically, even; Posner, like Holmes, exudes a nihilistic philosophy & style-- arrays against one another conflicting empirical studies on the impact of concealed-weapons laws on violent crime (uncharacteristically, Posner misses a supporting citation: to an "expert consensus" report from the National Academy of Sciences, which declares such evidence utterly inconclusive).

Posner concludes:

A gun is a potential danger to more people if carried in public than just kept in the home. But the other side of this coin is that knowing that many law-abiding citizens are walking the streets armed may make criminals timid. Given that in Chicago, at least, most murders occur outside the home, Chicago Police Dep’t, Crime at a Glance: District 1 13 (Jan.–June 2010), the net effect on crime rates in general and murder rates in particular of allowing the carriage of guns in public is uncertain both as a matter of theory and empirically. “Based on findings from national law assessments, crossnational comparisons, and index studies, evidence is insufficient to determine whether the degree or intensity of firearms regulation is associated with decreased (or increased) violence.” Robert A. Hahn et al., “Firearms Laws and the Reduction of Violence: A Systematic Review,” 28 Am. J. Preventive Med. 40, 59 (2005); cf. John J. Donohue, “The Impact of Concealed-Carry Laws,” in Evaluating Gun Policy Effects on Crime and Violence 287, 314–21 (2003). “Whether the net effect of relaxing concealed-carry laws is to increase or reduce the burden of crime, there is good reason to believe that the net is not large…Based on available empirical data, therefore, we expect relatively little public safety impact if courts invalidate laws that prohibit gun carrying outside the home, assuming that some sort of permit system for public carry is allowed to stand.” Philip J. Cook, Jens Ludwig & Adam M. Samaha, “Gun Control After Heller: Threats and Sideshows from a Social Welfare Perspective,” 56 UCLA L. Rev. 1041, 1082 (2009)....

In sum, the empirical literature on the effects of allowing the carriage of guns in public fails to establish a pragmatic defense of the Illinois law. Bishop, supra, at 922–23; Mark V. Tushnet, Out of Range: Why the Constitution Can’t End the Battle over Guns 110–11 (2007). Anyway the Supreme Court made clear in Heller that it wasn’t going to make the right to bear arms depend on casualty counts. 554 U.S. at 636. If the mere possibility that allowing guns to be carried in public would increase the crime or death rates sufficed to justify a ban, Heller would have been decided the other way, for that possibility was as great in the District of Columbia as it is in Illinois.

So ... I was planning to write something along the lines, "the statistical debate has been won by the gun-rights team, which has (a) effectively fought the statistical battle to a standstill & (b) succceeded in getting courts to impose a liberty-preserving standard of proof on those who want to restrict guns. Point (b) -- who should bear the burden of proof -- is an interesting question in a liberal society, yet people will still go on and on about statistics blah blah."  Or some such.

But now this... And I don't know what to say. Except that statistics really are beside the point.

They are beside the point because they are genuinely inconclusive.

They are beside the point because they genuinely don't engage what psychologically motivates people's positions here.

And they are beside the point because they ignore the real moral issues, which are ones of social meaning:

  • What does it say about what people value when they want to own a cache of military-style armaments such as a "Glock," a "SIG Sauer handgun," and a "Bushmaster .223-Caliber Assault rifle"? (These were weapons owned, apparently, by the shooter's mother, his first victim, who was incorrectly reported to be a kindergarten teacher at the school where the shooting spree took place; guess she was also an avid "self-defense" enthusiast?)   
  • What does it say about us when we permit other people to make money -- lots of it -- satisfying the appetite of ordinary people to own the tools of the trade of those whose profession is killing?

  • Yet what would it say about us if in saying this with our law we couldn't also acknowledge that owning various types of guns are also, for many, enmeshed with a host of very different cultural meanings--ones relating to personal virtues like self-reliance, honor, and responsibility; and to social roles and practices that intimately connect them to people, past and present, whom they do value, and very appropriately so?
  • What would it say about our commitment to liberal principles if those of us who don't belong to communities in which guns have these meanings (and in fact are puzzled by them) refused to acknowledge (or just couldn't see; the two are connected) that those who do belong to them understandably see many kinds of gun control as expressing hostility and contempt for their values? Understandably, because in fact, many (not all!) of those who advocate gun control are motivated by (or are simply profiting from) exactly that sort of ugly, illiberal sentiment?

This is a conflict of cultural meaning that must be negotiated by law. And it should be negotiated in a way that is consistent with liberal political principles, which impose on citizens of diverse understandings the duty to show they are committed to accommodating one another, and to resisting making use of laws (whether handgun bans or "stand your ground" provisions) as expressive symbols of dominance over one another.

This is complicated.  Much more complicated than the convoluted (and utterly inconclusive) multivariate regression analyses with which those involved in the "statistics" debate beat each other over the head.

And more complicated than simply mocking those who think statistics (or "history") can solve this issue.

We don't need nihilism in our public discourse. We need genuine liberal statesmanship.

We don't need a Holmes. We need a Lincoln.


A multivariate regression analysis of CRT performance & ideology, plus a preliminary diatribe against mindlessly overspecified regression models

My most recent paper, Ideology, Motivated Reasoning, and Cognitive Reflection, reports data on the relationship between ideology and the disposition to use high-level, "System 1" information processing, as opposed to intuitive, low-effort, heuristic-drive "System 2," as measured by the Cognitive Reflection Test (CRT).  

I report that there really isn't any. That's sort of surprising in light of all the attention being paid to the neo-Authoritarian Personality literature, which asserts that conservativism is characterized by closed-mindedness, aversion to complexity, and the like.

I know reviewers will want to know, "but if one controls for ..." so I've prepared a multivariate regression that includes ideology along with various other individual characteristics (gender, race, education, income, and religiosity) that have been shown to correlate with CRT scores.  Still nothing!

See for yourself by clicking on the thumbnail to the left.

But in fact, I view this analysis as pretty close to worthless -- if one is really trying to figure out if there is an association between ideology and cognitive reflection.  The idea of  "controlling for" these sorts of characteristics in order to measure the "independent" impact of "ideology" is nonsense: it treats "ideology" as some sort of disembodied "essence" inside of people, when in fact it is one facet of an integrated package of attributes that cohere with one another and are all indicators of a latent type of shared identity or style.

I've been planning for a while to go ballistic on the idea of "overspecified" regressions -- and what sort of mistake in thinking (or failing to think, really) they reflect. This isn't really the right vehicle for getting the point across, so I anticipate coming back to this topic at some point.

But as a preview, here is a short text that I prepared to go with this genuinely fantastic multivariate regression analysis of ideology & CRT.  I'm sure the note will be revised -- or dropped altogether -- before I submit the paper for publication:

Finally, a multivariate regression model that included all of these predictors was tested. That analysis can be seen as assessing the effect of ideology on CRT scores “controlling for” gender, race, education, income, and religiosity. It is doubtful, however, that such a “model” bears any meaningful relationship to reality. In the world we live in, people come in packages of demographic and political orientations, which correlate in ways suggestive of various latent forms of identity. Thus, “partialing” out the covariance of gender, race, religion, education, and income in order to estimate the “independent” effect of ideology creates a model of something (either individual characteristics disconnected from people, or people who can be randomly endowed with combinations of characteristics) not actually observed on planet earth (Berry & Feldman 1985, p. 48; Cohen, Cohen, West &  Aiken 2003, p. 419; Gelman & Hill 2006, p. 187). Absent some appropriate aggregation of all these variables into a valid latent-variable measure, a zero-order correlation furnishes a more valid estimate of the influence of subjects' political outlooks on their CRT scores than does the coefficient for that predictor in a model that treats ideology and all of these other characteristics as independent, right-hand side variables in a multivariate regression (Lieberson 1985, pp. 14-43). But for the benefit of those who prefer to regard multivariate regression as a magical black box for capturing the “causal” effect of phantom essences, as opposed to a statistical tool for measuring the relationship of valid measures of real-world phenomena, this blunderbuss analysis shows the the coefficient for Conserv_Repub remained trivially different from zero and nonsignificant (b = 0.06,  p = 0.33). When party self-identification was substituted for Conserv_Repub, that variable continued to predict an increase in CRT score as subjects’ identification with the Republican party intensified, but the effect was reduced and was only marginally significant (b = 0.10, p = 0.06).


Berry, W.D. & Feldman, S. Multiple regression in practice. (Sage Publications, Beverly Hills; 1985).

Cohen, J., Cohen, P., West, S.G. & Aiken, L.S. Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences, Edn. 3rd. (L. Erlbaum Associates, Mahwah, N.J.; 2003).

Gelman, A. & Hill, J. Data Analysis Using Regression and Multilevel/Hierarchical Models. (Cambridge University Press, Cambridge ; New York; 2007).

Lieberson, S. Making it count : the improvement of social research and theory. (University of California Press, Berkeley; 1985).


Query on climate change, culture, ideology & identity-protective cognition

A thoughtful person asks:

I’ve come across your work while trying to make sense of climate change denial. I find your analysis very interesting (along with Flynn et al. 1994, Finucane et al. 2007, and McCright and Dunlap 2011) because it offers a compelling explanation for what seems like a curious social dynamic.

However, there’s something I don’t quite follow, and with your kind forbearance I hope I may ask you a question. Your 2007 Journal of Empirical Legal Studies paper sketches out a synthesis of cultural risk perception and identity-protective cognition. From that I was expecting to see how group identity (conservative, Republican) and world vision (hierarchical, individualistic) somehow mutually reinforced each other in the climate change arena. In fact, though, that seems not to be the case. Instead it is the hierarchical and individualistic world vision itself that cognition seeks to protect. Indeed, your regression 4 in Table 2 (p. 483) if anything seems inconsistent with my expectation, since both "Conservative" and "Democrat" are highly significant, whereas if these substantially overlapped with the hierarchical white male dummy (so to speak), the coefficients would have been insignificant. Is your view that Conservative and Democrat are independent (from the white male effect) determinants of views on climate change? The narrative in McCright and Dunlap directly linking climate change skepticism to conservatism appeals to me, but I'm not sure if it's consistent with your own perceptions and/or findings.

Thanks very much, and thanks for your very interesting paper.

This is my response. Anyone want to add anything? 

I believe that people have unobservable latent predispositions that they acquire as a result of one or another social influence. The thing to do is find observable indicators that one has good reason to believe correlate with those dispositions, combine them into reliable scales, and use those measures to test hypotheses about who sees what & why, & about what sorts of communication strategies are geared to promoting open-minded engagement with information by people of diverse predispositions.  "Republican, Democrat, liberal, conservative, hierarch, egalitarian, individualist," etc are all candidate indicators. Which ones to combine to form scales depends on which latent-variable measurement strategy most instructively enables explanation, prediction & prescription.  

For more info, click on links below; & let me know if you have additional questions or if you have reactions, comments etc.


Yow--lots of great comments on "science literacy vs. climate-change science literacy"

I was going to post something, but it's not as interesting or important as the points that people made in response to yesterday's post on science literacy & climate-change science literacy.  So check those out! And add more.



Science literacy vs. "climate science literacy"

This is in the department "recurring misunderstanding that I should say something about in a single place so that I can simply refer people to it."

Last May, CCP researchers published a study in Nature Climate Change presenting evidence suggesting that political controversy over climate change in the US cannot be attributed to any sort of deficit in the public's comprehension of science.

As science literacy and numeracy  (a technical reasoning disposition associated with more discerning perception of risk) increase, members of the general public do not converge in their perceptions of the risks posed by climate change. Instead, they become even more culturally polarized.

This finding fit the hypothesis that individuals can be expected to engage information in a manner that fits their interest in forming and maintaining beliefs that reflect their membership in, and loyalty to, important affinity groups.

Competing positions on climate change, unfortunately, are now conspicuously associated with opposing cultural groups. Being out of line with one's group on this issue exposes an individual to a social cost, whereas forming a mistaken view on the science of climate change has zero impact on the risk that individual, or anyone or anything she cares about, faces, insofar as one individual's personal behavior (as consumer, voter, public discussant, etc.) has no material effect on the climate.

One doesn't have to be a rocket scientist to figure out what side of the issue one's cultural group is on in a debate like the one over climate change. But if one is, well, not a rocket scientist, but someone who has an above-average command of basic science and an above-average ability to make sense of fairly complicated technical and quantitative information, then one necessarily has skills --an ability to search out  supportive evidence, fight off counterarguments, etc.-- that one can use to be even more successful at forming and persisting in group-convergent beliefs.

The survey data reported in the Nature Climate Change study supported this conjecture. The experimental findings in the most recent CCP study -- on ideology, motivated reasoning, and cognitive reflection--supply even more support for it.

Now, the response to the Nature Climate Change study that I have in mind says, "Wait -- you didn't measure climate change literacy! Regardless of their worldviews, if people knew more about climate change science they surely would converge on the best understanding of the risks that climate change poses!"

That response is in fact a non sequitur.

Yes, of course, people who are "climate science literate," by definition, understand and accept the best scientific evidence on climate change. 

The whole point of the study, though, was to test hypotheses about why members of the general public haven't converged on that evidence -- or why, in other words, they aren't uniformly climate-science literate.  

We measured their general science literacy to assess the (widespread) claim that a general deficit in science comprehension explains this particular aspect of confusion about science.  What we found -- that members of the general public who display the greatest general science comprehension are the most culturally polarized on climate change risks -- is flatly inconsistent with that claim.

Imagine we had measured "climate change literacy" instead and used it to predict "climate change risk perception." We would have found that the former predicts the latter quite well -- because in fact, they are, analytically, the same thing.  

But then we'd still be left with the key question -- what explains deficits in "climate science literacy"?  By measuring general science literacy--something that is analytically distinct from climate change risk perception--we were able to help show that one common conjecture about that -- that people are not "climate change science literate" because they can't comprehend basic science -- is inconsistent with empirical evidence.

If one genuinely wants to explain public conflict over climate change, one has to offer and test explanations that don't just amount to redescribing the phenomenon.

And if the goal is to promote public recognition of the best available evidence on climate change -- and other societal risks -- then the sort of science illiteracy we need to remedy relates to our collective ability to protect our science communication environment from the sorts of toxic cultural meanings that make it individually rational for ordinary citizens -- including the most science literate ones -- to pay more attention to what positions on risk say about who they are than to whether those positions are true.


Kahan, D. Why we are poles apart on climate change. Nature 488, 255 (2012).

Kahan, D.M., Peters, E., Wittlin, M., Slovic, P., Ouellette, L.L., Braman, D. & Mandel, G. The polarizing impact of science literacy and numeracy on perceived climate change risks. Nature Climate Change 2, 732-735 (2012).

Kahan, D.M., Wittlin, M., Peters, E., Slovic, P., Ouellette L.L., Braman, D., Mandel, G. The Tragedy of the Risk-Perception Commons: Culture Conflict, Rationality Conflict, and Climate Change. CCP Working Paper No. 89 (June 24, 2011).

Ideology, Motivated Reasoning, and Cognitive Reflection, CCP Working Paper 107 (Nov. 29, 2012)

Peters, E., Västfjäll, D., Slovic, P., Mertz, C.K., Mazzocco, K. & Dickert, S. Numeracy and Decision Making. Psychol Sci 17, 407-413 (2006).




Cultural cognition is not a bias -- and the corruption of it is no laughing matter!

Well, I feel sort of bad for coming across as gleeful in reporting that further analysis of data confirmed Indepdendents, just like politial partisans, display ideologically motivated reasoning.  A commentator (Metamorph, aka "Metamorph") called me out on that.

My punishment is to write 500 times ...

1. Cultural cognition is not a bias (parts one and two).

2. It's the science communication environment, stupid -- not stupid people!

3. Cultural cognition is not a bummer (parts one and two).



WSMD? JA!, episode 3: It turns out that Independents are as just partisan in cognition as Democrats & Republicans after all!

This is the third episode in the insanely popular CCP series, "Wanna see more data? Just ask!," the game in which commentators compete for world-wide recognition and fame by proposing amazingly clever hypotheses that can be tested by re-analyzing data collected in one or another CCP study. For "WSMD?, JA!" rules and conditions (including the mandatory release from defamation claims), click here.

Okay, so I was all freaked out by the discovery that Independents are more reflective, in terms of the Cognitive Reflection Test scores, than partisans and wondering if this signified that somehow that Independents are these magical creatures who don't become even more vulnerable to ideologically motivated reasoning as their disposition to engage in analytical, System 2 reasoning becomes more pronounced (one of the findings of the latest CCP study).

Enticed by my promise to share the Nobel Prize in whatever 4 or 5 disciplines would surely award it to us for unravelling  this cosmic mystery, Isabel Penraeth (aka "Gemeinshaft Girl) and NiV (aka "NiV") told me to just calm down and use some System 2 thinking. Did it ever occur to you, NiV asked with barely concealed exasperation, that the problem might be that Independents are members of a "cultural in-group" that evades the dopey 1-dimensional left-right measure used in the study? Yes, you fool, added Geminschaft Girl, have you even botherd to see whether Independents behave at all differently from Partisans (let's use that term for those who identify as either Republicans or Democrats) when their worldviews are measured with the "CCR group-grid scale?"

Doh! Of course, this is the right way to figure out if there's really any difference in how Independents and Partisans process information. 

The basic hypothesis of the study was that ideologically motivated reasoning is a consequence of a kind of "identity-protective cognition" that reflects the stake people have in forming perceptions of risk and other policy-relevant facts consistent with the ones that predominate in important affinity groups.

This is actually the core idea behind cultural cognition generally. Usually, too -- as in always before now, really-- our studies have used "cultural worldview" scales, derived from the "group-grid framework of Mary Douglas, to measure the motivating group commitments that we hypothesized drive identity-protective cognition on climate change, gun control, nuclear power, the HPV vaccine, Rock 'n Roll vs. Country, & like issues.

We do that, I've explained, because we think the cultural worldview measures are better than left-right measures. They are more discerning of variations in the outlooks of ordinary, nonpartisan folk, and thus do a better job of locating the source and magnitude of cultural divisions on risk issues.

The reason I used right-left in the most recent study was that I wanted to maximize engagement with the researchers whose interesting ideas motivated me to conduct it. These included the Neo–Authoritarian Personality scholars, whose work is expertly synthesized in Chris Mooney's Republican Brain. They all use right-left measures, which, like I said, I don't think are as good as cultural-worldview ones but are (as I've explained before) plausibly viewed as alternative indicators of the same latent motivating predispositions.

So for crying out loud, why not just see how Independents compare with Partisans when instead of right-left ideology cultural worldviews are used as the predictor in the motivated-reasoning experiment described in the study?! Of course, I have the data on the subjects cultural worldviews; like the participants in all of our studies, they were part of a large, nationally diverse subject pool recruited to take part in cultural cognition studies generally.

As I'm sure you all remember vividly, the experiment tested whether subjects would show motivated reasoning in assessing evidence of the "validity" of Shane "No limit video poker world champion" Frederick's gold-standard "System 1 vs. System 2" Cognitive Reflection Test.  Subjects were assigned to one of three conditions: (1) a control group, whose members were told simply that psychologists view CRT as valid test of open-mindedness and reflection; (2) a "skeptic-is-biased" condition, whose members were told in addition that "climate skeptics" tend to get lower CRT scores (i.e., are more closed-minded and unreflective); and (3) a "nonskeptic-is-biased" condition, whose members were told that "climate believers" get lower scores (i.e., are more closed-minded and unreflective). 

As hypothesized, subjects polarized along ideological lines in patterns that reflected their disposition to fit their assessment of scientific informtion--here on a test that measures open-mindedness and reflection--to their ideological commitments. So relative to their counterparts in the control, more liberal, Democratic subjects were more likely to deem the CRT valid, and more conservative Republican ones to deem it invalid, in the "skeptic-is-biased" condition; these positions were flipped in the "nonskeptic-is-biased condition." Moreveover, this effect was magnified by subjects' scores on the CRT test--i.e., the more disposed they are to use analytical rather than heuristic-driven reasoning, the more prone subjects are to ideologically motivated reasoning.  

Necessarily, though, Independents, didn't show such an effect (how could they, logically speaking? they aren't left or right to a meaningful degree) and they happened to score a bit higher than Partisans (Dems or Repubs) on CRT. Hmmmm....

But Independents, just like Democrats and Republicans, have cultural outlooks. So I reanalyzed the study data using the cultural cognitoin "Hierarchy-egaltiarian" and "Individualist-communitarian" worldview scales.

Because climate change is an issue that tends to divide Hierarch Individualists (HIs) and Egalitarian Communitarians (ECs), my principal hypotheses were (1) that HIs and ECs would display motivated reasoning effects equivalent to the ones of conservative Republicans and liberal Democrats, respectively, and (2) that this effect would increase as subjects CRT reflectiveness scores increased. The competiting additional hypotheses: (3a) that Independents wouldn't behave any differently in this respect than Partisans; and (3b) that Independents would be shown to be magic, supherhuman (possibly outerspace alien) beings who are immune to motivated cognition.

I had my money (a $10,000 bet made w/ Willard, a super rich guy who doesn't pay any income taxes) on 3a. Independents, like Democrats and Republicans, have cultural worldviews; why wouldn't they be motivated to protect their cultural identities just like everyone else?

Results? Hypotheses (1) and (2) were confirmed. When I just looked at subjects defined in terms of their worldviews, I observed the expected pattern of polarization. Indeed, HIs and ECs reacted in an even more forcefully polarizing manner to the experimental manipulation than did conservative Republicans and liberal Democrats, an effect that should come as no surprise because the culture measures are indeed better--i.e., more discerning--measures of the group dispositions that motivated biased processing of information on risk and other policy-relevant facts.

Next, I compared the size of this culturally motivated reasoning effect for Partisans and Independents, respectively.  The regression model that added the appropriate variables for being an Independent did add explanatory power relative to the model that pooled Indepedents and Partisans. But the effect was associated almost entirely with the tendency of Independents to polarize more forcefully in the "skeptic-is-biased" condition. The same basic pattern--HIs and ECs polarizing in the expected ways, and magnification of that effect by higher CRT scores--obtained among both Partisans and Independents.

 You can see that there are some small differences, ones that reflect the relationship I described between being an Independent and being assigned to the "skeptic-is-biased" condition.  But I myself don't view these differences as particularly meaningful; when you start to slice & dice, you'll always see something, so if it wasn't something you were looking for on the basis of a sensible hypothesis, more than likely you are looking at noise.

So I say this is corroboration of hypothesis (3a): Independents are just as partisan in their assessment of information that threatens their cultural identities as political Partisans. I'm done being freaked out!

But hey, if you disagree, tell me! Come up with an interesting hypothesis about how Independents are "different" & I'll test it with our data, if I can, in another episode of WSMD? JA!

 WSMD? JA! episode 1

WSMD? JA! episode 2

Ideology, Motivated Reasoning, and Cognitive Reflection, CCP Working Paper 107 (Nov. 29, 2012)


What does the science of science communication say about communicating & expanding interest in noncontroversial but just really really cool science?

I got this really thoughtful email from Jason Davis, who is doing graduate work in journalism at the University of Arizona, and who operates Astrsosaurus, an interesting-looking science journalism site:

I just finished watching your recent science communication talk for the Spitfire Strategies' speaker series on YouTube. . . .

I'm curious to hear your thoughts on communicating "non-controversial" science. There seems to be no shortage of plausible, but not necessarily correct, ideas on addressing polarized subjects like global warming, stem cell research, nuclear power, etc. But what about something more innocuous, like spaceflight? Do you think the "you tell me what works and what doesn't" approach you emphasize in your lecture is equally valid in these areas? Or since we're treading on less-controversial turf, are we back to a deficit approach? Or perhaps it's naive to assume any science issue can be communicated without controversy?

As an example, I'm personally fascinated by the vast unknowns in our own solar system, but I know not everyone shares my enthusiasm. To make some gross assumptions here, if we had more public support for NASA, perhaps its budget would be increased, and perhaps we would have more spacecraft uncovering the mysteries of Jupiter and Saturn's moons. So, are we to assume that more effective science communication could close that gap? If so, what should that communication look like? Or should I just concede that not everyone is interested in these things, and convincing someone to care about the moon Enceladus is akin to someone else trying convincing me to care about, say, Hollywood gossip?...

Jason Davis

Here is my response, which I think is OK, but definitely could be improved upon -- by others who have thoughts, insights, and experience.  For one thing, looking back on this, I can see that I sort of avoided the "how to generate interest" question & instead focused on "how to satisfy the appetite of culturally diverse people who are curious to know what is known" -- which I view as the critical mission that that science journalism plays in the Liberal Republic of Science. So feel free to supplement my response in comments section (I'd like to know answers to Jason's questions, too!).

My response:

1. I think this is a very very different sort of science communication issue. The biggest fallacy that the science of science communication addresses is that there is no need for a science of science communication -- sound science communicates itself. But the second biggest is that the science of science communication is one thing -- that the same insights into how communicate probabilistic information to an individual trying to make an informed decision about a medical procedure are the same ones that will "solve" the climate change dispute.  Science communication is 5 things +/- 2.

2.  One of the things it is is systematic, empirical inquiry into how to make what's known to science known to curious people. Many many people in the Liberal Republic of Science are thrilled and awed by what the use of our collective intelligence has revealed to our species about the workings of the universe; it is part of what makes this political regime so good that it invests resources to produce information to satisfy that interest in knowing what's known, and that many of our smartest & most creative people are excited to play the translation role that science journalism involves.

3.  The state of knowledge on this part of science communication is in good shape -- it is in much better shape than the part of our scientific knowledge that relates to protecting the quality of the science communication environment from toxic meanings that disable citizens' ordinarily reliable sense of who knows what.  Indeed, this part of science knowledge -- the part that involves making what's known by science known to curious people -- is woven into the sophisticated and successful craft of science journalists & related communicators.

4.  But I still think there are ways in which the use of scientific inquiry-- indeed, the incorporation of scientific tools, insights, methods into the craft of science journalism -- could make this sort of science communication better. One problem, which I address here, is that I think the resource of accessible and entertaining communications of what is known to science is not as readily available to all cultural groups in our society.  I am curious to know if you think I'm right in this hunch; surely you are in a position to tell me -- you are thoughtful & are dedicating your life to this field.



Three theories of why we see ideological polarization on facts

Explaining the phenomenon of political conflict over risk and related policy-consequential facts is, of course, the whole point -- or the starting point; the "whole point" includes counteracting it, too-- of the Cultural Cognition Project.

But what's being explained admits of diverse characterizations, and in fact tends to get framed and reframed across studies in a way that enables the basic conjecture to be probed and re-probed over & over from a variety of complementary angles (and supplementary ones too)

Yes, simple obsessiveness is part of what's going on. But so is validity. 

No finding is ever uniquely determined by a single theory. One makes a study as singularly supportive as possible of the "basic account" from which the study hypothesis derived. But corroboration of the hypothesis can't by itself completely rule out the possibility that something else might have generated the effect observed in a particular study.

The way to deal with this issue is not to argue until one's blue in the face w/ someone who says, "well, yes, but maybe it was ..."; rather it is to do another study, and another, and another, in which the same basic account is approached and re-approached from different related angles, enabled by slightly different reframings of the basic conjecture. Yes, in each instance, "something else" -- that is, something other than the conjecture you meant to be testing -- might "really"  explain the result. But the likelihood that "something else" was  "really" going on -- and necessarily something different every time; if there's one particular alternative theory that fits every single one of your results just as well as your theory, then you are doing a pretty bad job in study design! -- becomes vanishingly more remote as more and more studies that all reflect your conjecture reframed in another way way keep piling up.

The framing of the latest CCP study, Ideology, Motived Reasoning, and Cognitive Reflection, is a reframing in that spirit.  

The study presents experimental evidence that supports the hypotheses that ideologically motivated reasoning is symmetric or uniform across different systems of political values and that it increases in intensity as individuals' disposition to engage in conscious and systematic information processing-- as opposed to intuitive, heuristic-driven information processing-- increases.  

Those findings lend support to the "basic account" of cultural cognition: that political polarization over risk reflects the entanglement of policy-relevant facts in antagonistic social meanings; fixing the science communication problem, then, depends on disentangling meaning and fact.

But the story is told here as involving a competition between three "theories" of how dual-process reasoning, motivated cognition, and ideology relate to each other.  That story is meant to be interesting in itself, even if one hasn't tuned into all the previous episodes.

Here is the description of those theories in the paper; see if you can guess which one is really "cultural cognition"!

a. Public irrationality thesis (PIT). PIT treats dual-process reasoning as foundational and uses motivated cognition to explain individual differences in risk perception. The predominance of heuristic or System 1 reasoning styles among members of the general public, on this view, accounts for the failure of democratic institutions to converge reliably on the best available evidence as reflected in scientific consensus on issues like climate change (Weber 2006). Dynamics of motivated cognition, however, help to explain the ideological character of the resulting public controversy over such evidence. Many of the emotional resonances that drive system 1 risk perceptions, it is posited, originate in (or are reinforced by) the sorts of affinity groups that share cultural or ideological commitments. Where the group-based resonances that attach to putative risk sources (guns, say, or nuclear power plants) vary, then, we can expect to see systematic differences in risk perceptions across members of ideologically or culturally uniform groups (Lilienfeld, Ammirati, Landfield 2009; Sunstein 2007).

b.  Republican Brain hypothesis (RBH). RBH—so designated here in recognition of the synthesis constructed in Mooney (2012); see also Jost & Amado (2011)—treats the neo–authoritarian personality findings as foundational and links low-quality information processing and motivated cognition to them. Like PIT, RBH assumes motivated cognition is a heuristic-driven form of reasoning. The mental dispositions that the neo–authoritarian personality research identifies with conservative ideology—dogmatism, need for closure, aversion to complexity, and the like—indicate a disposition to rely predominantly on System 1. Accordingly, the impact of ideologically motivated cognition, even if not confined to conservatives, is disproportionately associated with that ideology by virtue of the negative correlation between conservativism and the traits of open-mindedness, and critical reflection—System 2, in Kahneman terms—that would otherwise check and counteract it (e.g., Mooney 2012;  Jost, Blaser, Kruglanski & Sulloway 2003; Kruglanski 2004; Thórisdóttir & Jost 2011; Feygina, Jost & Goldsmith 2010; Jost, Nosek & Gosling 2008).

It is primarily this strong prediction of asymmetry in motivated reasoning that distinguishes RBH from PIT. PIT does predict that motivated reasoning will be correlated with the disposition to use System 1 as opposed to System 2 forms of information processing. But nothing intrinsic to PIT furnishes a reason to believe that these dispositions will vary systematically across persons of diverse ideology.

c.  Expressive rationality thesis (ERT). ERT lays primary emphasis on identity-protective motivated reasoning, which it identifies as a form of information processing that rationally advances individual ends (Kahan, Peters, Wittlin, Slovic, Ouellette, Braman & Mandel 2012). The link it asserts between identity-protective cognition, so conceived, and dual-process reasoning creates strong points of divergence between ERT and both PIT and RBH.

One conception of “rationality” applies this designation to mental operations when and to the extent that they promote a person’s ends defined with reference to some appropriate normative standard. When individuals display identity-protective cognition, their processing of information will more reliably guide them to perceptions of fact congruent with their membership in ideologically or culturally defined affinity groups than to ones that reflect the best available scientific evidence. According to ERT, this form of information processing, when applied to the sorts of facts at issue in polarized policy disputes, will predictably make ordinary individuals better off. Any mistake an individual makes about the science on, say, the reality or causes of climate change will not affect the level of risk for her or for any other person or thing he cares about: whatever she does—as consumer, as voter, as participant in public discourse—will be too inconsequential to have an impact. But insofar as opposing positions on climate change have come to express membership in and loyalty to opposing self-defining groups, a person’s formation of a belief out of keeping with the one that predominates in hers could mark her as untrustworthy or stupid, and thus compromise her relationships with others. It is therefore “rational” for individuals in that situation to assess information in a manner that aligns their beliefs with those that predominate in their group whether or not those beliefs are correct—an outcome that could nevertheless be very bad for society at large (Kahan 2012b).

It is important to recognize that nothing in this account of the individual rationality of identity-protective cognition implies that this process is conscious. Indeed, the idea that people will consciously manage what they believe about facts in order to promote some interest or goal extrinsic to the truth of their beliefs reflects a conceptually incoherent (and psychologically implausible) picture of what it means to “believe” something (Elster 1983). Rather the claim is simply that people should be expected to converge more readily on styles of information processing, including unconscious ones, that promote rather than frustrate their individual ends. At least in regard to the types of risks and policy-relevant facts typically at issue in democratic political debate, ordinary people’s personal ends will be better served when unconscious modes of cognition reliably focus their attention in a manner that enables them to form and maintain beliefs congruent with their identity-defining commitments. They are thus likely to display that form of reasoning at the individual level, whether or not it serves the collective interest for them to do so (Kahan et al. 2012).

Individuals disposed to resort to low-level, System 1 cognitive processing should not have too much difficulty fitting in. Conformity to peer influences, receptivity to “elite” cues, and sensitivity to intuitions calibrated by the same will ordinarily guide them reliably to stances that cohere with and express their group commitments.

But if individuals are adept as using high-level, System 2 modes of information processing, then they ought to be even better at fitting their beliefs to their group identities. Their capacity to make sense of more complex forms of evidence (including quantitative data) will supply them with a special resource that they can use to fight off counterarguments or to identify what stance to take on technical issues more remote from ones that that figure in the most familiar and accessible public discussions.

ERT thus inverts the relationship that PIT posits between motivated cognition and dual-process reasoning. Whereas PIT views ideological polarization as evidence of a deficit in System 2 reasoning capacities, ERT predicts that the reliable employment of higher-level information processing will magnify the polarizing effects of identity-protective cognition (Kahan et al. 2012).

Again, the argument is not that such individuals will be consciously managing the content of their beliefs. Rather it is that individuals who are disposed and equipped to make ready use of high-level, conscious information processing can be expected to do so in the service of their unconscious motivation to form and maintain beliefs that foster their connection to identity-defining groups.

ERT’s understanding of the source of ideologically motivated reasoning also puts it into conflict with RBH. To begin, identity-protective cognition—the species of motivated reasoning that ERT understands to be at work in such conflicts—is not a distinctively political phenomenon. It is likely to be triggered by other important affinities, too—such as the institutional affiliations of college students or the team loyalties of sports fans. Unless there is something distinctive about “liberal” political groups that makes them less capable of underwriting community attachment than all other manner of group, it would seem odd for motivated reasoning to display the asymmetry that RBH predicts when identity-protective cognition operates in the domain of politics.

In addition, because RBH, like PIT, assumes motivated reasoning is a feature of low-level, System 1 information processing, ERT calls into question the theoretical basis for RBH’s expectation of asymmetry. Like PIT, ERT in fact suggests no reason to believe that low-level, System 1 reasoning dispositions will be correlated with ideological or other values. But because ERT asserts that high-level, System 2 reasoning dispositions magnify identity-protective cognition, the correlations featured in the neo–authoritarian-personality research would, if anything, imply that liberals—by virtue of their disposition to use systematic reasoning—are all the more likely to succeed in resisting evidence that challenges the factual premises of their preferred policy positions. Again, however, because ERT is neutral on how System 1 and System 2 dispositions are in fact distributed across the population, it certainly does not entail such a prediction.



What does "Lincoln" mean?

Saw Lincoln last night.

On 2d try. The 1st time theater was sold out.  

So this time I got my tickets in advance. But because I made the "mistake" of showing up 10 mins after "screen time" -- still a good 15 mins before end of the annoying previews -- I had to sit in the first row of an overpacked theater.   Overpacked w/ ordinary people in an ordinary central CT "suburban" (what passes for that in CT) multiplex.

The full house burst into applause at end, and the theater didn't empty until all the credits had run out...

The movie was beautiful & moving.

Learning (the hard way?) that so many other people -- ones with nothing particular in common with me except for belonging to the same vast and vastly diverse society -- also found the movie so beautiful & moving was even more so.



Okay, now *this* is the Liberal Republic of Science!

Gets the point across so much more succinctly-- and inspiringly!



A surprising (to me) discovery: reflective Independents...

The analyses I did for my latest paper—on ideology, cognitive reflection, and motivated reasoning—really surprised me in one respect.

They didn’t surprise me altogether. Indeed, they corroborated the hypothesis (one that also was explored in the CCP study of science comprehension & climate change polarization) that people who are more disposed to use System 2 reasoning (conscious, analytical, reflective) are more likely to to selectively credit or discredit evidence in patterns that fit their ideological predispositions.  This is contrary to how most people thing heuristic-driven, System 1 reasoning contributes to public confusion and controversy on issues like climate change.

But what did surprise me was the finding that self-identified Independents are more reflective—more disposed to use System 2 rather than System 1 reasoning.  I assumed people who were in middle were just less reflective.  The difference isn’t huge (and actually, no one, of any particular political orientation or non-orientation demonstrates a high degree of reflection on Shane Frederick's gold-standard CRT test), but it’s there.

It also follows from the analyses that are in the paper that Independents display less motivated reasoning than partisans. Of course, that’s sort of a logical thing; if they don’t have a predisposition, they can’t be fitting their interpretation of evidence to it. But I think there’s more to it than that.

Why am I surprised? My experience in doing studies has caused me to form the impression that people who are “in the middle” on measures of cultural or ideological predispositions are sort of like statistical noise—random, unreliable--& not that important for figuring out what is going on, at least if the signal you get from people w/ a more choate sense of identity is a clear one.

Well, it looks anyway, like the Independents are not simply inert or confused. They are reflective people, engaging information of political significance in a non-ideological way.  That’s something to try to figure out, not dismiss.  What are they thinking? Who the hell are they?

At this point, I’m not suffering any great intellectual crisis.  I suspect if I thoughtfully engage the data a bit more, I’ll discover something that, without necessarily making this finding unimportant, reveals that it it poses no particular problem for the basic hypothesis behind the study (which is that individuals rationally engage information on societal risks in a manner that reflects their interest in forming and maintaining group connections).

But I’m curious. Also a bit excited and anxious; maybe I’m missing something really important.

What I’m going to do for now is think for a bit. Also read and re-read some other things (including John Bullock's great study on need for cognition and partisanship).  And try to form some interesting hypotheses about what the “Reflective Independent” datum might mean. Then I’ll see if there is a way to test those hypotheses, at least provisionally, with this data set.

Like I said, I don’t think I’ll find anything here that makes me think I have to adjust my thinking in a major way. But I want to approach this minor nuggest of surprise in a way that wouldn’t obscure the possibility that just beneath it is a deep deposit of information that would liberate me from the intellectually destitute state of unrecognized ignorance.

So to start this inquiry:

Do you have ideas about this little datum? What to make of it; how to explore its signficance?  

Fine to tell me, too, if you think this was “obvious” for reason x, y, & z; but do realize that you could have been assigned to one condition in a “many worlds” experimental design that includes another condition in which my doppelgänger has just blogged, “See! Independents are less reflective!,” and in which yours is typing up the response, “Of course, that was obvious!


New paper: Ideology, Motivated Reasoning, and Cognitive Reflection: An Experimental Study

Okay, here's the paper I mentioned yesterday. 

I want to make this as good as it can be, so comments please (either in comments field or to me by email).

 Ideology, Motivated Reasoning, and Cognitive Reflection: An Experimental Study


Social psychologists have identified various plausible sources of ideological polarization over climate change, gun violence, national security, and like societal risks. This paper reports a study of three of them: the predominance of heuristic-driven information processing by members of the public; ideologically motivated cognition; and personality-trait correlates of political conservativism. The results of the study suggest reason to doubt two common surmises about how these dynamics interact. First, the study presents both observational and experimental data inconsistent with the hypothesis that political conservatism is distinctively associated with closed-mindedness: conservatives did no better or worse than liberals on an objective measure of cognitive reflection; and more importantly, both demonstrated the same unconscious tendency to fit assessments of empirical evidence to their ideological predispositions. Second, the study suggests that this form of bias is not a consequence of overreliance on heuristic or intuitive forms of reasoning; on the contrary, subjects who scored highest in cognitive reflection were the most likely to display ideologically motivated cognition. These findings corroborated the hypotheses of a third theory, which identifies motivated cognition as a form of information processing that rationally promotes individuals’ interests in forming and maintaining beliefs that signify their loyalty to important affinity groups. The paper discusses the normative significance of these findings, including the need to develop science communication strategies that shield policy-relevant facts from the influences that turn them into divisive symbols of identity. 



Download paper


Some opinionated reflections on design of motivated reasoning experiments

Am tuning up a working paper--"Ideology, Cognitive Reflection, & Motivated Reasoning" -- that reports experiment results relating to "ideological symmetry" of motivated cognition as well as the relationship between motivated cognition & dual-process reasoning theories. Probably post in day or 2 or 3...

But here's a piece of the paper. It comes out of the methods section & addresses issues relating to design for motivated-reasoning experiments:

Testing for vulnerability to motivated reasoning is not straightforward. Simply asking individuals whether they would change their mind if shown contrary evidence, e.g., is inconclusive, because motivated reasoning is unconscious and thus not reliably observed or checked through introspection.

Nor is it satisfactory simply to measure reasoning dispositions or styles—whether by an objective performance test, such as CRT, or by a subjective self-evaluation one, like Need For Cognition. None of these tests has been validated as a predictor of motivated cognition. Indeed, early work in dual-process reasoning theory—research predating Kahneman’s “System 1”/“System 2” framework—supported the conclusion that motivated reasoning can bias higher-level or “systematic” information processing as well as lower-level, heuristic processing (Chen, Duckworth & Chaiken 1999; Giner-Sorolla & Chaiken 1997).

For these reasons, experimental study is more satisfactory. Nevertheless, proper experiment design can be complicated too.

One common design involves furnishing subjects who disagree on some issue (e.g., climate change or the death penalty) with balanced information and measures whether they change their positions. The inference that they are engaged in motivated reasoning if they do not, however, is open to dispute. For one thing, the subjects might have previously encountered equivalent information outside the context of the experiment; being exposed to the same information again would not furnish them with reason to alter their positions no matter how open-mindedly they assessed it. Alternatively, subjects on both sides of the issue might have given open-minded consideration to the evidence—indeed, even given it exactly the same weight—but still failed to “change their minds” or to reach a shared conclusion because of how strongly opposed their prior beliefs were before the experiment.

Variants of this design that assess whether subjects of opposing ideologies change their positions when afforded with counter-attitudinal information on different issues are even more suspect. In those instances, it will not only be unclear whether subjects who stuck to their guns failed to afford the information open-minded consideration. It will also be unclear whether the counter-attitudinal information supplied respectively to the opposing sides was comparable in strength, thereby defeating any inference about the two groups’ relative disposition to engage in motivated reasoning.

It is possible to avoid these difficulties with an experimental manipulation aimed at changing the motivational stake subjects have in crediting a single piece of evidence. In Bayesian terms, the researcher should be measuring neither subjects' priors nor their posteriors but instead their likelihood ratios--to determine whether subjects will opportunistically adjust the significance they assign to information in a manner that promotes some interest or goal collateral to making an accurate judgment.

For example, subjects of diverse ideologies can be instructed to determine whether a demonstrator in a video—represented in one condition as an “anti-abortion protestor” and in another an “gay-rights protestor”—“blocked” or “screamed in the face” of a pedestrian trying to enter a building. If the perceptions of subjects vary in a manner that reflects the congeniality of the protestors’ message to the subjects’ ideologies, that would be convincing evidence of motivated reasoning. If the film of the protestors’ behavior is itself evidence relevant to some other issue—whether, say, the protestors broke a law against use of “coercion” or “intimidation”—then the impact of ideologically motivated reasoning will necessarily be biasing subjects’ assessment of that issue in directions congenial to their ideologies (Kahan, Hoffman, Evans, Braman & Rachlinski 2012).

In such a design, moreover, it is the subjects’ ideologies rather than their priors that are being used to predict their assessments of evidence conditional on the experimental manipulation. This element of the design bolsters the inference that the effect was generated by ideological motivation rather than a generic form of confirmation bias (Kahan, Jenkins-Smith & Braman 2011).

Such a design also enables straightforward testing of any hypothesized asymmetry in motivated reasoning among subjects of opposing ideologies. The corroboration of motivated reasoning in this design consists of the interaction between the experimental manipulation and subjects’ ideology: the direction or magnitude of the weight assigned to the evidence must be found to be conditional on the manipulation, which determined the congruence or noncongruence of the evidence with subjects’ ideologies. The hypothesis that this effect will be asymmetric—that it will, say, be greater among more conservative than liberal subjects, as RHB would assert—is equivalent to predicting that the size of the interaction will vary conditional on ideology. Such a hypothesis can be tested by examining whether a polynomial model—one that posits a “curved” rather than a “linear” effect—confirms that the magnitude of the interaction varies in the manner predicted and furnishes a bitter fit than a model that treats such an effect as uniform across ideology (Cohen, Cohen, West & Aiken 2003).


"I endorse this message ...": video of lecture bits

I gave a talk on Nov. 2 for Spitfire Communications, an organization that counsels groups that fund and participate in public-advocacy communications. Subject was "science communication problem," and argument was that it can be solved only by integrating evidence-based methods into science communications practice. Slides here.

Spitfire has posted the talk. What's more, they've posted a set of expertly edited excerpts (45 secs to 3 mins in length), each of which addresses a discrete theme. They seem almost like political-campaign advertisement spots -- although obviously, I'd have to be running for a very strange office & making a case to a very unusual sort of electorate... But a testament to their editing skills that they were able to catch me uttering single sentences less than 5 mins in length!



Full lecture

On motivated reasoning

On identity-protective cognition

On "public irrationality" thesis

Why spokespeople matter

"Closing thoughts" on the science communication problem


Giving thanks for ... Popper!

true, just got big dose of Popper in the Liberal Republic of Science series, but gratitude to him can't be overstated, right?