follow CCP

Recent blog entries
popular papers

What Is the "Science of Science Communication"?

Climate-Science Communication and the Measurement Problem

Ideology, Motivated Cognition, and Cognitive Reflection: An Experimental Study

'Ideology' or 'Situation Sense'? An Experimental Investigation of Motivated Reasoning and Professional Judgment

A Risky Science Communication Environment for Vaccines

Motivated Numeracy and Enlightened Self-Government

Ideology, Motivated Cognition, and Cognitive Reflection: An Experimental Study

Making Climate Science Communication Evidence-based—All the Way Down 

Neutral Principles, Motivated Cognition, and Some Problems for Constitutional Law 

Cultural Cognition of Scientific Consensus
 

The Tragedy of the Risk-Perception Commons: Science Literacy and Climate Change

"They Saw a Protest": Cognitive Illiberalism and the Speech-Conduct Distinction 

Geoengineering and the Science Communication Environment: a Cross-Cultural Experiment

Fixing the Communications Failure

Why We Are Poles Apart on Climate Change

The Cognitively Illiberal State 

Who Fears the HPV Vaccine, Who Doesn't, and Why? An Experimental Study

Cultural Cognition of the Risks and Benefits of Nanotechnology

Whose Eyes Are You Going to Believe? An Empirical Examination of Scott v. Harris

Cultural Cognition and Public Policy

Culture, Cognition, and Consent: Who Perceives What, and Why, in "Acquaintance Rape" Cases

Culture and Identity-Protective Cognition: Explaining the White Male Effect

Fear of Democracy: A Cultural Evaluation of Sunstein on Risk

Cultural Cognition as a Conception of the Cultural Theory of Risk

« Vaccine risk perceptions and risk communication: study conclusions & recommendations | Main | Finally: decisive, knock-down, irrefutable proof of the ideological symmetry of motivated reasoning »
Wednesday
Apr092014

More on "Krugman's symmetry proof": it's not whether one gets the answer right or wrong but how one reasons that counts 

Okay, I've finally caught my breath after laughing myself into state of hyperventilation as a result of reading Krugman's latest proof (this is actually a replication of an earlier empirical study on his part) that ideologically motivated reasoning is in fact perfectly symmetric with respect to right-left ideology.

Rather than just guffawing appreciatively, it's worth taking a moment to call attention to just how exquisitely self-refuting his "reasoning" is!

There's the great line, of course, about how his "lived experience" (see? I told you, he's doing empirical work!) confirms that motivated cognition "is not, in fact, symmetric between liberals and conservatives."

But what comes next is an even more subtle -- and thus an even more spectacular! -- illustration of what it looks like when one's reason is deformed by tribalism: 

Yes, liberals are sometimes subject to bouts of wishful thinking. But can anyone point to a liberal equivalent of conservative denial of climate change, or the “unskewing” mania late in the 2012 campaign, or the frantic efforts to deny that Obamacare is in fact covering a lot of previously uninsured Americans?

Uh, no, PK. I mean seriously, no.

The test for motivated cognition is not whether someone gets the "right" answer but how someone assesses evidence.

A person displays ideologically motivated cognition when, instead of weighing evidence based on criteria related to its connection to the truth, he or she credits or dismisses it based on its conformity to his or her ideological predispositions.

Thus, if we want to use public opinion on some issue -- say, climate change -- to assess the symmetry of ideologically motivated reasoning, we can't just say, "hey, liberals are right, so they must be better reasoners."

Rather we must determine whether "liberals" who "believe" in climate change differ from "conservatives" who "don't" in how impartially they weigh evidence supportive of & contrary to their respective positions. 

How might we do that?  

Well, one way would be to conduct an experiment in which we manipulate the ideological motivation people with "liberal" & "conservative" values have to credit or dismiss one and the same piece of valid evidence on climate change.  

If "liberals" (it makes me shudder to participate in the flattening of this term in contemporary political discourse) adjust the weight they give this evidence depending on its ideological congeniality, that would support the inference that they are assessing evidence in a politically motivated fashion.  

If in aggregate, in the real world, they happen to "get the right" answer, then they aren't to be commended for the high quality of their reasoning.  

Rather, they are to be congratulated for being lucky that a position they unreasoningly subscribe to happens to be true.

And vice versa if the "truth" happens (on this issue or any other) to align with the position that "conservatives" unreasoningly affirm regardless of the quality of the evidence they are shown.

That Krugman is too thick to see that one can't infer anything about the quality of partisans' reasoning from the truth or falsity of their beliefs is ... another element of Krugman's proof that ideological reasoning is symmetric across right and left!

For in fact, "the 'other side' is closed-minded" is one of the positions that partisans are unreasoningly committed to. 

One of the beliefs that they don't revise in light of valid evidence but rather use in lieu of truth-related criteria to assess the validity of whatever evidence they see.

This proposition is supported by real, honest-to-god empirical evidence -- of the sort collected precisely because no one's personal "lived experience" is a reliable guide to truth.

That PK is innocent of this evidence is-- another element of his proof that ideological reasoning is symmetric across right and left!

As is his unfamiliarity with studies that use the design I just suggested to test whether "liberals" are forming their positions on climate change and other issues in a manner that is free of the influence of politically motivated reasoning.  Not surprisingly, these studies suggest the answer is no.

But does that mean that all liberals who believe in climate change believe what they do because of ideologically motivated cognition? Or that only someone who is engaged in that particular form of defective reasoning would form that belief?

If you think so, then, despite your likely ideological differences, you & Paul Krugman have something in common: you are both very poor reasoners.

PrintView Printer Friendly Version

EmailEmail Article to Friend

References (1)

References allow you to track sources for this article, as well as articles that were written in response to this article.

Reader Comments (66)

Like you say, it's not just a question of whether you get the answer right or wrong - but of *why* you think so, and whether your method was rational/scientific. As I've been saying of all these surveys and studies of scientific knowledge and the effect of various factors on it - they all ask what you believe, they none of them ask the subjects *Why?*.

But there is another obvious flaw he misses. Given that the theory predicts that both sides think they're right and the other side is wrong, how does thinking you're right and the other side is wrong disprove this theory? Maybe - for example - liberals are *wrong* about global warming, and global warming is in fact the example he asks for of a bit of scientific knowledge that liberals generally get wrong?

Both sides symmetrically are predicted to believe there is asymmetry, so either side seeing asymmetry confirms the prediction. I can confirm that, seeing the symmetrically reflected asymmetry from this side, Krugman's comments are even funnier!

-

I'm very impressed that you part persuaded Chris Mooney. I tried for years, and only got the point through to him a couple of times. Maybe there *is* hope.

April 9, 2014 | Unregistered CommenterNiV

A strong form of democratic theory argues that factual knowledge itself is an important component to citizens being able to form opinions and hold political leaders accountable (See Delli Carpini & Keeter 1992). I agree that people may get to facts through suspect reasoning, through simply accepting information from a trusted elite, or through a more thorough process of understanding an issue and accepting true facts, and these processes matter. However, as an empirical matter, I think that when you're testing motivated reasoning, it's useful to have some "true facts" as a benchmark to measure whether the reasoning that people use is biased. It is true whether or not you like it that the current unemployment rate is 6.7% and that it's fallen steadily over the last 4 years, and people should be able to reflect that fact back on a survey even as they disagree about what to do about unemployment. I don't necessarily agree with Krugman that motivated reasoning is asymmetrical, but one way of testing that would require presenting facts or asking people to evaluate a situation that has a "right answer" that would be uncomfortable to people with strong ideology or partisan loyalties on both sides and seeing their reactions. The Bartels piece here looks at whether partisans can accurately reflect the state of the economy both after the Reagan and Clinton presidencies and sees no asymmetries in bias (http://link.springer.com/article/10.1023/A:1021226224601#page-1).

April 10, 2014 | Unregistered CommenterShana Gadarian

@Shana:

I agree w/ everything you say. There is value, all else equal, in citizens having an apprehension of the facts -- or beliefs consistent with the best evidence. Of course, they are more likely to converge on the best evidence if they reason well.

I agree too that a powerful way to test symmetry of motivated reasoning would be to design an experiment in which one manipulates the ideological significance of a problem that has a "right" answer & see ir partisans "get the right" answer conditional on it being ideologically congenial.

We do exactly that in Motivated Numeracy & Enlightened Self-Government.

That was the paper that was featured in the Ezra Klein article & that Krugman dismisses because it is contrary to his "lived experience." Krugman's dismissal of evidence that those who agree with him are as subject to bias as those who don't is also a form of motivated reasoning that has been demosntrated experimentally.

I hope someday everyone will recognize the poisonous effect of Krugman's style of discourse on the quality of our political culture.

April 10, 2014 | Registered CommenterDan Kahan

"Krugman's dismissal of evidence that those who agree with him are as subject to bias as those who don't is also a form of motivated reasoning that has been demosntrated experimentally."

I don't know. I thought his reasoning on that part was actually reasonable, given the incorrect premises he was using resulting from earlier motivated reasoning. What he's asking is, if conservatives and liberals are equally inclined to bias, why are liberals right and conservatives wrong about everything? Where are the scientific topics that liberals are wrong about?

He's taken the hypothesis, predicted consequences, and then attempted to examine reality to see if the predictions are true. He's got a puzzle, because he was expecting to be able to list several liberal equivalents of evolution and climate change and he couldn't. So far as the hypothesis itself goes, he's done exactly the right thing. He hasn't rejected it out of hand because "liberals can't be biased". He hasn't tried to pick holes in the method, or call the authors cranks. He calls it "a genuine intellectual puzzle", that he is "troubled" by.

The problem is that he hasn't followed through all the consequences of the hypothesis - possibly because it's a consequence that you have been a little reticent about promoting. (And for understandable reasons.) It implies it's very likely that on some of the battleground issues where liberals and conservatives disagree, the conservatives are right and the liberals are wrong and the liberals can't see it.

It's true people can reach the right conclusion for the wrong reasons - as you say they can just get lucky - but is it credible that they got lucky on every single one of the battleground issues? His argument is not simply that liberals are factually right and conservatives factually wrong on climate change, but that there is no major scientific topic he knows of like climate change in which things are vice versa.

(Of course, since I think climate change is such a topic, you can just imagine how amusing I think that reasoning is.)

The problem is that in testing the claim that liberals are equally subject to biased reasoning, he's taking his liberal standpoint as the baseline "truth" to compare its predictions against. That some of this must be wrong as well is an additional consequence of the theory that hasn't been heavily emphasised so far.

In reasoning through consequences to come to a conclusion on something, we normally assume that our own reasoning is correct. It's a much harder problem to work out how to achieve valid reasoning using an invalid reasoner, like trying to use a broken calculator that sometimes gives wrong answers to do sums. If it's your only means of calculating, what are you supposed to check your calculations against to test them?

April 11, 2014 | Unregistered CommenterNiV

Definitely agree on the points you make here. But isn't Krugman actually putting forward a separate idea, that hasn't been tested by your research?

This is the idea that, while liberals and conservatives as individuals might have equal tendencies for motivated reasoning, something in the cultural/discourse norms of conservatives makes it possible for more extreme beliefs to become widespread and "normal".

So if you put an individual liberal or conservative in the experimental situation - as your studies did - they perform equally badly. But if you try and popularize a crazy idea (i.e. disprovable via relatively obvious facts) widely, among the broader population, something about conservative discourse culture/norms makes it more likely to take root and be seen as tolerable, whereas something about liberal culture makes it more likely to be shut down and denounced as ridiculous.

I'm not sure I agree with this idea, but just thought I would point out it's a different idea than the individual-level cognitive processes your studies looked at.

April 11, 2014 | Unregistered CommenterNeil Stenhouse

"... something about conservative discourse culture/norms makes it more likely to take root and be seen as tolerable, whereas something about liberal culture makes it more likely to be shut down and denounced as ridiculous."

Hmm. Conservatives are more liberal and liberals are more intolerant?

For the sake of argument, it's worth considering. But I'm dubious, because not only are people's tendencies towards motivated reasoning symmetrical, their perceptions of the outcome are symmetrical, too. Conservatives believe that liberals are more inclined to believe crazy, easily disprovable things: that they have an unscientific tendency to trust socially approved authorities, that they're more intolerant of dissenting views and so more inclined to lock themselves into groupthink, that they prefer simplistic explanations with authoritarian solutions, and that they're wildly inconsistent, able to take opposing positions on fundamental principles simultaneously, depending on what is being attacked/defended. And that one of their crazier notions is the strength of their belief in their own infallibility, that they are right and conservatives are wrong about everything, which conservatives find ironic given the long list of things the left have been tragically wrong about.

But of course all this is to be expected, given that the prediction of the symmetry hypothesis is that *both* sides will see the situation as asymmetric.

The test of this idea - that conservatives are culturally more inclined to accept crazy notions - is to test it on scientific statements that are *not* political battlegrounds. Dan's done that on numerous topics, and found that they're not. Left and Right are equally intelligent, equally sensible, equally scientifically literate, cognitively reflective, numerate, etc. And we appear to be equally fallible - a lot of those results record the often shockingly poor understanding the general public on all sides have of science. We only notice it, and care about it, on certain topics where politics gets involved, but its a very widespread phenomenon.

It is for example shocking how many of the people who pride themselves on their scientific belief in the impending global warming catastrophe have no idea even how the greenhouse effect actually works. Start asking them technical questions about adiabatic lapse rates and the differential equations for two-box models, and they stare at you blankly. (Before anyone says anything, most sceptics are not much better.) So they're not believing as the result of any personal scientific or technical knowledge, having looked at the arguments and evidence and coming to their own judgement. They believe because they've heard that's what "science" says, and people who believe what science says are intellectually superior to people who don't. They follow the herd.

Sometimes that works, and sometimes the whole herd is "following the herd". It is in any case a perfectly symmetrical mechanism that doesn't give any a priori reason to think either side is any better than the other. Each side sees the other in very similar terms. We know our perceptions are biased by being on one side or the other, so we can't tell just by looking. Why should we believe in a hypothetical difference by an unknown mechanism we can't explain and that we have no evidence of, merely to maintain a comfortable (and unconvincingly convenient) hypothesis that all the evidence we do have stands against? It sounds very much like motivated reasoning, yes? Which is of course further evidence that we're really all the same.

April 11, 2014 | Unregistered CommenterNiV

I do not believe Krugman is a poor reasoner. That is a dangerous claim since it discounts the intentions side of the argument. Krugman is in fact a very smart and educated reasoner who is simply willing and talented enough to lie and cheat convincingly enough when he thinks the ends do justify the means.

The argument I have against Krugman is not on scientific or reasoning grounds but on moral ones. The man is not stupid, silly or funny. He's evil.

I am aware that making claims about intentions is dangerous grounds. But in the case of Krugman his life and work do demonstrate knowledge and intellect. He is not one to be able to claim ignorance credibly.

April 12, 2014 | Unregistered CommenterZK

With an eye to the larger picture, do you not concede that there is both journalistic and academic incentive in the context of domestic politics to adopt a 'pox on both houses' stance (roughly on par with incentive to arrive at 'counter-intuitive' results)? Presumably one should be interested both in how (likely to be) accurate and in how (subjectively) well-grounded various held beliefs turn out to be. But, unless one takes care to clearly distinguish the two, there is a legitimate concern that, by focusing on the apparent symmetry as to the latter, one might imply to one's audience that symmetry exists as to the former? In fairness, I think it's Krugman's failure to separate these two that leads to his response. However, his response is one that has, in my assessment, been conditioned by bad tendencies in journalism and scholarship -- whether in terms of accuracy of analysis or skewing of research priorities -- that are quite real and quite dangerous. And the existence of those bad tendencies might place a burden on advocates of symmetry of cognitive bias to make clear that they are not suggesting symmetry as to (apparent) accuracy of held beliefs (which, it seems, does not exist).

April 17, 2014 | Unregistered CommenterRDD

The herd following the herd leads to a stampede. Or maybe the proverbially lemmings off a cliff.

That sort of "thinking" doesn't work well for anyone or any group.

Unfortunately, the educational system in the country seems to be doing it's best to turn out more and more graduates who "think" like that than not.

Krugman's just a pointer to the symptom like a canary in a coal mine.

April 17, 2014 | Unregistered Commenterjakee308

Some of these comments are fantastic!

but is it credible that they got lucky on every single one of the battleground issues?

hahahahahaha

April 17, 2014 | Unregistered CommenterJ. Locke

@Shana
You're wrong about unemployment. What you fail to grasp is that part of the reason it has fallen is because some people have given up looking for employment (some have done so because they retired and some have simply given up on finding a job. This is to say nothing of how you don't comprehend certain economic pronciples such as how there are often peaks and valleys. There is a more than substantial possibility that this is just a peak in economic activity.

Also Global Warming is busted. It's climate change now. Nothing like complete vagueness to make a theory both plausible and unfalsifiable while still calling it a theory.

April 17, 2014 | Unregistered CommenterBen

What we can do in order to make ourselves always right on an issue is to assign an indefensible position to the other side. Call it "conservative denial of climate change" rather than the belief that man contributes little to any factors that might cause change. Say that conservatives don't believe Obamacare is helping people when virtually nobody thinks that.

There are many liberals who believe the 2000 election was stolen--I don't need to assign any false beliefs to them to believe myself that they are incorrect.

Anyway, Krugman assumes any academic writing on this subject is just too polite to say what everybody knows--conservatives are dumb--so he'll come out and say it. Doing so, he demonstrates exactly what Kahan was really proving.

April 17, 2014 | Unregistered Commenterspongeworthy

NiV, liberals are certainly as ideologically blinded about nuclear power and genetically modified organisms (GMO) in food as conservatives are about climate change.

April 17, 2014 | Unregistered Commenterbrian

One element of advocacy regarding 'Global Warming' is frequently overlooked is that we do not have two 'symmetric' sets of consequences to the results of the debate. In this case the advocates for action, the 'Liberals' are demanding we 'do something' that imposes great costs. The 'Conservatives' are not advocating for action, many are not even advocating inaction, but are in fact being demonized for simply requiring the Liberal advocates prove their case. Certainly the burden is upon those that advocate such costly change to prove the benefits and illustrate those benefits out weigh the enormous costs... costs that always seem to provide such great rewards to the advocates for those expenses.

April 17, 2014 | Unregistered CommenterDANEgerus

"Climate change" is an interesting case because it is precisely how you evaluate evidence that is most important, due to the low signal-to-noise ratio in everything from the paleo climate record to future climate projections to impact studies -- even the instrumental temperature record has so much error that adjustments since 2000 have turned the U.S. twentieth-century cooling trend into a warming trend. In such situations it is exceptionally easy to find the data to support one's preconceived or partisan notions.

Even the basic terms are fraught with conflations and confusions. When someone says "97% of scientists believe in manmade climate change" are they referring to post-LIA warming trend, CO2 levels, the effects of aerosols, the accuracy of IPCC projections, strong CO2 feedbacks, the need for emissions control policies? Some of these even virtually all skeptics agree with, some of them are controversial even among believers.

"Where are the scientific topics that liberals are wrong about?" GMOs, nuclear power, resource depletion, economic incentive effects, racial crime statistics, heck scientific epistemology itself is viewed as an arbitrary social construct by much of the left.

April 17, 2014 | Unregistered CommenterTallDave

This is fascinating stuff.

I'm a libertarian who hates both parties. What does that make me?

My sense is that cl