follow CCP

Recent blog entries
popular papers

What Is the "Science of Science Communication"?

Climate-Science Communication and the Measurement Problem

Ideology, Motivated Cognition, and Cognitive Reflection: An Experimental Study

'Ideology' or 'Situation Sense'? An Experimental Investigation of Motivated Reasoning and Professional Judgment

A Risky Science Communication Environment for Vaccines

Motivated Numeracy and Enlightened Self-Government

Ideology, Motivated Cognition, and Cognitive Reflection: An Experimental Study

Making Climate Science Communication Evidence-based—All the Way Down 

Neutral Principles, Motivated Cognition, and Some Problems for Constitutional Law 

Cultural Cognition of Scientific Consensus

The Tragedy of the Risk-Perception Commons: Science Literacy and Climate Change

"They Saw a Protest": Cognitive Illiberalism and the Speech-Conduct Distinction 

Geoengineering and the Science Communication Environment: a Cross-Cultural Experiment

Fixing the Communications Failure

Why We Are Poles Apart on Climate Change

The Cognitively Illiberal State 

Who Fears the HPV Vaccine, Who Doesn't, and Why? An Experimental Study

Cultural Cognition of the Risks and Benefits of Nanotechnology

Whose Eyes Are You Going to Believe? An Empirical Examination of Scott v. Harris

Cultural Cognition and Public Policy

Culture, Cognition, and Consent: Who Perceives What, and Why, in "Acquaintance Rape" Cases

Culture and Identity-Protective Cognition: Explaining the White Male Effect

Fear of Democracy: A Cultural Evaluation of Sunstein on Risk

Cultural Cognition as a Conception of the Cultural Theory of Risk

« What does "believing/disbelieving in" add to what one knows is known by science? ... a fragment | Main | "Repugnance" & reasoned deciscionmaking ... a fragment »

"According to climate scientists ..." -- WTF?! (presentation summary, slides)

Gave talk at the annual Association for Psychological Science on Sat.  Was on a panel that featured great presentations by Leaf Van Boven, Rick Larrick & Ed O'Brien. Maybe I'll be able to induce them to do short guest posts on their presentations, although understandably, they might be shy about become instant world-wide celebrities by introducing their work to this sites 14 bilion readers.

Anyway, my talk was on the perplexing, paradoxical effect of "according to climate scientists" or ACS prefix (slides here).

As 6 billion of the readers of this blog know-- the other 8 have by now forgotten b/c of all the other cool things that have been featured on the blog since the last time I mentioned this--attributing positions on the contribution of human beings to global warming, and the consequences thereof, to "climate scientists" magically dispels polarization on responses to cliimate science literacy questions.

Here's what happens when "test takers" (members of a large, nationally representative sample) respond to two such items that lack the magic ACS prefix:

Now, compare what happens with the ACS prefix:

Does this make sense?

Sure. Questions that solicit respondents’ understanding of what scientists believe about the causes and consequences of human-caused global warming avoid forcing individuals to choose between answers that reveal what they know about what science knows, on the one hand, and ones that express who they are as members of cultural groups, on the other.

Here's a cool ACS prefix corollary:

Notice that the "Nuclear power" question was a lot "harder" than the "Flooding" one once the ACS prefix nuked (as it were) the identity-knowledge confound.  Not surprisingly, only respondents who scored the highest on the Ordinary Science Intelligence assessment were likely to get it right.

But notice too that those same respondents--the ones highest in OSI--were also the most likely to furnish the incorrect identity-expressive responses when the ACS prefix was removed.

Of course! They are the best at supplying both identity-expressive and  science-knowledge-revealing answers.  Which one they supply depends on what they are doing: revealing what they know or being who they are. 

The ACS prefix is the switch that determines which of those things they use their reason for.

Okay but what about this: do rspts of opposing political ordinations agree on whether climate scientists agree on whether human-caused climate change is happening?

Of course not!

In modern liberal democratic societies, holding beliefs contrary to the best available scientific evidence is universally understood to be a sign of stupidity. The cultural cogniton of scientific consensus describes the psychic pressure that members of all cultural groups experience, then, to form and persist in the belief that their group’s position on a culturally contested issue is consistent with the best avaialbel scientific evidence.

But that's what creates the "WTF moment"-- also known as a "paradox":

Um ... I dunno!

That's what I asked the participants--my fellow panelists and the audience members (there were only about 50,000 people, because were scheduled against some other pretty cool panels) to help me figure out!

They had lots of good conjectures.

How about you?

PrintView Printer Friendly Version

EmailEmail Article to Friend

Reader Comments (15)

The phrasing is important. "Most experts" vs "According to climate scientists". The second form suggests an objectively aggregated value or collective. The first form allows the participant to filter/select from those members in an subjective manner. Naturally, the experts criteria will co-align with their identity beliefs and is just going to collapse back down into an identity expression.

May 31, 2016 | Unregistered CommenterT. Chen

At first glance it seems asking someone to mark True/False vs asking if they agree or disagree could have an impact. Asking if something is True/False seems to be putting the focus on the thing/question. Asking if you agree or disagree is putting the focus on them. Are you the kind of person that agrees with this? I can feel the impulse to disagree with something just because I don't like it, even if I think it's true. That is, I want to say it disagrees with me, it doesn't sit well with me, my narrative, my sense of the world, maybe even my team.

Second, to add to what T. Chen said very well, the difference in "according to climate scientists" and "most expert climate scientists" could be significant. Calling them "experts" is kind of an ambiguous distinction. Are there climate scientists that are not experts? As a result I get the feeling that 'expert' is making this a loaded question, sort of reminding me that these people are experts, which I then want to rebel against, or at least it sets off my spidey-sense. The phrase is generally much more ambiguous..."Most" agree, "become warmer", "few decades"...perhaps that ambiguity gives our ability to reason away things we don't like more freedom. Or perhaps it is more threatening in it's ambiguity. "If I agree to this broad phrase, maybe it could be used against me and my closely held beliefs." Whereas something specific about scientific consensus around coasting flooding isn't so damning to my view.

May 31, 2016 | Unregistered CommenterTrevor Geise

Anything I say is, of course, retrofitting and most likely untestable, but I think the phrasing of "most experts agree" puts the emphasis on whether the person thinks there's a consensus, whereas "according to climate scientists..." just asks them to spit back what they've heard about conventional climate change wisdom.

I'm most curious, now, about whether you can "boil the frog" with these questions. Start by asking them the de-polarizing version, get them to commit to a position, then ask the polarizing version. Maybe with one intermediate step (whatever that would be...maybe "according to most climate science experts..."). See if their answer changes (and if it does, ask them why?).

May 31, 2016 | Unregistered CommenterMW


You want to hurt frogs?...

Look at the n's. What will happen to the frogs, you sadist?

Is your hypothesis that someone would say, "Yes, *according to climate scientists,* human-caused global warming will caused flooding, but the truth is *most* expert climate scientists don't accept that human-caused global warming is even happening."

Also, what is the liberal who is high in science comprehension thinking when she says "false" on NUCLEAR? Is thinking, but "they are wrong! They are in pocket of industry! Shills!"

BTW, sorry to have nelgected your disgust questions... I'll answer them presently (been a bit distracted here)

May 31, 2016 | Registered CommenterDan Kahan

@Trevor & @T.Chen--

Maybe the word "expert" is the answer. But my sense is that the proportions here is in line w/ results from variously worded items collected elsewhere. Will check!

Do you agree that

1. It is sensible, in interest of parsimony & minimizing the lack of explanatory discipline associated with ad hockery, to start by trying to come up w/ a plausible mechanism for ACS effect that sees it working the *same way* for "Ds" & "Rs"? That is, not to start w/ an account that says that ACS was working on way for Rs in FLOODING but another way for Ds in NUCLEAR?

2. Also, if we accept (1), at least for now, then that we should also be trying to figure out a plausible solution to the "paradox" that fits the one-size-fits-all-ideologies account of ACS?

I'm not saying the explanation that you are proposign (I think you might not really be in 100% agreement!) doesn't fit (1) & (2) but I'm not sure. My thought is, let's start w/ (1). Then let's see what, consistent w/ (1) solves the paradox.

Otherwise, I think there is too strong a temptation to approach the issue as if it were about "explaining why Republicans don't agree witih me/democrats/scientists" etc. There's more going on here than that

May 31, 2016 | Registered CommenterDan Kahan


I think there's a decent chance the frogs will boil! People will say that according to climate scientists, global warming will cause flooding, so then they're locked in, and they will say experts agree that human-caused global warming is happening. But I'm far, far from certain. I could see them still understanding the questions very differently, even if they're juxtaposed.

I don't think the high-OSC liberal who gets the prefix and answers "false" on nuclear is thinking much about the experts at all. I think the prefix frames it as a knowledge quiz instead of a moment for cultural expression. Hasn't that been your idea all along?

Like, take these two quizzes:
Quiz 1
Which of the following plays did Shakespeare write? (check all that apply)
a. Hamlet
b. Mother Courage and Her Children
c. Romeo & Juliet
d. A Midsummer Night's Dream
e. The Way of the World

Quiz 2
Although conventional historical wisdom says William Shakespeare was the author of a number of plays, such as Hamlet, Romeo & Juliet, and A Midsummer Night's Dream, many people question whether a single person named Shakespeare actually wrote these plays, or whether they were written by a number of people and just attributed to Shakespeare. Do you think Shakespeare wrote those plays, or do you think the story is more complicated than your teachers told you?

People taking Quiz 1 -- even natural skeptics, conspiracy theorists, etc. -- are probably not going to be thinking in the least about the question posed in Quiz 2. They're just being asked to show off what they know. So they'll check the "correct" plays. If they were presented with Quiz 2 instead, they might have a completely different answer.

May 31, 2016 | Unregistered CommenterMW


So what's your *theory* of ACS?

The 1st paragraph seems to imply that the people who say they don't think there is consensus are being disingenous but ACS is so subtle that it flushes the out -- & once exposed they can't run back & hide behind the veneer of feigned disbelief. Is that your view?

It isn't your view of the high-OSI liberal, though, right? Why does she repsond differently when she is doing a knowledge quiz? Is she *doing* something different when she responds to the quiz, ACS question than when she responds to the non-ACS question? Something different w/ her high OSI?

If so, isn't the conservative doing the same thing -- or the same different things -- when she answers the w/ & w/o ACS prefix versions of the questions the "correct" answer for qwhich is identity-threatening?

Why do you think all that adds up to being "locked in," ttrapped etc., in relation to the consensus question?

What are the competing theories? What are the ocmpeting predictions? What can we infer from one result or another?...

May 31, 2016 | Registered CommenterDan Kahan


I don't think anyone's being disingenuous (as I don't think anyone answering either Shakespeare quiz would be disingenuous).

For now, at least, I'm buying into your cognitive dualism idea (at least I think I am...I still don't totally understand it). ACS puts people into the mindset of "this is a knowledge quiz; tell us what you've heard" and insulates them from the question of whether climate change is real or not -- a question where their answers would have expressive implications. The "most experts agree" question doesn't have that effect: what "most experts agree" on is probably the right answer, so they're prompted to give the answer that expresses their own belief of whether climate change is a real risk.

The reason I'm curious about the frog boiling is that I think the juxtaposition might force people to see these questions as related, whereas taken in isolation they would have very different reactions to them. If they were to recognize the questions as related, they would feel locked in. If they didn't and still understood them to be asking very different things -- and I do think that's a strong possibility -- then they wouldn't.

I wouldn't distinguish between the high-OSI liberal who varies on nuclear depending on whether ACS is there and the high-OSI conservative who varies on climate change. I'm not quite sure what you're getting at with the comparison.

I don't remember: Did you try ACS on the exact phrasing of the "most experts agree" question? Essentially: "According to climate scientists, human activity has caused the temperature to rise..."? If not, I'd wonder if ACS itself isn't magic, and it's the context of the "how much do you know about global warming?"-style quiz. Testing that would be easy: just ask the ACS super-question out of context and see whether subjects react more like other ACS subjects or like "most experts agree" subjects. If they reacted more like "most experts agree" subjects, that would support the idea that they're trying to display their knowledge in the current ACS battery.

May 31, 2016 | Unregistered CommenterMW


We didn't ask the ACS out of context; but that would likely disrupt sense that subjects were being "tested." The "magic" of ACS, though, is pretty apparent insofar as subjects were "tested" wt the same battery of tiems but w/o ACS prefix responded in politically polarized way.

The "most experts agree" item didn't use "according to climate scientists." Do you think if the items in quiz were worded "most climate sientists agree..." instead of "according to climate sicntists," the results would have been different?

Do you think conservatives *disagree" with the scientists in ACS condition? But libs "agree"? Or Libs disagree too if the item is not identity congruent (as in case of NUCLEAR)?

I think they both *agree* with the scientists when they are being tested. The same way that the Kentucky Faremer does when he is being a Farmer or the climate disibelieving resdients of SE Fla do when they are having the "right" rather than the "wrong" conversation about climate change.

But it's pretty clear to both what the question is when one asks whether there is scientific consensus on human-caused global warming....

On frogs, how long does it take the Kentucky Farmer to get from his tractor tohis living room?

June 1, 2016 | Registered CommenterDan Kahan

"Of course! They are the best at supplying both identity-expressive and science-knowledge-revealing answers. Which one they supply depends on what they are doing: revealing what they know or being who they are. The ACS prefix is the switch that determines which of those things they use their reason for."

Or alternatively, they might just be answering the question as it was posed.

Consider the "According to new-age aromatherapist astrologers..."-prefix. People will give opposite answers depending on whether they're asked what *they* believe is true, or what they think NAAAs believe is true. Is the distinction here really between "who they are" and "what they know"?

"In modern liberal democratic societies, holding beliefs contrary to the best available scientific evidence is universally understood to be a sign of stupidity. The cultural cogniton of scientific consensus describes..."

There's a big hole in the logic here!

"Best available scientific evidence" is not equal to "scientific consensus" (in the sense of "what most scientists say"). That's trivially obvious - when any scientific discovery is made, at the instant of discovery there's only *one* scientist with the best scientific evidence. And there are plenty of other reasons why "what most scientists say" might not be the best scientific evidence going.

"Is your hypothesis that someone would say, "Yes, *according to climate scientists,* human-caused global warming will caused flooding, but the truth is *most* expert climate scientists don't accept that human-caused global warming is even happening.""

Or how about "Yes, *according to climate scientists,* human-caused global warming will caused flooding, but the truth is that natural coastal deposition and erosion is complicated and orders of magnitude more powerful than the effect of sea level rise. First year geology students are taught that land rises as well, as rivers erode mountains and deposit silt on plains and deltas, and coral islands grow upwards to meet the surface. Satellite surveys have shown a number of islands claimed to be threatened by sea level rise are actually growing in area! It's a case of climate scientists opining outside their own field of expertise."

(I'm not saying that's what really happens in most people's heads - I'm giving it as a hypothetical example to see if your logic really follows.)

So, with this hypothesis, the subject is setting "what most scientists say" against "the best available scientific evidence" and coming down on the side of the latter. The "climate scientists say..." prefix isn't triggering them to answer based on who they are, but on who the climate scientists are. We all know that's what climate scientists say. They wouldn't survive long as climate scientists if they didn't. But they're obviously wrong.

*If* that's what people were thinking, then would you agree that the "what they know" vs "who they are" dichotomy is not in operation here?

"How can the same proposition - according to climate scientists, humans are causing global warming - both provoke expressions of identity and unconfound identity and knowledge?"

Maybe because you're assuming that everyone agrees that "what most scientists say" is the same thing as "best scientific evidence", and they are still changing their answer despite knowing that they disagree with most scientists. The alternative is that identity decides *whether they agree* that "what most scientists say" is the same thing as "best scientific evidence".

People whose identity-favoured answer agrees with scientists can accept Argument ad Verecundiam. Those whose identity-favoured answer disagrees with scientists go with the scientific evidence instead.

Thus - someone who likes nuclear power might accept "most scientists say" on whether it causes climate change - someone who dislikes it will point out (correctly) that reactors are made of concrete, the manufacture of which emits CO2 - i.e. the scientific evidence.

Likewise - someone who likes climate change mitigation policies might accept "most scientists say" on flooding - someone who dislikes it will point out (correctly) that land can rise as fast as the sea - i.e. the scientific evidence.

It's not the only hypothesis - you can even have several *different* mechanisms operating for different questions, so expecting the same mechanism to have to apply to both questions is another hole in the logic. But it's entirely possible for the same mechanism to explain both observations.

June 1, 2016 | Unregistered CommenterNiV

Questionnaires are too blunt a tool to work out what people are thinking on contentious issues and create a ‘what are they after’ response. Both helpful and unhelpful.

As NiV points out nuclear power stations use a lot of energy and concrete in construction. Does the question mean any affect or net affect? ‘Few decades’ global warming might mean two and there hasn’t been much global air warming in the last 20 years, especially depending upon which ‘expert’ you ask. By ‘global warming’ do you mean any warming at all or dangerous warming? Are the ‘experts’ really expert or the best we have at the moment? Does ‘flooding of many coastal regions’ mean soon or in hundreds of years? Are we talking managed retreat or catastrophic failure of existing boundaries? Are you adjusting for natural sea level rise of about 15cm and geological effects? Are you using the IPCC figure of about 1m by 2100 or some figure of your own? Define ‘many’. And so on.

By giving the right answer, does the questioned person feel their answers will be used in a way that they don’t agree with eg build more nuclear power stations or add more tax to fossil fuels. Will they just tick the wrong box as a protest or a joke? Even using the word 'experts' engenders a reaction. Is the questioner asking 'are you pretending to be smarter than an expert?' To which some will defer and others will challenge. Personally I won’t complete any surveys until I know what the researchers want to do with the results after a certain person tried to make it look like sceptics were conspiracy nuts. Especially as I and most sceptics never saw the survey in the first place.

June 2, 2016 | Unregistered CommenterTinyCO2

My inclination is to agree with people who think that the word "expert" is causing problems here. If you're polarized on a subject, an expert on it is just someone credentialed who agrees with you about it.

In other words, even if "most climate scientists" are [ {SCRD-cats and won't commit their reputations to their own results}/{in their own pocketbooks and really ought to know better about drawing alarmist conclusions} ], "most expert climate scientists" will agree with me.

June 2, 2016 | Unregistered Commenterdypoon


If the pcts here are same as in study that uses alternative wording -- say "From what you’ve heard or read, do scientists generally agree that the earth is getting warmer because of human activity, or do they not generally agree about this?" -- would you accept that as evidence that "expert" is not source of effect?

June 3, 2016 | Registered CommenterDan Kahan



I said "no" to your question, but that's the right test for the stated issue, so you're right. "Expert" is not source of effect, it's simply the focal point for the underlying sentiment. What is that sentiment?

I examined my feelings closely. I think the common question is, "Well, which darned scientists are you speaking of, the ones who agree with me or the ones who don't?" By asking the question in either wording about consensus or general agreement, you're forcing people to make a choice between responding on behalf of one of two (or possibly more than two) different societies to which they belong, the larger one or their more immediate one, in which two different social facts are accepted.

If that above hypothesis is true, then the reason people are able to produce identity-unconfounded answers with "According to climate scientists," is that something about that wording allows the respondent to feel safe responding with their judgment of "climate scientists" as an other-group, as opposed to an own-group or an opposing group. It removes any accusation of affiliation.

June 5, 2016 | Unregistered Commenterdypoon

@Dan Kahan

> Do you agree that
> 1. It is sensible, in interest of parsimony & minimizing the lack of explanatory discipline associated with ad hockery, to start by trying to come up w/ a plausible mechanism for ACS effect that sees it working the *same way* for "Ds" & "Rs"?

The 'expert' phenomenon is exhibited in such other flipped cases like GMO and and fracking. So it's the same way for both Ds and Rs. This is my anecdotal experience at least. I've seen really odd discounting/selection of experts in my time.

> 2. Also, if we accept (1), at least for now, then that we should also be trying to figure out a plausible solution to the "paradox" that fits the one-size-fits-all-ideologies account of ACS?"

One potential adjoining mechanism to this is the idea that as technical knowledge increases (loosely related to OSI), one is more likely to have 'outs' for agree with the consensus. In the specific case of climate change for instance, measurement uncertainties plays a large part in Rs dissent. While these are valid concerns (likewise there are valid concerns regarding GMOs), the difference between Rs, Cs, and Ds boils down to the extent to which these concerns influence their priors. As a C myself, measurement uncertainty nudges me a tiny bit off the consensus. For a D, this factor is unlikely even on the radar. A R will weigh it more against the consensus. This is a rational process, although the selected weighting is intuitive and varies among us.

I think I have difficulty in agreeing with any model where these actors are consciously duplicitous. Every actor here recognizes that consensus is merely the best state of current knowledge. Depending on the case, some actors also think that the best state of current knowledge is lacking. Those actors that have a higher knowledge level will be more confident to diverge from the consensus if they rationalize it. Those actors that have a lower knowledge level will look upward to their peers above them on the same side but lacking the intricate knowledge required to go against consensus, hedge their bets which we see as lower divergence.

June 5, 2016 | Unregistered CommenterT. Chen

PostPost a New Comment

Enter your information below to add a new comment.

My response is on my own website »
Author Email (optional):
Author URL (optional):
Some HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>