follow CCP

Recent blog entries
popular papers

What Is the "Science of Science Communication"?

Climate-Science Communication and the Measurement Problem

Ideology, Motivated Cognition, and Cognitive Reflection: An Experimental Study

'Ideology' or 'Situation Sense'? An Experimental Investigation of Motivated Reasoning and Professional Judgment

A Risky Science Communication Environment for Vaccines

Motivated Numeracy and Enlightened Self-Government

Ideology, Motivated Cognition, and Cognitive Reflection: An Experimental Study

Making Climate Science Communication Evidence-based—All the Way Down 

Neutral Principles, Motivated Cognition, and Some Problems for Constitutional Law 

Cultural Cognition of Scientific Consensus

The Tragedy of the Risk-Perception Commons: Science Literacy and Climate Change

"They Saw a Protest": Cognitive Illiberalism and the Speech-Conduct Distinction 

Geoengineering and the Science Communication Environment: a Cross-Cultural Experiment

Fixing the Communications Failure

Why We Are Poles Apart on Climate Change

The Cognitively Illiberal State 

Who Fears the HPV Vaccine, Who Doesn't, and Why? An Experimental Study

Cultural Cognition of the Risks and Benefits of Nanotechnology

Whose Eyes Are You Going to Believe? An Empirical Examination of Scott v. Harris

Cultural Cognition and Public Policy

Culture, Cognition, and Consent: Who Perceives What, and Why, in "Acquaintance Rape" Cases

Culture and Identity-Protective Cognition: Explaining the White Male Effect

Fear of Democracy: A Cultural Evaluation of Sunstein on Risk

Cultural Cognition as a Conception of the Cultural Theory of Risk

« Cultural cognition? Oy! What's a science journalist supposed to do?... | Main | Year in review for CCP research, including the conservation-of-perplexity principle »

Still another metacognition question 

How about this one, which is a classic in study of critical reasoning? What's answer & more importantly what percent of general public get it right? Why don't 100% get the correct answer? How do self-described "tea party" membes do (whatever happened to those guys?) Answers anon . . . . 

PrintView Printer Friendly Version

EmailEmail Article to Friend

Reader Comments (8)

Dan -

I have some questions for you.

What's answer & more importantly what percent of general public get it right?

Why do you think it is important to know what % of the general public gets the right answer? I'm guessing that you believe that we can understand something about how the world works on the basis of what % of people get it what might that be?

Why don't 100% get the correct answer?

I would think that there are a lot of reasons, but perhaps one of the most operative explanations is that when people are presented with such a problem, they are worried about being judged as stupid, and thus shortchange their full analytical process when they find an answer that "seems" right.

How do self-described "tea party" membes do (whatever happened to those guys?)

What do you think might be meaningful about singling out the %'s for that group compared to other groups? I think that evaluating Tea Partiers' beliefs on some issues is relevant - for example, I think that it is meaningful that some polling shows that as a group they are highly confident that the climate isn't being meaningfully affected by ACO2 emissions, and that they have enough of an understanding of the science to be able to make that determination. I think that's meaningful because those data have a direct political implication. Do you think that we can understand something important, in a political sense, from the % of Tea Partiers who get the correct answer to this problem?

December 31, 2016 | Unregistered CommenterJoshua

OK, one more -

When you're deleting, could you delete the embarrassing #2 comment also? In rereading, I have no idea myself what I was trying to ask.

December 31, 2016 | Unregistered CommenterJoshua

@Joshua-- just joking about tea party. I swear I'll never again post any information about how they do on critical-reasoning tests.

Why is kniowing what pct of public more interesting? I just think it is interesting to see what sorts of problems are hard & which ones not.

Likely we can learn lots about world, too, if we construct a good critical reasoning test & then look at how it relates to culturally contested issues.

December 31, 2016 | Registered CommenterDan Kahan

I seem to recall reading that the percentage of people who get this right, differs depending on whether or not it is presented as an abstract problem, or a problem related to cheating. I think the example given was something like "you can only drink alcohol if you're at least 21. There are four people in the bar. One is grey-beareded and wrinkled, with a drink in front of him that might or might not have alcohol in it. One is grey-bearded and wrinkled with no drink. One looks young, with no drink. One looks young, with a drink in front of them. Whose id/drink do you need to check?"

The idea being that people found this problem (or something close to it, I might have the details wrong) trivial, but a more abstract problem not about cheating, most people got wrong. Have you heard of the research I'm referring to? I can't recall the source right now but I could if you want it, but perhaps you already know it?

January 1, 2017 | Unregistered CommenterRoss Hartshorn

Ross -

This is really interesting:

January 1, 2017 | Unregistered CommenterJoshua

emphasis added:


Evolutionary psychologists Leda Cosmides and John Tooby (1992) identified that the selection task tends to produce the "correct" response when presented in a context of social relations.[10] For example, if the rule used is "If you are drinking alcohol then you must be over 18", and the cards have an age on one side and beverage on the other, e.g., "16", "drinking beer", "25", "drinking coke", most people have no difficulty in selecting the correct cards ("16" and "beer").[10]

In a series of experiments in different contexts, subjects demonstrated consistent superior performance when asked to police a social rule involving a benefit that was only legitimately available to someone who had qualified for that benefit.[10] Cosmides and Tooby argued that experimenters have ruled out alternative explanations, such as that people learn the rules of social exchange through practice and find it easier to apply these familiar rules than less-familiar rules.[10]

According to Cosmides and Tooby, this experimental evidence supports the hypothesis that a Wason task proves to be easier if the rule to be tested is one of social exchange (in order to receive benefit X you need to fulfill condition Y) and the subject is asked to police the rule, but is more difficult otherwise. They argued that such a distinction, if empirically borne out, would support the contention of evolutionary psychologists that human reasoning is governed by context-sensitive mechanisms that have evolved, through natural selection, to solve specific problems of social interaction, rather than context-free, general-purpose mechanisms.[10]

In this case, the module is described as a specialized cheater-detection module.[10] Davies et al. (1995) have argued that Cosmides and Tooby's argument in favor of context-sensitive, domain-specific reasoning mechanisms as opposed to general-purpose reasoning mechanisms is theoretically incoherent and inferentially unjustified.[11]


It would be interesting to know what Davies et al. think explains the better performance with the beer/age setup.

January 1, 2017 | Unregistered CommenterJoshua

@Joshua & Ross-- embedding the task in familiar problem definitely helps. Take a look at

Gigerenzer, G. & Hug, K. Domain-specific reasoning: Social contracts, cheating, and perspective change. Cognition 43, 127-171 (1992).

January 2, 2017 | Registered CommenterDan Kahan

Dan -

Thanks for that link.

Such an interesting situation. say:

==> embedding the task in familiar problem definitely helps. ==>

But it seems to me that the article you linked suggests something somewhat different than that:


Second, cassava root is not familiar to North American and European subjects, nor is the connection between cassava root and tattoo, nor can memory provide counter-examples from experience. Availability theory thus predicts a low percentage of P & not-Q responses for the cassava rule. This contrasts sharply with the prediction from both SC and PRS
theory. In the transportation rule, however, the propositions, the relation, and even counter-examples are familiar although familiarity may vary with the subjects’ geographical location). In any case, availability theory should predict that there will be far more P & not-Q responses in the familiar transportation problem than in the unfamiliar cassava problem. Cosmides, however, found that the reverse holds.


Seems to me that they're arguing that it isn't familiarity that makes a difference but a domain specific aspect, and in that sense, a "domain specific approach to reasoning." As I get it, they're arguing against "familiarity" as an explanation (asn explanation that was suggested by Kahnemann).

Anyway, IMO, the article and the theorizing it describes, it seems to me, misses an important aspect: the "affective" aspect of how comfort-level affects reasoning. Familiarity" doesn't have to be restricted to concrete components of context (beer/age vs. casave/age) <Strong>or a domain-restrictive "social contract" or "non-SC permission rule" structure. I think that the "familiarity" of a "domain" could more generally refer to "something that people are comfortable with." People are generally familiar with the setup of the beer/age cards because they have encountered such scenarios in the past. I would guess that it doesn't have to be a social contract scenario, but something that puts people at ease to the point where they can more comfortably engage in probabilistic reasoning without feeling like an idiot. In other words, it is the "familiarity" of the social contract scenario that is more relevant than the actual "social contract-i-ness" of the rule structure. And that familiarity is with a more general nature of the scenario than what would be delimited by familiarity, or the lack thereof, with casava.

Of course, I also doubt that there is any single explanation for why a lot more people get the beer/age setup than the transportation/location setup.

January 9, 2017 | Unregistered CommenterJoshua

PostPost a New Comment

Enter your information below to add a new comment.

My response is on my own website »
Author Email (optional):
Author URL (optional):
Some HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>