## Meta probabilistic thinking quiz...

About what percentage of US population will get correct answer (i.e., all 3 correct)? (a) 0-10%; (b) 11%-to 25%; (c) 26% to 50%; (d) 51%-75%; or (e) 76% to 100%?

Will post answer later today

The *quiz* answer is "67%" for every question. The "meta" question answer: just shy of 2% of general population gets this one right.

## Reader Comments (21)

Dan -

What is your source?

@Joshua--

for what? the problem? Margolis, H. Dealing with risk : why the public and the experts disagree on environmental issues (University of Chicago Press, Chicago, IL, 1996).

The 2%? Data I collected this month. As one would predict, modal answer is "50%, 50%. 67%"

That's a clever one. It's like a reverse Monty Hall.

@Zachary-- strongly recomend reading Margolis's account of how people think about this problem. There are like 400 explanations for the right result & at some point, w/ one of them, person to who said 50-50-67 will finally "see" why answer is 67-67-67. But what did they *see*? Margolis says "the pattern" -- the logical explanation comes after the seeing, not before

I've always wondered how much of the problem is linguistic. How clear is it that what you're asking for is P(reverse blue | obverse blue) instead of P (both sides of chip blue | one side of chip blue)? The difference between those is the weight that you'll observe each chip, which is what people who argue for the {1/2, 1/2, 2/3} answer "forget" to put in to their calculation, and I think it's caused by the phrase "you pick a chip".

==> How clear is it that what you're asking for is P(reverse blue | obverse blue) instead of P (both sides of chip blue | one side of chip blue)? ==>

I think that there's something like that going on for me.

I can abstractly "understand" the answer if I chart all the six possibilities, eliminate the 2 red/red configurations, eliminate the red up/blue down possibility, and see that two of the 3 remaining possibilities have a blue up/blue down configuration. I can also abstractly "understand" that my original error (in thinking 50%, 50%, 67%) is that I'm not accounting for two possible configurations (instead of just one) of the blue/blue chip, but can't get past thinking that if I've pulled out a chip, and I can eliminate the red/red, I've got two possible chips left and an equal likelihood that the chip on the table in front of me is blue up/blue down, or blue up/red down.

And I think that the language issue is somewhat at hand. If I were asked how many possible configurations could there be to create that scenario, I (think I) would have likely given the correct answer, but when I am asked about the "chance]s]" I am engaged with a more circumscribed scenario. It reminds me of the criticisms of Piaget's work on establishing when children understand conservation of matter, that what seems like a cognitive issue is perhaps a language issue.

--snip--

Piaget and Inhelder (1974) conducted several experiments to test out the child’s conservation of various properties such as matter, weight and volume. In the conservation of matter experiment, 180 children between the ages of four and six were given two lumps of modeling clay, one of which was in the size of a ball and the child was told to construct an identical ball from the other lump. Once the child is certain that both balls are identical, the experimenter proceeded to either roll one of them into a sausage-like shape or flatten it to form a disc. The child is then questioned if the two objects were still identical in terms of weight, matter or volume. In addition, the child must justify their answers in order to properly identify the child’s understanding of conservation. Approximately a third of the children agreed that matter was conserved but not weight and volume while 55 out of the 180 displayed no knowledge of conservation at all. Piaget believes that during this stage, the child cannot distinguish between conservation of matter, weight or volume. Piaget is convinced that the reason preoperational children are unable to grasp the concept of conservation is because of the fact that direct perception precedes intellectual operations. The preoperational child does not understand the transformation presented to him and does not possess the operations, which is an action that is reversible and interiorized, necessary to understand these transformations (Piaget & Inhelder, 1974). Due to this lack of understanding of transformation, the child compares the objects’ initial and final stage (direct perception), ignoring the transformation it underwent (Piaget, 1964, as cited in Gauvain & Cole, 1997).

Several aspects of these experiments have come under severe criticism. One of these aspects is the language usage of the experimenter, which Micheal Siegal (2003) termed ‘conversational confusion’. Flavell (as cited in Green, Ford & Flamer, 1971) believes that it is difficult to determine if the child’s response in Piaget’s conservation tasks is really a display of his understanding of conservation or merely a linguistic error in understanding. He also suggests that there is a possibility the child is responding based on what the child believes the experimenter would expect him to say or that a “trick question” (p. 202) was presented. Several research have been done to see the effect of questioning and some of these research have indicated that a change in questioning methods yields results opposing that of Piaget’s experiments (Siegal, 2003). A study conducted by McGarrigle and Donaldson (1974, as cited in Eysenck, 2004) changed the method of questioning by instead using a toy called ‘Naughty Teddy’ to carry out the transformation and proceeded to ask the children if there was any changes between the initial and final state. This experiment yielded a higher percentage of preoperational children capable of conserving. However, it may be possible that because the child does not notice the transformation and is expected to believe it was incidental, the results yielded were a false-positive. In another study conducted by Marion Lamos (1970), she adopted two different methods of questioning: the neutral form (similar to that of Piaget’s experiments where children were asked if the initial and final state were the same) and negative suggestive questioning where children were asked ‘which one is bigger? or ‘which one is longer?’. The results maintained that regardless of which type of questioning is adopted, the number of children aged six who were capable of conserving were significantly lesser than their older counterpart, thus agreeing with Piaget’s view that conservation does not occur until the stage following preoperational – concrete operations. (Lamos, 1970).

--snip--

Or at least that's what I would like to think, because it's more gratifying than just acknowledging my limited intelligence.

Dan -

Since it appears that the Margolis paper is paywalled, can you post some salient excerpts?

This is it. The difficulty here is that you, along with everyone reasonable in the world, think of a chip as a single physical object, whereas mathematically what's actually being sampled here are sides of chips.

Do you find this completely equivalent problem easier to understand?

It's easy for us to see that there are 2 gay men and 1 straight man, for we think of each man as an individual by default. It's not as easy to think of two blue sides with blue reverses as "two distinct individuals" when they're back to back on the same chip.

In fact, I'm willing to bet real money that if Dan were to test these alternative question wordings, he would find a statistically significant difference in response correctness.

@Dypoon & Joshua-- I think the acid test here for question validity is the relationship the "correct" answer has with scoring well on critical reasoning tests like CRT; if people who do well w/ critical reasoning problems do well w/ this one, then that's strong evidence for treating it as a valid measure of the same disposition. If they do no better or worse than those who get lower scores, it's a dog question.

Stanovich calls this the "Spearman's postive manifold" test.

dypoon -

Perhaps your alternative problem gets to the root of

my"error" in answering Dan's problem, although I'll guess that I'm not smart enough to figure out whether or not it does.With your problem, I kind of see two different tasks possible. The first task (which quite likely is not the one that would jump out at me) is to estimate the chances, that if any randomly selected man were asked to point to his partner, he would point to another man. In that case, I'd likely say 67%. Clearly, two of the (three) men would point to a man and one of the (three) men would point to a woman.

The second task (which is the one that would jump out at me) is to pick a particular man, and then to (1) imagine asking him to point to a partner, and (2) THEN estimating the chances THAT man would be pointing to a man. In response to the second prompt, I'd likely say 50%. The task has been circumscribed.

Yes, your alternative problem helps to clarify my error, but I also think that there is a time sequence element involved in my confusion, which would lead to me providing an "erroneous" answer if I'm conceiving of the task as being the second task (which I think would be likely).

-------------

To revisit Dan's problem to help explain my confusion...

I think of the tasks at hand as being something on the order of...

------------------

There are TWO chips in a cup. (As I reconstructed the problem in trying to solve it, the existence of the red/red chip is irrelevant). One with a blue dot on both sides and one with a blue dot on on side and a red dot on the other side.

(1) Pull out a chip. "What is the color facing up?"

[answer: "blue"]

(2) "What is the chance that the other side of THAT chip is blue?"

I don't instinctively think of the task being presented as to estimate the chances that a randomly selected chip would have two blue sides. That seems to me like different task then the one being asked.

-----------------------------------------------

I see that problem/task as being essentially the same as:

There are TWO chips in a cup. One is blue on one side and red on the other side. The second chip is blue on one side and BLACK on the other side.

(1) Pull out a chip. "What color is facing up?"

[answer: "blue"]

(2) "What is the chance that the other side of that chip is BLACK?"

-------------------

Sorry I speak English. I am told that I have one chip with one side blue. The red red is not in play. This is a two chip problem.

So I either have blue blue or blue red. So the revere has a 50% chance of red and 50% chance of blue.

QED 50%

The probability is being asked AFTER the first selection has already been made.... or is there another variety of English only spoken by statisticians?

ATB

Steve

Steve,

You are correct. There may be confusion caused by the wording of the problem. Presumably those who think the answer to all three questions is 67% have mistakenly concluded that (i) and (ii) are identical to (iii). However this is not the case for the problem as stated, for which the correct answer is indeed 50%,50%,67%. Presumably they're failing to realise that (i) and (ii) are conditional.

One has to state problems like this very carefully.

It's written without looking suppose the face up side is blue. That's only one reason to phrase that way: to catch people wrong. Why anyone should pick a chip and not look at it? It's easy to understand why in answering the question people forget to take into account that they are not supposed to look.

Hey all. My wife (with 2 degrees in statistics) tried to explain it using Bayes theorem and lost me very quickly. Here's my quick explanation:

You RANDOMLY picked the chip. It has a blue side up. Every time someone RANDOMLY picks the blue/blue chip you place it blue side up. Every time someone RANDOMLY picks the blue/red chip there is only a 50% chance it will be placed blue side up. Half the time it will be placed red side up and would be rejected from our question. If you have a blue side up chip it is therefore TWICE as likely it's the blue/blue chip. Therefore 66%

It would be 50% if someone chose a chip for you. I wonder how many of the 2% guessed it without understanding.

That answer is way too long.

When you pull a blue, the possibilities are

B1/B2, B2/B1, B3/R1

R1/R2, R2/R1, R3/B1

Help??

@Joshua-- It would be a copyright violation for me to publish the relevant chapter from Margolis's book.

@everyone else-- if you think the answer is 50-50-67, you are in good company. But you are wrong. Find a friend who gets why & have her explain it to you. Somewhere between 1st & 50th explanation, it will dawn on you that the right answer is indeed 67-67-67.

My favorite is that if you threw away the red chip, you'd have B/B & B/R. There are twice as many ways for B/B to be the blue-up chip than there are for the B/R to be it. So if you drew randlomly from those 2 100 times, you'd expect the B/B to be "blue up" 50 times & the B/R chip to be "blue up" only 25 times. 50 is 2x25. So there is 2x as high a chance anytime you pick a "blue up" that it is a B/B outcomes than that it is a B/R one.

But if that doesn't do it for you, that's okay. There are 49 explanations left

@Roger-- You draw a blue-up chip. Eliminate from your sample space the Rx/Rx outcomes & the R/B in which Red is on the upside-- since it is stipulated that you have a blue-up chip. What's left are 2 B/B outcomes and only 1 B/R. Since there are twice as many ways for the observed result if the chip is B/B, the probability it is the B/B chip is 67%. Voila.

Joshua's reasoning is correct. The lack of understanding comes from not factoring in you are pulling your own chip.

"Here is a chip which is blue side up. What is the chance that the other side of THIS chip is blue?"

...is the question that I think most people (myself included), who have a modicum of sense that is, think they're answering. It is the act of the draw which adds the confusion. I don't put this down to a lack of understanding of the statistics, more to misunderstanding the setup. The Monty Hall draw adds confusion in a similar way.

@Drdrew-- it makes no difference if the chip is drawn randomly by you or another person. This isn't the Monty Hall problem, where Monty's opening a door gives you information.

@Joshua-- you "can't get out of your head" that there are two chips for which upside could be blue. But those chips don't have equal likelihood of being the blue-up chip. One has 2 ways it can be that chip & other has only 1. It's a simple conditional probability problem: you start out w/ 1:1 odds on either chip. Since blue is certain to come up i"blue" if draw, and B/R is only 50% likely to to be Blue if drawn, the likelihood ratio is 2 (100% is 2x 50%) in favor of B/B. Multiply prior of 1:1 for B/B. by 2, the Likelihood ratio, and you have 2:1 odds for B/B .