The Stockholm syndrome in action: I find Lodge view more persuasive as 3-day conference goes on
Back from Stockholm. Here’s a delayed postcard:
So in my talk, I presented 4 points—
--aided with discussion of 2 CCP studies (Kahan, Peters et al. 2017; Kahan, D., Landrum, A., et al 2017) (slides here).
As previously mentioned, Milton Lodge was among the collection of great scholars who participated in the Wenner-Gren Foundation’s “Knowledge resistance and how to cure it“ symposium. (Lodge also gets conference “outstanding teacher” award for conducting a tag-team-style presentation with one of his students, who did a great job).
I had the honor of being on the same panel as Lodge, who summarized his & Taber’s own body of research (2013) on politically motivated reasoning. Lodge definitely understood the thrust of my remarks (likely aided by reading it in various forms elsewhere) meant that he and I “had a disagreement.”
That disagreement boils down to how we should view the complicity of “System 2” reasoning in politically distorted information processing. Lodge & Taber (2013) push hard the view that once a partisan has been endowed with motivations that run in one direction or the other, it’s confirmation bias—a system 1 mechanism—that does all the distorting of information processing.
My & my collaborators’ position, in contrast, is that individuals who are high in System 2 reasoning have a more fine-tuned “System 1” reasoning capacity that unconsciously discerns the types of situations in which the use of “System 2” need to be brought to bear to solve an information-processing problem in politically congenial terms. Once engaged, partisans’ “System 2” will generate decision-making confabulations for dismissing evidence that blocks the result they are predisposed to accept.
We had a very brief exchange on this in connection with the motivated numeracy (MN) paper. Persuaded to an extent by what Lodge was saying, I agreed that the MN result would likely be as consistent with his position as with ours if the result was a consequence of high-numeracy subjects “tuning out” and lapsing into congenial heuristic reasoning when confronted with information that, improperly interpreted, supported positions on gun control at odds with subjects’ political affiliations & outlooks.
In contrast, the study results would lean our way if the high-numeracy subjects were being alerted by unconscious System 1 sensibilities to use System 2 to rationalize away information that they did recognize as contrary to their political predispositions.
I think on reflection that the design of the MN study doesn’t furnish a lot of light on which interpretation is correct.
But I’d also say that our interpretation—that highly proficient reasoners were using their cognitive advantage to reject evidence as flawed when it challenged their viewpoint-- was consistent with other papers that examined motivated system 2 reasoning (including Kahan, 2013).
Anyway, it takes only one thoughtful engagement of this sort to make a 3-day conference worthwhile. And this time I was lucky enough to be involved in more than one, thanks to the conference organizers who really did a great job.
References
Reader Comments (16)
link drop:
https://phys.org/news/2017-09-hypescience-prevalent.html
I'm (still) trying to think of ways that the skin rash gun control test could be modified to uncover more about the mechanism involved.
Suppose that one hides the values of the various data fields - along with values of other related but not relevant data fields (as controls), such that the subject has to request that the value be displayed, and describes the test as requiring the subject to request the minimum of such displays that they need to get the correct answer (motivate this by docking them points or reward money per value displayed). This way, one would have some audit trail of what is going on in the minds of subjects.
The high numeracy subjects wouldn't know in the gun control test which way the test result was going to go (for or against their preference) prior to displaying the necessary values. Would they request the necessary values, and then still fail to use them? Or would they decide not to request the values? Or would the fact that they are making their intentions known to the tester cause them to use the values they request correctly more often (than in the all-values-visible original test) even in the against-preference case?
@Jonathan-- tell me what you think the alternative mechanisms are that could generate the patterns observed in the data; presmuably then it is possible to think of ways to tweak the design to generate results that avoid the confound I gather you perceive
One alternative is that high numeracy subjects can determine to a good enough approximation using only system 1 which way the gun control test is likely to go, and then only finish the calculation properly (system 2) if it is likely to go their preferred way.
Another alternative is that high numeracy subjects are dis-motivated to properly use system 2 (and base rate data) in only the case when their system 1 (without base rates) calculation of the result is for their preference in a test that might go against their preference (the gun control but not the skin rash test).
Another alternative is that high numeracy subjects use system 2 and base rates in all cases to determine which way the gun control test will go. They're consciously "acting dumb" when they claim to get the preferred answer by not taking into account base rates.
@Jonathan--
I agree that this
<<Another alternative is that high numeracy subjects are dis-motivated to properly use system 2 (and base rate data) in only the case when their system 1 (without base rates) calculation of the result is for their preference in a test that might go against their preference (the gun control but not the skin rash test)">>
might be what's going on. But I don't think it would be conscious as opposed to unconscious lapse into heuristic strategy, in that case.
If selecting "ideologically congenial answer no matter what" was strategy, then low numeracy subjects would look like geniuses whenver that answer was (unknown to them) the right answer.
Other hypothesis is taht high-numeracy know that the "right" answesr is culturally uncongenial & so (unconsciously) search for a defect in study design but only in gun condition
Surely there are ways to test that -- & test debiasihg strategies that fit one or the other explanation....
But I don't think it would be conscious as opposed to unconscious lapse into heuristic strategy, in that case.
I agree. I am surprised you think this might be what's going on, because it sounds to me more in line with Lodge's theory than yours (Stockholm syndrome lingering maybe?). It's a failure of a system 1 "alarm" to go off at the appropriate time to signal the subject that their non-base-rate calculation is flawed, and this failure is due to the appearance of a preferred "culturally congenial" conclusion after doing just the initial non-base-rate calculation.
If selecting "ideologically congenial answer no matter what" was strategy, then low numeracy subjects would look like geniuses whenver that answer was (unknown to them) the right answer.
I didn't suggest that as an alternative - because I think the current test already rules it out.
Other hypothesis is taht high-numeracy know that the "right" answesr is culturally uncongenial & so (unconsciously) search for a defect in study design but only in gun condition
OK - that's an unconscious variant of my "acting dumb" 3rd alternative. They get the right answer via system 2, see that it is not their preference, and something steps in - whether it's conscious "acting dumb" or unconscious nudge to "search further for design defect".
Mostly, I'm interested if they ever consciously had the right answer.
Jonathan:
Your three alternatives are all different mechanisms via which cultural bias towards a preferred answer may be working, albeit your first and second look very similar to me and your third is via a conscious preference. However, in the prior thread you (and NiV too) suggested that the test needed to be altered in order to weed out a proposed significant effect that wasn't related to cultural bias. Have you abandoned this position?
The effect is highly unlikely to be conscious. The long-evolved capability of parts our our brain to deceive other parts regarding group deceptions, which capability is enabled / disabled by an individual's cultural values, is well established, notwithstanding a string of mysteries regarding the how and the necessary architecture. Indeed without this capability a host of major group deceptions would not exist, and hence essentially neither would strong cultural entities like religions or extremist political groups. So regarding the culturally aligned issue of gun-control, there would have to be something special and different going on for this ubiquitous effect to be exceeded by conscious lying. That doesn't mean it isn't happening, and neither is the distinction likely black and white anyhow (lying to ourselves is likely achieved on a analogue scale). Nor is it easy to test for. But I think it's reasonable to ask why we'd think large slices of the (numerate) public (assuming the sample is representative) would *consciously* lie about the figures they'd worked out.
Andy,
Have you abandoned this position?
No. I still think it is possible that the bias involved is not confined to identity protection, and so might be triggered in identity neutral but highly unexpected outcomes. Or at least that one can't conclude on the current evidence that the bias has identity protection as its purpose.
The effect is highly unlikely to be conscious.
I cautiously agree. However, there may be some that got the right answer and suppressed it consciously - and that would be interesting. Consider the similarity to the Solomon Asch conformity tests.
But, conscious or not, and purposely identity protective or not, there is still the question of whether the high numeracy subjects ever had (or were even well on their way towards) the right answer (properly accounting for base rate), and then suppressed it for some reason (conscious or not), vs. never had (nor were even attempting to get) the right answer in the non-preferred case in the gun control test.
Jonathan:
"I still think it is possible that the bias involved is not confined to identity protection, and so might be triggered in identity neutral but highly unexpected outcomes. Or at least that one can't conclude on the current evidence that the bias has identity protection as its purpose."
But it is cultural alignment that creates the expectation gap here. If an identity neutral yet strong expectation gap produces a similar result, this would not detract at all from the identity protective cause in this case.
"Consider the similarity to the Solomon Asch conformity tests."
What similarity? There is no direct conformance pressure in privately answering a survey question.
"...there is still the question..."
Indeed the how is still a very valid question.
Andy,
If an identity neutral yet strong expectation gap produces a similar result, this would not detract at all from the identity protective cause in this case.
My interest here is quite possibly just one of taxonomy. Is it right to call it "identity protection" if it just incidentally protects identity when identity aligns with expectation? True, identity may align with expectation almost all the time, but not vice versa. If the bias always aligns with expectation, what is then gained by claiming that the bias is identity protective instead of saying that expectation is often identity protective (which seems less controversial) and the bias tracks expectation (which seems also less controversial)?
What similarity? There is no direct conformance pressure in privately answering a survey question.
Sorry for the ambiguity. I wasn't referring to the presence of conformity pressure in Asch. I assumed that there might be some internalized analog going on here, and was instead thinking of how such can induce even obviously consciously wrong answers from some. I recall that some Asch subjects confessed after the test that they were just following the crowd - showing that for them it was entirely conscious. Maybe some of Dan's subjects are thinking something like "You're not going to get me to say gun control does X when I want to believe it does the opposite!".
Jonathan:
"...what is then gained..."
A provisional step in understanding, rather than a clouding by the introduction of apparently needed considerations for an alternate effect that is not named and / or a flaw that is not identified.
"...which seems also less controversial..."
What do you believe is controversial here?
The effect is revealed via a filter of political poles, so the correlation is not 'incidental'.
One should take care not to apply excessive taxonomy, it may simply obscure useful concepts.
All our steps may be regarded as provisional. Even refusing the provisional is fine, but you have no alternative candidate and no suggestion for how Dan may adjust his test to weed out your apparently significant yet unspecified other effects. Even refusing the provisional without a candidate is fine, if instead you can clearly point to a serious flaw in Dan's test procedure or the logic chain behind his provisional hypothesis. But these appear not to be forthcoming either.
If you have a strong case for why an effect other than cultural bias may be the agent behind Dan's results in this study, why not just state it, and whatever associated fingerprints you believe need to be looked for? Or alternatively, say what specifically is wrong with Dan's test that might allow any such effect (even without identification of same) through? Legitimate questions about how the effect occurs, and the potential involvement of cognitive machinery that may also be involved with non-IP processing, in no way points away from the study's effect correlation with cultural alignment, or towards any other possible candidate. Nor likewise does pointing out that the effect may be a subset of other human effects, whatever these are. And one day the 'how' might reveal that 'identity protection' is not the best descriptor for the effect, yet likewise a provisional name does not invalidate the observed effect and its correlation with political poles.
In short, what clearly and simply is your objection about attribution of the effect to cultural bias / identity protection?
"Maybe some of Dan's subjects are thinking something like "You're not going to get me to say gun control does X when I want to believe it does the opposite!"."
Possibly some are. Yet if so there's little option but to conclude that cultural bias is even more strongly driving people i.e. towards consciously giving a wrong answer, rather than merely subconsciously doing so (there is no direct conformity pressure in answering a survey question, and the effect is revealed by a filter of political poles). Given that the ability to subconsciously lie to ourselves seems to be a long-evolved feature of brain architecture, it would seem far more likely that this is what's happening for most. At any rate, the test as it stands does not give insight on that question.
Andy,
One thing Dan has tested before might qualify as a differentiator here: HBV vs. HPV. Suppose we construct a test of HBV vs. HPV in the same style as the conservative half of the skin rash vs. gun control test. However, instead of asking whether HBV and HPV work or not, we ask whether they increase promiscuity or not. That's for two reasons: We first of all have no direct proof that HPV lowers cervical cancers yet (the results won't be in for another decade or so), which some subjects might be aware of. Secondly, the identity protective "excuse" in the HPV case is (often) the potential to increase promiscuity and not its failure to work - so that matches the gun control test better, as the "excuse" is what is tested directly.
I am hypothesizing that conservatives may have a bias against gun control reducing crime based on what are somewhat relevant non-identity experiences, but they have no relevant experience about HPV - so any bias there is purely identity protection.
Suppose that this HBV vs. HPV test shows no greater incidence of failure to use base rates between HBV and HPV in either the increasing promiscuity or non-increasing scenarios, unlike the skin rash vs. gun control test.
Would that be enough to make you doubt that the bias that disables high numeracy skills is identity protective?
Suppose we also did that "grass is green" test (high expectation, low identity) with something like aspirin and pain relief, and that did show a bias that disables high numeracy skills in the unexpected case, to a degree similar to the non-preferred gun control cases.
Would you still claim that the result in the gun control test is due to an identity protective bias?
On the opposing side, if the HPV test shows bias, but the aspirin test does not, I would be much more inclined to believe that the gun control test bias is due to identity protection instead of experience.
If both aspirin and HPV tests show bias, then we're back to splitting hairs, I think.
Jonathan:
"I am hypothesizing that conservatives may have a bias against gun control reducing crime based on what are somewhat relevant non-identity experiences."
From just where are you proposing that a majority of US conservatives would be receiving NON-identity related experiences on gun control? The whole point about a 'polluted SC environment', as Dan phrases it, is that there is no significant non-identity related source feeding the ordinary general public, at least when such pollution has occurred over the long-term for a major socially contested issue. Emergent polarization is inclusive of messaging on main communication channels, and hence the dominant information the public receives is all from culturally aligned sources. So there are no significant non-identity experiences for the great majority. However, following through on your propostion that such non-biased sources of experience on gun control were readily available, why would Dem / Libs not be able to access these sources just as well as Rep / Cons? Which would eventually lead to both poles converging upon the same (non-biased) conclusion? If the available experiences are not in the context of cultural identity, then they cannot by definition be part of biased Rep / Con or Dem / Lib networks or issued by the talking heads of either side.
As far as I recall, Dan used the HBV / HPV case to show the polarization in the latter and lack of in the former, due to their different entry modes into the public space. I can't remember the level of polarization for HPV, but if it is at least similar to the level for gun control, then yes indeed I'd expect a similar result of the numerate and identity challenged folks fluffling the test. Any issue that is strongly polarized on the Dem / Lib - Rep / Con axis should produce a very similar result for an equivalent test, e.g. climate change would be another.
"Suppose we also did that "grass is green" test (high expectation, low identity) with something like aspirin and pain relief..."
Tests based around these examples may lack so much credibility (just way too obvious) that you may get some 'this is silliness' responses that could skew the results. Nevertheless, I'm sure such a test could be formed more subtly. But the result of value from Dan's study is not that strong pre-conceptions cause numerate folks to fluff such tests, but that cultural bias is powerful enough to create the strong enough pre-conceptions (obviously in opposite directions for opposite poles). If you pick an issue that has no (or trivial) cultural identity related implications, you may well find a significant effect for the numerate folks, but this cannot *even in principle* show difference in a test which is switching between identity affirmation and identity challenge specifically to pick up the movement between these conditions. There will be no movement; i.e. any effect will be the same whatever the condition.
"...if the HPV test shows bias, but the aspirin test does not, I would be much more inclined to believe that the gun control test bias is due to identity protection instead of experience."
So any sufficiently polarized issue should show the same effect, i.e. *movement* between the results for the numerate, hence including the HPV issue if indeed it satisfies this condition. The aspirin test cannot even in principle produce opposite effects for each political alignment, unless you are suggesting that the Dem / Libs have a radically different opinion about aspirins, or grass being green, than the Rep / Cons. And you are not, because this is exactly *why* you want to pick them. So doing this exercise does not tell us anything for or against Dan's hypothesis about IP as a cause. It only suggests to us, if positive, that this IP bias probably works by inducing a similar level of conviction to that derived from experience. Not unuseful, yet we know already that strong cultural alignment induces strong convictions.
Andy,
From just where are you proposing that a majority of US conservatives would be receiving NON-identity related experiences on gun control?
Maybe this is one of the ways we are misunderstanding each other: they get non-identity related experiences about guns and gun ownership. The non-identity related experiences about guns and gun ownership may come from the urban/rural divide we've previously discussed, and so correlate with left/right because mediated by urban/rural. Urban dwellers get a chronic view of gun related crime on the evening news. Rural dwellers get a view of guns as necessary tools for hunting, defending property and life, etc.. (Anecdotally, I personally know both urban righties that hate guns and rural lefties that love guns). Is there some reason to believe that the bias would only be triggered due to views expressly about gun control and not instead about guns and gun ownership?
The aspirin test cannot even in principle produce opposite effects for each political alignment
This is certainly a misunderstanding. I am not suggesting that the aspirin test can be left/right polarized. Also, I am not suggesting that there is a left-side analog to the HPV/HBV distinction. Hence these two issues cannot produce the same matrix that Dan produced with gun control. But, they can each produce half, which is all that is necessary to test my hypothesis. Actually, we really only need one quarter of the matrix - the absence of expected effect for righties quadrant - we don't need presence of opposite-to-expected effect (also because unlikely anyone would believe that HPV would lower promiscuity, or that aspirin would increase pain).
So we have:
1. aspirin test on righties, expected pain relief vs. no effect
2. gun control test on righties, expected raises crime vs. no effect
3. HPV/HBV test on righties., expected raises promiscuity vs. no effect
I am unsure if all 3 would produce the same bias, or if only 1&2 would, or only 2&3 would. I guess it is even possible that only 2 produces the bias.
I would label the bias "identity protective" if 2&3 showed the bias, and 1 did not. If they all did, I still think it is somewhat misleading to call the bias "identity protective" because it certainly isn't in 1. If only 2 did, then I also think it is somewhat misleading to call the bias "identity protective" because it certainly isn't in 3.
Oops - 3 should be:
3. HPV test on righties., expected raises promiscuity vs. no effect
The HBV would just be an additional control, like the skin rash tests.
“Maybe this is one of the ways we are misunderstanding each other:”
Not for this point at least. I believe I do get where you are coming from, yet nevertheless I think your perspective is wrong. Despite much higher figures for gunshot incidents in the US than say the UK, only a very small fraction indeed of the US general public, whether urban or rural, *actually* fires a gun in defense of their property. Even for the very few having done so, this experience provides diddly squat to inform them in an unbiased fashion of the totality of the social impact of gun controls. However, a large proportion of both the urban and rural populations have strong opinions about using guns to defend their property, and the pros and cons of same for themselves and society, which opinions are cultural and polarized. And notwithstanding necessary rural management practices, hunting for any sizeable fraction of the population is essentially a cultural pursuit, and hence so is defense or challenge to its continuance. For the vast majority of conservatives your point does not in any way amount to an experience of reality, which means a receival of culturally *unbiased* knowledge and understanding on the net social impacts of the issue, as these exist at the present time. Nor are liberals in the general public receiving of same. As noted above, for a long-term socially conflicted issue with high polarization, the public is largely isolated from unbiased information, but very exposed to cultural conflicts on the issue in the media. Even where correct info happens to exist, it is in a cultural frame. There may well be an urban / rural slant to the level of beliefs, this seems highly plausible. Yet as also noted above this merely means that cultural identity is more complex and subtle than left - right political axis only, and has some contributions from various axes. Religiosity is another component for instance, and in the US its influence strongly colors the political axis and vice versa. But it remains the case that the left - right axis is a strong indicator of the main affiliations on gun-control, which are a feature of cultural identity.
“So we have: 1… 2… 3…”
It is not valid to present a sub-sample result as though it applies to the whole sample *in the same manner* as other listed sub-sample results, when this is not the case. This leads to the invalid assumption that all of the listed results form a set (i.e. have the same characteristics), when this is not so. The wrongly presented result does not support the assumption of a set. Below is an equivalent of what you are doing:
1) A majority of conservatives do not go nude in public
2) A majority of conservatives do not attend public rallies sponsored by the Democratic party
3) A majority of conservatives vote for the Republican party.
[Wrong] conclusion: 2) and 3) are not demonstrated as being due to conservative character, because 1) is not, and 1 to 3 are a set.
When using items like 2) and 3) in a simple one axis comparison, the typically unexpressed logical context (i.e. unless stated otherwise) is that the rest of the sample does *not* behave the same way. E.g. in this example case the liberals in the sample (and by extension the whole population) would behave more or less oppositely. However, for result 1) the liberals behave identically! Hence result 1) neither supports nor invalidates anything about conservative characteristics.
In your own list and assuming a significant result for 1), you are identically attempting to use this result to say that 2) and 3) are not due to IP because 1) is not and the list is a set. But the list is not a set, and this exercise is just as invalid as above!
Your actual item 3) based on HPV is more complex. Yet I think your approach may have incorporated another confounding factor. From what I recall on Dan's data, HPV is a polarized issue in the US, hence there will be bias on *both* sides. I.e. there *is* a left-right analog. But by using the cultural reason / excuse for rejection as a filter, instead of merely support or opposition, you morph the framing of the issue into something that looks much more like it is just one-sided. But this is an artifact of that filter. For some cultural conflicts one side may indeed be mostly 'right', as determined for instance by the ultimate perspective of future history. For other conflicts, maybe neither side will be proved right. But even where one side does happen to be mostly right, morphing the frame to perceive bias on one side only, obscures the fact that many on the 'right' side will be right for the wrong reason. I.e. not because they have an objective grasp of the truth, but merely because they believe in their cultural side. Hence you will then be surprised when cultural effects are detected on the 'right' side, which you thought should not have bias effects. Depending on the level of polarization (I don't recall), I would expect a test based on HPV to get similar results to gun control (despite potentially more Lib / Con asymmetry in the data Dan combines to make up the total identity challenge / support positions). I think a wording based closer to just support or opposition would maximize cultural loyalty effects in the responses to see this, notwithstanding that questions about promiscuity would be useful to other ends.
"I would label the bias "identity protective" if 2&3 showed the bias, and 1 did not. If they all did, I still think it is somewhat misleading to call the bias "identity protective" because it certainly isn't in 1"
In truth I'm not certain how HPV would work out, not least as I haven't looked back at Dan's prior data on this (and HBV is not conflicted). However, per above it is invalid to use 1) in the way you attempt here. So whatever 1) shows, it doesn't support your objections.