Still another cognitive-style steel cage match: CRT vs. AOT go "head to head" on belief in climate change & belief in evolution
The carnage continues! SCS (aka "Science Curiosity Scale") is taking a rest after having bashed its way to the top of the open-mindedness rankings, but this week we bring you CRT vs. AOT in a match arranged by the Don King of the cognitive-style steel cage match world, Jonathan Corbin!
If You Open Your Mind Too Much Your Brain Might Fall Out, But At Least It’ll Have a Parachute to Soften the Landing
Frank Zappa said, “A mind is like a parachute. It doesn’t work if it is not open” (though Thomas Dewar may have coined the phrase). This is the motivation behind the psychological scale measuring actively open-minded thinking (AOT; Baron, 2008). AOT is a self-report scale meant to measure the tendency to which an individual seeks out information that conflicts with one’s own beliefs. So, simply having an open mind is only half of what this scale tries to measure – the other half is the propensity to look for information that disagrees with your current beliefs. At first look, AOT seems like a silver bullet in terms of understanding why some people seem so resistant to scientific information that threatens their beliefs.
Recent work by Dan Kahan and colleagues has shown that another individual difference measure – Science Curiosity – has been shown to relate to increased acceptance of human-caused global warming regardless of political affiliation. Whereas performance measures like the Cognitive Reflection Test (which measures some combination of impulse control/inhibition and mathematical ability) and measures of scientific knowledge predicted increased polarization on politically charged scientific issues like climate change, science curiosity predicted the opposite! As soon as I saw this result, I was immediately curious about how the AOT would do in such a comparison. The obvious prediction is that AOT should perform just like science curiosity – an increased predilection for seeking out information that disagrees with one’s beliefs should definitely predict increased acceptance of human-caused climate change!
Dan was nice enough to direct me to his publicly available dataset in which they measured climate change beliefs as well as AOT (along with CRT, science knowledge, and many other variables), allowing us to test the hypothesis that individuals higher in AOT should be more accepting of climate change regardless of political affiliation. As you’ve probably guessed if you read Dan’s previous post, it turns out that AOT was more similar to performance measures like the CRT, showing greater polarization with higher scores on the scale.
So, unfortunately it appears to be the case that AOT is not the silver bullet that I once thought it could be. Perhaps, rather than Zappa’s quote of the mind as a parachute, I should be looking to Tim Minchin, who said, “If you open your mind too much, your brain might fall out.” To further explore this pattern, I looked at another contentious topic – evolution. Rather than examining political identification, for this analysis, I relied on religiosity (given that there is also a reason for many highly religious individuals to deny evolution as an identity protective measure). The other reason I looked at religiosity is that there is a lot of AOT research linking higher religiosity with lower AOT. This is interpreted as evidence that greater religiosity is associated with a heavier reliance on associative intuition (or “going with your gut”) as opposed to deliberative thinking (Gervais & Norenzayan, 2012; Pennycooke et al., 2013). Few (if any) other studies collect nationally representative samples with such a large number of participants, so Kahan’s ordinary science intelligence dataset allowed us to test whether greater AOT in religious individuals relates to increased acceptance of evolution.
Results show a similar pattern to the climate change question, with CRT and AOT behaving similarly in that higher AOT failed to predict greater acceptance of evolution in the highly religious.
If there is any consolation, it is that we can say that higher AOT in the highly religious did not predict decreased belief in evolution. However, this data certainly does not give hope for the prediction that belief should increase with greater AOT among the highly religious. Similar to political identity and climate change, whereas the overall relationship between AOT and belief in Evolution remains positive, broken down by religiosity, the picture quickly becomes more complicated.
By now, you are probably asking yourself whether there is any real difference between the CRT and AOT. Definitionally, they are distinct (though expected to share variance), however, so far I haven’t given you much data to encourage that belief. Well first of all, there is other research out there to support a difference. For example, Haran, Ritov, and Mellers (2013) examined both AOT and CRT scores in relation to forecasting accuracy and information acquisition (basically what predicts how much information you’re willing to take in as well as your accuracy in predicting an outcome related to such information). They demonstrated that AOT predicted superior forecasting over and above any effect of CRT (and this was mediated by information acquisition).
We can also look for differences in the ordinary science intelligence dataset that we previously examined. Rather than looking at individualls' belief in evolution, I analyzed level of agreement with the statement ““From what you’ve read and heard, is there solid evidence that the average temperature on earth has been getting warmer over the past few decades, or not?”. This question differed from the last, in that it does not ask about agreement with human-caused climate change – it only asks if there is solid evidence based on “what you’ve read and heard”. The data showed that there was no main effect of CRT and no interaction between CRT and political affiliation (political affiliation did predict agreement with conservatives less likely to agree than liberals). However, AOT did show a significant relationship, predicting greater agreement.
So, where does this leave us? It seems that although AOT is likely distinct from performance measures like the CRT, it falls into the same trap when it comes to science issues that generate conflicts with individuals’ identities. Despite the fact that AOT is meant to measure one’s propensity toward seeking out belief-inconsistent information, it fails to predict higher levels of agreement with evidence-based claims that cue these identities.
Given the final analysis reported here (and the literature as a whole), claiming that the result boils down to measurement error is probably incorrect. It is more likely that one’s propensity to seek out information (particularly information that conflicts with one’s beliefs) is simply insufficient in countering the strength of cultural identity in swaying reasoning. With regards to the evidence for human-caused climate change, there is an enormous amout of information available online. Simply see the following website for a list of arguments in favor and against human-caused climate change. This seems to be the perfect resource for someone high in AOT. However, a lot of these arguments on both sides are technical, and it is possible that someone high in AOT may not be satisfied with trusting experts’ interpretation of the evidence, and would rather judge for themselves. The need to judge for themselves mixed with the desire to come to conclusions that support one’s identity could very well increase polarization (or at the very least lead to no increase in support for those who’s identities support disagreement). (It is worth the reminder that these are post-hoc explanations that require testing).
So, is Zappa correct in that an open mind a parachute or should we listen to Minchin who says that it is a recipe for losing one’s brain? Well, the answer (because it is psychology) is--it depends! When dealing with non-polluted science topics you should expect a positive relationship between AOT and agreement (maybe above and beyond performance measures like the CRT). However, once you throw in the need to protect one’s identity, AOT is not going to be the solution. So, why is science curiosity different from AOT? Perhaps science curiosity is less about belief formation and more of a competing identity. Whereas AOT is focused on how someone forms and changes beliefs, science curiosity is simply the need to consume scientific information. Maybe instead of trying to throw information at people hoping that it’ll change their minds, we should start fostering a fascination with science.
Baron, J. (2008). Thinking and Deciding. Cambridge University Press.
Gervais, W. M., & Norenzayan, A. (2012). Analytic thinking promotes religious disbelief. Science, 336(6080), 493-496.
Haran, U., Ritov, I., & Mellers, B. A. (2013). The role of actively open-minded thinking in information acquisition, accuracy, and calibration. Judgment and Decision Making, 8(3), 188.
Kahan, D. M. (2016). ‘Ordinary science intelligence’: a science-comprehension measure for study of risk and science communication, with notes on evolution and climate change. Journal of Risk Research, 1-22.
Pennycook, G., Cheyne, J. A., Barr, N., Koehler, D. J., & Fugelsang, J. A. (2014). Cognitive style and religiosity: The role of conflict detection. Memory & Cognition, 42(1), 1-10.