follow CCP

Recent blog entries
Wednesday
Dec172014

We need a CRT 2.0! And IRT should be used to develop it

I really really really like the Cognitive Reflection Test--or "CRT" (Frederick 2005).

The CRT is a compact three-item assessment of the disposition to rely on conscious, effortful, "System 2" reasoinng as opposed to rapid, heuristic-driven "System 1" reasoning.  An objective or performance-based measure, CRT has been shown to be vastly superior to self-report measures like "need for cognition" ("agree or disagree-- 'thinking is not my idea of fun'; 'The notion of thinking abstractly is appealing to me' . . .") in predicting vulnerability to the various biases that reflect over-reliance on System 1 information processing  (Toplak, West & Stanovich 2011).

As far as I’m concerned, Shane Frederick deserves a Nobel Prize in economics for inventing this measure every bit as much Daniel Kahneman deserved his for systematizing knowledge of the sorts of reasoning deficits that CRT predicts.

Nevertheless, CRT is just not as useful for the study of cognition as it ought to be. 

The problem is not that the correct answers to its three items are too likely to be known at this point by M Turk workers—whose scores exceed those of MIT undergraduates (Chandler, Mueller & Paolacci 2014).

This is what CRT score distribution looks like when test is administered to normal people (i.e., not M Turk workers, ivy league college students, people who fill out surveys at on-line sites that solicit study subjects who want to learn their CRT scores, etc)Rather the problem is that CRT is just too darn hard when used to study legitimate study subjects.

The mean score when it is administered to a general population sample is about 0.65 correct responses (Kahan 2013; Weller, Dieckmann, Tusler, Mertz, Burns & Peters 2012; Campitelli & Labollita, 2010).

The median score is 0.

Accordingly, if we want to study how individual differences in System 1 vs. System 2 reasoning styles  interact with other dynamics—like motivated reasoning—or respond to interventions designed to improve engagement with technical information, then for half the population CRT necessarily gives us zero information.

Unless one makes the exceedingly implausible assumption that there's no variance to measure among this huge swath of people, this is a severe limitation on the value of the measure.

I've addressed this previously on this blog but I had occasion to underscore and elaborate on this point recently in correspondence with a friend who does outstanding work in the study of cognition and who (with good reason) is a big fan of CRT.

Here are some of the points I made:

I don’t doubt that CRT measures the disposition to use System 2 information processing more faithfully than, say, Numeracy [a scale that assesses proficiency in quantitative reasoning]. 

But the fact remains that Numeracy outperforms CRT in predicting exactly what CRT is supposed to predict—namely vulnerability to heuristic biases (Weller et al. 2012; Liberali 2012). Numeracy is getting a bigger piece of the latent disposition that CRT measures—and that's strong evidence of the need for a better CRT.

Or consider the Ordinary Science Intelligence assessment, “OSI_2.0,” the most recent version of a scale I've been working on to measure a disposition to recognize and give appropriate effect to scientific information relevant to ordinary, everyday decisions (Kahan 2014).  

Cognitive reflection is among the combination of reasoning proficiencies that this (unidimensional) disposition comprises.

But for sure, I didn't construct OSI_2.0 to be "CRT_2.0.”  I created it to help me & others do a better job in investigating how to asses the relationship between science comprehension and dynamics that constrain the effectiveness of public science communication.

With Item Response Theory, one can assess scale reliability continuously along the range of the underlying latent disposition (DeMars 2010).  Doing so for OSI_2.0, it can be seen that what CRT contributes to OSI_2.0’s measurement precision is concentrated at the very upper end of the range of the "ordinary science intelligence" aptitude:

 

This feature of CRT can be shown to make CRT less effective at what it is supposed to do—viz., predict individual differences in the disposition to resist over-reliance on heuristic processing.

The covariance problem is considered diagnostic of that sort of disposition (Stanovich 2009, 2011). Those vulnerable to over-reliance on heuristic processing tend to make snap judgments based on the relative magnitudes of the numbers in “cell A” and either “cell B” or “cell C” in a 2x2 contingency table or equivalent. Because they don't go to the trouble of comparing the ratio of A to B with the ratio of C to D, people draw faulty inferences about the significance of the information presented (Arkes & Harkness 1983).

As it should, CRT predicts resistance to this bias (Toplak, West & Stanovich 2011).

But not as well as OSI_2.0.

Consider:

These are scatter plots of performance on the covariance problem (N = 400 or so) in relation to OSI_2.0 & CRT, respectively, w/ lowess regression plots superimposed.

The crook in  profile of the OSI_2.0 plot compared to the flat, boring profile of CRT shows that the former has superior discrimination (that is, identifies in a more fine-grained way how differences in reasoning ability affect the probability of getting the right answer).

Relatedly, the interspersing of the color-coded observations on the OSI_2.0 scatter plot shows how CRT is dividing people into groups that are both under- & over-inclusive w/r/t to proficiencies that OSI_2.0 is sorting out more reliably.

Or more concretely still, if I had only CRT, then I'd predict that  there is only a 40% probability that someone who is +1 on OSI_2.0-- just short of "1" on CRT -- would get the covariance problem correct, when in fact the probability such a person will get the right answer is about  60%. 

Similarly, if I used CRT to predict how someone at +1.5 on OSI_2.0 is likely to do on the problem, I'd predict about a 50% probability of him or her selecting the correct response -- when in fact the probability of a correct response for that person is closer to 75%.

Essentially, I'm going to be as satisfied with CRT as I am in OSI_2.0 only if  my interest is to predict performance of those who score either 2 or 3 on CRT -- the 90th percentile or above in a general population sample. 

But as can be seen from the OSI_2.0 scatter plot, it’s simply not the case that there’s no variance in people’s vulnerability to this particular heuristic bias in the rest of the population.  A measure that can't enable examination of how so substantial a fraction of the population thinks should really disappoing cognitive psychologists, assuming their goal is to study critical reasoning in human beings generally.

click on me-- your CRT score will instantly jump 2 points!Now, it's absolutely no surprise that OSI_2.0 dominates CRT in this regard: the CRT items are all members of  the OSI_2.0 scale, which comprises 18 items the covariance structure of which is c consistent with measurement of a unidimensional latent disposition.  So of course it is going to be a more discerning measure of whatever it is CRT is itself measuring -- even if CRT_2.0 isn't faithfully measuring only that, as CRT presumably is.

 

But that’s the point: we need a “better” CRT—one that is as tightly focused as the current version on the construct the scale is supposed to measure but that gets at least as big a piece of the underlying disposition as OSI_2.0, Numeracy or other scales that outperform CRT in predicting resistance to heuristic biases.

For that, "CRT 2.0" is going to need not only more items but items that add information to the scale in the middle and lower levels of the disposition that CRT is assessing.  IRT is much more suited for identifying such items than are the methods that those working on CRT scale development now seem to be employing.

I could certainly understand why a researcher might not want a scale with as many as 18 items. 

But again IRT can help here: use it to develop a longer, comprehensive battery of such items, ones that cover a large portion of the range of the relevant disposition.  Then administer an "adaptive testing" battery that uses strategically selected subsets of items to zero in on any individual test-taker’s location on the range of the measured “cognitive reflection” disposition (DeMars 2010).  Presumably, no one would need to answer From Mueller, Chandler, & Paolacci, Soc'y for P&SP, 1/28/12more than half dozen in order to enable a very precise measure of his or her proficiency -- assuming one has a good set of items in the adaptive testing battery.

Anyway, I just think it is obvious that researchers here can and should do better--and not just b/c MTurk workers have all learned at this point that the ball costs 5 cents!

References

Arkes, H.R. & Harkness, A.R. Estimates of Contingency Between Two Dichotomous Variables. J. Experimental Psychol. 112, 117-135 (1983).

Campitelli, G. & Gerrans, P. Does the cognitive reflection test measure cognitive reflection? A mathematical modeling approach. Memory & Cognition, 1-14 (2013).

Chandler, J., Mueller, P. & Paolacci, G. Nonnaïveté among Amazon Mechanical Turk workers: Consequences and solutions for behavioral researchers. Behavior research methods 46, 112-130 (2014).

DeMars, C. Item response theory (Oxford University Press, Oxford ; New York, 2010).

Frederick, S. Cognitive Reflection and Decision Making. Journal of Economic Perspectives 19, 25-42 (2005).

Kahan, D.M. Ideology, Motivated Reasoning, and Cognitive Reflection. Judgment and Decision Making 8, 407-424 (2013). 

Kahan, D.M. "Ordinary Science Intelligence: A Science Comprehension Measure for Use in the Study of Science Communication, with Notes on "Belief in" Evolution and Climate Change. CCP Working Paper No. 112 (2014).

Liberali, J.M., Reyna, V.F., Furlan, S., Stein, L.M. & Pardo, S.T. Individual Differences in Numeracy and Cognitive Reflection, with Implications for Biases and Fallacies in Probability Judgment. Journal of Behavioral Decision Making (2011).

Stanovich, K.E. Rationality and the reflective mind (Oxford University Press, New York, 2011).

Stanovich, K.E. What intelligence tests miss: the psychology of rational thought (Yale University Press, New Haven, 2009).

Toplak, M., West, R. & Stanovich, K. The Cognitive Reflection Test as a predictor of performance on heuristics-and-biases tasks. Memory & Cognition 39, 1275-1289 (2011).
 

Weller, J.A., Dieckmann, N.F., Tusler, M., Mertz, C., Burns, W.J. & Peters, E. Development and testing of an abbreviated numeracy scale: A Rasch analysis approach. 

Saturday
Dec132014

Weekend update: More on the wisdom of SE Fla's *political science* of climate change

From correspondence with friends . . .

As you know, I think the science-communication brilliance of the SE Fla Regional Climate Compact is its recognition that constructive public engagement w/ climate change doesn't depend on identifying "magic words" or "frames," or on finding charismatic "conservative messengers" (Hank Paulsen? seriously?).

Rather it depends on creating and protecting a conversation that enables diverse citizens to apply their reason to protecting their shared way of life as opposed to a conversation that forces them to use their reason to protect the status of their particular cultural group & their own personal standing within it....

Subject of course as always to revising my understanding as I learn more, I'm convinced that that the mission for serious empirical researchers is now to help all those who don't yet get the "right conversation"principle to understand the importance of it; & to help those who already do get it to have all the information they need to create & protect that conversation as effectively as they can.

Here is something that strikes me as produced by some smart folks in the latter category. I've had some occasion to observe, w/ both casual & structured empirics, what is going on in Australia. I think Australia is as close to US as any other country (I love watching their tv shows)! And not surprisingly, it has its own SE Floridas.


Wednesday
Dec102014

Project disentanglement ... a fragment

from something I'm working on . . . 

Project Disentanglement

I. CCP is currently involved in a series of interlocking initiatives. Spanning a variety of  settings, these initiatives are animated by a common objective: the extrication of science from cultural conflict by use of the science-communication disentanglement principle.

II. Cultural conflict over what is known by science is not the norm. It occurs only when risks and like facts become entangled in antagonistic cultural meanings, which effectively transform positions on them into badges of membership in opposing groups. In such circumstances, the interest individuals have in protecting their connections to others with whom they share important social ties can exceed the personal stake they have in forming beliefs consistent with the best available evidence. It can thus become individually rational—albeit collectively disastrous—for people to use their reason to maintain beliefs consistent with the ones predominant in their cultural groups (Kahan 2012). Indeed, polarization rooted in this dynamic—known as identity-protective cognition—is most intense among individuals  highest in science literacy (Kahan 2013; Kahan, Peters et al. 2012).

III. The only means to neutralize identity-protective cognition is to dispel the conflict cultural diverse individuals experience between recognizing valid science and forming beliefs that express their defining commitments (Kahan in press). The disentanglement principle describes the fundamental imperative of effective science communication under such circumstances: to free reasoning individuals from having to choose between knowing what is known by science and being who they are.

IV. “Project disentanglement” is dedicated to enabling science communication professionals to implement the disentanglement principle. The Project contemplates two sets of complementary practical research initiatives.

V. The climate-science education initiative will focus on teaching of climate science at the secondary-school level. The disentanglement principle is in fact derives from classic studies on teaching evolution to high school students (Lawson & Worsnop 2006). Such research showed it was possible—indeed, indispensable—to divorce the opportunity to learn  evolutionary science from the psychological experience of being forced to “assent” to propositions inimical to religious students’ defining commitments.

The climate-science education initiative will adapt these techniques to secondary-school climate-science education. The same tension between recognizing what’s known to science and maintaining fidelity to defining cultural commitments is now widely recognized as threatening education in this critical area of science, too. Working with education researchers, CCP is devising project-based learning materials, on the theory that rooting instruction in familiar local issues is distinctively suited to disentangling climate-science  knowledge from the antagonistic meanings that pervade the climate debate nationally.

VI. While intrinsically valuable, the climate-science education initiative is also expected to generate insights of value for research on the disentanglement principle in local political decisionmaking. The evidence-based science communication initiative is committed to furnishing science-communication support services to local governments pursuing adoption of environmental and conservation policies (Kahan 2014).

Communication strategies featuring the disentanglement principle have been the central focus of  the Southeast Florida Science Communication Initiative, a collaborative partnership between CCP and the Southeast Florida Regional Climate Compact. The four member Counties (Broward, Miami-Dade, Monroe, and Palm Beach) have generated widespread public support for a multifaceted Climate Action Plan despite the high degree of cultural polarization that characterizes public opinion on climate change in the region, just as it does in the rest of the U.S. (Kahan in press).

Just as we anticipate that insights gleaned from the climate-education initiative can be used to advance the aims of programs like the Southeast Florida Evidence-based Science Communication Initiative, so we believe that research in the setting of local decisionmaking can support development of effective climate-science education in secondary schools. Indeed, appropriate project-based learning programs in area high-schools can be seamlessly integrated into larger science-communication packages used to support public engagement with valid science in local decisionmaking. Positive impressions of the effectiveness of project-based learning can materially contribute to the disentanglement of scientific knowledge and identity in the community at large as its diverse members deliberate on how to meet the environmental challenges the face.

VII.  The toll that a polluted science communication environment exacts on human reason is in fact one of the principal impediments to the use of science to protect our natural environment.  But we can use reason to protet reason. Research on the science-communication disentanglement principle is critical to the development of a new ethos of science communication environment protection.

References

Kahan, D. (2012). Why we are poles apart on climate change. Nature, 488, 255.

Kahan, D. M. (2013). Ideology, Motivated Reasoning, and Cognitive Reflection. Judgment and Decision Making, 8, 407-424.

Kahan, D. M. (2014). Making Climate-Science Communication Evidence-Based—All the Way Down. In M. Boykoff & D. Crow (Eds.), Culture, Politics and Climate Change. New York: Routledge Press.

Kahan, D. M., Peters, E., Wittlin, M., Slovic, P., Ouellette, L. L., Braman, D., & Mandel, G. (2012). The polarizing impact of science literacy and numeracy on perceived climate change risks. Nature Climate Change, 2, 732-735.

Kahan, D.M.(in press).Climate science communication and the Measurement Problem. Advances in Pol. Psych.

Lawson, A. E., & Worsnop, W. A. (2006). Learning about evolution and rejecting a belief in special creation: Effects of reflective reasoning skill, prior knowledge, prior belief and religious commitment. Journal of Research in Science Teaching, 29(2), 143-166.

 

Friday
Dec052014

CCP's "Evidence-based Science Communication Initiative" (EBSCI)

In recent years, the field of science communication has been marked by both progress and frustration.  On one hand, basic research has yielded a wealth of new insights into the processes by which scientific information is acquired and interpreted by the public.  On the other, increasingly elaborate and costly initiatives to communicate scientific information have spectacularly failed to dispel cultural conflict over climate change and other disputed science issues.

The reason the science of science communication is yet to generate real-world benefits, we believe, is that it is yet to genuinely set foot in the real world.

Click to read more ...

Tuesday
Dec022014

On (confused, confusing) "belief-fact" distinction -- a fragment

From revised version of The Measurement Problem: 

 As used in this paper, “believe in” just means to “accept as true.” When I use the phrase to characterize a survey item relating to evolution or global warming, “belief in” conveys that the item certifies a respondent’s simple acceptance of, or assent to, the factual status of that process without assessing his or her comprehension of the evidence for, or mechanisms behind, it. I do not use “belief in” to align myself with those who think they are making an important point when they proclaim that evolution and climate change are not “mere” objects of “belief” but rather “scientifically established facts.” While perhaps a fitting retort to the schoolyard brand of relativism that attempts to evade engaging evidence by characterizing an empirical assertion as “just” the “belief” or “opinion” of its proponent,  the “fact”–“belief” distinction breeds only confusion when introduced into grownup discussion. Science neither dispenses with “belief” nor distinguishes “facts” from the considered beliefs of scientists. Rather, science treats as facts those propositions worthy of being believed on the basis of evidence that meets science’s distinctive criteria of validity. From science’s point of view, moreover, it is well understood that what today is appropriately regarded as a “fact” might not be regarded as such tomorrow: people who use science’s way of knowing continuously revise their current beliefs about how the universe works to reflect the accumulation of new, valid evidence (Popper 1959).


Monday
Dec012014

Distrust of "trust in science" measures--crisis solved? 

As interesting things come in over the transom, I put them in a pile--right next to the transom--marked "to read." 

At this point, the pile is taller than the transom itself! I'm not joking!

And just this second I have descended the ladder after placing this newly arrived item on top of the pile:

Trust in science and scientists can greatly influence consideration of scientific developments and activities. Yet, trust is a nebulous construct based on emotions, knowledge, beliefs, and relationships. As we explored the literature regarding trust in science and scientists we discovered that no instruments were available to assess the construct, and therefore, we developed one. Using a process of data collection from science faculty members and undergraduate students, field testing, expert feedback, and an iterative process of design, we developed, validated, and established the reliability of the Trust in Science and Scientist Inventory. Our 21-item instrument has a reliability of Cronbach's alpha of .86, and we have successfully field-tested it with a range of undergraduate college students. We discuss implications and possible applications of the instrument, and include it in the appendix.

At the present rate, I should be able to read it by April 22, 2019.

But I'm sort of eager to know what it says sooner than that.  That's because of all the recent discussion arising from recent posts (e.g., here, here, here, & here) on "trust in science"/"confidence in science"/"anti-science"/"we all love science!" measures.

The upshot of all that discussion seems, in my mind at least, to be this: there just isn't any validated measure of "trust in science/scientists" item or scale of the sort that one could use to support reliable inferences in a correlational study.  

Us vs. them: we all love science!!!!!! (click & see)There are, on the one hand, a bunch of "general science affect measures" ("on a scale of 1 to a billion, how 'cool' is science?"; "on a scale of 10^45 to 10^97, how much do you love science?") that all seem to show that everyone, including "anti-science" conservatives and religious fundamentalists who deny the earth goes around the sun, reveres science.

On the other, there are "domain-specific science affect measures" that ask "how much do you trust scientists who say things like global warming is happening/gm foods are yummy/what's good for 'GM' [i.e., General Motors] is good of Amerika" etc. These find, not surprisingly, that the answer depends on what one's attitude is toward global warming/gm foods, industry etc. That's because domain-specific trust items are measuring the same thing as items that measure attitudes toward (including "risk perceptions of") the thing in question: namely, some general affective, yay-or-boo orientation toward whatever it is (global warming, gm foods, industry, etc).

Proposed survey item: "This figure shows (A) 'we all love science,' (B) 'dramatic decline' in conservative 'trust' in science, or (C) researchers need better 'trust in science' measures." Click to respond--& see how your choice match up against others (assuming you aren't first person to click)People who are passionate about the hypothesis that "distrust in science" explains controversy over science-informed policy issues such as, oh, global warming, distrust the "general affect" measures; they are "missing" some more subtle form of ambivalence, they conjecture, that people won't admit to or necessarily even be able to detect through self-inspection.

A reasonable reaction, certainly.

But there's a problem if those same people then whip out data using the "domain-specific affect" measures to support their view.  Because in that case, the evidence that "distrust in science or scientists" causes one or another science-informed policy controversy among "Hierarchs" & "egalitarians," "Republicans" & "Democrats," "born again Christians" & "atheists" -- persons who all swear they love science-- will consist of a correlation between two measures of one and the same thing.

That's called a tautology, which can be useful for some things but not for drawing causal inferences.

So is there anyway out of this dilemma?

Anway to solve this crisis of confidence/erosion of trust in measures of "distrust" in science/scientists?

Maybe this study is the solution!

But like I said, it'll be years before I can figure that out on my own (if I ever do; it's only a matter of time before the pile of materials sitting next to the transom topples over and crushes me . . . ).

Can any of you, the 14 billion readers of this blog, help out me & all the others too busy to get to this interesting looking study right now by taking a look & filing a report in the comments?

Thanks, fellow citizens!

Friday
Nov282014

Group conflict and risk perceptions: two accounts

This is just the first post in a series to address a very small question that I’m sure we can quickly dispose of.

But here’s the question:

I’m sure the vast majority of you need no further explanation.  But for newbies, this is a “tweet” from “Fearless Dave” Ropeik, the public risk perception expert who correctly believes it is irrational to worry about anything.  Likely you all remember the discussion we recently had about how Fearless Dave had his kids go over & play with the nextdoor neighbors’ children when they had Ebola because he figured it was much better for his kids to get the disease when they were young than when they were grown ups.  Of course—this is the perfect System 2 rationality we all aspire to!

But anyway, what he’s asking is—why do cultural affinities (like being an “egalitarian communitarian” as opposed to a “hierarch individualist”) make such a big difference in perceptions of the risk of climate chanage, or owning a handgun, or nuclear energy?

Fearless Dave doesn’t mean why as in “what are the mechanisms that generate such big disparities in the proportion of people of one type who believe that human beings are heating up the climate & the proportion of another type who believe that?”; he’s quite familiar with (and a very lucid expositor and insightful interpreter of) all manner of work on risk perception, including the research that shows how people of opposing identities conform all manner of information—from their intepretation of data to their assessments of arguments to their perception of the expertise of scientists to what they observe with their own eyes—to the position that predominates in their group.

What he wants to know is why these cognitive mechanisms are connected to group identities.  Why are people so impelled to fit their views to their groups'? And why do the groups disagree so intently?

Is there, Fearless Dave wonders, some sort of genetic hard wiring having to do with the evolutionary advantages, say, that “Democratic” or “nonreligious” cavepeople & “Republican” “religious” cavepeople got from forming opposing estimates of the risk of being eaten by a a sabre tooth tiger on the savannah--and then going to war w/ each other over their disagreement?

Really good question.

I don’t know.

But I and a few others twitterers offered some conjectures:

Now probably this exchange needs no explanation either.

But basically, I and Jay Van Bavel are disagreeing about the reason cultural identities generate conflicting perceptions of risk and like facts.  

Or maybe we aren’t.  It’s hard to say.

While Twitter is obviously the venue most suited for high-quality scholarly interaction, I thought I’d move the site of the exchange over to the CCP Blog--so that you, the 12 billion regular readers of this blog (for some reason 2 billion people unsubscribed after my last post!),  could participate in it too.

Just to get the ball of reasoned discussion rolling, I’m going to sketch out two competing answers to Fearless Dave’s question: the “Tribal Science Epistemologies Thesis” (TSET) and the “Polluted Scicomm Environment Thesis” (PSET). The answers aren't "complete" even on their own terms, but they convey the basics of the positions they stand for and give you a sense of the attitudes behind them too.

TSET. People are by nature factional. They use in-group/out-group distinctions to organize all manner of social experience—familial, residential, educational, occupational, political, recreational (“f***ing Bucky Dent!”).  The ubiquity of this impulse implies the reproductive advantage it must have conferred in our formative prehistory. Its permanence is testified to by the unbroken narrative of violent sectarianism our recorded history comprises.

The mechanisms of cultural cognition reflect our tribal heritage. The apprehension of danger in behavior that deviates from a group’s norms fortifies a group’s cohesion. Imputing danger to behavior characteristic of a competing group’s norms helps to stigmatize that group’s members and thus lower their status.  Cultural cognition thus reliably converts the fears and anxieties of a group’s members into the energy that fuels that group’s drive to dominate its rivals.

In a democratic political order, these dynamics will predictably generate cultural polarization. Opposing positions on societal risks (climate change, gun ownership, badger infestation)  supply conspicuous markers of group differentiation. Democratically enacted policies endorsing o rejecting those positions supply evocative gestures for remarking the relative status of the groups that hold them.

Nothing has really changed.  Nothing ever will. 

PSET. Cultural conflict over risk and related facts is not normal. It is a pathology peculiar to the pluralistic system of knowledge certification that characterizes a liberal democratic society. 

Individuals acquire their understanding of what is known to science primarily through their everyday interactions with others who share their basic outlooks. Those are the people they spend most of their time with, and the ones whose professions of expertise they can most reliably evaluate. Because all self-sustaining cultural groups  include highly informed members and intact processes for transmitting what they know, this admittedly insular process nevertheless tends to generate rapid societal convergence on the best available evidence.  

But not always. The sheer number of diverse groups that inhabit a pluralistic liberal society, combined with the tremendous volume of scientific knowledge such a society is distinctively suited to generating, makes occasional states of disagreement inevitable.

Even these rare instances of nonconvergence are likely to be fleeting.

But if by some combination of accident, misadventure, and strategic behavior, opposing perceptions of risk become entangled in antagonistic cultural meanings, dissensus is likely to endure and feed on itself. The material advantage any individual acquires by maintaining her standing within her cultural groups tends to exceed the advantage of holding personal beliefs in line with the best evidence on societal risks. As a result, when people come to regard  positions on risk as badges of membership in one or another group, they will predictably use their reason to persist in beliefs that express their cultural identities.

This identity-protective variant of cultural cognition is the signature of a polluted science communication environment.  The entanglement of risks in antagonistic cultural meanings disables human reason and deprives the citizens of the Liberal Republic of Science of their political regime’s signature benefits: historically unprecedented civil tranquility and a stock of collective knowledge bountiful enough to secure their well-being from all manner of threat, natural and man-made.

But we can use our reason and our freedom to overcome this threat to our reason and our freedom.  Dispelling the toxin of antagonistic cultural meanings from our science communication environment is the aim of the science of science communication—a “new political science for a world itself quite new.”

So? Which is closer to the truth—TEST or PSET? 

What are the key points of disagreement between them? What might we already know that helps us to resolve these disagreements, and what sorts of evidence might we gather to become even more confident?

What are the alternatives to both TEST and PSET? Why might we think they are closer to the truth? How could we pursue that possiblity through observation, measurement, and inference?

And what does each of the candidate accounts of why “group affiliation” has such a profound impact on our perception of risk and like facts imply about the prospects for overcoming the barrier that cultural polarization poses to making effective use of scientific knowledge to promote our ends, individual and collective?

BTW, why do I say "closer to the truth" rather than "true"? Because obviously neither TEST nor PSET is true, nor is any other useful answer anyone will ever be able to give to the Fearless Dave's question. The question isn't worth responding to unless the person asking means, "what's a good-enough model of what's going on--one that gives me more traction than the alternatives in explaining, predicting, and managing things?"

So ... what's the answer to Fearless Dave's question? Do TEST & PSET help to formulate one?

Thursday
Nov272014

Liberals trust in Supreme Court plummets! Less than 25% of them would agree to have Steve Breyer housesit for them when they want on vacation!

Actually, I think all we can say is that neither liberals nor consersvatives hold the U.S. Supreme Court in as high regard as they both hold scientists.

But the Court shouldn't feel bad.  Nearly everyone is less respected than scienitstis.

 

 

 

Tuesday
Nov252014

Don't make free, reasoning people choose between learning posterior predictive model checking & *being who they are*!

Holy smokes--  former Freud expert & current stats legend Andrew Gelman is using the "disentanglement principle" to teach Bayesian statistics to frequentists! I'm not kidding!

For crying out loud, if he can pull that off, then surely science communicators can overcome cultural polarization on climate change.

 

Tuesday
Nov252014

"Conservatives lose faith in science over last 40 years"--where do you see *that* in the data? 

Note: Special bonus! Gordon Gauchat, the author of PSPS, wrote a reflective response that I've posted in a "followup" below.  I can't think or write as fast he does (in fact, I'm sort of freaked out by his speed & coherence), but after I think for a bit, I'll likely add something, too, since it is the case, as he says, that we "largely agree" & I think it might be useful for me to be even clearer about that, & also to engage some of the other really good interesting points he makes.

 This is a longish post, & I apologize for that to this blog’s 14 billion regular readers.  Honestly, I know you are all very busy.

To make it a little easier, I’m willing to start with a really compact summary.

But I’ll do that only if you promise to read the whole thing. Deal?

Okay, then.

This post examines Gordon Gauchat’s Politicization of Science in the Public Sphere, Am. Sociological Rev., 77, 167-187 (2012).

PSPS is widely cited to support the proposition that controversy over climate change reflects the “increasingly skeptical and distrustful” attitude of “conservative” members of the general public (Lewandowsky et al. 2013).

Is that supposed to be an elephant? Looks more like a snuffleupagus--everyone knows they don't believe in science (it's reciprocal)This contention merits empirical investigation, certainly.

But the data analyzed in PSPS, an admittedly interesting study!, don’t even remotely support it.

PSPS’s analysis rests entirely on variance in one response level for a single part of a multiple-part survey item.  The reported changes in the proportion of survey takers who selected that particular response level  for that particular part of the single item in question cannot be understood to measure “trust” in science generally or in any group of “scientists.”

Undeniably, indisputably cannot.

Actually—what am I saying? 

Sure, go ahead and treat nonselection of that particular response level to that one part of the single survey item analyzed in PSPS as evincing a “decline” in “trust of scientists” for “several decades among U.S. conservatives” (Hmielowski et al. 2013).

But if you do, then you will be obliged to conclude that a majority of those who identify themselves as “liberals” are deeply "skeptical" and “distrustful” of scientists too.  The whole nation, on this reading of the data featured in PSPS, would have to be regarded as having “lost faith” in science—indeed, as never having had any to begin with.

That would be absurd. 

It would be absurd because the very GSS survey item in question has consistently found—for decades—that members of the US general public are more “confident” in those who “run” the “scientific community” than they are in those who “run” “major companies,” the “education” system, “banks and financial institutions,” “organized religion,” the “Supreme Court,” and the “press.”

For the entire period under investigation, conservatives rated the “scientific community” second among the 13 major U.S. institutions respondents were instructed to evaluate.

If one accepts that it is valid to measure public "trust” in institutions by focusing so selectively on this portion of the data from the GSS "confidence in institutions" item, then we’d also have to conclude that conservatives were twice as likely to “distrust” those who “run . . . major companies” in the US as they were to “distrust” scientists .

That’s an absurd conclusion, too. 

PSPS’s analysis for sure adds to the stock of knowledge that scholars who study public attitudes toward science can usefully reflect on.

But the trend the study shows cannot plausibly be viewed as supporting inferences about the level of trust that anyone, much less conservatives, have in science.

That’s the summary.  Now keep your promise and continue reading.

A. Let’s get some things out of the way

Okay, first some introductory provisos

1. I think PSPS is a decent study.  The study notes a real trend & it’s interesting to try to figure out what is driving it.  In addition, PSPS is also by no means the only study by Gordon Gauchat that has taught me things and profitably guided the path of my own research.  Maybe he'll want to say something about how I'm addressing the data he presented (I'd be delighted if he posted a response here!).  But I suspect he cringes when he hears some of the extravagant claims that people make--the playground-like prattle people engage in--based on the interesting but very limited and tightly focused data he reported in PSPS.

2. There’s no question (in my mind at least) that various “conservative” politicians and conflict entrepreneurs have behaved despicably in misinforming the public about climate change. No question that they have adopted a stance that is contrary to the best available evidence, & have done so for well over a decade.

3. There are plenty of legitimate and interesting issues to examine relating to cognitive reasoning dispositions and characteristics such as political ideology, cutural outlooks, and religiosity. Lots of intriguing and important issues, too, about the connection between these indicators of identity and attitudes toward science.  Many scholars  (including Gauchat) and reflective commentators are reporting interesting data and making important arguments relating to these matters.  Nevertheless, I don’t think “who is more anti-science—liberals or conservatives” is an intrinsically interesting question—or even a coherent one.  There are many many more things I’d rather spend my time addressing.

But sadly, it is the case that many scholars and commentators and ordinary citizens insist there is a growing “anti-science” sensibility among a meaningful segment of the US population.  The “anti-science” chorus doesn’t confine itself to one score but “conservatives” and “religious” citizens are typically the population segments they characterize in this manner.

Advocates and commentators incessantly invoke this “anti-science” sentiment as the source of political conflict over climate change, among other issues.

Those who make this point also constantly invoke one or another “peer reviewed empirical study” as “proving” their position.

And one of the studies they point to is PSPS.

Because I think the anti-science trope is wrong; because I think it actually aggravates the real dynamics of cultural status competition that drive conflict over climate science and various other science-informed issues; because I think many reasonable people are nevertheless drawn to this account as a kind of a palliative for the frustration they feel over the persistence of cultural conflict over climate change; because I think empirical evidence shouldn’t be mischaracterized or treated as a kind of strategic adornment for arguments being advanced on other grounds; because I have absolutely no worries that another scholar would resent my engaging his or her work in the critical manner characteristic of the process of conjecture and refutation that advances scientific understanding; and because only a zealot or a moron would make the mistake of thinking that questioning what conclusions can appropriately be drawn from another scholar’s empirical research, criticizing counterproductive advocacy, or correcting widespread misimpressions is equivalent to “taking the side of” political actors who are misinforming the public on climate change, I’m going to explain why PSPS does not support claims like these:

 Have they actually read the study? click to see what they say ...

B. Have you actually read PSPS?

It only takes about 5 seconds of conversation to make it clear that 99% of the people who cite PSPS have never read it.

They don’t know it consists of an analysis of one response level to a single multi-part public opinion item contained in the General Social Survey, a public opinion survey that has been conducted repeatedly for over four decades (28 times between 1974 and 2012).

Despite how it is characterized by those citing PSPS, the item does not purport to measure “trust” in science. 

It is an awkwardly worded question, formulated by commercial pollsters in the 1960s, that is supposed to gauge “public confidence” in a diverse variety of (ill-defined, overlapping) institutions (Smith 2012):

I am going to name some institutions in this country. As far as the people running these institutions are concerned, would you say you have a great deal of confidence, only some confidence, or hardly any confidence at all in them?

a. Banks and Financial Institutions [added in 1975]

b. Major Companies

c. Organized Religion

d. Education

e. Executive Branch of the Federal Government

f. Organized Labor

g. Press

h. Medicine

i. TV

j. U.S. Supreme Court

k. Scientific Community

l. Congress

m. Military

For the period from 1974 to 2010, PSPS examines what proportion of respondents selected the response “a great deal of confidence” in those “running” the “Science community.”

 

As should be clear, the PSPS figure above plots changes only in the “great deal of confidence” response. 

I’m sure everyone knows how easy it is to make invalid inferences when one examines only a portion rather than all of the response data associated with a survey item

Thus, I’ve constructed Figures that make it possible to observe changes in all three levels of response for both liberals and conservatives over the relevant time period: 

As can be seen in these Figures, the proportion selecting “great deal” has held pretty constant at just under 50% for individuals who identified themselves as “liberals” of some degree (“slight,” “extreme,” or in between) on a seven-point ideology measure (one that was added to the GSS in 1974).

Among persons who described themselves as “conservatives” of some degree, the proportion declined from about 50% to just under 40%.  (In the 2012 GSS—the most recent edition—the figures for liberals and conservatives were 48% and 40%, respectively. I also plotted pcts for "great deal" in relation to the relevant GSS surveys "yesterday" in this post.)

The decline in the proportion of conservatives selecting “great deal” looks pretty continuous to the naked eye, but using a multi-level multivariate analysis (more on that below), PSPS reported finding that the decline was steeper after the election of Ronald Reagan in 1980 and George W. Bush in 2006.

That’s it.

Do you think that these data justify conclusions like "conservatives' trust in science has declined sharply," "conservatives have turned on science," "Republicans really don't like science," "conservatives have lost their faith in science," "fewer conservatives than ever believe in science," etc?  

If so, let me explain why you are wrong.

C.  Critically engaging the data

1. Is everyone anti-science?

To begin, why should we regard the “great deal of confidence” response level as the only one that evinces “trust”?

“Hardly any” confidence would seem distrustful, I agree.

But note that the proportion of survey respondents selecting “hardly any at all” held constant at under 10% over the entire period for both conservatives and liberals.

Imagine I said that I regarded that as inconsistent with the inference that either conservatives or liberals “distrust” scientists.

Could you argue against that?

Sure.

But if you did, you’d necessarily have to be saying that selecting “some confidence” evinces  “distrust” in scientists.

If you accept that, then you’ll have to conclude that a majority of “liberals” distrust scientists today,  too, and have for over 40 years.

For sure, that would be a conclusion worthy of headlines, blog posts, and repeated statements of deep concern among the supporters of enlightened self-government.

But such a reading of this item would also make the decision to characterize only conservatives as racked with “distrust” pathetically selective.

2.  Wow--conservative Republicans sure “distrust” business!

You’d also still be basing your conclusion on only a small portion of the data associated with the survey item.

Take a look, for example, at the responses for Major companies”: 

It’s not a surprise, to me at least, that conservatives have had more confidence than liberals in “major companies over the entire period.

I’m also not that surprised that even conservatives have less confidence in major companies today than they did before the financial meltdown.

But if you are of the view that any response level other than “a great deal of confidence” evinces “distrust,” then you’d have to conclude that 80% of conservatives today “distrust” our nation’s business leaders.

You’d also have to conclude that conservatives are twice as likely to trust those “running . . . the scientific community” as they are to trust those “running . . . major companies.”

I’d find those conclusions surprising, wouldn’t you?

But of course we should be willing to update our priors when shown valid evidence that contradicts them. 

The prior under examination here is that PSPS supports the claim that conservatives “don’t believe in science,” "have turned on science," “reject it," have "lost their faith in it," have been becoming "increasingly skeptical" of it "for decades,"  etc.

The absurdity of the conclusions that would follow from this reading of PSPS---that liberals and conservatives alike "really don't like science," that conservatives have so little trust in major companies that they'd no doubt vote to nationalize the healthcare industry, etc. -- is super strong evidence that it's unjustifiable to treat the single response level of the GSS "confidence" item featured in PSPS as a litmus test of anyone's "trust" in science.

3.  Everyone is pro-science according to the data presented in PSPS

What exactly do response to the GSS “confidence” item signify about how conservatives and liberals feel about those “running” the “Scientific community”?

Again, it’s always a mistake to draw inferences from a portion of the response to a multi-part survey item.  So let’s look at all of the data for the GSS confidence item.

The mean scores are plotted separately for “liberals” and “conservatives. The 13 institutions are listed in descending order as rated by conservatives-- i.e., from the institution in which conservatives expressed the greatest level of confidence to one in which they expressed the least in each period. 

The variance in selection of the "great deal" response level analyzed in PSPS is reflected in the growing difference between liberals' and conservatives' respective overall "confidence" scores for "the Scientific Community."

Various other things change, too.

But as can be seen, during every time period—including the ones in which Ronald Reagan and G.W. Bush were presidents—conservatives awarded “Science community” the second highest confidence score among the 13 rated institutions.  Before 1990, conservatives ranked the “science community” just a smidgen below “medicine”; since then, conservatives have vested more confidence in the “military.”

Conservatives rated the “science community” ahead of “major companies,” “organized religion,” “banks and financial institutions,” and “education,” not to mention “organized labor,” the “Executive Branch of the Federal Government” (during the Reagan and G.W. Bush administrations!), Congress, and “TV” throughout the entire period!

Basically the same story with liberals.  They rated the “science community” second behind “medicine” before 1990, and first in the periods thereafter.

So what inference can be drawn?

Certainly not that conservatives distrust science or any group of scientists.

Much more plausible is that conservatives, along with everyone else, hold science in extremely high regard.

That’s obvious, actually, given that the “Confidence” item sets up a beauty-contest by having respondents evaluate all 13 institutions.

click on me for thanksgiving treat! mmmmmm!But this reading—that conservatives, liberals, and everyone else has a high regard for science—also fits the results plainly indicated by a variety of other science-attitude items that appear in the GSS and in other studies.

It’s really really really not a good idea to draw a contentious/tendentious conclusion from one survey item (much less one response level to one part of a multi-part one) when that conclusion is contrary to the import of numerous other pertinent measures of public opinion.

4. Multivariate analysis

The analyses I’'ve offered are very simple summary ones based on “raw data” and group means.

There really is nothing to model statistically here, if we are trying to figure out whether these data could support claims like "conservatives have lost their faith in science" or  have become “increasingly skeptical and distrustful” toward it. If that were so, the raw data wouldn't look the way it does.

Nevertheless, PSPS contains a multivariate regression model that puts liberal-conservative ideology on the right-hand side with numerous other individual characteristics.  How does that cut?

As much as I admire the article, I'm not a fan of the style of model PSPS uses here.

E.g., what exactly are we supposed to learn from a parameter that reflects how much being a "conservative" rather than a "liberal" affects the probability of selecting the "great deal" response "controlling for" respondents' political party affiliation?

Overspecified regressions like these treat characteristics like being “Republican,” “conservative,” a regular church goer, white, male, etc. as if they were all independently operating modules that could be screwed together to create whatever sort of person one likes.

In fact, real people have identities associated with particular, recognizable collections of these characteristics.  Because we want to know how real people vary, the statistical model should be specified in a way that reflects differences in the combinations of characteristics that indicate these identities--something that can’t be validly done when the covariance of these characteristics is partialed out in a multivariate regression (Lieberson 1985; Berry & Feldman 1985).

But none of this changes anything.  The raw data tell the story. The misspecified model doesn’t tell a different one—it just generates a questionable estimate  of the difference in likelihood that a liberal as opposed to a  conservative will select “great deal” as the response on "Confidence" when assessing those who "run ... the Scientific Community” (although in fact PSPS reports a regression-model estimate of 10%--which is perfectly reasonable given that that's exactly what one observes in the raw data).

5. Someone should do a study on this!

There’s one last question worth considering, of course.

If I’m right that PSPS doesn’t support the conclusion that conservatives have “lost faith” in science, why do so many commentators keep insisting that that’s what the study says?  Don’t we need an explanation for that?

Yes. It is the same explanation we need for how a liberal democracy whose citizens are as dedicated to pluralism and science as ours are could be so plagued by unreasoning sectarian discourse about the enormous stock of knowledge at its disposal.

Refs

Berry, W.D. & Feldman, S. Multiple Regression in Practice (Sage Publications, Beverly Hills, 1985).

Gauchat, G. Politicization of Science in the Public Sphere, Am. Sociological Rev., 77, 167-187 (2012)

Hmielowski, J.D., Feldman, L., Myers, T.A., Leiserowitz, A. & Maibach, E. An attack on science? Media use, trust in scientists, and perceptions of global warming. Public Understanding of Science  (2013).

Lewandowsky, S., Gignac, G.E. & Oberauer, K. The role of conspiracist ideation and worldviews in predicting rejection of science. PloS one 8, e75637 (2013).

Lieberson, S. Making it count : the improvement of social research and theory (University of California Press, Berkeley, 1985).

Smith, T.W. Trends in Confidence in Institutions, 1973-2006. in Social Trends in American Life: Findings from the General Social Survey Since 1972 (ed. P.V. Marsden) (Princeton University Press, 2012).

Sunday
Nov232014

Weekend update: Does marching around in costumes help overcome cultural polarization? Comparative data might help answer this Q

Someone should do some research on the effectiveness of this "communication strategy" as a means of extinguishing the cultural conflagration surrounding the issue of badger infestation in UK.  Who knows-- what's learned might help us to learn to defuse other hyper-charged risk issues in US, like GM foods and raw milk.

Those close to the effort have a strong sense-- the kind that certainly is unlikely to lead them astray-- that costumed marchers banging symbols, blowing whistles & waving signs will spur the government to act within days!  But I suppose we should wait for data before drawing any firm conclusions.

Two additional things to note here.

One is the novel idea to use of ideologically diverse vouchers.  Seeing these two well-known representatives of opposite sides of the UK political spectrum (it doesn't exactly run from left-to-right; it's more up & downish) is definitely something that communicators on other complex science issues like climate change should consider trying--it's really quite astonishing that they haven't!  I really don't see how this could fail to work.

The other is to tie the divisive issue of badger infestation to a "solution" on which there is already widespread consensus in the UK -- vaccination!  

We all know, e.g., that conservatives as well as liberals love making money!  Why not "frame" climate-change mitigation as an opportunity to profit from innovative "green marketing solutions"?!  I bet that would work! Why has no one even thought to try it? Weird!

Saturday
Nov222014

Weekend update: Decisive proof of "conservative distrust in science"? You tell me...

This worked pretty well before, so why not try again?

Recently I posted some data on right-left political outlooks, religion, and positions on disputed science issues and asked you, the 14.33 billion readers of this blog, to say what inferences the data support.  The responses, including one submitted by Chris Mooney in his Washington Post Wonkblog, were really interesting, and led to an informative set of exchanges, which continued when I finally added my own assessment.  I'm pretty confident the discussion wouldn't have been as enlightening had I offered my own views of the data in the original post.

Well, here's some more data! 

These are all from the General Social Survey data set, 1973-2012.  The question are ...

Do these data support inferences on the cause of public controversy over science issues such as climate change?  In particular, how do they bear on the commonplace claim that such controversy originates in a growing "distrust"of or hostility toward science associated with right-of-center political outlooks?

Well, you tell me!  I'll say nothing, nothing for at least 48 hours (±6 hrs).

Well?  

Okay, I said I would hold back for a bit on my own conclusions, but here are a few reactions to these very data from others, just to prime the pump of reasoned engagement with evidence:

***


***

 

 Hey--once again: prize to commentator who offers "best" interpretation (as determined by expert consensus survey)!

Monday
Nov172014

ICT eats RAT & CAT for breakfast: More (and more data on) religiosity, political predispositions, and "anti-science"

Okay—astonishingly, I’ve held my fire (bitten a hole straight through my lower lip, which fell off during a poker tournament this weekend, in the process) for over a week on the CAT vs. RAT fight.

The answer, in my view, is that both CAT & RAT are barking up the wrong tree!

Neither conservative ideology nor religiosity has been shown to predict a greater anti-science disposition than the other by the evidence presented. And indeed, that  evidence, plus some more, suggests that it's a mistake to think either of them is connected to  such a disposition at all.

For those of you just tuning in (site traffic suggests only 10 billion readers for the original post; apparently there was a climate-change induced net outage in the Netherlands Antilles, where there is a very strong CCP following), the question was, “What ‘explains’ public science conflict—political predispositions or religiosity?”

The inspiration for posing the question was an intriguing study that pinned the blame on religion. CCP blog readers viewed the study as methodologically dubious.

But the question was interesting so I decided to try to help us think about it by gathering data and presenting models that seemed responsive to commentators’ concerns.

I characterized the two positions that the original study seemed to be pitting against each other as the “Conservative-science Antipathy Thesis,” or CAT, which identifies antagonism between conservative or right-leaning ideology toward science as the source of public conflicts over climate change and various other science-informed policy issues; and the “Religion-science Antipathy Thesis,” or RAT, which states that religious animosity toward science is the cause.

I used appropriately modeled data from CCP and from the Pew Research Center studies to try to remedy shortcomings in the study that inspired the question—and then asked you, the loyal, perfectly rational 14 billion readers of this blog (or at least the 10 billion who managed to get through and submit response) to say what you made of the evidence.

I’d say Chris Mooney offered the best response, a conclusion I validated by doing an “expert consensus survey." He has been awarded the prize that was offered (he chose the Synbio Ipad—the very last one in stock).

The three issues that were featured in the original study (the one I tried to remedy the methodological defects of)—were climate change skepticism, disbelief in evolution, and opposition to federal support for stem-cell research.  Political predispositions and religiosity both seemed to predict these attitudes but in ways that varied in degree and that interacted with one another in diverse patterns.  CM thus concluded:

So what's the upshot? Obviously, both politics and religious beliefs contribute to science resistance, and the relative influence of one over the other varies on an issue-by-issue basis. The role of religion is very strong on the evolution issue, far weaker on the climate issue, and somewhere in between on the stem cell issue. And if you picked other issues to examine, you would assuredly find different results yet again.

What this exercise underscores, most of all, is that when people deny science, they do it because they think it conflicts with their personal identity. But many elements go into each of our identities, with both politics and religion constituting vital components for many people.

In light of this, it really doesn't make much sense to assert the power of one over the other.

Yup, for sure I agree with that.

But I’d go further: the evidence presented helped to reveal that neither CAT nor RAT is a very well supported.

In a mistake that is pervasive in the study of public attitudes toward science, the original study constructed its sample of observations in a manner that presupposed a generalized anti-science sensibility is the explanation for conflicts over evolution, climate change, evolution, etc.

But that’s a seriously contested issue too!

ICT—the “identity-protective cognition thesis”—is a major alternative to both RAT & CAT (Sherman & Cohen 2006). On this account, when policy relevant facts become entangled in antagonistic cultural meanings, people start to see individuals’ positions on them as badges of membership in & loyalty to opposing cultural groups. As a kind of identity self-defense, then, they begin to process information relating to these facts in a manner that conforms their beliefs to the positions that are dominant in their group (Kahan 2010, 2012).

RAT & CAT predict, respectively, that “religiosity” and “conservatism” can be shown to offer the best "explanation" for science-hostile positions generally.

It’s not clear that ICT will take any view on the relative influence of religion & conservatism in science disputes. Indeed, for the reasons CM stated, I think it’s strange to imagine that one could meaningfully specify cultural identities in the US in a way that split religiosity and political commitments apart.

But  ICT (or at least the variant I find most compelling) does join issue with both RAT and CAT on whether disputes over science can should be attributed to any particular cultural group's “anti-science” dispositions.

Being “liberal” and even “nonreligious” are both integral to self-defining commitments of certain people. Accordingly, where positions on some science-informed policy or other matter becomes entangled in antagonistic cultural meanings, we should expect liberal and nonreligious individuals, as well as conservative and religious ones, to display the signature forms of motivated reasoning that distort their perception of the best available science (Kahan, Peters, Dawson & Slovic 2013).

ICT is a combatant in the  “great asymmetry thesis debate,” which asks whether motivated reasoning on science-informed policy issues is concentrated in one end of the political spectrum or instead spread evenly across it is the(Crawford 2013; Brandt & Crawford 2013; Kahan 2013; Mooney 2012).

It was a serious defect in the study that inspired this exercise that it didn’t include in its observations any disputed science issues that might show neither conservatism nor religiosity is distinctively “anti-science.”

By doing so, it assumed particular answers to the question it purported to be investigating.

To remedy this defect, I added another disputed science issue: nuclear power.

That’s one where individuals whose cultural identities are more secular and left-leaning are typically understood to be the ones disposed to adopt “science-skeptical” or “science hostile” positions.

Nuke attitudes--click (but only from behind lead barrier)Indeed, in the data I presented, liberalism and nonreligiosity predicted a negative reaction both to building new nuclear power plants and to scientific research on nuclear energy.

That's more consistent with ICT--and its position on the ideological symmetry of motivated reasoning (Kahan 2013)-- than with either RAT or CAT.

A related point: if a researcher wants to do a valid test of whether disputed science issues are a consequence of one or another group's supposed "anti-science" disposition, then he or she definitely should not rely on simple correlations between the disputed issues and group identities.

Yes, conservatism and religion are associated with hostility to stem-cell research, climate skepticism, and disbelief in evolution.

But to treat that as evidence that conservatism and religion are anti-science and that that is what causes disputes on these issues presupposes that these positions are all explained by some sort of anti-science sensibility rather than something else.

To avoid this obvious error (an instance of selecting on the dependent variable), the researcher has to have a way of measuring whether groups are “pro-“ or “anti-science” independently of their positions on climate skepticism, disbelief in evolution, and opposition to stem-cell research.

What do they show?  You tell me! (Click on either for more detail.)

But I will tell you in the meantime what inference I draw from them: (a) that the US public is overwhelmingly pro-science; and (b) that any differences associated with politics and religiosity both are ambiguous and, more importantly, trivial in magnitude. 

These sorts of responses—and there are many more items in these data sets that support the dame conclusions (one should in fact look at all, not just one, if one is trying to figure out what they signify!)—are inconsistent with the inference that either conservatism or religiosity is antithetical to science, and hence inconsistent with the assumption that the correlation of these characteristics with climate skepticism,  disbelief in evolution, or opposition to stem cell research evinces an anti-science orientation.  Accordingly, it is even less sensible to think that one could look at these issues to say which one is “more” anti-science than the other.

Those who attribute disagreement with their views on science disputes to their opponents’ “anti-science” dispositions don’t come off looking especially “pro-science” themselves when they fail  to use evidence in a  scientifically valid way.

Refs

Brandt, M. J., & Crawford, J. (2013). Replication-Extension of 'Not for All the Tea in China!' Political Ideology and the Avoidance of Dissonance-Arousing Situations' (Nam, Jost, & Van Bavel, 2013, Plos One).  Retrieved from http://ssrn.com/abstract=2365281 or http://dx.doi.org/10.2139/ssrn.2365281

Crawford, J. T. (2012). The ideologically objectionable premise model: Predicting biased political judgments on the left and right. Journal of Experimental Social Psychology, 48(1), 138-151.

Kahan, D. (2010). Fixing the Communications Failure. Nature, 463, 296-297.

Kahan, D. (2012). Why we are poles apart on climate change. Nature, 488, 255.

Kahan, D. M. (2013). Ideology, Motivated Reasoning, and Cognitive Reflection. Judgment and Decision Making, 8, 407-424.

Kahan, D. M., Peters, E., Dawson, E., & Slovic, P. (2013). Motivated Numeracy and Englightened Self Government. Cultural Cognition Project Working Paper No. 116.

Mooney, C. (2012). The Republican Brain: The Science of Why They Deny Science--And Reality. Hoboken, NJ: John Wiley & Sons. 

Sherman, D. K., & Cohen, G. L. (2006). The Psychology of Self-defense: Self-Affirmation Theory Advances in Experimental Social Psychology (Vol. 38, pp. 183-242): Academic Press. 

Monday
Nov102014

What accounts for public conflict over science--religiosity or political predispositions? Here are some data: you declare the winner in this RAT vs. CAT fight!

Outsourcing my critical reading (i.e., just plain reading!) of this article worked really well.  Given the great points that came out in the comments, I don’t think there’s any value added in my offering a full assessment of the paper, which reported the results of a study that analyzed less than ideal data with a questionably specified statistical model from which the authors seemed to draw very debatable inferences.

But I do think it might be interesting to explore, at least to a degree, what one might learn about the authors’ research questions if one applied a valid statistical model to data that could support some reasonable inferences.

Basically, the authors purport to find that “religious variables,” but not “partisan identification,” predict a general hostility to science as manifested by attitudes toward climate change, evolution, and stem-cell research.  They treat this finding as suggesting reason to doubt claims that partisan political predispositions (e.g., Gauchat 2012), shaped by elite discourse (e.g., Mooney 2005; Brulle et al. 2012), account for public conflict over science issues.

Because the authors believe that  religiosity bears a greater share of the responsibility for such conflicts than is normally appreciated, let’s call this the “Religion-science Antipathy thesis” (RAT).

The main “political” competitor the authors advert to is one that attributes such conflict to antipathy—perhaps psychologically grounded (Mooney 2012), perhaps economically (Brulle et al., 2012), or maybe both—between a conservative political orientation and science. Let’s call this the Conservativism-science Antipathy Thesis” (CAT).

So here’s what I’m going to do.  I’ve compiled a bunch of observational (i.e., survey!) data and modeled them in a way that I think arguably bears on the relative plausibility of RAT and CAT. 

But exercising a credulity-defying degree self-restraint, I am going to refrain, for at least 24 hrs, from telling you what sorts of conclusions I think these data support. 

In that period, you, the 14 billion readers of this blog, will be afforded the exclusive opportunity to specify and defend your own inferences

How many of the myriad other “cultural cognition blogs” out there do you think would have the necessary levels of self-confidence and respect for their readers to surrender their “first word” prerogative?  That’s right—not a one!

Okay, then.

Let’s start with global warming.

You deserve to see raw data! Assert your rights!As you can see, this figure plots the probability of agreeing that there is “solid evidence” for human-caused global warming in relation to right-left political outlooks conditional on one’s level of “religiosity.”  Political outlooks and religiosity are measured with separate multi-item scales. (Because partisan self-identification and liberal-conservative ideology are both indicators of the same thing—an unobserved or latent political disposition—it is really not a good idea to treat them as “independent” right-hand side variables in a multivariate regression.)  The colored hashmarks are the 0.95 confidence intervals for the predicted probability at the indicated point on the left-right political outlook scale.

If you want to “see” the regression model or the “raw data,” then click on the specified thumbnails in the margin.

As you can tell, there’s an interaction between religiosity and political outlooks: the contribution that moving left in outlook makes to acceptance of climate change is bigger the less religious one is.

If you get only this, demand a refund!But I won’t say anything more than that!  What this signifies in the battle between RAT and CAT is your call to make!

Next, let’s look at belief in evolution.  Same model, used now to examine the impact of political outlooks on belief in evolution conditional on religiosity.

Okay... you get the idea!Significance? You tell me!

Next, support for stem-cell research. 

Actually, I’ve never collected data on this topic.  So I popped open a canned dataset that has such data: the super great 2009 Pew public attitudes toward science survey.

I again constructed a political outlook scale by aggregating response to partisan self-identification and liberal-conservative ideology items.

Pew didn’t have all the same items from which I constructed the religiosity scale in the previous models.  So I constructed one using self-reported church attendance (one of the items that I did have in my CCP data set), self-identified “born again” evangelical status, and a “non-religious” self-identification variable that separated out persons who self-identified as agnostics or atheists from those who reported affiliation with any religious denomination.

How good a measure is this?  I wasn’t sure, so I came up with a method to externally validate it. 

It turns out the Pew survey also has measures for global-warming acceptance and belief in evolution (the authors of the study that inspired this exercise should have used the Pew dataset rather than the 2006 GSS dataset, which lacked a genuine measure of global-warming acceptance).  When I used the Pew religiosity scale and the right-left political outlook measures as predictors of these beliefs, the Pew religiosity scale behaved very comparably to the CCP-dataset religiosity scale in the modelsreported above. That struck me as pretty good evidence that the Pew scale is tapping into pretty much the same unobserved or latent disposition being tapped into by the CCP religiosity scale.

Here’s the result for stem-cell funding:

What do you think?

These are the three issues—global-warming acceptance, belief in evolution, and support for stem-cell research—that the article we read used to test RAT vs. CAT.

But some of you pointed out that disbelief in evolution and opposition to stem-cell research are arguably the sorts of positions one might expect highly religious individuals to form independent of any sort of general hostility to science. For that reason, one might conclude they don’t supply as clean a test of RAT vs. CAT as, say, climate-change acceptance, where generalized "science hostility" is less likely to be confounded with issue-specific religious concerns.

So, I decided to add one more issue to try to make the fight more fair: nuclear power!

The great Pew study had two nuclear-power items, positions on which I also modeled in relation to political outlooks conditional on religiosity:

So there you go.

Infer away!

Refs

Brulle, R.J., Carmichael, J. & Jenkins, J.C. Shifting public opinion on climate change: an empirical assessment of factors influencing concern over climate change in the US, 2002–2010. Climatic Change 114, 169-188 (2012).

Gauchat, G. Politicization of Science in the Public Sphere. American Sociological Review 77, 167-187 (2012).

Mooney, C. The Republican Brain: The Science of Why They Deny Science--And Reality (John Wiley & Sons, Hoboken, NJ, 2012).

Mooney, C. The Republican war on science (Fine Communications/MJF Books, New York, NY, 2009).

 

Sunday
Nov092014

Weekend update: cognitive illiberalism--what is it? & what does it have to do with the Constitution?

Now & again people ask me what I mean by the term "cognitive illiberalism." That's reasonable; I often use that term w/o stopping to spell it out. That's because I & my collaborators have already done so in various places. But of course people join conversations in progress all the time, & their participation is impeded by unfamiliar, specialized terms that those who've been participating for a longer period have constructed to condense information of recurring significance.  It wouldn't make much sense for those who are parties to an ongoing conversation to deny themselves the efficiency of this device just so new entrants could follow along without confusion; indeed, the conversations that would result would be so burdened by throat-clearing re-elaboration of all that has come before that no one would have the time or patience to take part in them. But what does make sense is for the parties to such conservations to help reduce the cost of joining by contributing to the stock of reasonably accessible materials that a curious person can consult to recover the content that is being compacted into these terms of art. So here is an excerpt from Kahan, D. M., Hoffman, D. A., Braman, D., Evans, D., & Rachlinski, J. J. (2012). They Saw a Protest : Cognitive Illiberalism and the Speech-Conduct Distinction. Stan. L. Rev., 64, 851-906, that helps explicate the concept of "cognitive illiberalism" and its significance to enforcement of the individual liberty provisions of the Constitution. In the future, I'll hyperlink to it, refer to it, etc., when I think will help out someone who might be interested in what I'm saying but who (reasonably enough!) has never heard of "cognitive illiberalism."

Introduction

 In a 1950s social psychology experiment, students from two Ivy League colleges were instructed to evaluate a series of controversial officiating calls made during a football game between their respective schools. Researchers found that the students, from both institutions, were much more likely to perceive error in the penalty assessments imposed on their school’s team than in those imposed on their rival’s. The students’ emotional stake in affirming their loyalty to their institutions, researchers concluded, had unconsciously shaped what they had seen when viewing events captured on film. This study is now recognized as a classic demonstration of “motivated cognition,” the ubiquitous tendency of people to form perceptions, and to process factual information generally, in a manner congenial to their values and desires.

Motivated cognition poses an obvious hazard for law. Sports fans are permitted—even expected—to be partisan. But legal decisionmakers must be neutral. Just as the integrity of a sporting contest would be undermined by unconscious favoritism on the part of the referee, so the legitimacy of the law would likewise be compromised if legal decisionmakers, as a result of motivated cognition, unwittingly formed perceptions of facts that promoted the interests and values of groups with whom they had an affinity.

This effect could be particularly subversive of constitutional law. The Free Speech, Equal Protection, and Due Process Clauses all mandate governmental evenhandedness.  Within  their  respective domains,  each  forecloses the  state from privileging particular affiliations, ways of life, or points of view and mandates that law be justified by its contribution to secular interests—physical security,  public  health,  economic  prosperity—valued by  all  citizens.  But  if decisionmakers (particularly adjudicators) unconsciously apply these provisions to favor outcomes congenial to favored ways of life, citizens who adhere to disfavored ones will suffer the same array of disadvantages for failing to conform that they would in a regime expressly dedicated to propagation of a sectarian orthodoxy. This distinctively psychological threat to constitutional ideals, which we will refer to as “cognitive illiberalism,” has received relatively little attention from commentators or jurists.

We performed an experimental study designed to help assess just how much of a threat cognitive illiberalism poses to constitutional ideals. The study focused on a discrete and recurring task in constitutional law: discernment of the line between “speech” and “conduct” for purposes of the First Amendment. Embodied in a variety of doctrines, the speech-conduct distinction aims to assure that coercive regulation is justified on grounds unrelated to governmental or public hostility to disfavored ideas. Most importantly, the speech-conduct distinction has historically played, and continues to play, a vital function in preventing the government from invoking its responsibility for maintaining “public order” to disguise suppression of impassioned political dissent. Our study furnishes strong evidence that this function is indeed highly vulnerable to the power of motivated cognition to shape decisionmakers’ perceptions of the facts that mark the speech-conduct boundary.

The features of the speech-conduct distinction that make it susceptible to this influence, moreover, are shared by a host of other constitutional doctrines. The study results thus highlight the need to fortify constitutional theorizing with psychological realism. Normatively ideal standards for enforcing the Constitution are of little value if applying them defies the capacities of constitutional decisionmakers. * * *

 [III.]B.  Cognitive Illiberalism and the Constitution

The practical motivation for this study was to focus attention on the danger that cognitive illiberalism can pose to constitutional law. We use this term to refer to the vulnerability of political and legal decisionmakers to betray their commitment to liberal neutrality by unconsciously fitting their perceptions of risk and related facts to their sectarian understandings of the good life. This is the dynamic, we believe, that transforms seemingly empirical debates over how to protect the environment, promote public health, and secure the nation from external threats into occasions for divisive group-based status competition. Our study results show how readily constitutional decisionmaking can become infected by this pathology.

In our subjects, cognitive illiberalism eviscerated the line between “speech” and “conduct.” The speech-conduct distinction can be seen as one doctrinal device courts employ to test whether a regulation conforms to liberal prohibitions on governmental promotion of a moral or political orthodoxy: by requiring that a regulation be shown to promote a governmental interest independent of hostility to any particular idea, the teleological conception of the speech-conduct divide assures that law is used to pursue secular goods of value to all citizens regardless of their cultural outlooks.

Enforcing this test, however, necessarily requires decisionmakers to make critical determinations of fact: in the case of a mass demonstration, for example, did the protestors intend to intimidate or only persuade? Were the protestors simply expressing impassioned dissent, or did they impose themselves on members of the public in anassaultive or invasive manner (e.g., “screaming in their faces”)? Were onlookers genuinely frightened of physical assault, or merely angry, offended, or possibly even ashamed by exposure to the protestors’ message? Did law enforcement actors intervene to preempt incitement to violence or only to quell a public backlash propelled by animosity toward the demonstrators’ point of view?

For our subjects, the answers were decisively shaped by the congruence between the protestors’ message and the subjects’ own cultural worldviews. As a result, in the course of certifying that the law was free of culturally partisan influence, they ended up infusing it with exactly that.

Other First Amendment doctrines also seem vulnerable to this type of subversion. * * *

click me! Please!Indeed, we suspect this point can be generalized to constitutional theory as a whole. As discussed, the First Amendment can be integrated into a general theory that reads the Constitution as implementing the liberal prohibition on state endorsement of partisan conceptions of the good life. Like the First Amendment, the Equal Protection and Due Process Clauses require courts to “strictly scrutinize” proffered secular rationales—public health, deterrence of criminal violence, national security, and the like—to “flush out” the impact, conscious or unconscious, of regulators’ animosity toward those whose identity or values defy dominant norrms. But if legal decisionmakers, like everyone for else, are unconsciously motivated by their cultural affiliations, then they—like everyone else—are more or less likely to see challenged laws as contributing to the attainment of secular ends depending on whether those laws affirm or denigrate their own cultural commitments. Angry denunciations of judges who have thrown their lot in with one or the another of the belligerents in the American “culture wars” is itself a form of status conflict characteristic of cognitive illiberalism.

Some legal commentators (and historically certain jurists) have criticized constitutional standards that “balance” constitutional liberties against “compelling interests,” such as national security, public order, and diversity. The phenomenon of culturally motivated cognition vindicates their anxiety that such “tests inevitably become intertwined with the ideological predispositions of those doing the balancing.” But our study results suggest that these commentators are too quick to assume that their preferred alternative to balancing—such as the “teleological conception” of the speech-conduct distinction, the “anticaste” principle, the liberal “harm” criterion, and the like—will necessarily avoid such entanglement. The primary implication of our study— the main message we are trying to get across—is that constitutional theorists have paid too much attention to explicating the normative content of various free speech standards and too little to the psychology of enforcing them.

Thursday
Nov062014

"Religion, not political predispositions or political elite discourse, generates conflict over science" Seriously?!!!

Okay, I'll get to this but not for a bit.  Maybe one or more of our 14 billion readers can read it in meantime and report in comments field?

As you can see from the abstract, the basic claim is that neither political predispositions nor the positions of political elites contribute much to conflicts over science relative to the contribuiton that religion makes.

I'll admit that I have priors very strongly opposed to this thesis.  But I'll do my best not to let those infect my likelihood ratio as I examine the authors' evidence.

In the meantime, I'd be grateful as I'm sure billions of others would be for anyone else's assessment.

Wednesday
Nov052014

Still more on the concept/value of a "science of science journalism"

From correspondence with a friend:

You mentioned you were eager to learn what I had in mind about how to use the science of science communication to improve science journalism.  I'm sure you can guess what I'd say: you tell me -- & I'll measure!  

We've talked about this philosophy, of course.  I think all the professions that traffic in the dissemination of what's known by science can benefit from the use of science's signature methods to improve their craft.  Not b/c those methods furnish a substitute for the exercise of professional judgment or craft sense; but b/c they are suited for generating information -- and inspiring informative action -- that those with professional judgment would recognize as valuable.  

These methods are uniquely suited for doing that, I think, on questions that experienced professionals themselves recognize as having competing plausible answers. In that situation, there will be no progress through more & more talk, in the form, say, of perennial panels that rehash the opposing positions yr after yr at professional conferences, as predictably entertaining as those are!

What's needed are appropriate tests -- ones designed to generate observations the nature of which will give the professionals at least some more reason than they had previously for crediting one or another of their competing surmises.

Those tests are unlikely to definitively resolve any particular disputed issue! 

But they can be expected to infuse new information into science journalists' own continuous process of professional self-assessment.  They can be expected, too, to inspire particular practitioners to try something new in their work, generating outcomes that can themselves supply a basis for additional reflective assessment.

As a result, the ongoing critical engagement of science journalists with their own craft norms will unfold in a manner that these professionals will themselves find more satisfying.

But if you ask me what to do, then you are not fully grasping what I'm saying!  

I am not of your profession; I don't have your craft sense, your professional judgment.  

There are some things that I can do, using my own craft judgment and skill to the best of my ability, that will give you relevant information.  I can do an experiment, for example, designed to pit two of your plausible conjectures against each other & generate the information that would give you more reason to view one or the other as more likely true than you previously had.  

But you must tell me what the plausible conjectures are. 

You must tell me whether the design I have crafted is such that it really will generate a result that those w/ professional judgment would regard as supporting the sort of inference I'm describing. 

And most importantly of all, once we are done with that experiment, you must tell me what you think can be done in the real world, the particulars of which were stripped away in our study so we could be confident we knew what was happening and why, to reproduce the effect observed in the lab.  At which point -- I will again help by measuring: that is, by applying my knowledge to figure out how to fit to your real-world activity some apparatus for collecting observations on the basis of which you will have more reason than you otherwise would have had to think that what you are doing is or isn't working.

After that--or better still over the course of the process, at the various stages at which there are observations to share--you will go to the professional conference & describe what you have been up to. And everyone will talk about what can be learned.  Professional judgment will continue to evolve in the way that it always has --in response to members’ reflective engagement with their shared experiences--but now with the benefit of this additional input on a disputed issue that had been resisting resolution with the information previously at hand.

This is what it is like to have a genuine evidence-based culture within a profession. 

To have people from outside your profession do stylized studies & then purport to tell you what do is not.  Not seeing that is, I think, is  one reason that science journalists report getting little value from events like the Sackler Science of Science Communication symposia. You actually should be dissatisfied if researchers who do what I do--conduct studies designed to explore the relative significance of alternative mechanisms thought to be of consequence for one or another aspect of science communication -- tell you "here's what to do"; b/c they don't know  how to connect that relevant research to practice & shouldn't pretend to (really really really shouldn't; I think it is in fact unethical for them to peddle “how to” advice manuals and the like to science communicators—rather than being clear on the need for evidence-based practice “all the way down”). 

But in turn, you shouldn't expect that sort of counsel from them!  You have the situation sense that is essential to figuring out how to translate the relevant lab studies into practices that might plausibly link up with what the studies have identified to be the relevant mechanisms; at which point, there is again a role to be played by those who measure. 

So -- don't ask what the science of science communication can do for your; instead ask, "What can I do with the science of science communication for myself."

I am saying only that making this sort of evidence-based practice a part of the professional culture of science journalism -- along with all the other professions that traffic in disseminating what's known by science -- will make the evolution of its members' professional judgment better by their members' own lights.

That's a hypothesis!  I'm happy to help anyone in these professions test it.

--Dan

p.s. I’ve addressed this before; there is a groundhog-day quality to discussing the need for & character of the “science of science communication."  But that’s okay, b/c you actually can sometimes make the same day a bit better or more complete than it was last time -- & can hyperlink to things that still seem to make sense.

Anyway, here are some relevant posts:

Tuesday
Nov042014

How *cognitive* adaptation relates to mitigating a polluted science communication environment

I've been corresponding with a friend whom I -- & many others -- regard as an extraordinary climate-science communicator (& whose skills in this regard are matched by the depth of her civic virtue). In addition to how she manages to communicate so successfully, we have been discussing my view of  how big an impact her efforts, if they could be enlarged in scale, could be expected to have in reducing public conflict over climate change.  My position is complicated; and I'm not nearly as good a science communicator as she is! But I thought I'd share my best efforts so that the 14 billion readers of this blog might also tell me how to improve my communication of this point, and of course whether it is a point that merits wider communication.

I don't disagree w/ anything you are saying about how to engage people whose cultural identities are threatened by information on global warming.  On the contrary, I think there is a tremendous amount to be learned from your example about how to counteract this dynamic in settings in which educators and others are helping people understand the basic mechanisms of climate change and the weight of the scientific evidence on its causes & consequences.

What I'm less sure about is the connection between promoting comprehension in those settings & promoting more constructive engagement with climate science in our national political life.  

Essentially, the science communication environment has become polluted with antagonistic cultural meanings that transform "positions" on global warming into badges of membership in & loyalty to competing cultural groups. Those meanings effectively disable the faculties that diverse citizens use, very successfully most of the time, to align their own decisionmaking (personal & collective) with the best available evidence.  

I see you as implementing, as it were, a kind of cognitive adaptation strategy.  By proving to suspicious listeners that you harbor no hostility to their identities, you create conditions in which people do what they normally do with their reason--use it to make sense of complex things &, even more important, to reliably recognize what’s known by science.

Again, I view that as enormously important-- & will say more in a moment about why.

But the fact remains that what you are doing doesn't actually repair the polluted science communication environment.  

You are making it possible for people to reason within that environment when you yourself are presenting information to them.  But when you finish, the antagonistic meanings that make global-warming positions into symbols of membership in opposing cultural groups persist in the world in which those people live.  

Those meanings will certainly continue to shape the perceptions of those you didn’t get a chance to talk to-- who outnumber those you did by orders of magnitude.  

But even more important, those meanings are also likely to continue shaping how the people you did talk to engage the climate issue in democratic political life. That's because the positions people adopt on climate change in that realm aren’t caused by any deficit in their understanding of how climate works; on the contrary, they are a consequence of just how keenly perceptive they are of what stances on global warming express about people's group identities.

Observational studies support this: individuals who display the highest degree of science comprehension generally & climate science comprehension in particular are the most culturally polarized over whether human-caused climate change is even occurring.  That question, on a survey & in our political life, measures who they are--whose side they are on--not what they know.

My surmise, then, is that even though the people you have communicated with have genuinely learned something--& likely now enjoy a comprehension level that puts them at the very top of the scale for public "climate literacy”-- they'll continue to orient themselves toward the issue of global warming in a way that evinces their group identity.

That means, among other things, that someone who otherwise possesses the requisite sorts of values & the formidable degree of intelligence that these citizens appropriately look for in selecting congressional representatives is unlikely to enjoy a very long career in national politics if that person decides to make addressing the risks of climate change one of his or her priorities. 

That would be the equivalent of the obtuse Presidential candidate who decides to highlight his support for gun control in advertisements in West Virginia because a pollster has just handed him a survey showing that a majority of its residents favor it.  He doesn't get that what stances on gun control say is much more important to citizens than what restrictions on guns would do.  For West Virginianians, the decision of a candidate to make gun control one of his key positions "says a lot about who you are and who you aren't"--and the only message his ads  will succeed in communicating is that "he's not one of us.'' (If only someone had told him!)

As long as the positions that people--ordinary citizens or politicians--adopt on climate change convey "who they are," the issue will continue to polarize culturally diverse groups, no matter how much their members know about the dangers of failing to address global warming.

That means we need a science communication environment mitigation strategy.  We need to staunch the "us-vs.-them" sources of pollution, which emanate from both sides.  We need to detoxify that environment, so that reasoning people & their representatives don't face the sorts of conditions that in fact make it perfectly rational for them to form climate-change positions that express who they are instead of what they know.

As I said, I don't think creating conditions in which people can learn & comprehend is sufficient to do that.

Nevertheless, creating such conditions is vitally important.  

It is vitally important, first, because it has intrinsic value: people who want to be able to know what is known by science should be enabled to know that.  Science communicators who virtuously respond to this need should learn how to do what you do.

Second, doing what you do is vital because even amidst the toxic conditions that stifle constructive national policymaking on climate change, many people will need to make consequential decisions that should be informed by the best available evidence.

No matter what happens at the national level, e.g., people will have to make collective decisions about how to adapt  to climate change--not cognitively but physically.  

They'll have to decide individually, too, how to make all manner of adjustments in their private affairs to reflect  a changing climate. Think, e.g., of farmers in the midwest who, despite "not believing" in human-caused climate change, are in fact very interested to hear about the latest climate modeling forecasts of the USDA & EPA (also the work that firms are doing to create genetically modified crops that will fare better in changed climate conditions).

Communicators need to know how to convey this information to these actors, too -- and can learn something from you about how to do it.

Third, what you are doing is vital because it is making us smarter about how science communication works.

Again, I don't think that the success you are achieving in helping individuals to learn about climate science by itself offsets the dynamics that make climate change polarizing, and that stifle exploitation of our scientific knowledge in national policymaking.  

But you are showing in one very important setting how to disentangle the question "who are you, whose side are you on?" from "what do we know about how the world works?"  

By systematically studying how you & other communicators (particularly educators) are able to achieve this effect, we can learn a tremendous amount about the dynamics of "disentanglement" generally & thus figure out more quickly what sorts of things we should do to reproduce that effect in our politics

In sum, the work you do not only fills me with admiration & awe. It also fills me with hope & excitement, and with a sense of motivation & direction in my own research, my highest aspiration for which is that it will contribute to formation of a science-communication culture that embodies your skill & knowledge.

But I do think that the benefit we can get from learning how to do what you do will depend on getting a lot of other people to recognize that improving popular comprehension of climate science won't in itself do much to resolve the cultural conflict over global warming.

On the contrary, we need to decontaminate our science communication environment of antagonistic cultural meanings so that we can get the benefit of what you & others are doing to help citizens comprehend what science knows.

Monday
Nov032014

"A science of science journalism & filmmaking" vs. "Throw strikes & keep 'em off the bases" (lecture synopsis & slides)

I haven't been faithfully reporting on recent talks, workshops etc.  But fortunately, James Bell, who attended one recently, did a great writeup!  

The event was a panel at the Jackson Hole Wildlife Film Festival.  The two other panelists were Naomi Oreskes and Carl Zimmer.

Katie Carpenter's award-winning documentary! It's soooo good!In addition, the moderator for the panel was Katie Carpenter, my collaborator in the CCP's ongoing project to supply evidence-based science communication support to the Southeast Florida Climate Compact.  

Just the day before, the Festival had awarded Katie's documentary Battle for the Elephants the prize for "Best Environmental & Conservation Science Program"!  So of course she did a great job directing our panel while also contributing her own insights to the discussion.

Read Bell's excellent writeup if you want a blow-by-blow.

The only thing that it occurs to me to add concerns the relationship between Carl Zimmer's talk & mine. There was a bit of point-counterpoint to it.

My basic message -- surprise surprise-- was that science filmmakers & journalists could benefit by using empirical methods to refine and extend their craft norms (slides here).

The gist of Carl's talk, to paraphrase Stanley Fish, was that a science journalist doesn't use a science of science of communication; he or she is a science of science communication. By training and experience, science journalists acquire a form of professional judgment -- one not amenable to quantitative specification-- distinctively suited to discerning how to make what's known by science accessible to curious members of the public. 

Zimmer was Martinez, & I Berkow. Click it!I agree with him!

Or at least I agree with him that no set of methods, empirical or otherwise, can be a substitute for the facility Carl described, which for the most part operates tacitly and automatically as science journalists do what they do.

But the situation sense of science filmmakers, like that of other professionals, is neither static nor impervious to the conscious reflection of those who exercise it.

On the contrary, professional judgment evolves through the interactions of a profession's members, as they accumulate, observe, and share their experiences--formally via training, informally by conversation, and semi-formally through cool events like the Jackson Hole Film Festival!

My claim would be that that process would be enriched by access to empirical information generated with the input of science journalists and filmmakers for the specific purpose of addressing important questions of craft that they themselves recognize as admitting of multiple, competing plausible answers.

The studies would be unlikely to definitely resolve such issues.

But the results would give those participating in professional exchange an additional source of evidence they all agreed was relevant.  Such studies could also be expected to spark insight in individual science journalists and filmmakers, whose use of the study results to inform their actions would thereafter furnish even more material for collective assessment.

So for sure empirical methods are no substitute for professional judgment. But they can supply professionals with information that they themselves will value for the contribution it makes to the exercise of their professional judgment and to the accumulation of shared experiences through which such judgment is formed and transmitted.

Or at least that is my hypothesis! I'd bet (say, $10,000) that even Carl would agree it's worth testing.

& if he and other professional science journalists or filmmakers decided to try such an experiment, I and other scientists of science communication would  be honored to help them design and carry out studies in service of their continuing mission to perfect their craft.

Click to read James Bell's report of talks & Q&A

Friday
Oct312014

"Who am I? Why am I here?" My (ongoing) trip to West Point

At West Point yesterday & today, where I'm giving talks & today co-teaching a criminal law class.

The military, it seems to me, is an institution that is ruthlessly self-evaluative & remarkably unambivalent -- to point of lacking any self-consciousness of the attitude it has adopted -- about use of empirical methods of self-assessment.

The questions & discussions are great & there are tons of really smart people here thinking about how to teach critical thinking & cultivate professional judgment.

The educational enviornment here is, I think, a token of how successfully the US military has adapted its practices and outlooks to the political culture of the Liberal Republic of Science.

I'm not an historian, of course, but it does seem to me that unpardonable damage has been done to our military by a civilian leadership that lacked these very commitments to empirical self-evaluation & liberal principles of self-government.

Some lecture slides:

What is 'cultural cognition'?  I'll show you!

“Motivated System 2 Reasoning”: Rationality in a Polluted Science Communication Environment