follow CCP

Recent blog entries

"They saw a statutory ambiguity"--judges & motivated reasoning

“Yesterday” I mentioned that I and my collaborators—David Hoffman, Danieli Evans, Judge Eugene Lucci, and Katherine Cheng—had a new paper out, and that I’d say more “today.”

So here is the first of a planned 73 blog posts on the paper, which describes a study we did on whether judges’ decisionmaking is vulnerable to distortion by cultural cognition.

By now, all 14 billion readers of this blog, along with the remaining 18 other persons in the world (the ones whose internet connections were disconnected for failing to pay comcast on a timely basis), know about Hastorf & Cantril’s classic paper “They Saw a Game.”

H&C found that when students from two Ivy League colleges were shown a film of a football game between their two schools, students selectively perceived the referee to be making correct or mistaken penalty calls depending on whether those calls were beneficial or detrimental to their school’s team.

This was the first finding of “motivated reasoning,” which refers to the tendency of individuals to conform their assessments of all manner of information—from brute sense impressions to assessments of logical arguments to evaluations of empirical evidence—to some end or goal independent of factual accuracy.

In H&C, that goal was the experience of solidarity with their school.

“Cultural cognition” is a form of motivated reasoning that manifests itself in individuals conforming their perceptions of risk or other policy relevant facts to propositions that promote the status of, or their standing in, their own cultural group.  Cultural cognition, studies suggest, makes a substantial contribution to public conflicts over climate change, gun control, marijuana legalization and other putative societal risks.

One of the things that and I and my colleagues are curious about is whether cultural cognition influences—and in this context, we’d say biases—legal decisionmaking.

One the papers in which we examine this issue is called “They Saw a Protest.” 

In that study, we showed the subjects—a group of 200 members of the public drawn from a demographically diverse panel of US adults—a videotape of a political protest. We told them the protestors were suing the police for breaking up their demonstration in violation of the protestors’ free speech rights.

The police, we explained, were justifying their dispersing of the protestors on the ground that the protestors were threatening and intimidating bystanders, and preventing their access to a building.  The protestors denied this, asserting that they were peacefully chanting and  directing comments at the bystanders in a nonthreatening, lawful fashion.

All the subjects had to do was, in the role of jury, decide whose position—the protestors or the police—the tape supported.

Half the subjects, however, were told that the protesters were anti-abortion activists demonstrating in front of an abortion clinic.  The other half were told that the protestors were college students demonstrating at against “Don’t ask, don’t tell” (for those of you too young to remember, the long-time policy that excluded open Lesbians and Gays from the US military) in front of the college recruitment center, where the military was interviewing students interested in signing up for service.

Consistent with the influence of cultural cognition—and reminiscent of the result from the classic “They Saw a Game” study—we found that the study subjects polarized along cultural lines. 

Those who had opposing cultural outlooks (“egalitarian communitarian” vs. “hierarchical individualist,” or “egalitarian individualist vs. hierarchical communitarian”) who were assigned to the same experimental condition—who, in other words, thought that they were watching the same kind of protest—disagreed about whether they saw the protestors blocking pedestrians, threatening them with their signs, shouting in their faces, etc.  Those who had the same cultural outlooks but who were assigned to different experimental conditions (thought they were watching different kinds of protests) disagreed with each other about what had occurred.

Not a happy outcome for the law. The First Amendment is supposed to protect the right of citizens to express themselves no matter how objectionable anyone finds their message, so long as those citizens don’t cross the line from speech to intimidation. But if decisionmakers are motivated by cultural cognition to attribute intimidating behavior to those whose messages offend the decisionmakers’ values, then this constitutional guarantee won’t be worth much!

But these decisionmakers were ordinary citizens playing the role that ordinary citizens often do—juror—in a legal proceeding.

Would judges be subject to the same sort of bias?

That’s the question our new study was designed to help answer.

The paper is entitled “ ‘Ideology’ or ‘Situation Sense’? An Experimental Investigation of Motivated Reasoning and Professional Judgment,” a more appropriate might have been “They Saw a Statutory Ambiguity.”

In the study, we had the subjects—a group of 800 members of the public and 700 law students, lawyers and judges (represented in roughly equal proportions) play the role of judge in two cases.

Both cases featured statutory ambiguities: in one (“Littering”) whether leaving reusable plastic water bottles in the desert constituted “depositing debris” in a protected wildlife preserve; and in the other (“Disclosure”) whether a police officer who knew he was releasing confidential investigatory information but didn’t realize doing so was illegal “knowingly violating” a law against making such disclosures.

Again, we had experimental manipulations.

click me! I'll be your best friend if you do!In Littering, the alleged violator was either a group of construction workers building a border, who left the bottles in the desert to drink water from as the build a fence to keep undocumented immigrants from illegally crossing the Mexico-Texas border; or an immigrant-aid group who put the bottles in the desert in anticipation that the water would be consumed by the illegal immigrants.

In Disclosure, the party receiving the leaked information was either a family-planning center counseling individuals on where to obtain abortions or a religious family-planning center counseling individuals on alternatives to the same.

Those experimental manipulations were ones irrelevant to proper interpretation of the statutes at issue (or so we believed).  But they were ones that could still provoke the sort of bias associated with cultural cognition, which could be expected to bias decisionmakers’ interpretations toward finding violations where such outcomes were consistent with rather than contrary to their cultural values.

We found exactly that pattern of biased decisionmaking in members of the public.

But not in the judges: the judges converged on outcomes (no violation in Littering; violation in Disclosure) regardless of which version of the problem they considered and regardless of their cultural values.

In other words, the judges got a cultural-cognition clean bill of health. They decided the cases free of the sort of influences that, the study showed, could be expected to generate group-favoring biases in ordinary citizens.

click me. Or you suck.The judges resisted that bias, moreover, even though they too displayed signs of biased information processing on culturally contested risks.

In another part of the study, the judges divided along exactly the same cultural lines as members of the public on the risks of climate change, marijuana legalization, and a host of other issues.

The resistance the judges displayed to cultural cognition, then, was very specific to their legal reasoning.

Lawyers, by the way, were pretty much identical to judges in their responses to both the legal reasoning and risk-perception portions of the study.

Law students were in between the members of the public (who again, were biased in both types of responses), on the one hand, and the lawyers and judges, on the other.

So that’s what we found.

In future posts, I’ll say more about the theoretical and practical motivations for the study, the methods we used to analyze the results, and the implications of the study for assessment of the performance not only of judges but of professionals generally, including climate scientists and others who specialize in assessing societal risks.

 [Ah shit! Because I was concentrating on writing on this I didn’t notice that they were boarding my connecting flight from Detroit to Boulder where I’m supposed to a lecture on this study. . . . Whoops!  I’ll pay more attention when the next flight is getting ready to go.]


Science of Science Communication 2.0, Session 11.1: Teaching science in a polluted science communication environment, part 2 -- Climate

It's that time again!  "Science of Science Communication 2.0" session 10. Reading list here, & study/discussion questions below.  

Share your wisdom please (don't hoarde & be selfish!)


Are judges politically biased? New paper, new study, new methods, new body piercings--new new new!

Will have more to say about this new study "tomorrow"-- but if anyone wants to get a head start in commenting/questioning/qualifying/annihilating, dive in.

“Ideology” or “Situation Sense”? An Experimental Investigation of Motivated Reasoning and Professional Judgment

Dan M. Kahan, David Hoffman, Danieli Evans, Neal Devins, Eugene Lucci, and Katherine Cheng 


This paper reports the results of a study on whether political predispositions influence judicial decisionmaking. The study was designed to overcome the two principal limitations on existing empirical studies that purport to find such an influence: the use of nonexperimental methods to assess the decisions of actual judges; and the failure to use actual judges in ideologically-biased-reasoning experiments. The study involved a sample of sitting judges (n = 253), who, like members of a general public sample (n = 800), were culturally polarized on climate change, marijuana legalization and other contested issues. When the study subjects were assigned to analyze statutory interpretation problems, however, only the responses of the general-public subjects and not those of the judges varied in patterns that reflected the subjects’ cultural values. The responses of a sample of lawyers (= 217) were also uninfluenced by their cultural values; the responses of a sample of law students (n = 284), in contrast, displayed a level of cultural bias only modestly less pronounced than that observed in the general-public sample. Among the competing hypotheses tested in the study, the results most supported the position that professional judgment imparted by legal training and experience confers resistance to identity-protective cognition—a dynamic associated with politically biased information processing generally—but only for decisions that involve legal reasoning. The scholarly and practical implications of the findings are discussed.



Weekend update: Science of Science Communication 2.0 -- anticipating synthetic biology

Once again, Tamar Wilner shares illuminating insights with a response paper, this one on Session 9, the readings for which were designed to help us formulate advice on how science communicators should promote constructive public engagement with synthetic biology (in the event the public ever does engage).

I'm reproducing the beginning of her post below (no more than is consistent with the federal Copyright Act's "fair use" provisions; everything that happens on this blog, in contrast to the ones of our so-called "competitors," is in strict compliance with the law), and then linking over to her site for the rest.

There's already great discussion going on in connection with the "virtual class" post, so post your own thoughts there!

to keep reading ...


Pakistani Dr & Kentucky Farmer cause uprising of agitated reflection at Annenberg Center!

Gave talk yesterday at Annenberg Public Policy Center (where I will be a fellow next spring) on “Science Communication Measurement Problem.” 

Was a super great audience, brimming with knowledge, intelligence & curiosity.

Slides here.

I’ve given talks before on the Measurement Problem & its significance for science communication.  

But in this one for the first time I gave a pretty central place to the “Pakistani Dr” paradox—the apparently simultaneous belief & disbelief in one or another scientific proposition (human evolution, human-caused climate change).

Indeed, Everhart & Hameed’s Pakistani Dr only arrived after the “Measurement Problem” study was done, to try to help me answer the question a perceptive audience member asked after I gave a lecture at RMIT University last summer. . . .  The Dr’s helped a lot, but I’m for sure I remain perplexed.

The audience members yesterday were aroused and agitated by him, and particularly by his buddy the Kentucky Farmer

There was the usual impulse to try to explain away the paradox—one’s involving either specifying the propositions believed/disbelieved in more fine-grained ways (“micro- vs. macro-evolution”; “scientists say that, but they are wrong”) or positing unrevealed attitudes (“he doesn’t really disbelieve evolution—he’s just saying that”; “FYATHYRIO” ; “hypocritical selfish bastard acting on basis of self-interest” etc.) that dissolve the apparent contradiction.

That's understandable. It’s everyone’s first instinct, and isn’t necessarily the wrong answer!  But as I tried to explain, I think we should resist the impulse to accept those “solutions” too readily, lest they preempt valid empirical inquiry into the range of plausible hypotheses.

Actually, as far as I could tell, everyone readily agreed with me when I raised that point.

I, of course, found myself engaged in a kind of “cheerleading” for my favorite conjecture—the “pragmatic dualism” position, I guess I’d call it (not b/c that is a very good label but b/c it’s as good as anything else I can think of for now).

On this account, the appearance of contradiction reflects a mistaken model of how “beliefs” figure in reasoning. 

The mistaken model is that “beliefs” are mental objects akin to factual or empirical propositions that can be identified exclusively with their states-of-affairs referents: “natural history of humans”  or “scientific theory of same originating in work of Darwin"; “global temperature  trends over last decade” and “impact of burning fossil fuels on the same.”

It makes sense to treat “facts” (essentially) that way for purposes of scientific inquiry, and “beliefs” about them as summaries of our assessment of the best available scientific evidence, I agree.

But “inside of people’s heads” it doesn’t make sense to think of “beliefs” being isolated proposition bits switched to either a “true” (“1”) or “false”(“0”) position.

Rather, “beliefs” in states of affairs are always parts of a bundles of intentional states that include not just assessments of such propositions but also affective reactions to them that reflect their significance and that incline one to particular courses of action (Damasio 2010; Lewandwosky 2000; Elga & Rayo 2014).

“Knowing that” is always part of a “knowing how”—for psychological purposes.

There is no way, on this account, to individuate a belief as a “mental object” abstracted from the action-enabling bundles of intentional states that they are part of.

Because beliefs can’t be individuated independently of the actions they enable, then there’s no necessary “contradiction” in both “believing” and “not believing” propositions about external states of affairs.  There would be a contradiction only if the kinds of things that a person is enabled to do by the bundles of intentional states that contain those opposing "beliefs" themselves interefered with one another.

Hameed’s Pakistani Dr is enabled to be a doctor—enabled to practice medicine and experience sense of identity as a part of a science-based profession—by believing in evolution.

He is also enabled to be a part of a certain religious community by disbelieving that particular account of the natural history of human beings.

What’s the problem, he keeps asking us?  I am both of those things—and there’s no tension, in the life I lead (in the society in which I live) in doing so.

Similarly for Kentucky Farmer.  He is enabled to be a certain kind of person—a “hierarchical individualist,” let’s say—by “disbelieving human-caused climate change.”  But he is also enabled to be a successful farmer by “believing in human-caused climate change”—by using, in fact, the best available information on how human activity is affecting the climate so that he can make sensible decisions about his farming practices (no-till farming, crop-rotation, use of genetically modified seeds, etc.) and about conducting his commercial operations (buying crop-failure insurance, etc.).

Big deal, he says. I do both of those things—and they fit together for me just fine.

I don’t know if this is right.  I’d like to figure out experiments for testing this & other plausible conjectures about “what’s going on in their heads.”

But one thing that I realized might be making people resist this account at the workshop wasn’t the implausibility of it.

On the contrary, it was the very likelihood that this might be exactly what is happening.

The objection, for some, I think, was less to the apparent “contradiction” in the Kentucky Farmer’s “beliefs” (he came in for the most critical attention at the workshop).  Rather it was to what he was being enabled to do by his particular “knowing that”/“knowing how” clusters on climate change.

People were distressed, in particular, by the absence in him of a bundle of action-enabling intentional states containing “belief in climate change” that was geared toward impelling him to demand a particular set of policies relating to the mitigation via putting restrictions on various sorts of commercial and market behavior in the US and other countries.

If Ky Farmer had "belief in human-caused climate change" within a cluster of intentional states that impelled him to demand that, I doubt these critics would have cared much if he, like the vast majority of people who have that bundle of intentional states, actually didn't know even the most rudimentary aspects of climate change science.

In other words, at least some people weren't really objecting to the "irrationality" of the Kentucky Farmer's beliefs. They just didn’t like the person that his “beliefs” were rationally enabling him to be.

They are entitled to feel that way!

But I do think it is useful to recognize that that's what the objection is.

Or in any case, it occurred to me that this might be one way to make sense of how others were making sense of the Kentucky Farmer and the Pakistani Dr.

I could be wrong about that.

As I said, I’m perplexed—and curious what others think (and of course for guidance to treatments of these issues by thoughtful people who have already investigatd them)!


Elga A & Rayo, A. Fragmentation and information access. Working paper (2014). 

Damasio, A.R. Self comes to mind : constructing the conscious brain (Pantheon Books, New York, 2010). 

Lewandowsky, S. & Kirsner, K. Knowledge partitioning: Context-dependent use of expertise. Memory & Cognition 28, 295-305 (2000).



Science of Science Communication 2.0, Session 10.1: Teaching science in a polluted science communication environment, part I -- Evolution

It's that time again!  "Science of Science Communication 2.0" session 10. Reading list here, & study/discussion questions below.  



On self-deception & motivated reasoning: Who is fooling whom?

From something I'm working on....

Who is fooling whom?

Identity-protective cognition is a species of motivated reasoning that consists in the tendency of people to conform disputed facts (particularly ones relevant to political controversies) to positions associated with membership in one or another affinity group. I will present evidence—in the form of correlational studies, standardized assessment tests, and critical-reasoning experiments—that show that identity-protective cognition is not a consequence of over-reliance on heuristic information processing. On the contrary, proficiency in one or another aspect of critical reasoning magnifies individuals’ tendency to selectively credit evidence in a manner that conforms to the position associated with their group identity. The question I want to frame is, Which of these two conclusions is more supportable: that  individuals who engage in this form of information processing are using their reason to fool themselves; or that we (those who study them) are fooling ourselves about what these individuals are actually using their reason to do?


"You -- talking to me? Are *you* talking to *me?" Actually, no, I'm not; the data don't tell us how any individual thinks (or that your side is either "biased" or "right").

A thoughtful correspondent writes:

I am a physician . . .  I was reading an article on Vox debunking the theory which states that more information makes people smarter. This article referenced your study concluding that those with the most scientific literacy and technical reasoning ability were less likely to be concerned about climate change and the safety of nuclear energy.

I read the paper which shows this quite nicely.

I am confused about the conclusions. I scored a perfect score on the science literacy test and on a technical reasoning test as well. I do not believe climate change is a settled science and I believe nuclear power is the safest form of reliable energy available.

The conclusion that I am biased by my scientific knowledge is strange.

In medical experiments data are scientifically gathered and tabulated. Conclusions are used as a way to explain the data. Could an alternate conclusion be reached that scientific and reasonable people downplay the danger of climate change and nuclear power precisely because we are well informed and able reason logically? It seems just as likely a conclusion as the one you reached yet it was never discussed.

My response:

Thanks for these thoughtful reflections. They deserve a reciprocally reflective and earnest response.

1st, I don't think the methods we use are useful for explaining individuals. In the study you described, they identify in large samples patterns that furnish more support than one would otherwise have for the inference that some group-related influence or dynamic is at work that helps to explain variance in genera.  

One can then do additional studies, experimental in nature (like this & this), that try to help to furnish even more support for the inference -- or less, since that is what a valid study has to be in the position to do to be valid. 

But once one has  done that, all one has is an explanation for some portion of the variance in groups of people.  One doesn't have an explanation all the variance (the practical & not & merely "statistical" significance of which is what a reflective person must assess). One doesn't have an instrument that "diagnoses" or tells one why any particular individual believes what he or she does.  

And most important of all you don't have a basis for saying anyone on any of the issues one is studying is "right" or "wrong": to figure that out, do a valid study on the issue on which people like this disagree; then do another & another & another & another. And compare your results w/ others doing the same thing.

2d, I don't believe the dynamic we are looking at is a "bias" per se. Things are more complicated than, at least for me!

I'm inclined to think that the dynamics that we observe generating polarization in our studies are the very ones that normally enable people to figure out what is known by science.

They are also the very same processes that enable people to effectively use information for another of their aims, which is to form stances and positions on issues that evince commitments that they care about and that connected them to others.  That is a matter that is cognitively demanding as well -- & of course one that that most people, even ones who don't get a perfect score on "science comprehension" tests, possess the reasoning proficiency that it takes to perform it.  

What to make of the situations, then, in which that same form of reasoning generates states of polarization on facts that admit of empirical inquiry is a challenging issue -- conceptually, morally & psychologically?  This is very perplexing to me!

I suspect sometimes it reflects the experience of a kind of interference between or confounding of mental operations that serve one purpose and those that serve another.  That in effect, the "science communication environment" has become degraded by conflicts between the stake people have in knowing what's known & being who they are.  

At others, times it might simply be that nothing is amiss from the point of view of the  people who are polarized; they are simply treating being who they are as the thing that matters most for them in processing information on the issue in question. . . .

3d, notwithstanding all this, I don't think our studies admit of your "alternate conclusion": that "scientific and reasonable people downplay the danger of climate change and nuclear power precisely because we are well informed and able reason logically."  

The reason is that that's not what the data show.   They show that those highest in one or another measure of science comprehension are the most polarized on a small subset of risk issues including climate change.  

That doesn't tell us which side is "right" & which "wrong."

But it tells us that we can't rely on what would otherwise be a sensible heuristic -- that the answer individuals with those proficiencies are converging on is most likely the right answer.  Because again, those very people aren't converging; on the contrary, they are the most polarized.

Many people write to me suggesting that an "alternative explanation" for our data is that "their side" is right.  

About 50% of the time they are part of the group whose group is "climate skeptical" & the other half of the time the one that is "climate nonskeptical" (I have no idea what terms I'm supposed to be using for these groups at this point; if they hold a convention and vote on a preferred label, I will abide by their decisions!).

I tell them every time that can’t actually be what the data are showing—for all the reasons I’ve just spelled out.   

Some fraction (only a small one, sadly), say "ah, yes, I see."  

 I can't draw any inferences, as I said, about the relationship between their "worldviews" & how they are thinking.  

I have no information about their scors on "science comprehension" or "critical reasoning" tests.

But at that point I can draw an inference about their intellectual character: that they possess the virtue of being able and willing to recognize complexity.


Science of Science Communication 2.0, Session 9.1: Emerging technologies part II -- synthetic biology!

Were back from spring break (learning things is hard & we need lots of rest & time to recover as we go along).

Time for  "Science of Science Communication 2.0" session 9. Reading list here, & study/discussion questions below.  

Have at it!



Professional judgment--in risk perception & in law: Dual process reasoning and science communication part 3...

This is from something I've been working on.  For a long time.  The paper of which it is a part will be posted soon.  

But for now I am treating it as the final installment of a 3-part series on the relevance of dual-process reasoning theories to science communication. As I'm sure all 14 billion regular readers of this blog recall, the first installment appeared on July 19, 2013, and the second on July 24, 2013.  

Even as of that period, I had been working on this project for a long time. . . .

II. Information Processing, Pattern Recognition, and Professional Judgment

Legal training and practice can reasonably be understood to cultivate proficiency in conscious, analytical forms of reasoning. Thus, the work on “motivated System 2 reasoning”—the tendency of conscious, effortful information processing to magnify identity-protective cognition—might in fact be regarded as supplying the strongest support for the conjecture that unconscious cultural partisanship can be expected to subvert judicial neutrality.

Nevertheless, when judges decide cases, they are not merely engaging in conscious, effortful information processing: they are exercising professional judgment. Professional judgment consists, essentially, in habits of mind—conscious and effortful to some degree, but just as much tacit and perceptive—that are distinctively fitted to reasoning tasks the nature of which falls outside ordinary experience. Indeed, it is characterized, in many fields, by resistance to all manner of error, including ones founded on heuristic information processing, that would defeat the special form of decision that professional judgment facilitates.

The dominant scholarly account of professional judgment roots it in the dynamic of pattern recognition. Pattern recognition consists in the rapid, un- or pre-conscious matching of phenomena with mentally inventoried prototypes. A ubiquitous form of information processing, pattern recognition is the type of cognition that enables human beings reliably to recognize faces and read one another’s’ emotions. But it is also the basis for many forms of highly specialized forms of expert decisionmaking. Highly proficient chess players, for example, outperform less proficient ones not by anticipating and consciously simulating a longer sequence of potential moves but by more reliably perceiving the relative value of different board positions based on their prototypical affinity to ones learned from experience to confer an advantage to one player or another. Likewise, the proficiency of aerial photography analysts consists in their tacit ability to discern prototypical clusters of subtle cues that allow them to cull from large masses of scanned images ones that profitably merit more fine-grained analysis. Forensic accountants must use the same form of facility as they combing through mountains of records in search of financial irregularities or fraud.

Expert medical judgment supplies an especially compelling and instructive example of the role of pattern recognition. Without question, competent medical diagnosis depends on the capacity to draw valid inferences from myriad sources of evidence that reflect the correlation between particular symptoms and various pathologies—a form critical reasoning that figures in System 2 information processing. But studies have shown that an appropriately attuned capacity for pattern recognition plays an indispensable role in expert medical diagnosis, for unless a physician is able to form an initial set of plausible conjectures—based on the match between a patient’s symptoms and an appropriately stocked inventory of disease prototypes—the probability that the physician will even know to collect the evidence that enables a proper diagnosis will be unacceptably low.

The proposition that pattern recognition plays this role in professional judgment generally is most famously associated with Howard Margolis. Focusing on expert assessment of risk, Margolis described a form of information processing that differs markedly from the standard “System 1/System 2” conception of dual process reasoning. The latter attributes proficient risk assessment to an individual’s capacity and disposition to “override” his or her unconscious System 1 affective reactions with ones that reflect effortful System 2 assessments of evidence. Margolis, in contrast, suggests an integrated and reciprocal relationship between unconscious, perceptive forms of cognition, on the one hand, and conscious, analytical ones, on the other. Much as in the case of proficient medical diagnosis, expert risk assessment demands reliable, preconscious apprehension of the phenomena that merit valid analytical processing. Even then, the effective use of data generated by such means, Margolis maintains, will depend on the risk expert’s reliable assimilation of such evidence to an inventory of prototypical representations of cases in which the appropriate data were given proper effect. Of course, the quality of an expert’s pattern recognition capacity will depend heavily on his or her proficiency in conscious, analytical reasoning: that form of information processing, employed to assess and re-assess successes and failures over the course of the expert’s training and experience, is what calibrates the experts’ perceptive faculty.

To translate Margolis’s account back into the dominant conception of dual-process reasoning, System 2 gets nowhere—because it is not reliably activated—without a discerning System 1 faculty of perception. The reliability of  System 1, however, reflects the contribution System 2 makes to the process of continual self-evaluation that imparts perceive judgments with their reliability.

Karl Llewellyn suggested an account of the reasoning style of lawyers and judges very much akin to Margolis’s view of the professional judgment for risk experts. Although Llewellyn is often identified as emphasizing the indeterminacy of formal legal rules and doctrines, the aim of his most important works was to explain how there could be such a tremendously high degree of consensus among lawyers and judges on what those rules and doctrines entail. His answer was “situation sense”—a perceptive faculty, formed through professional training and experience, that enabled lawyers and judges to reliably assimilate controversies to “situation types” that include within them their own proper resolutions. Llewellyn discounted the emphasis on deductive logic featured in legal argumentation. But he did not dismiss such reasoning as mere confabulation; in his view, lawyers and judges (legislators, too, in drafting rules) employed formal reasoning to prime or activate the “situation sense” of other lawyers and judges—the same function that Margolis sees it as playing in professional discourse among risk experts and indeed in any setting in which human being resort to it.

Margolis also identified the role that pattern recognition plays in professional judgment to explain expert-public conflicts over risk. Lacking the experience and training of experts, and hence the stock of prototypes that reliably guide expert risk assessment, members of the public, he argued, were prone to one or another heuristic bias. By the same token, the experts’ access to those prototypes reliably fixes their attention on the pertinent features of risks and inure them to the features that excite cognitive biases on the part of the lay public.

Based on the role of pattern recognition in professional judgment, one might make an analogous claim about judicial and lay judgments in culturally contested legal disputes. On this account, lawyers’ and judges’ “situation sense” can be expected to reliably fix their attention on pertinent elements of case “situation types,” thereby immunizing them from the distorting influence that identity-protective cognition exerts on the judgments of legally untrained members of the public. It is thus possible the professional judgment of the judge, as an expert neutral decisionmaker, embodies exactly the form of information processing most likely to counteract identity-protective reasoning, including the elements of it magnified by System 2 reasoning.


What can we learn from (a) studying public perceptions of the risks of technologies the public hasn't heard of & (b) studying studies that do that?

Tamar Willner has posted another very perceptive and provocative essay in reaction to the readings for Science of Science Communication 2.0, this time in relation to Session 8, on “emerging technologies.”  I’ve posted the first portion of it, plus a link to her site for continuation.

She also posed a very interesting question in the comments about an experiment that CCP did on nanotechnology risk perceptions.  I’ve posted my answer to her question below the excerpt from her own post.

1. Tamar Wilner on studying perceived risks of emerging technologies ...

read rest of post

2. Q&A on a CCP study of nanotechnology risk perceptions

Tamar's question:

[I]n your paper (Cultural Cognition of the Risks and Benefits of Nanotechnology) you say, “The ‘cultural cognition’ hypothesis holds that these same patterns [cultural polarization] are likely to emerge as members of the public come to learn more about nanotechnology.” But in your blog you repeatedly make the point that only a minority of public science topics end up getting polarized - that such polarization is “pathological” in its rarity. Why then did you hypothesize that such a pattern would be likely to emerge for nanotech?

I noticed that you start to address this later in the paper when you say, “At the same time, nothing in our study suggests that cultural polarization over nanotechnology is inevitable…” and point out that proper framing can help people to extract factual information. Does this indicate that the passages used in your study employed framing likely to encourage polarization? They seem to use pretty neutral language, to me. What about them makes them polarizing - and is it possible that some polarizing language is unavoidable? For example it seems like just talking about "risks of a new technology" tape into certain egalitarian/communitarian sensibilities, but since that's exactly what the topic of discussion is, I don't see how you would avoid it.

My response

This is a great question.  It raises some important general issues & also gives me a chance to say some things that how my own views of the phenomenon of cultural contestation over risk have evolved since performing the study.

The main motivation for the study, actually, was a position that we characterized as the “familiarity hypothesis”: that as people learned more about nanotechnology, their views were likely to be positive.

This was an inference from the a consistent survey finding that although only a small percentage of the public reports having heard of nanotechnology, those who say they have tend to express very favorable views about the ratio of benefits to risks that it is likely to involve.

That inference is specious: there is obviously something unusual about people who know about a technology 80% of the rest of the public is unfamiliar with; it reflects poor reasoning not to anticipate that whatever is causing them to become familiar with a novel technology might also dispose them to form a view that others who lack their interest in technology might fail to form when they eventually learn about a novel form of science.

Our hypotheses, largely corroborated by the study, was that those who were already familiar with nanotechnology (or actually, simply saying they were familiar; the surveys were using self-report measures) were likely people with a protechnology “individualist” cultural outlook, and that when individuals with anti-technology “egalitarian communitarian” ones were exposed to information on nanotechnology they would likely form more negative reactions.

Okay, fine.

But Tamar’s perceptive question is why did we expect people unfamiliar with a technology to react at all when exposed to such a small amount of info?

As she notes, only a small minority of potentially risky technologies excite polarization.  People tend to overlook this fact b/c the they understandably fixate on those and ignore the vast majority of noncontroversial ones.

My answer, basically, is that I don’t think the research team really had a good grasp of that point at the time we did the study.  I know I didn’t! 

I think, actually, that I really did mistakenly believe that culturally infused and hence opposing reactions to putative risk sources was “the norm,” and that it was therefore likely our subjects would polarize in the way they did.

Looking back, I’d say the reason it was reasonable to expect subjects would polarize is that the study was putting them in the position of consciously evaluating risks and benefits.

Check out these polarized & nonpolarized risks -- all you need to do is click! C'mon! It won't hurt!On the vast majority of putative risk sources on which there isn’t any meaningful level of polarization—from pasteurization of milk to medical x-rays to cell phone radiation to high-power transmission lines etc.—people don’t consciously think anything; they just model their behavior on what they see other people like them doing; when they do so, it’s rare for them to observe signs that give them reason to think there is anything to worry about.

Perfectly sensible approach, in my view, given how much more information known to science it makes sense to use in our lives than we have time to make sense of on our own.

But as I said, the study subjects were being prompted to do conscious risk assessment.  Apparently, in doing that, they reliably extracted from the balanced risk-benefit information culturally affective resonances that enabled them to assimilate this novel putative risk source—nanotechnology—to a class of risks, environmental ones, on which members of their group are in fact culturally polarized. 

Being made to expect, in effect, that there would be an issue here, the subjects reliably anticipated too what position “people like them” culturally speaking would likely take.

This interpretation raises a second point on which my thinking has evolved: the external validity of public opinion studies of novel technologies.

This was (as Tamar’s excellent blog post on the readings as a whole discusses) a major theme of the readings.  Basically, when pollsters ask people their views on technological risks about which members of the public have never heard and don’t have discussions about in their daily lives, they aren’t genuinely measuring a real-world phenomenon. 

They are, in effect, modeling how people react to the strange experience of being asked questions about something they have not thought about.  To pretend that one can draw inferences from that to what actual people in the world are truly thinking is flat-out bogus. Serious social science researchers know this is a mistake; news-maker and advocacy pollsters either don’t or don’t care.

One can of course try to anticipate how people—including ones with different cultural outlooks might react to an emerging technology when they do learn about it.  Indeed, I think that is a very sensible thing to do; the failure to make the effort can result in disaster, as it did in the case of the HPV vaccine!

But to perform what amounts to a risk-perception forecasting study, one must use an experimental design that it is reasonable to think will induce in subjects the reaction that people in the real world will form when they learn about the technology—or could form depending on how they learn about it.  That is what one is trying to model.

A simple survey question—like the one Pew asked respondents about GM foods in its recent public attitudes studycannot plausibly be viewed as doing that.  The real-world conditions in which people learn things about a new technology will be much richer—much more dense with cues relating to the occasions for discussing an issue, the setting in which the discussion is being had, and the identity and perceived motivations of the information sources—than are accounted for in a simple survey question.

I think it is possible to do forecasting studies that reasonable people can reasonably rely on.  I think our HPV vaccine risk study, e.g., which tried to model how people would likely react depending on whether they learned about the vaccine in conditions that exposed them to cues of group-conflict or not was like that.

But I think it is super hard to do it.

Frankly, I now don’t think our nanotechnology experiment design was sufficiently rich with the sorts of contextual background to model the likely circumstances in which people would form nanotechnology risk perceptions!

The study helped to show that the “familiarly hypothesis,” as we styled it, was simplistic.  It also supported the inference that it was possible people might assimilate nanotechnology to the sorts of technological-risk controversies that now polarize members of different groups. 

But the stimulus was too thin to be viewed as modeling the conditions in which that was actually likely to happen..

We should be mindful of hindsight bias, of course, but the fact that nanotechnology has not provoked any sort of cultural divisions in what is now approach two decades of its use in commercial manufacturing helps show the limited strength of inferences on the likelihood of conflict that can be drawn from experiments like the one we did. 

As Tamar notes, we were careful in our study to point out that the experimental result didn’t imply that conflict over nanotechnology was “inevitable” or necessarily even “likely.”

But I myself am very willing—eager even—to acknowledge that we viewed the design we used as more informative than it could have been expected to be about the likely career of nanotechnology.

I have acknowledged this before in fact. 

In doing so, too, I pointed out that that doesn’t mean studies like the ones we and other researchers did on nanotechnology risk perceptions weren’t or aren’t generally useful.  It just means that the value people can get from those studies depends on researchers and readers forming a valid understanding of what designs of that sort are modeling and what they are not.  

In order for that to happen, moreover, that researchers must reflect on their own studies over time to see what the fit between them and experience tells them about what is involved in modeling real-world processes in a manner that is most supportive of real-world inferences.

Speaking for myself, at least, I acknowledge that, despite my best efforts, I cannot guarantee anyone I will always make the right assessment of the inferences that can be drawn from my studies.  I can promise, though, that when I figure out that I didn’t, I’ll say so—not just to set the record straight but also to help enlarge understanding of the phenomena that it is in fact my goal to make sense of.

Of course, if a cultural conflagration over nanotechnology ignites in the future, I suppose I’ll have to acknowledge the “me” I was then then had a better grasp of things than the “me” I am now; I doubt that will happen—but life, thank goodness, is filled with surprises!


Is "shaming" an effective way to counteract biased information processing? A preliminary investigation

So far, there's been no improvement in the subject's defective information processing.

But data collection involving this subject and others is ongoing.



Submerged ... 

But will surface in near future -- w/ results of new study ....

Prize for anyone who correctly predicts what it is about; 2 prizes if predict result.


Science of Science Communication 2.0, Session 8.1: Emerging technologies part I

Wow--time flies, doesn't it? Especially when every other week of class is cancelled due to snow.

But in any case-- it's that time again!  "Science of Science Communication 2.0" session 8 reading list here, & study/discussion questions below.  

Have at it!


Weekend update: Science of Science Communication 2.0: on CRED "how to" manual & more on 97% messaging

Here are some contributions to class discussion.  The first is from Tamar Wilner, who offers reactions to session 7 readings.  I've posted just the beginning of her essay and the linked to her page for continuation. The second is from Kevin Tobia (by the way, I also recommend his great study on the quality of "lay" and "expert" moral intuitions), who addresses "97% messaging," a topic initially addressed in Session 6 but now brought into sharper focus with a recently published study that I myself commented on in my last post.

Health, ingenuity and ‘the American way of life': how should we talk about climate?

Tamar Wilner

Imagine you were

  1. President Obama about to make a speech to the Nation in support of your proposal for a carbon tax;
  2. a zoning board member in Ft. Lauderdale, Florida, preparing to give a presentation at an open meeting (at which members of the public would be briefed and then allowed to give comments) defending a proposed set of guidelines on climate-impact “vulnerability reduction measures for all new construction, redevelopment and infrastructure such as additional hardening, higher floor elevations or incorporation of natural infrastructure for increased resilience”;
  3. a climate scientist invited to give a lecture on climate change to the local chapter of the Kiwanis in Springfield, Tennessee; or
  4. a “communications consultant” hired by a billionaire, to create a television advertisement, to be run during the Superbowl, that will promote constructive public engagement with the science on and issues posed by climate change.

Would the CRED manual be useful to you? Would the studies conducted by Feygina, et al., Meyers et al., or Kahan et al. be? How would you advise any one of these actors to proceed?

First, some thoughts on these four readings.

The CRED Manual: well-intentioned, but flawed2015-02-27_10-42-04

Source material: Center for Research on Environmental Decisions, Columbia University. “
The Psychology of Climate Change Communication: A guide for scientists, journalists, educators, political aides, and the interested public.”

When I first read the CRED manual, it chimed well with my sensibilities. My initial reaction was that this was a valuable, well-prepared document. But on closer inspection, I have misgivings. I think a lot of that “chiming” comes from the manual’s references to well-known psychological phenomena that science communicators and the media have tossed around as potential culprits for climate change denialism. But for a lot of these psychological processes, there isn’t much empirical basis showing their relevance to climate change communication.

Of course, the CRED staff undoubtedly know the literature better than I do, so they could well know empirical support that I’m not aware of. But the manual authors often don’t support their contentions with research citations. That’s a shame because much of the advice given is too surface-level for communications practitioners to directly apply to their work, and the missing citations would have helped practitioners to look more deeply into and understand particular tactics.

continue reading


“Gateway beliefs” and what effect 97% consensus messaging should have

Kevin Tobia

A new paper reports the effect of “consensus messaging” on beliefs about climate change. The media have begun covering both this new study and some recent criticism. While I agree with much of the critique, I want to focus on a different aspect of the paper, the idea of a “gateway belief.” Thinking closely about this suggests an interpretation of the paper that raises important questions for future research – and may offer a small vindication of the value of 97% scientific consensus reporting. You be the judge.

First, what is a “gateway” belief? In this context, what is it for dis/belief in scientific consensus on climate change to be a “gateway” belief? You might think this means that some perceived level of scientific consensus is necessary to have certain beliefs in climate change, worry in climate change, belief in human causation, and support for public action. You can’t have these beliefs unless you go – or have gone – “through the gate” of perceived scientific consensus.

But here’s what the researchers actually have to say about the “gateway” model prediction:

We posit that belief or disbelief in the scientific consensus on human-caused climate change plays an important role in the formation of public opinion on the issue. This is consistent with prior research, which has found that highlighting scientific consensus increases belief in human-caused climate change. More specifically, we posit perceived scientific agreement as a “gateway belief” that either supports or undermines other key beliefs about climate change, which in turn, influence support for public action. ... Specifically, we hypothesize that an experimentally induced change in the level of perceived consensus is causally associated with a subsequent change in the belief that climate change is (a) happening, (b) human-caused, and (c) how much people worry about the issue (H1). In turn, a change in these key beliefs is subsequently expected to lead to a change in respondents’ support for societal action on climate change (H2). Thus, while the model predicts that the perceived level of scientific agreement acts as a key psychological motivator, its effect on support for action is assumed to be fully mediated by key beliefs about climate change (H3). 

There are a couple things worth noting here. First, if the ultimate aim is increasing support for public action, you don’t have to go “through the gate” of the gateway belief model at all. That is, what is really doing the work is the set of (i) beliefs that climate change is happening, (ii) beliefs it is human-caused, and (iii) how much people worry about it. If we could find some other way to increase these, we could affect public support without needing to change the perceived level of science consensus (though, whether that change might itself affect the perceived level of scientific consensus is another interesting and open question).

Second – and more importantly – it is not the case that there is some “gateway belief” in perceived scientific agreement that is required to have certain beliefs in climate change or human causation, worry, or support for action. The model merely predicts that all of these are affected by changes to the perceived level of scientific agreement. Thus, it is incorrect to draw from this research that the “gateway belief” of perceived scientific consensus is (as analogy might suggest) some necessary or required belief that must be held in order to belief in human-caused/climate change, worry about it, or support for action. Instead, the gateway belief is something like a belief that tends to have or normally has some relation to these other climate beliefs.

When we put it this way, the “gateway belief” prediction might start to seem less interesting: there’s one belief about climate change (perceived consensus among scientists) that is a good indicator of other beliefs about climate change. But, what is more intriguing is the further claim the researchers make: this gateway belief isn’t just a good indicator of others, but increasing the gateway belief will cause greater belief in the others. Perceiving scientific non-consensus is the gateway drug to climate change denial!

This conception of the gateway belief illuminates a subtle feature of the researchers’ prediction. Recall:

we hypothesize that an experimentally induced change in the level of perceived consensus is causally associated with a subsequent change in the belief that climate change is (a) happening, (b) human-caused, and (c) how much people worry about the issue ....

The hypothesis here is that changing the level of perceived consensus causes changes in these other climate beliefs.

Some might worry about the relatively small effects found in the study. But if we recognize the full extent of the researchers’ prediction (that increased perceived consensus will raise the other beliefs AND that decreased perceived consensus will lower the other beliefs), one possibility is that some participants with very high pre-test consensus estimates (e.g. 99% or 100%) actually reduced their estimate in light of the consensus messaging – and that this affected their other beliefs. It might seem unlikely that many participants held an initial estimate of consensus upwards of 97% (even with a pre-test mean estimate of 66.98), but data on this would be useful.

There is a more plausible consideration that also weakens the worry about small effect size: would we really want participants (or people in the world) to change their beliefs about science in exact proportion to the most recent information they receive about expert consensus? To put it another way, the small effect size might not be evidence of the weakness of consensus messaging, but might rather be evidence of the measured fashion in which people weigh new evidence and update their beliefs.

What would be helpful is data on the number (or percent) of participants whose beliefs increased from pre to post test. This would help distinguish between two quite different possibilities: 

  1. very few participants greatly increased their beliefs in climate change after reading the consensus message
  2. many participants moderately increased their beliefs in climate change after reading the consensus message
There is no good way to distinguish between these from the data provided so far, but which of these is true is quite important. If (1) is true, consensus messaging can only be offered as a method to appeal to the idiosyncratic few. And we might worry that the few people responding in this extreme way are, in some sense, overreacting to the single piece of evidence they just received. If (2) is true, this might provide a small redemption of the “consensus messaging campaign.” A little consensus messaging increases people’s beliefs a little bit (and, since this is from just one message, the small belief change is quite reasonable).

Of course, even if (2) is true, much more research will be required before deciding to launch an expensive messaging campaign. For instance, suppose we discover that the climate beliefs of people with very high initial beliefs in consensus (“97+% initial believers”) are actually weakened when they are exposed to 97% messaging. Even if people who are initially “sub-97% initial believers” change their beliefs in light of 97% messaging, we should ask whether this trade-off is beneficial. There are a number of relevant considerations here, but one thought is that perhaps reducing someone’s climate change belief from 100% to 99% is not worth an equivalent gain elsewhere from, say, 3% to 4%. For one, behaviors and dispositions may increase/decrease non-proportionally. 100 to 99 percent belief change might result in the loss of an ardent climate action supporter, while change from 3 to 4 percent might result in little practical consequence. These are all open questions! 


"the strongest evidence to date" on effect of "97% consensus" messaging

There's a new study out on effect of "97% consensus" messaging.

Actually, it is a new analysis of data that were featured in an article published a few months ago in Climatic Change.

The earlier paper reported that after being told that 97% of scientists accept human-caused climate change, study subjects increased their estimate of the percentage of scientists who accept human-caused climate change.

The new paper reports results, not included in the earlier paper, on the effect of the study's "97% consensus msg" on subjects' acceptance of climate change, their climate change risk perceptions, and their support for responsive policy measures.

The design of the study was admirably simple: 

  1. Ask subjects to characterize on a 0-100 scale their "belief certainty" that climate change is occurring, that it is caused by humans, that it is something to worry about, and that something should be done about it;
  2. tell the subjects that “97% of climate scientists have concluded that human-caused climate change is happening”; and
  3. ask the subjects to characterize again their "belief certainty" that climate change is occurring, that it is caused by humans, that it is something to worry about, and that something should be done about it.

Administered to a group of 1,104 members of the US population, the experiment produced these results on the indicated attitudes:

So what does this signify?

According to the authors, 

Using pre and post measures from a national message test experiment, we found that all stated hypotheses were confirmed; increasing public perceptions of the scientific consensus causes a significant increase in the belief that climate change is (a) happening, (b) human-caused and (c) a worrisome problem. In turn, changes in these key beliefs lead to increased support for public action.

I gotta say, I just don't see any evidence in these results that the "97% consensus msg" meaningfully affected any of the outcome variables that the authors' new writeup focuses on (belief in climate change, perceived risk, support for policy).

It's hard to know exactly what to make of  the 0-100 "belief certainty" measures. They obviously aren't as easy to interpret as items that ask whether the respondent believes in human-caused climate change, supports a carbon tax etc.

In fact, a reader could understandably mistake the "belief certainty" levels in the table as %'s of subjects who agreed with one or another concrete proposition. To find an explanation of what the "0-100" values are actually measurements of, one has to read the Climatic Change paper-- or actually, the on-line supplementary information for the Climatic Change paper.

Weirdly, the authors simply don't report how the information affected the proportion of subjects who said they believe in climate change, human-caused or otherwise! If the authors have data on %s who believed in climate change before & after etc, I'm sure readers would actually be more interested in those....

But based on the "belief certainty" values in the table, it looks to me like the members of this particular sample, were, on average, somewhere between ambivalent and moderately certain about these propositions before they got the "97% consensus msg."

After, they got the message, I'd say they were, on average,  ... somewhere between ambivalent and moderately certain about these propositions.

From "75.19" to "76.88" in "belief certainty": yes, that's "increased support for policy action," but it sure doesn't look like anything that would justify continuing to spend milions & millions of dollars on a social marketing campaign that has been more or less continuously in gear for over a decade with nothing but the partisan branding of climate science to show for it.

The authors repeatedly stress that the results are "statistically significant."

But that's definitely not a thing significant enough to warrant stressing.

Knowing that the difference between something and zero is "statistically significant" doesn't tell you whether what's being measured is of any practical consequence.  

Indeed, w/ N = 1,104, even quantities that differ from zero by only a very small amount will be "statistically significant."

The question is, What can we infer from the results, practically speaking? 

A collection of regression coefficients in a path diagram can't help anyone figure that out.

Maybe there's more to say about the practical magnitude of the effects, but unfortunately the researchers don't say it.

For sure they don't say anything that would enable a reader to assess whether the "97% message" had a meaningful impact on political polarization.

They say this: 

While the model “controls” for the effect of political party, we also explicitly tested an alternative model specification that included an interaction-effect between the consensus-treatments and political party identification. Because the interaction term did not significantly improve model fit (nor change the significance of the coefficients), it was not represented in the final model (to preserve parsimony). Yet, it is important to note that the interaction itself was positive and significant (β = 3.25, SE = 0.88, t = 3.68, p < 0.001); suggesting that compared to Democrats, Republican subjects responded particularly well to the scientific consensus message.

This is perplexing....

If adding an interaction term didn't "significantly improve model fit," that implies the incremental explanatory power of treating the "97% msg" as different for Rs and Ds was not significantly different from zero. So one should view the effect as the same.

Yet the authors then say that the "interaction itself was positive and significant" and that therefore Rs should be seen as "respond[ing] particularly well" relative to Ds. By the time they get to the conclusion of the paper, the authors state that "the consensus message had a larger influence on Republican respondents," although on what --their support for policy action? belief in climate change? their perception of % of scientists who believe in climate change? -- is not specified....

Again, though, the question isn't whether the authors found a correlation the size of which was "significantly different" from zero.

It's whether the results of the experiment generated a practically meaningful result.

Once more the answer is, "Impossible to say but almost surely not."

I'll assume the Rs and Ds in the study were highly polarized "before" they got the "97% consensus msg" (if not, then the sample was definitely not a valid one for trying to model science communication dynamics in the general population). 

But because the authors don't report what the before-and-after-msg "belief certainty" means were for Rs and Ds, there's simply no way to know whether the "97% consensus msg's" "larger" impact on Rs meaningfully reduced polarization.

All we can say is that whatever it was on, the "larger" impact the msg had on Rs must still have been pretty darn small, given how remarkably unimpressive the changes were in the climate-change beliefs, risk perceptions, and policy attitudes for the sample as a whole.


The authors state that their "findings provide the strongest evidence to date that public understanding of the scientific consensus is consequential." 

If this is the strongest case that can be made for "97% consensus messaging," there should no longer be any doubt in the minds of practical people--ones making decisions about how to actually do constructive things in the real world-- that it's time to try something else.

To be against "97% consensus messaging" is not to be against promoting public engagement with scientific consensus on climate change.

It's to be against wasting time & money & hope on failed social marketing campaigns that are wholly disconnected from the best evidence we have on the sources of public conflict on this issue.




Had "comments disabled" for yesterday's post on  Session 7 of virtual "Science of Science Communication 2.0"

Was wondering why there weren't the usual 7,000+ "student" comments!

The sole purpose of this post is to announce that the comments feature for that one is now enabled.  Because that's the logical place for discussion, I'm disabling comments here.


Science of Science Communication 2.0, Session 7.1: communicating climate science part 2

It's that again! Another session -- # 7 --of virtual "Science of Science Communication 2.0"

Reading list here

Imagine you were

  1. President Obama about to make a speech to the Nation in support of your proposal for a carbon tax; 

  3. a zoning board member in Ft. Lauderdale, Florida, preparing to give a presentation at an open meeting (at which members of the public would be briefed and then allowed to give comments) defending a proposed set of guidelines on climate-impact “vulnerability reduction measures for all new construction, redevelopment and infrastructure such as additional hardening, higher floor elevations or incorporation of natural infrastructure for increased resilience”; 

  5. a climate scientist invited to give a lecture on climate change to the local chapter of the Kiwanis in Springfield, Tennessee; or 

  7. a “communications consultant” hired by a billionaire, to create a television advertisement, to be run during the Superbowl, that will promote constructive public engagement with the science on and issues posed by climate change.

Would the CRED manual be useful to you? Would the studies conducted by Feygina, et al., Meyers et al., or Kahan et al. be?  How would you advise any one of these actors to proceed?


Some other places to to find discussion

Couple of posts elsewhere worth checkiout out today.

1st is Tamar Wilner's great "response paper" for Science of Science Communication course session 6. She asks whether the evidence for "97% consensus messaging" bears critical scrutingy

2d is a post by me on Washington Post Monkey Cage discussing our recently published paper on geoengineering and "two-channel science communication."

One thing that bums me a bit about the post is that they edited out material at end explaining that the experiment was a model, not a proposed "communication framing strategy," and that the value of such a study is in guiding field studies.

Also not thrilled with headline--I don't study science communication to teach people how to "change skeptics' minds"; I do studies to show how to communicate science in a manner that enables people to decide for themselves what to make of it.

Oh well...





Weekend update: Hard questions, incomplete answers, on the "disentanglement principle"

I have written a few times now about the “disentanglement principle”—that science communicators & educators must refrain from “making free, reasoning people choose between knowing what’s known by science and being who they are.”  The Measurement Problem paper uses empirical evidence to show how science educators and communicators have “disentangled” identity and knowledge on issues like evolution & climate change, and proposes a research program aimed at perfecting such techniques.

In a comment on a recent post, Asheley Landrum posed a set of penetrating and difficult questions about “disentanglement.”  I thought they warranted a separate blog, one that I hoped might, by highlighting the importance of the questions and the incompleteness of my own answers, motivate others to lend their efforts to expanding our understanding of, and ability to manage, the problem of identity-knowledge entanglement.

Asheley's comment:

I'm really interested in the idea of disentangling identity from knowledge. However, I wonder to what extent that really can be done. Take, for instance, the conflation of belief in evolution versus knowledge of evolution that you've described. Does it matter if multiple cultural identies recognize that the theory of evolution states humans evolved from earlier species of mammal if they do not accept (or believe) it to be extremely likely to be true? Is our goal as scientists (and science communicators) to make sure that people simply know what a theory is comprised of but not worry about whether the public buys it?

Also, once a topic becomes politicized, is it possible to truly disentangle that topic from people's cultural identies? I feel like new work is showing how we can potentially stop topics from becoming politicized in the first place, but once a topic becomes entangled with cultural identy, the mere mention of it may trigger motivated cognition. Is it something that will pass with time? For instance, we've seen public perception shift on a myriad of social issues (e.g. Interracial marriage, now gay marriage). Is this a result of time or a change in the narrative surrounding the topics? Does changing the narrative surrounding certain science topics change eventually change how entangles that topic is with regard to cultural identity?

My response:


Good questions. I certainly don't have complete answers.

But I'd start by sorting out 3 things.

1st, is "non-entanglement" possible?

2d, can entanglement be undone?

3d, is the goal of the science communicator/educator “belief” or “knowledge”?

1. Is non-entanglement possible?

I take this to mean, is it possible to create conditions where people don't have to choose between knowing what's known and being who they are?

Answer is, of course.

For one thing, the issue never arises for most issues -- ones for which it very well could have.

E.g., identity and knowledge were "entangled" on HPV vaccine but not the HBV vaccine.

The former—as a result of factors that were perfectly foreseeable and perfectly avoidable—traveled a path into public awareness that generated conditions, persisting to this day, that distort and disable the normally reliable faculties parents use to make informed decisions about their children’s health.

Likewise, there's no entanglement between identity and knowledge and GMO in US - - even though there is in Europe.

There's no entanglement on vaccine safety in US -- although there might well be if we don't stifle the evidence-unencumbered misrepresentations about the extent of vaccine risk perceptions and the relationship between them and cultural styles.

So the first thing is, there is nothing necessary about entanglement; it happens for reasons we can identify; and if we organize ourselves appropriately to use the science of science communication, we can certainly avoid this reason-eviscerating state of affairs.

Another pointeven when positions on risks and other facts become entangled in antagonistic cultural meanings--turning them into symbols of cultural identity--it still is possible to create conditions of science communication that free people from having to choose between knowing and being who they are!

You advert to this in raising evolution. We know from empirical evidence that it is possible to teach evolution in a manner that doesn't make religious students choose between knowing and being who they are and that when the right mode of teaching (one focusing on simply valid inference from observation), they can learn the modern synthesis just as readily as students who say they "do believe" in evolution (and who invariably don't know anything about natural selection, random mutation, and genetic variance).

We need similar pedagogical strategies for teaching evolutionary science—and able teachers and researchers are busy at work on this problem.

I think, too, we are seeing successful disentanglement strategies being used in local government to promote evidence-based climate policymaking aimed at adaptation.

We need to study these examples and learn the mechanisms they feature and how to harness and deploy them to promote knowledge.

2. Can entanglement be undone?

As I mentioned, in most cases, the entanglement problem never arises-- as in case of HBV vaccine or GMO foods.

But if entanglement occurs-- if antagonistic meanings become attached to issues turning positions on them into symbols of identity--can that condition itself be neutralized, vanquished

This is different, I think, from asking whether, in a polluted science communication environment, it is possible to "disentangle" in communicating or teaching climate science or evolution, etc.

The communication practices that make that possible are in the nature of "adaptation" strategies for getting by in a polluted science communication environment.

The question here is whether it possible to decontaminate a polluted science communication environment.

I think this is possible, certainly. I suppose, too, I could give you examples where this seems to have happened (e.g., on cigarette smoking in US).

But the truth is, we know a lot less about how risks and like facts become entangled in antagonistic meanings, and about how to “adapt” when that happens, than we do about how to clear the science communication environment of that sort of pollution once it becomes contaminated by it.

We need more information, more evidence.

But the practical lesson should be obvious: we must use all the knowledge at our disposal, and summon all the common will and attention we can, to prevent pollution of the science communication environment in the first place (a critical issue right now for childhood vaccines).

3.  Is the goal of the science communicator/educator “belief” or “knowledge”? 

Finally, you raise the issue of what the “goal” of science communication and education is—“knowledge” vs. “belief”?

My own sense is that the “knowledge”-“belief” dichotomy here reflects at least two forms of confusion

One is semantic. It’s the incoherent idea that there is some meaningful distinction between the objects of “belief” and the objects of “knowledge” and that “science” deals with the latter.

I’ve discussed this before. It’s not worth going into again, except to remark that those who think they are making an important point when they assert “it’s not a belief—it’s a fact!”need to seek guidance from the two patron saints of science’s theory of causal inference—the Rev. Thomas Bayes and Sir Karl Popper—to get some remedial instruction on how empirical proof expands our knowledge (by furnishing valid observational evidence on the basis of which we update our current beliefs about how the world works).

The other confusion is more complicated. It's certainly not a cause for embarrassment, but not grasping it is certainly a cause for concern.

The nature of the mistake (I'm still struggling, but am pretty sure at this point that this is the nub of the problem) is to believe that, as a psychological matter, it makes sense to individuate people's "beliefs" (or items of "knowledge") independently of what those people are doing.

Consider Hameed’s Pakistani Dr.  He says he doesn’t “believe in” evolution: “Allah created man! We did not descend from monkeys!”

Yet he tells you “of course” he depends on evolutionary science in his practice as an oncologist, where he uses insights from this field to screen patients for potential cancer risks. 

Of course” medical research relies on evolutionary science, too, he says—“consider stem cell research!”

If we say, “but isn’t that inconsistent—to say you ‘disbelieve’ in evolution but then make use of it in those ways as a Dr?,” he thinks we are being obtuse.

And he is right.

As Everhart & Hameed helps us see, there are two different “evolutions”: the one the Dr rejects in order to be a member of a religious community; and the one he accepts in order to be a doctor and a member of a scientific-knowledge profession.

The idea that there is a contradiction rests on a silly model that thinks individuals’ “beliefs” (or what is “known” by them) can be defined solely with reference to states of affairs or bodies of evidence in the world.

In the mind, “beliefs” are intentional states--often compound ones, consisting of assent to various factual propositions but also pro- or con- affective stances, and related propensities to action--that are yoked to role-specific actions

As a psychological matter, then, what people “believe” or “know” cannot a be divorced from what use they make of the same.

Being a member of a religious community and being as a member of the medical profession are integrated elements of the Pakistani Dr's identity. 

As a result, there’s no contradiction between the Pakistani Dr. saying he “disbelieves” in evolution when he is “at home” (or at the Mosque), where the set of intentional states that signifies allows him to be the former, and that he “believes in” it when he is “at work,” where the set of intentional states that signifies allows him to be a member of a science-informed profession.  

He knows that the evolution he accepts and the one he rejects both refer to the same account of the natural history of human beings that originates in work of Darwin; but the one he accepts and the one he rejects are "completely different things" b/c connected to completely different things that he does.  

It's confusing, I agree, but he's not the one who is confused-- we are if we can't get grasp the point knowing that can't be disconnected, psychologically, from knowing how!

Still, the Pakistani Dr is is lucky to live a life in which the two identities that harbor these competing beliefs have no reason to quarrel.

For members of certain religious communities in the US–and as Hameed notes, for more and more Islamic scientists and scientists in training in Europe—that's not so.

There is a social conflict, one foisted on them by illiberal cultural status competition, that makes it impossible to use their reason to be both members of cultural communities and science-trained professionals at the same time.

So what should the goal of the science communicator and teacher be?

It should be to make it possible for people to recognize and give effect to scientific knowledge in order to do the things the doing of which require such knowledge: like being scientists; like being successful members of other professions, including, say, agriculture, that depend on knowing what science knows; like being a good parent; like being a member of a self-governing community, the well-being of which turns on it making science-informed policy choices; and like being curious, reflective people who simply enjoy the awe and pleasure of being able to participate in comprehending the astonishing insights into the mysteries of nature that our species has gained by using the signature methods of scientific inquiry.

If a science teacher or communicator thinks that it his or her job “to “get people to say they ‘believe in’ ” evolution or climate change independently of enabling them to do things that depend on making use of the best available evidence, then he or she is making a mistake.

If that a science educator thinks that he or she is doing a good job because people say they “believe in” those things even though they couldn’t pass a high-school biology test on natural selection, and think that CO2 is poisonous to plants, then he or she is incompetent.

And if he or she is trying to get people who use scientific knowledge related to climate change or evolution or anything else to say they “believe in” it when the only purpose that would serve is to force them to denigrate who they are, then he or she has become a source, witting or unwitting, of the science communication pollution that entangles identity and knowledge and that disables free and reasoning democratic citizens from making use of all the scientific knowledge at their disposal.

Page 1 ... 2 3 4 5 6 ... 30 Next 20 Entries »