follow CCP

Recent blog entries
popular papers

What Is the "Science of Science Communication"?

Climate-Science Communication and the Measurement Problem

Ideology, Motivated Cognition, and Cognitive Reflection: An Experimental Study

'Ideology' or 'Situation Sense'? An Experimental Investigation of Motivated Reasoning and Professional Judgment

A Risky Science Communication Environment for Vaccines

Motivated Numeracy and Enlightened Self-Government

Ideology, Motivated Cognition, and Cognitive Reflection: An Experimental Study

Making Climate Science Communication Evidence-based—All the Way Down 

Neutral Principles, Motivated Cognition, and Some Problems for Constitutional Law 

Cultural Cognition of Scientific Consensus
 

The Tragedy of the Risk-Perception Commons: Science Literacy and Climate Change

"They Saw a Protest": Cognitive Illiberalism and the Speech-Conduct Distinction 

Geoengineering and the Science Communication Environment: a Cross-Cultural Experiment

Fixing the Communications Failure

Why We Are Poles Apart on Climate Change

The Cognitively Illiberal State 

Who Fears the HPV Vaccine, Who Doesn't, and Why? An Experimental Study

Cultural Cognition of the Risks and Benefits of Nanotechnology

Whose Eyes Are You Going to Believe? An Empirical Examination of Scott v. Harris

Cultural Cognition and Public Policy

Culture, Cognition, and Consent: Who Perceives What, and Why, in "Acquaintance Rape" Cases

Culture and Identity-Protective Cognition: Explaining the White Male Effect

Fear of Democracy: A Cultural Evaluation of Sunstein on Risk

Cultural Cognition as a Conception of the Cultural Theory of Risk

« Weekend update: geoengineering and the expanding confabulation frontier of the "climate communication" debate | Main | Weekend update: Another helping of evidence on what "believers" & "disbelievers" do & don't "know" about climate science »
Friday
Sep122014

How should science museums communicate climate science? (lecture summary & slides)

I had the great privilege of participating in a conference, held at the amazing Museum of Science in Boston, on how museums can engage the public in climate science.  Below are my remarks--as best as I can remember them a week later.  Slides here.

You are experts on the design of science-museum exhibits.

I am not. Like Dietram, I study the science of science communication with empirical methods. 

I share his view that there are things he and I and others have learned that are of great importance for the design of science museum exhibits on climate change.

If you ask me, though, I won’t be able to tell you what to do based on our work—because I am not an expert at designing museum exhibits. 

But you are

So if in fact I am right to surmise that insights gleaned from the scientific study of science communication are relevant to design of climate science exhibits, you should be able to tell me what the implications of this work are for your craft.

I will thus share with you everything I know about climate science communication.

I’ve reduced it all to one sentence (albeit one with a semi-colon):

What ordinary members of the public “believe” about climate change doesn’t reflect what they know; it expresses who they are.

The research on which this conclusion rests actually originates in the study of public opinion on evolution.

One thing such research shows is that there is in fact no correlation whatsoever between what people say they believe about evolution and what they know about it.  Those who say they “believe” in evolution are no more or less likely to understand the elements of the modern synthesis—random mutation, genetic variance, and natural selection—than those who say they “don’t.” 

Indeed, neither is likely to be able to give a sufficiently cogent account of these concepts to pass a high school biology test.

Another thing scholars have learned from studying public opinion on evolution is that what one “believes” about it has no relationship to how much one knows about science generally.

I’ll show you some evidence on that.  It consists in the results of a science literacy test that I administered to a large nationally representative sample.

Like a good knowledge assessment should, this science comprehension instrument consisted of a set of questions that varied in difficulty.

Some, like “Electrons are smaller than atoms—true or false” were relatively easy: even an individual whose score placed him or her at the mean comprehension level, would have had about a 70% chance of getting that one right.

Other questions were harder: “Which gas makes up most of the Earth's atmosphere? Hydrogen, Nitrogen, Carbon Dioxide, Oxygen?”  Someone of mean science comprehension would have only about a 25% chance of getting that one.

If one looks at the item-response profile for “Human beings, as we know them today, developed from earlier species of animals—true or false?,” an item from the NSF’s Science Indicators battery, we see that it’s difficult to characterize it as either hard or difficult.  At the mean level of science comprehension, there is about a 55% chance that someone with an average level of science comprehension will get this one correct. But the probability of getting it right isn’t much different from that for respondents whose science comprehension levels are significantly lower or significantly higher than average.

The reason is that the NSF Indicator Evolution item isn’t a valid measure of science comprehension for a general-population sample of test takers. 

Its item-response profile looks sort of like what one might expect of a valid measure when we examine the answers of those members of the population who are below average in religiosity (as measured by frequency of prayer, frequency of church attendance, and self-reported importance of religion): that is, the likelihood of getting it right slopes upward as science comprehension goes up.

But for respondents who are above average in religiosity, there is no relationship whatsoever between their response to the Evolution item and their science comprehension level.

In them, it simply isn’t measuring the same sort of capacity that the other items on the assessment are measuring. What it’s measuring, instead, is their religious self-identity, which would be denigrated by expressing a “belief in” evolution. 

Among the ways one can figure this out, researchers have learned, is to change the wording of the Evolution item: if one adds to it the simple introductory clause, “According to the theory of evolution,” then the probability of a correct response turns out to be roughly the same in relation to varying levels of science comprehension among both religious and nonreligious respondents.

The addition of those words frees a religious respondent from having to choose between expressing who she is and revealing what she knows. It turns out she knows just as much—or just as little, really, since, as I said, responses to this item, no matter how they are worded, give us zero information on what the respondent understands about the theory of evolution.

But good high school teachers, empirical research show, can impart such an understanding just as readily in a student who says she “doesn’t believe in” evolution as those teachers can in a student who says he “does.” But the student who said she didn’t “believe in” evolution at the outset will not say she does when the course is over.

Her skillful teacher taught her what science knows; the teacher didn’t make her into someone else.

Indeed, insisting that students profess their “belief in” evolution, researchers warn, is the one thing guaranteed to prevent the religiously inclined student from forming a genuine comprehension of how evolution actually works.  If one forces a reasoning individual to elect between knowing what is known by science and being who she is, she will choose the latter.

The teacher who genuinely wants to impart understanding, then, creates a learning environment that disentangles information from identity, so that no one is put in that position.

What researchers have learned from empirical study of the teaching of evolution can be extended to the communication of climate science.

To start, just as it would be a mistake (is a mistake made over and over by people who ought to know better) to treat the fraction of the population who says they “disbelieve in” evolution as a measure of science comprehension in our society, so it is a mistake to treat the fraction who say they “disbelieve” in human-caused climate change as such a measure.

My collaborators and I have examined how people’s beliefs about climate change relate to their science comprehension, too.  Actually, there is a connection: as culturally diverse individuals’ scientific knowledge and reasoning proficiency improve, they don’t converge in their views about the impact of human activity on global temperatures.  Instead they become even more culturally polarized. 

Because what one “believes” about climate change is now widely understood to signify one’s membership in and commitment to one or another cultural group, and because their standing in these groups are important to people, individuals use all manner of critical reasoning ability, experiments show, to form and persist in beliefs consistent with their allegiances.

But that doesn’t necessarily mean that individuals who belong to opposing cultural groups differ in their comprehension of climate science.  This can be shown by examining how individuals of diverse outlooks do on a valid climate science comprehension assessment.

To design such an instrument, I followed the lead of the researchers who have studied the relationship between “belief in” evolution and science comprehension. They’ve established that one can measure what culturally diverse people understand about evolution with items that unconfound or disentangle identity and knowledge.  Like the evolution items that enable respondents to show what they know without making affirmations that denigrate who they are, the items in my climate literacy assessment focus on respondents’ understanding of the prevailing view among climate scientists and not on respondents' acceptance or rejection of climate change “positions” known to be highly correlated with cultural and political outlooks.

Some of these turn out to be very easy. Encouragingly, even the test-taker of mean climate-science comprehension is highly likely (80%) to recognize that adding  CO2 to the atmosphere increases the earth’s temperature.

Others, however, turn out to be surprisingly hard: there is only a 30% chance that someone of average climate-science comprehension believes that adding CO2 emissions associated with burning fossil fuels have been shown by scientists to reduce photosynthesis in plants.

Obviously, someone who gets that  CO2 is a “greenhouse gas” but who believes that human emissions of it are toxic to the things that grow in greenhouses can’t be said to comprehend much about the mechanisms of climate science.

Nevertheless, a decent fraction of the test takers from a general population sample turned out to have a very accurate impression of climate scientists’ current best understandings of the mechanisms and consequences of human-caused global warming.  Not so surprisingly, these were the respondents who scored the highest on a general science comprehension assessment.

Moreover, there was no meaningful correlation between these individuals’ scores and their political outlooks.  “Conservative Republicans” who displayed a high level of general science comprehension and “liberal Democrats” who did both scored highly on the climate assessment test.

Nevertheless, those who displayed the highest scores on the test were not more likely to say they “believed in” human-caused global warming those who scored the lowest. On the contrary, those displayed the greatest comprehension of science’s best prevailing understandings of climate change were the most politically polarized on whether human activity is causing global temperatures to rise.

In other words, what ordinary members of the public “believe” about climate change, like what they “believe” about evolution, doesn’t reflect what they know; it expresses who they are.

The reason our society is politically divided on climate change, then, isn’t that citizens have different understandings of what climate scientists think.  It is that our political discourse, like the typical public opinion poll survey, frames the “climate change question” in a manner that forces them to choose between expressing who they are, culturally speaking, and revealing and acting on what they know about what is known.

This is changing, at least in some parts of the country.  Despite being as polarized as the rest of the country, for example, the residents of Southeast Florida have, through a four-county compact, converged on a comprehensive “Climate Action Plan,” consisting of 100 distinct adaptation and mitigation measures.

People in Florida know a lot about climate.  They’ve had to know a lot, and for a long time, in order to thrive in their environment.

Like the good high-school teachers who have figured out how to create a classroom environment in which curious and reflective students don’t have to choose between knowing what’s known about the natural history of humans and being who they are,  the local leaders who oversee the Southeast Florida Climate Compact have figured out how to create a political environment in which free and reasoning citizens aren’t forced to choose between using what they know and being who they are as members of culturally diverse communities.

Now what about museums?  How should they communicate climate science?

Well, I’ve told you all I know about climate science communication: that what ordinary members of the public “believe” about climate change doesn’t reflect what they know; it expresses who they are.

I’ve shown you, too, some models how of science-communication professionals in education and in politics have used evidence-based practice to disentangle facts from the antagonistic cultural meanings that inhibit free and reasoning citizens from converging on what is collectively known.

I think that’s what you have to do, too.

Using your professional expertise, you have already made museums a place where curious, reflective people of diverse outlooks go to satisfy their appetite to experience the delight and awe of apprehending what we have come to know by employing science’s signature methods of discovery.  

You now need to assure that the museum remains a place, despite the polluted state of our science communication environment generally, where those same people can go to satisfy their appetite to participate in what science has taught us and is continuing to teach us about the workings of our climate and the impact of human activity upon it.

You need, in short, to be sure that nothing prevents them from recognizing that the museum is a place where they don’t have to choose between enjoying that experience and being who they are.

How can you do that?

I don’t know.  Because I am not an expert in the design of science museum exhibits.

But you are—and I am confident that if you draw on your professional judgment and experience, enriched with empirical evidence aimed at testing and refining your own hypotheses, you will be able to tell me.  

 I have a strong hunch, too, that what you will have to say will be something other science-communication professionals will be able to use to promote public engagement with climate science in their domains, too.

 

PrintView Printer Friendly Version

EmailEmail Article to Friend

Reader Comments (3)

I believe that to sort out what sort of educational methods are effective, we need to be upfront about basicly conflicting objectives:

First, a public museum needs to be welcoming to all comers. Thus, your first slide above is correct, science museums should not force museum visitors to chose between what they believe and knowing what scientist know. Everyone should feel free to enjoy the really cool exhibits.

On the other hand, I think that it needs to be acknowledged that museum scientist/curators are not running a circus along the lines of BT Barnum. Its not just about the "Oh! Cool! reaction and collecting the entrance fees. They really are motivated by a desire to create an informed public. The measure of this though, is not just to have a public with a sound base of science facts. But more importantly that such awareness leads to citizens who can and will be capable of acting intelligently in the manner in which they might use that knowledge to inform decision making outside of the museum walls.

In an example having mostly to do with evolution, The Denver Museum of Nature and Science, for example, has had private tour groups that use their facilities for their own purposes of re-enforcing religious dogma: http://abcnews.go.com/Nightline/FaithMatters/Story?id=4467337&page=1 and http://www.bctours.org/dinosaur-tour.html. This is ok. As a result of publicity generated by news stories such as the one above, the Museum staff went out of its way to ensure that visitors to the museum needed to be respectful of other visitors and confrontations between ideologies was inappropriate.

On the other hand, museum staff has also worked hard to ensure that its displays are directly engaging, with lots of opportunities for visitors, especially children, to participate with scientists in hands on fashion. See: http://www.dmns.org/exhibitions/current-exhibitions/prehistoric-journey/ and for younger kids: http://www.dmns.org/exhibitions/current-exhibitions/discovery-zone/.

As described in the post above, museums should be places where scientists convey their feelings of excitement curiosity and awe that come from the practice of science, and in as much as possible give visitors a sense of participating in that process. Museums like the one in Denver have transformed through the years from being strictly displays (here is a stuffed polar bear) to more thoughtful and participatory exhibits. A lot of this change has to do with what we now know about the science of science communication.

Museum curators most definitely should want visitors, all visitors, to go away more thoughtful about their own ideas and belief system than when they arrived.

I think that this process fits the old adage that "You can lead a horse to water, but you can't make him drink". This does not mean that we should delude ourselves into imagining that we can proceed, in our interactions with others, as if we don't think that drinking that water is a really good idea.

September 12, 2014 | Unregistered CommenterGaythia Weis

"The reason our society is politically divided on climate change, then, isn’t that citizens have different understandings of what climate scientists think. It is that our political discourse, like the typical public opinion poll survey, frames the “climate change question” in a manner that forces them to choose between expressing who they are, culturally speaking, and revealing and acting on what they know about what is known."

Is "understandings of what climate scientists think" the same thing as "what they know about what is known"?

If you ask people "What do most creationists think..." about evolution, you will likely get everyone to converge on a common answer in a similar way. Does it reflect what people know about what is known, though?

"Despite being as polarized as the rest of the country, for example, the residents of Southeast Florida have, through a four-county compact, converged on a comprehensive “Climate Action Plan,” consisting of 100 distinct adaptation and mitigation measures."

Yes, but what does this prove? Climate sceptics all believe in weather. Mankind has always had to deal with weather-related disasters, and it makes perfect sense to take precautions against them. By promoting a "no regrets" approach, you can use climate change funding to enact the sort of weather defences that would benefit you anyway, whether the climate changes or not. Hurricanes will still happen, as they always have. Storm surges will still happen, as they always have. There will still be floods and droughts, heatwaves and freezes, as there always have. So we can be sure that money spent on measures intended to deal with them won't be wasted. And if you have to pay lip-service to the current political fashions to get politicians in government to route taxpayers' money your way, well, local politicians will still do that, as they always have.

So is it actually evidence of what they "know"?

"Museum curators most definitely should want visitors, all visitors, to go away more thoughtful about their own ideas and belief system than when they arrived."

Quite so.

So whatever you do to challenge people's beliefs on matters of controversy, you have to do to both sides. In Science, everything is open to challenge; and the best way to teach that is to do it.

"I think that this process fits the old adage that "You can lead a horse to water, but you can't make him drink". This does not mean that we should delude ourselves into imagining that we can proceed, in our interactions with others, as if we don't think that drinking that water is a really good idea."

The horse thinks drinking water is a good idea too, but doesn't want to drink the water from the pool you have led her to. There's another river over to your right, which the horse thinks is much tastier. But of course, it's you leading the horse, isn't it? The horse doesn't get to lead you. The horse doesn't get to decide, and has no choice, because you know better. She either drinks from the pool you select, or not at all.

There is an element, in this business of deciding what to do in museums, of regarding the public as like children in need of education. And of believing that we know better. It's a tiny bit elitist.

Do we really know any better? They're adults, just like us. Their knowledge and opinions compete or cooperate with ours on an equal footing in society. They clearly value science and knowledge - they've just paid money to get into a science museum after all! They might have their own ideas of what they want from a museum, or what they think would be good to show.

September 12, 2014 | Unregistered CommenterNiV

@Gaythia:

These are interesting comments.

I know of, but not enough about, the Denver MNS program.

My own view, though, is that the museum should not flinch or hesitate in presenting what science knows & exactly as science knows it for the benefit of any person who is interested in understanding what we have learned by using its signature methods.

Not everyone is interested. Some are even hostile.

But the population that experiences thrill and awe & pride in being able to apprehend what is known as a result of science is in fact very diverse. I know for a fact that it would break the heart of the sorts of museum professionals who attended this event to think that there was anyone who was prevented from satisfying his or her appetite for that sort of experience b/c doing so had become needlessly freighted with some meaning of cultural partisanship...

I'm not sure what this entails about how people should relate to their "belief systems" after visiting s museum. I'm sure I don't disagree with much of what you have in mind.

But I also have this feeling that Everhart & Hameed's Pakinstani Dr likely marvels at a museum display that features the contributions evolutionary science has made to the medical profession -- his profession -- and yet then goes "home," where he continues to "disbelieve" a theory of evolution that he'd tell you, with exasperation, is simply a "different thing" altogether.

I don't think I have any problem with that. Indeed, I think I have a problem with anyone who'd insist that those who communciate science --whether at the museum, inside the classroom, or in the democratic political arena -- do so in a way that tied the accessibility of what's known to someone's accepting or rejecting any particular "belief system."

September 12, 2014 | Registered CommenterDan Kahan

PostPost a New Comment

Enter your information below to add a new comment.

My response is on my own website »
Author Email (optional):
Author URL (optional):
Post:
 
Some HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>