1. SCS. This talk describes a tool for use in the evidence-based production of science films and related science media. The tool, the “Science Curiosity Scale” (SCS), enables individuals to be profiled in relation to their disposition to consume science-related media for personal edification and enjoyment. SCS also has other interesting, unexpected properties, the existence of which suggests the affinity between the craft of science filmmaking and the project to promote more constructive public engagement with science generally.
2. Why. (a) The book/movie Moneyball furnishes a useful backdrop for explication of the philosophy behind SCS. A supposed account of how statistical techniques were used to improve the general management of a professional baseball team, Moneyball rests on the premise that intuition and experience are unreliable guides for complex decisionmaking.
The philosophy behind SCS regards Moneyball’s premise as sheer, unadulterated bull shit. There is no substitute for craft sense—a perceptive faculty informed by immersion in professional norms and calibrated by personal experience—in complex decisionmaking, including science filmmaking.
But in science filmmaking as in other domains, the currency of craft sense is information. The premise of the “science of science communication,” of which SCS is a product, is that the methods of science can be used to augment the stock of information available to science filmmakers and other professional science communications so that they can exercise their craft sense in a manner that they can have reason to be even more confident will generate the outcomes they are seeking to produce.
(b) The most compelling sign that the science of science communication can be of value to science filmmakers is that they themselves so often disagree about how to conduct their craft. Some of the disagreements are general and recurring—the subject, in fact, of perennial panel discussions at annual gatherings like this one—while others are specific to the production of particular films. But whether systemic or episodic, issues that defy resolution on the basis of professionals’ collective judgment testify to their need for information beyond that to which they have ready access through their shared experience. In such circumstances, the empirical methods featured in the science of science communication aim not to supplant professional judgment but to aid it by generating information that those who possess such judgment would agree will help them to assess the relative plausibility of competing positions on the issues that divide them and thereafter supply the basis for action, the common assessment of which will promote the continued evolution of their common craft sense.
(c) The science curiosity scale was self-consciously designed in response to a disputed conjecture among science communication professionals: the missing audience hypothesis. At least some science documentary filmmakers believe that number of persons who view premier science films, on public television and in other venues, is smaller than it should be. Pointing to audience demographic disparities—ones founded in age, gender, region of the country, income, and even ideology—they surmise that there are correctible features of these offerings, collateral to their science content, that are unintentionally signaling to people with certain cultural identities that these materials are “not for them” and discouraging them from turning to these films to satisfy their interest in knowing what is known by science. Other science-communication professionals disagree: the size of the audience for premier science films, and its composition, they argue, are a simple reflection of how the taste for learning what is known by science is distributed in the general population. Call this the natural audience hypothesis explanation for the constrained appeal of premier science films.
Working with science filmmakers and related science-communication professionals on both sides of this issue, the CCP/APPC/TB science-of-science filmmaking team devised SCS to help resolve the impasse between proponents of these competing positions. The idea was to develop a measure of the disposition to seek out and consume science films and related science media for personal enjoyment. MAH and NAH make opposing predictions about what such a measure will show: NAH implies it will reveal that the taste for enjoyment of high-quality science films just is uneven in the general population, whereas NAH predicts that it will show that there are segments of the population whom high-quality science films are failing to engage notwithstanding those individuals' appetite to seek out and consume such material for personal enjoyment and edification.
3. What. SCS is a standardized assessment instrument. The idea behind it is to measure a latent or unobserved disposition to seek out and consume science-related material for personal enjoyment.
Such measures have a tortured history. Using absurd self-report measures, they invariable generate skewed results with no predictive validity. Many in the decision sciences had simply given up on the possibility of devising a valid science curiosity measure.
Our scale development strategy was geared to overcoming these difficulties. To avoid the “social desirability bias” associated with self-report measures, the scale embedded such items in a larger array of “interest” questions disguised as a social-marketing survey. It also used more reliable and objective behavioral and performance-based measures.
The resulting scale had very satisfactory psychometric qualities, meaning that its constituent items cohered in a manner that suggested they were measuring a real disposition that varied continuously and normally in the general population.
But most importantly of all, we were able to behaviorally validate it. That is, we were able to show that in fact it did very powerfully predict who would engage with science documentary films and who wouldn’t.
4. Who. SCS can be used to assess the MAH/NAH dispute in a couple of different ways. First, one can examine the distribution of SCS across demographic groups of interest. NAH implies we should see disparities—among racial, age, gender, and ideological groups—that reflect observed audience disparities for premier science films. It doesn’t seem that we do; SCS is remarkably uniform across the population.
Second, we can try to use SCS to explain observed disparities in audiences for science films. Here there is at least limited support for the MAH. Individuals who are more hierarchical and individualistic, conservative female, white & older seem to be less engaged with at least certain films that we tested than one might expect. Why might that be the case? That is something that can be tested in additional experimental studies using SCS.
The first concerns the engagement with evolution science films of individuals who don’t believe in evolution. There is only a modest discrepancy in the science curiosity of individuals who do and don’t believe in evolution. Moreover, we found that conditional on the same level of science curiosity, individuals who do and don’t believe in evolution have comparable levels of engagement with evolution-science films. They are not the missing audience!
The second WTF concerns the relationship between science curiosity and political information processing. We all know that Americans are polarized on a variety of science issues; many of us know what these divisions actually get worse as science comprehension increases. But turns out, surprisingly, that these divisions don’t get worse as science curiosity goes up; instead they abate. When we observed this, we decided to do an experiment to investigate. What we found was that unlike most individuals, those high in SCS willingly sought out information that was contrary to their political predispositions. This result plausibly explains why they are less polarized in general and why polarization among them, unlike in others, doesn’t go up as their science comprehension increases.
This result highlights the likely synergies between the use of scientific methods to study science communication across domains. What one learns in one is likely to have unexpected payoffs in others. Actually, for that reason the payoffs should be expected, although what they’ll be will be a surprise.
That sort of surprise is exactly what moves science curious people.