follow CCP

Recent blog entries
popular papers

What Is the "Science of Science Communication"?

Climate-Science Communication and the Measurement Problem

Ideology, Motivated Cognition, and Cognitive Reflection: An Experimental Study

'Ideology' or 'Situation Sense'? An Experimental Investigation of Motivated Reasoning and Professional Judgment

A Risky Science Communication Environment for Vaccines

Motivated Numeracy and Enlightened Self-Government

Ideology, Motivated Cognition, and Cognitive Reflection: An Experimental Study

Making Climate Science Communication Evidence-based—All the Way Down 

Neutral Principles, Motivated Cognition, and Some Problems for Constitutional Law 

Cultural Cognition of Scientific Consensus
 

The Tragedy of the Risk-Perception Commons: Science Literacy and Climate Change

"They Saw a Protest": Cognitive Illiberalism and the Speech-Conduct Distinction 

Geoengineering and the Science Communication Environment: a Cross-Cultural Experiment

Fixing the Communications Failure

Why We Are Poles Apart on Climate Change

The Cognitively Illiberal State 

Who Fears the HPV Vaccine, Who Doesn't, and Why? An Experimental Study

Cultural Cognition of the Risks and Benefits of Nanotechnology

Whose Eyes Are You Going to Believe? An Empirical Examination of Scott v. Harris

Cultural Cognition and Public Policy

Culture, Cognition, and Consent: Who Perceives What, and Why, in "Acquaintance Rape" Cases

Culture and Identity-Protective Cognition: Explaining the White Male Effect

Fear of Democracy: A Cultural Evaluation of Sunstein on Risk

Cultural Cognition as a Conception of the Cultural Theory of Risk

« Not very reflective tea-party/Republicans | Main | The value of civic science literacy »
Sunday
Dec152013

What is a "cultural style"? And some thoughts about convergent validity

what do you mean by cultural styles? As a qualitative researcher, that caught my eye! Thanks.

A commenter recently posed this question  in connection with a post from a while back. I thought the question was interesting enough, and the likelihood that others would see it or my response sufficiently remote, that I should give my answer in a new post, which I hope might prompt reflection from others.

My response:

That's a great question!

It goes to what it is that I think is being measured by scales like ours. I've addressed this to some extent before-- e.g., here & here & here & ...

But basically, we can see that on disputed risk issues, positions are not distributed randomly but instead correlated with reocognizable but not directly observable ("latent") group affinities that are themselves associated loosely with a package of individual characteristics and attitudes.

People who share particular group affiniteis, moreover, form clusters of positions across these issues ("earth not heating up" & "concealed  carry laws reduce crime"; "the death penalty doesn't deter murder" & "nuclear wastes can't be stored safety in deep geologic isolation") that can't possibly reflect links in the causal mechanisms involved and instead seem to reflect the identity-expressing equivalence of them.

The point of coming up w/ scales is to sharpen our perception of what these group affinities are & why those who share them see things the way they do -- to explain what's going on, in other words -- & also to enhance our power to predict and form prescriptions.

The term "cultural style" is, for me, a way to describe these affinities. I have adapted it from Gusfield. I & collaborators use the concept and say more about it and how it relates to Gusfield in various places.

“Unlike groups such as religious and ethnic communities[,] they have no church, no political unit, and no associational units which explicitly defend their interests,” but are nevertheless affiliated, in their own self-understandings and in the views of others, by largely convergent worldviews and by common commitments to salient political agendas. 

" 'They "posssess subcultures' " (id.) that
furnish coherent norms for granting and withholding esteem. "Examples of these are cultural generations, such as the traditional and the modern; characterological types, such as 'inner-directed and other-directed'; and reference orientations, such as 'cosmopolitans and locals.'" Many of the most charged social and political issues of the past century can be understood as conflicts between individuals who identify with competing cultural styles and who see their status as bound up with the currency of those styles in society at large.

Dan M. Kahan, The Secret Ambition of Deterrence, 113 Harv. L. Rev. 413, 442 (1999) (quoting Gusfield, who is himself quoting David Reisman, Karl Mannheim & C. Wright Mills-- yow! right after the quoted section, Gusfield discusses as an example Hofstadter's famous "Mugwump style").

Joseph Gusfield -- he rocks!BTW, I regard Gusfield as one of the most brilliant social theorists of our time. It is sad that he is not even more famous. But I suppose lucky, too, for me b/c it means I am able to play a more meaningful role in scholarly discussions by virtue of others not having the advantage of the perspective & insight that comes from reading Gusfield!

I like "cultural style" b/c it helps to reinforce that the orientation in question is relatively loose-- we are talking about a style here; not the sort of fine grained, highly particular set of practices & norms that, say, an anthropologist or sociologist might have in mind as "culture"  -- and also general -- a "style" doesn't reduce in some analytic sense to a set of necessary & sufficient conditions; it is a prototype.

You say you are a qualitative researcher. I take it then that you regard me as a "quantitative" one.  Fair enough.

But in fact, I see myself as just a researcher-- or simply a scholar. I want to understand things, and also to add to scholarly conversation by others who are interested in the same things as a way to reciprocate what I have learned from them.

To do that -- to learn; to add -- I figure out the method most suited to investigating questions of interest to me and invest the effort necessary to be able to use that method properly. Then I just get to it.

Any scholar who thinks that the methods he or she has learned should forever determine the questions he or she should answer rather than vice versa will, at best, soon become boring and, at worst, ultimately become absurd.

Actually, all valid methods, I'm convinced, are empirical in nature, since I don't believe one can actually know anything without being able to make observations that enable valid inferences to be drawn that furnish more reason to credit one account of a phenomenon than another (pending more of the same sorts of evidence, etc.).

I have found the sort of empirical methods that figure in the cultural cognition work very useful for this. And those methods, moreover, have evolved and been refined in various ways to try to meet challenges that we face in seeking to learn/add in the professional student way.

But in fact, I believe that the sorts of ethnographic, historical and related methods that figure in anthropological and sociological accounts and the fact-rich social theorizing that Gusfield has done to be very valid as well.

Indeed, there are few if any hypotheses that we have tested with the sorts of quantitative methods that figure in our cultural cognition work that aren't rooted in insights reflected in these more "qualitative" works.

Gusfield's account of the styles that contended over the issue of temperance--which he identifies as the same ones in conflict over various other issues, including many involving criminal deviancy lawsdrunk driving lawsanti-smoking laws, and other forms of risk regulation--is a source of inspiration for many of our conjectures, as I've indicated.

So is the work of Kristin Luker, whose understanding of the competing egaltiarian & hierarchic styles that impel conflict among women over abortion figured in our study of the white male effect and later in a study that I did of cultural contestation over rape law.

But there are many many other works of this sort that motivate & discipline our studies.

The disciplining consists in the fit between our study results and these accounts.  That correspondence helps to make the case that we really are measuring what we say we are measuring-- or modeling what we say we are modeling.

At the same time, our results give more reason to believe that the qualitative accounts are valid.

For any "qualitative style" (as it were) of empirical investigation, the issue of whether the researcher's own expectations shaped his or her observations rather than vice versa always looms menacingly overhead like a raised sword.

That we are able to build a simple empirical model that displays the characteristics--produces the results-- one would expect if the qualitative researcher's explanation of what's going on is true helps to shield the researcher from this sort of doubt.  I hope qualitiative researchers find value in that!

I am, of course, talking about the idea of convergent validity.

Every empirical method has limits that are in part compensated for by others.  When different approaches all generate the same result, there is more reason to believe not only that that what they are finding is true but that each of the individual approaches used to establish that finding were up to the job.

It's possible that a bunch of imperfect methods (the limitations of which are independent of one another) just all happened to generate the same result. But the more likely explanation is that they converged because they were in fact all managaing to get a decent-sized piece of the truth.

Would you like a more "Bayesian" analogy of how convergent validity validates?

You find something that looks a puzzle piece but aren't sure whether it is.  I find something that looks like a nearly complete puzzle--but also am unsure.  If we meet and discover that the former happens to fit into and seemingly complete the latter, you will have more reason for believing that the putative "puzzle piece" is in fact a puzzle piece. At the same time, I will have more reason for believing that my putative "incomplete puzzle" is truly an incomplete puzzle.  That's because the probability that a thing that isn't a puzzle piece would just happen to fit into a thing that isn't an incomplete puzzle is lower than the probability that the two things truly are "a puzzle piece" and "an incomplete puzzle" respectively.

To me convergent validity is the "gold standard." Or better the remedy for the sort of "gold standard" mentality that manifests itself in a chauvinistic insistence that there is only one genuinely valid one or even a single "best" for empirical investigation of social phenomena.

... Well, I am curious how this strikes you.

Useful? Eclectic? Confused?!

 

PrintView Printer Friendly Version

EmailEmail Article to Friend

Reader Comments (7)

I think everybody has a built in gold standard. I think its an especially good idea if the two people agree that gold standards get in the way of objectivity and that the gold standards of the two people mentioned above are in opposition. If they are the same, the possibility of lack of objectivity is higher.

I also believe in symmetry as a way to objectivity. For example, if a "heirarchical-individualist white male" effect is discovered (e.g. http://www.culturalcognition.net/display/ShowImage?imageUrl=/storage/Clipboard04.bmp?__SQUARESPACE_CACHEVERSION=1383749063473 ) then an entirely analogous but mirror image study of, say, an egalitarian-communitarian white male vs non-egalitarian white male, white female, minority male and minority female should be made, if only to illustrate that the mirror image effect doesn't exist. Is that a valid WSMD?-JA! ?

December 15, 2013 | Unregistered CommenterFrankL

@FrankL--

I agree entirely.

But is this --the symmetry, the ubiquity of the dynamics of which the "wmhi effect" is just one instance -not something that is stressed in the very account that my collaborators & I, at least, have given for the effect?

And I myself over & over& over & over -- in particular in response to those who mistake it for some sort of pathology distinctive of "conservative" thought -- in this blog?!

For a WSMD? JA! there'd need to be a more specific conjecture about the sort of observation that would support the inference in question. I'm all ears!

Maybe you'd like to take a look at this entry & identify what sort of test of the data would, if it disclosed evidence in support of "asymmetry," you'd assign a likelihood ratio > 1 in favor of the "asymmetry thesis"?

December 16, 2013 | Registered CommenterDan Kahan

@Dan - Just looking at the plot at http://www.culturalcognition.net/display/ShowImage?imageUrl=/storage/Clipboard04.bmp?__SQUARESPACE_CACHEVERSION=1383749063473 , I was wondering what density distributions for would look like for the following categories:

white communitarian egalitarian male
non-egalitarian white male
white female
minority male
minority female

It would be a mirror-image plot, so to speak, to see if there is a "white egalitarian-communitarian male" effect. Looking at the plot itself, I would not expect to see one, but looking at the very first plot at http://www.culturalcognition.net/blog/2013/11/5/a-snapshot-of-the-white-male-effect-ie-white-male-hierarch-i.html , I am struck by the mirror symmetry between the upper left and lower right quadrants.

December 18, 2013 | Unregistered CommenterFrankL

@FrankL:
I'll work on it.

December 19, 2013 | Unregistered Commenterdmk38

Kumbaya
I am apparently one of many Tea Partiers who respects your honesty/integrity. But let me add a characteristic I haven’t yet noticed in the posts: fearless from peer pressure.

I came in on the Tea Party scientific comprehension wave, and the surprise that made the Fox News segment worth investigating was not the content but the source: an honest academic leftie that sees the obvious and actually reports it.

Until discovering CCP, my main input from academia was reading the alumnae magazines from Penn and Johns Hopkins. Reports of study after study are biased toward leftist groupthink. Often I wholly endorse the methodology and regard the findings as supportive of my conservative views, only to encounter a disclaimer, “nothing in these results should be taken as….”

It reminds me of the Chinese Cultural Revolution’s forced “confessions” from the intelligentsia. Note that I said I find the reports biased. The studies are informative. The “tell” of bias is that I have only once encountered the disclaimer when the results were liberal-friendly. In that instance there were two disclaimers, the first cautioning the left, and the final cautioning the right.

I perceive organized religion, the left and academia as highly conformist. The first two I am tolerant of (partisan raison d’être). But I have a serious problem with stifled academic freedom and diversity (non-partisan raison d’être), even if the censorship is self-censorship and is willingly and cheerfully accepted.

So I am gleeful to see a forum (CCP) that finds it unnecessary to put a thumb on the scale. Did you happen to notice media outlets other than Fox reporting your Tea Party science comprehension correlation results? Yale?

Useful, eclectic, confused?
Useful: absolutely, to us both. For me, coming in at the middle of the movie, I needed to see the beginning to understand the dialogue. For you, this put in high relief the probity I thought I saw, discussed under Kumbaya. Credibility, already in evidence, soared.

Eclectic: not sure I understand the question. To the extent yes, it’s a good thing. To the extent no, no demerits warranted. I prefer multi-threaded analysis to single-, but you can’t get to multi- without singletons. Scope was appropriate.

Confused: me, no, you, I think so. I’m not confused, because your exposition was clear. Prima facie evidence of your confusion: the weakness of the correlations you are finding.

Don’t mistake that for disapproval—your confusion is appropriate for the tasks you are trying to accomplish: the population definitions are sloppy, the standards are suspect, and the influence of cultural “contaminators” is impossible to deny even while they are resistant to identification, much less quantification.

The expectation that profound insights relating to populations defined as imprecisely as the partisans do will be forthcoming from this swamp is overreach. To be anything but confused would be arrogant, an attribute of which I see no evidence.

Math is not science, it is precise language. Subject, verb, modifier, object. Variable, operation, factor, value. Past, present, future. T0, T1, expected.

Anything observable is theoretically quantifiable. The necessary adjustment is to determine the units of perception, e.g. change “red” light, to light in the wavelength range of 620 to 780 nanometers.

The virtue of expressing something mathematically is that the statement can be manipulated, if no rules are violated, without changing the meaning, to reveal relationships that were difficult to observe by inspection.

Precision is not always the way to go. Lack of precision is often more appropriate, and this is the place of natural language. This is as true for scientific communication as it is for any other. I would suggest that attempting to substitute precise language (math) in situations where any of the elements of mathematical presentation are better represented imprecisely is misleading and the conclusions are error prone.

Some grist for the mill?
1. Atheists are sure about something they cannot know. Agnostics are unsure about something they cannot know. Arrogance is unwarranted certainty. Humility is uncertainty and can be warranted or unwarranted.
2. Science is an organized religion. Practitioners believe as a matter of faith, largely ignorant of the observations, derivations, and decisions that resulted in the current state of the art. Both have canons and priests. When their faith is troubled practicioners refer to canons and priests for guidance and the priests quote canons to them as authoritative. When the priests find the canons inadequate, they expand the canonical law. A community is developed through conferences and rituals.
3. There is no inherent conflict between belief in science and belief in a higher power. A micro view versus a macro view. We have long seen a difference between “laws” governing particle movements seen through microscopes and telescopes. Our response is to try to reconcile these laws by developing a unified theory of the universe, not to reject physics. We can date the “big bang” to within a fraction of a second—no one disputes the universe was created. Darwin addresses the origin of the species, not the origin of life.
4. Belief in a higher power is humble (man is not the highest entity in the universe). Disbelief in a higher power is arrogant (nothing in my world was created by anyone much superior to me).

You said you don’t know anyone in the Tea Party. Pauline Kael didn’t know anyone that didn’t vote for McGovern. I don’t know anyone that rejects science.

December 22, 2013 | Unregistered CommenterTerry

@Terry:

Thanks for encouraging words & reflective comments.

This is actually the 2d 15-mins of fame for the "Tea party science comprehension" post. Like last time, when traffic originated in mischaracterizations of the blog post (it's content; "not its funding source as in this case!), the initial torrent of hate mail & calls has given way to lots of appreciative, engaged & engaging responses (many by email) from TP members who actually read the post & saw that the spirit of it was very different from what they had been led to believe (it happens all around -- those we rely on for information try to make us hate the "other side"; in doing that, they make the other side hate us).

In the end, the confirmation that people w/ values different from mine get excited to exchange ideas on intellectual interests that we have in common offsets the irritating occasion for the new contacts. But it also shows me that we -- people w/ diverse values, & serious disagreements about serious matters of consequence for our political life -- still have a common interest to in improving the quality of our deliberative environment so that we are not constantly made to form false impressions of the others' character & intentions.

Now, as you say, this is "mid conversation" for you -- but where else to start? Ever?!

You observe the small correlations. I assume you have in mind the ones identified in the tea party comprehension post? To me, those are interesting precisely b/c they are small -- meaningless, even -- b/c they are evidence that *alternatives* to the basic framework that I think makes the most sense (the most sense out of why there is conflict over decision-relevant science) are not correct.

That framework -- the cultural cognition theory -- posits that individuals will, when they are in an environment in which positions on contested risk issues become symbols of identity in & loyalty to competing affinity groups -- *use* their reason to conform the evidence they see to the positions that prevail in their group. This is "individually" rational; it connects them to the their group w/o "costing" them anything since their own views on the issues in question ("is earth heating up?" "do concealed carry laws increase crime?") etc. don't have any effect on policy. But it is a collective disaster since it prevents diverse citizens from converging on evidence important to their well being (even when that evidence doesn't uniquely determine policy, as is almost inevitably the case-- that's where values *do* come in).

The evidence in support of that theory is pretty strong, I think. In any case, the "correlations" from whcih inferences are drawn about its correctness are pretty healthy in size!

For a sampling, and as a way to get deeper into the conversation (you are already deep enough to offer any insight that occurs to you, btw), I recommend:

Kahan, D.M., et al. The polarizing impact of science literacy and numeracy on perceived climate change risks. Nature Climate Change 2, 732-735 (2012).

Kahan, D.M., Peters, E., Dawson, E. & Slovic, P. Motivated Numeracy and Englightened Self Government. Cultural Cognition Project Working Paper No. 116 (2013).

Kahan, D. Fixing the Communications Failure. Nature 463, 296-297 (2010).

Yes, I don't know anyone who rejects science either -- & haven't met any, and don't expect too, as my acquaintance with citizens of the Liberal Republic of Science who have used their reason & their freedom to form a common project called the "tea party" enlarges.

December 23, 2013 | Registered CommenterDan Kahan

Fixing the Communications Failure--I started with this one.

A subset of communications is advocacy—the communicator wants the recipient to change. Advocacy is essentially a sales pitch, and the test of the communications’ effectiveness is making the sale. The conclusion that employing of proven marketing principles helps make a sale should be neither surprising nor controversial.

Another subset of communications is dissemination—the communicator solely wants to inform the recipient. Evidence of the effectiveness of the communication is whether the recipients understand the information. Changing the recipients’ minds is not part of the test. To gather and disseminate information is to teach.

“As straightforward as these recommendations might seem, however, science communicators routinely flout them.” Why? Scientists do no conceive of themselves as pitchmen, but rather as educators. Their primary goal is to inform, whereas the advocates’ primary goal is to persuade.

Advocates might persuade some people simply by informing them (the educators’ approach). But the advocate may also be willing to persuade people by disseminating selective information, misinformation or by making an appeal to cultural or other biases, i.e., confusing the facts.

Salesmen and lawyers (advocates) are poorly perceived but scientists and teachers are respected. "Pushers” engender enmity that providers do not. Changing the modus operandi of educators to advocacy may cause a lack of respect. If scientists are perceived as advocates, the credibility of scientists may be diminished.

“Cultural cognition also causes…groups with opposing values often become more polarized, not less, when exposed to scientifically sound information.” I submit that perception of the source of the communication as a salesman or a scientist would be a least as strong a factor in determining the receptivity of the audience as whether the communicator is wearing a suit or denim shirt. [Testable assertion]

Thomas Sowell has estimated that stereotypes have about a .25 correlation with facts. I am reluctant to lead you to any interpretation of that statistic since you are more conversant with interpreting correlations than I. But I offer my interpretation: Accepting stereotypes will yield a statistically significant edge in being right over ignoring the stereotype (random acceptance), but accepting the stereotype as factual (acceptance only) will cause one to be wrong more often than one is right.

When considering a person’s facility with logic as a measure of how scientifically they think, consider the “logic” in the preceding statement. If you can reconcile those two positions with logic, I will be educated. More importantly, if you are more familiar with the work of Chris Mooney than the work of Thomas Sowell, you have a major mind-expanding treat in store.

December 28, 2013 | Unregistered CommenterTerry

PostPost a New Comment

Enter your information below to add a new comment.

My response is on my own website »
Author Email (optional):
Author URL (optional):
Post:
 
Some HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>