"They saw a statutory ambiguity"--judges & motivated reasoning
“Yesterday” I mentioned that I and my collaborators—David Hoffman, Danieli Evans, Judge Eugene Lucci, and Katherine Cheng—had a new paper out, and that I’d say more “today.”
So here is the first of a planned 73 blog posts on the paper, which describes a study we did on whether judges’ decisionmaking is vulnerable to distortion by cultural cognition.
By now, all 14 billion readers of this blog, along with the remaining 18 other persons in the world (the ones whose internet connections were disconnected for failing to pay comcast on a timely basis), know about Hastorf & Cantril’s classic paper “They Saw a Game.”
H&C found that when students from two Ivy League colleges were shown a film of a football game between their two schools, students selectively perceived the referee to be making correct or mistaken penalty calls depending on whether those calls were beneficial or detrimental to their school’s team.
This was the first finding of “motivated reasoning,” which refers to the tendency of individuals to conform their assessments of all manner of information—from brute sense impressions to assessments of logical arguments to evaluations of empirical evidence—to some end or goal independent of factual accuracy.
In H&C, that goal was the experience of solidarity with their school.
“Cultural cognition” is a form of motivated reasoning that manifests itself in individuals conforming their perceptions of risk or other policy relevant facts to propositions that promote the status of, or their standing in, their own cultural group. Cultural cognition, studies suggest, makes a substantial contribution to public conflicts over climate change, gun control, marijuana legalization and other putative societal risks.
One of the things that and I and my colleagues are curious about is whether cultural cognition influences—and in this context, we’d say biases—legal decisionmaking.
One the papers in which we examine this issue is called “They Saw a Protest.”
In that study, we showed the subjects—a group of 200 members of the public drawn from a demographically diverse panel of US adults—a videotape of a political protest. We told them the protestors were suing the police for breaking up their demonstration in violation of the protestors’ free speech rights.
The police, we explained, were justifying their dispersing of the protestors on the ground that the protestors were threatening and intimidating bystanders, and preventing their access to a building. The protestors denied this, asserting that they were peacefully chanting and directing comments at the bystanders in a nonthreatening, lawful fashion.
All the subjects had to do was, in the role of jury, decide whose position—the protestors or the police—the tape supported.
Half the subjects, however, were told that the protesters were anti-abortion activists demonstrating in front of an abortion clinic. The other half were told that the protestors were college students demonstrating at against “Don’t ask, don’t tell” (for those of you too young to remember, the long-time policy that excluded open Lesbians and Gays from the US military) in front of the college recruitment center, where the military was interviewing students interested in signing up for service.
Consistent with the influence of cultural cognition—and reminiscent of the result from the classic “They Saw a Game” study—we found that the study subjects polarized along cultural lines.
Those who had opposing cultural outlooks (“egalitarian communitarian” vs. “hierarchical individualist,” or “egalitarian individualist vs. hierarchical communitarian”) who were assigned to the same experimental condition—who, in other words, thought that they were watching the same kind of protest—disagreed about whether they saw the protestors blocking pedestrians, threatening them with their signs, shouting in their faces, etc. Those who had the same cultural outlooks but who were assigned to different experimental conditions (thought they were watching different kinds of protests) disagreed with each other about what had occurred.
Not a happy outcome for the law. The First Amendment is supposed to protect the right of citizens to express themselves no matter how objectionable anyone finds their message, so long as those citizens don’t cross the line from speech to intimidation. But if decisionmakers are motivated by cultural cognition to attribute intimidating behavior to those whose messages offend the decisionmakers’ values, then this constitutional guarantee won’t be worth much!
But these decisionmakers were ordinary citizens playing the role that ordinary citizens often do—juror—in a legal proceeding.
Would judges be subject to the same sort of bias?
That’s the question our new study was designed to help answer.
The paper is entitled “ ‘Ideology’ or ‘Situation Sense’? An Experimental Investigation of Motivated Reasoning and Professional Judgment,” a more appropriate might have been “They Saw a Statutory Ambiguity.”
In the study, we had the subjects—a group of 800 members of the public and 700 law students, lawyers and judges (represented in roughly equal proportions) play the role of judge in two cases.
Both cases featured statutory ambiguities: in one (“Littering”) whether leaving reusable plastic water bottles in the desert constituted “depositing debris” in a protected wildlife preserve; and in the other (“Disclosure”) whether a police officer who knew he was releasing confidential investigatory information but didn’t realize doing so was illegal “knowingly violating” a law against making such disclosures.
Again, we had experimental manipulations.
In Littering, the alleged violator was either a group of construction workers building a border, who left the bottles in the desert to drink water from as the build a fence to keep undocumented immigrants from illegally crossing the Mexico-Texas border; or an immigrant-aid group who put the bottles in the desert in anticipation that the water would be consumed by the illegal immigrants.
In Disclosure, the party receiving the leaked information was either a family-planning center counseling individuals on where to obtain abortions or a religious family-planning center counseling individuals on alternatives to the same.
Those experimental manipulations were ones irrelevant to proper interpretation of the statutes at issue (or so we believed). But they were ones that could still provoke the sort of bias associated with cultural cognition, which could be expected to bias decisionmakers’ interpretations toward finding violations where such outcomes were consistent with rather than contrary to their cultural values.
We found exactly that pattern of biased decisionmaking in members of the public.
But not in the judges: the judges converged on outcomes (no violation in Littering; violation in Disclosure) regardless of which version of the problem they considered and regardless of their cultural values.
In other words, the judges got a cultural-cognition clean bill of health. They decided the cases free of the sort of influences that, the study showed, could be expected to generate group-favoring biases in ordinary citizens.
The judges resisted that bias, moreover, even though they too displayed signs of biased information processing on culturally contested risks.
In another part of the study, the judges divided along exactly the same cultural lines as members of the public on the risks of climate change, marijuana legalization, and a host of other issues.
The resistance the judges displayed to cultural cognition, then, was very specific to their legal reasoning.
Lawyers, by the way, were pretty much identical to judges in their responses to both the legal reasoning and risk-perception portions of the study.
Law students were in between the members of the public (who again, were biased in both types of responses), on the one hand, and the lawyers and judges, on the other.
So that’s what we found.
In future posts, I’ll say more about the theoretical and practical motivations for the study, the methods we used to analyze the results, and the implications of the study for assessment of the performance not only of judges but of professionals generally, including climate scientists and others who specialize in assessing societal risks.
[Ah shit! Because I was concentrating on writing on this I didn’t notice that they were boarding my connecting flight from Detroit to Boulder where I’m supposed to a lecture on this study. . . . Whoops! I’ll pay more attention when the next flight is getting ready to go.]
Reader Comments (6)
I propose the non-binary theory of cultural cognition, whereby the conventional spectrum of egalitarian vs individualist is broken into distinct pieces, their denizens are labeled tribes, and given names. We'll call them the red tribe and the blue tribe. Their shared cultural biases are predictive of their response to survey questions.
But now that we're not limited to a binary spectrum of political beliefs, and can start pigeonholing people into tribal labels (really, it's so fun, why weren't we doing it before?) we have the freedom to take all the lawyers and judges and put them into the category "law tribe." The law tribe's shared biases are also predictive of their response to survey questions.
Would you like to learn more?
http://graphics8.nytimes.com/packages/pdf/national/jacobs.pdf
THE JOHN F. SONNETT MEMORIAL LECTURE
THE SECRET LIFE OF JUDGES
Dennis Jacobs (Chief Judge, United States Court of Appeals for the Second Circuit.)
Quoting only briefly:
This lecture is about bias, the judge’s inbred preference for outcomes controlled by proceduralism, the adversary system, hearings and experts, representation by lawyers, ramified complexity of doctrines and rules, multiple prongs, and all things that need and use lawyers, enrich them, and empower them vis-à-vis other sources of power and wisdom.
Looking forward to more posts.
I was wondering about the experimental conditions. I'm "motivated" to find a reason why they create a special case. (I became convinced of the bias of judges when SCOTUS switched sides on states' rights in Bush v. Gore).
Haven't found a fatal flaw yet to explain why your experimental condition won't generalize, but I'm working on it.
@Ryan
I'm not sure I udnerstand the first point, but I'm interpreting it to mean you are curious about why we use the cutlural worldview framework rather than simple right left. It's a matter that I've addressed quite often; here's a good place to start. There is also discussion in the paper that reprots this study.
On 2d point, I am taking it to be something that is in fact quite important but absent from the paper: that the value of convergence in the judgments of judges & lawyers depends on what it is they are converging on. That omission has to be remedied; I would have nothing but contempt for myself & others who teach law students if we made the mistake of confusing the reliability of lawyers' professional judgment for the validity of it.
@Joshua--
This is a bit like "you? talking to me?": if there is reason for thinking that judges have the capacity and disposition to make decisons w/o the distortion of ideologically motivated rasoning generally, that doesn't prove that any particular judge or group of them decided any particular case in an unbiased way....
Dan. Sure. Good point. Maybe I won't have to completely reevaluate my impression of Bush v. Gore after all! Whew. :-)
"I would have nothing but contempt for myself & others who teach law students if we made the mistake of confusing the reliability of lawyers' professional judgment for the validity of it."
Good man:)