By John Weston Parry, J.D.
[I]t is possible to identify the approximately 400,000 most problematic and 40,000 most dangerous individuals with severe psychiatric disorders . . . and provide [coerced] treatment for the most problematic and dangerous individuals. (E. Fuller Torrey, M.D.)
The difference between past and future . . . [is that the] past has taken place and is knowable . . . , whereas the future is up for grabs and is inscrutable, as if it were out of view. (Stephen Pinker, psychologist and linguist)
[P]redictions offered with great certainty and voluminous justification prove, when evaluated later, to [be] . . . the equivalent of monkeys tossing darts.
(paraphrasing Nate Silver)
In order to justify involuntary commitments, other types of forced detentions, and rights deprivations imposed exclusively on persons with mental disabilities deemed to be dangerous, the legal system has obscured and distorted social science and other empirical evidence. Otherwise a vast majority of these dangerousness determinations would have to be invalidated because they are unable to meet reasonable standards of proof. As law professor Peter Smith has observed, this type of courtroom illusion is accomplished by using legal fictions that are based on “judicial ignorance . . . [and] false factual suppositions in the service of other goals.” A prominent example of such a harmful legal fiction is allowing mental health professionals to make predictions of dangerousness based on the belief that those expert opinions will be adequately vetted in an adversarial setting.
There is overwhelming social science evidence which indicates that even in clinically-controlled environments using the best risk assessment methods currently available, properly trained mental health professionals—who are following sound empirical protocols and upholding the highest degree of ethics in their respective professions—are unable to consistently and reliably make accurate predictions about future dangerousness. Yet, in courtrooms throughout the United States—in both criminal and civil advocacy-oriented proceedings—this dangerousness evidence is accepted with minimal scrutiny, but considerable distortions.
The reason why we employ this legal fiction is Barefoot v. Estelle, a death penalty case in which the U.S. Supreme Court held that even though expert dangerousness evidence was shown to be inaccurate most of the time, it was admissible. A majority of the justices presumed, without any empirical evidence and little or no scholarly support, judges and juries would ensure that such evidence would be weighed properly in making these critical life and death decisions. Unfortunately, the opposite is true: pervasive social biases against people with mental disabilities and inadequacies in the advocacy system itself make these courtroom determinations about dangerousness substantially less reliable. Nevertheless, the Court’s spurious rationale has justified using these flawed dangerousness predictions in other courtroom settings as well.
Today, dangerousness determinations are the centerpiece of many types of proceedings, which include civil and quasi-civil inpatient and outpatient involuntary commitments; post-imprisonment commitments; conditional releases; sentencing determinations including the death penalty; pretrial detention; and decisions whether to try juveniles and other children as adults. The expansion of dangerousness proceedings has created a sprawling matrix of legal standards and procedures in every state and federal jurisdiction. Almost all of the relevant statutes and legal decisions utilize vague and often ambiguous terms to define dangerousness. Few if any of these definitions have been verified empirically.
Another problem is that courts tend to ignore or minimize the importance of a key component of these dangerousness determinations: whether the subject of these proceedings has a recognized mental disorder. Instead, this critical determination has been transformed into a question of whether a qualified mental health professional made the key diagnosis based on the most current version of the Diagnostic and Statistical Manual of Mental Disorders (DSM). If the expert is properly qualified and has relied upon the correct version of the DSM, that diagnosis is given very little scrutiny in our courtrooms when it is used in a dangerousness determination. In a legal system in which experts tend to agree with the position of those who are paying them at a rate that cannot be explained by anything other than confirmation bias, this lack of judicial scrutiny becomes particularly pernicious and irresponsible.
How psychiatric and psychological evidence and testimony are applied in the courtroom also reveals a clear double-standard. If such a diagnosis is going to be used to excuse or reduce criminal culpability, courts tend to deem it inadmissible or scrutinize it very carefully, often trying to manufacture reasons to exclude such evidence. On the other hand, if a diagnosis is being used to support or refute a dangerousness opinion, judicial scrutiny is almost nonexistent. In some circumstances dangerousness is established without any psychiatric or psychological diagnosis based on the subjective impressions of a judge or jury. Sanism and the related bias that people with mental disabilities are inherently dangerous strongly skew those outcomes towards a finding of dangerousness.
Moreover, no matter how these dangerousness determinations are made, when persons with mental disabilities are processed in the criminal justice system, they are likely to be incarcerated and otherwise deprived of their fundamental rights for longer periods of time than other criminal defendants charged with the same offenses. Reduced sentences based on a defendant’s diminished culpability have become increasingly less likely, so that today it should be considered a relatively rare event. People with mental disabilities also are deprived of their freedom and other fundamental constitutional rights in our civil justice system based on flawed predictions of dangerousness and the lack of proper care, treatment, and other essential services in their communities.
How legal fictions and other inequities operate in dangerousness determinations and why these judicial proceedings are unfair, unreliable, and anti-therapeutic is explained in Mental Disability, Violence and Future Dangerousness: The Myths Behind the Presumption of Guilt (Rowman & Littlefield, October 2013), Chapter 4 Predictions of Dangerousness in the Courtroom: Unreliable, Inaccurate and Misleading; and Chapter 5 Accusations Based on the Unknowable: Predictions of Dangerousness in Civil and Criminal Proceedings. Brief copyrighted excerpts without the references are reprinted below with permission of the publisher and author, along with reflections based on the book and other materials. Excerpts are in bold with italics.
Risk Assessment, Predictions of Dangerousness and Their Legal Consequences
[T]here have been significant advances in … risk assessment tools to predict dangerousness …. Unfortunately, being significantly better than in the past… is not nearly equivalent to being reliable, relevant, or accurate enough to reasonably satisfy required legal standards of proof…. A host of factors conspire to make dangerousness proceedings virtually worthless—and often very harmful—as methods for determining which people who have mental disabilities should be denied their fundamental rights because they might pose a risk to society. This is true whether or not these judicial determinations rely on predictions by forensic experts or the subjective impressions of judges and juries without such expert assistance.
Predictions of dangerousness, no matter how they are arrived at, are compromised by a slew of variables and intervening factors that make them particularly unreliable when they are used in judicial determinations. Furthermore, what happens to most people with mental disabilities when—as a result of these inherently flawed proceedings—they are indefinitely incarcerated or otherwise placed in indefinite government custody is unconscionable. Mistreatment and inhumane care are the hallmarks of the special consideration that these vulnerable individuals receive before, during, and after indeterminate periods of government custody and supervision—that may last a lifetime or even result in the death penalty—are ordered.
Making matters worse, the number of these flawed dangerousness proceedings are increasing due to two factors. First, there are many more potential respondents and defendants who are—and will be—diagnosed with mental disorders, conditions, or aberrations, in large part due to the alarming percentage of soldiers returning from combat with these conditions. In addition, our mental health system is being diminished by budget cuts. Second, dangerousness predictions are used in many more criminal, quasi-civil, and civil proceedings than in the past. Dangerousness has become the determining factor in civil and quasi-civil involuntary commitment proceedings; the death penalty and other sentencing dispositions; conditional release and outpatient commitment dispositions; pretrial detentions decisions; and most determinations as to whether a child should be tried as an adult.
Making matters even more challenging, complex, and complicated, dangerousness is defined differently depending on the legal circumstance involved and the jurisdiction. This has created a vast assortment of legal standards and procedures which boggle the mind. Virtually all of these standards and procedures tend to be somewhat vague, tautological, over-inclusive, and ambiguous. Thus, what dangerousness even means or is supposed to mean is almost impossible to nail down in order to test and evaluate whether it is being applied in a reasonable manner consistent with basic notions of relevance, accuracy, reliability, and validity. Further obscuring the relevance, accuracy, reliability, and validity of dangerousness determinations are fundamental problems with how mental disorders, conditions, or aberrations are diagnosed, as well as the advocacy system itself.
Yet, even when professional infighting and politics, adversarial tactics, and courtroom theatrics do not undermine the predictive reliability of the ultimate dangerousness determinations, the fact remains that even in clinical situations such predictions have only proven useful to show that individuals with mental disabilities do not present a high risk of acting violently. In other words, these predictions should only be used to rebut accusations of dangerousness. There is no reasonable predictive validity when dangerousness is utilized to establish a high risk of acting violently, even based on highly structured risk assessment methodologies. Moreover, the reliability and predictive validity, as low as it already is with regard to dangerousness, plummets when highly structured risk assessment methodologies are replaced by the types of subjective clinical or lay judgments that are typically used in dangerousness proceedings. Reliability and validity plummet again when these subjective dangerousness judgments and opinions are compromised by the hired gun syndrome, sanism, and the highly misleading presumption that people with mental disabilities are inherently dangerous.
Even in the best of circumstances, a dangerousness determination’s reliability is undermined by the number of variables…as well as the inadequacies of the information being utilized….[B]ehavioral experts…are basing their assessments…on second-, third-, and fourth-hand reports…some or many of which may be inaccurate, distorted, self-serving, or simply false. Statistically, it is very hard to believe that any dangerousness predictions can reliably meet even a preponderance of the evidence, much less clear and convincing standards of proof.
Also, most of these predictions have at least two critical elements, each of which must be taken into consideration in order to properly evaluate a determination’s predictive validity. First, there is the diagnosis of the existence of a mental disorder or condition; second that diagnosis must be properly linked to a separate risk of dangerousness. Statistically, the probability of each of these factors being accurate must be multiplied in order to establish the overall probability of the prediction being correct. If the diagnosis of a mental disorder is likely to be accurate 60% of the time, for example, and the risk of dangerousness is likely to be accurate 40% of the time, the overall likelihood of the prediction being correct is only about 24%. Unfortunately, even that low percentage appears to be enough for most judges to order an involuntary commitment, at least based on a social science survey of judges who make such decisions.
In many cases, however, there are three or more critical elements to be considered, each of which must be calculated in order to arrive at a statistically reliable result. Not surprisingly, there has never been a significant empirical study which has concluded that mental health professionals are able to consistently make risk assessments that are accurate more than half the time. Thus, the classic 1974 law review article, which analogized predictions of dangerousness to “flipping coins in the courtroom,” was understating the deficiencies, even given today’s modest advances in risk assessments. Furthermore, increasing ethical problems in both the legal and psychiatric professions have made such predictions even less reliable than in the past. The hired gun syndrome is one of the most serious ethical dilemmas that continue to grow worse, but there are others as well.
Diagnoses of Mental Disorders
Although diagnoses of mental disorders typically receive very little scrutiny in dangerousness proceedings, there are many reasons to independently question their reliability and accuracy. In these dangerousness cases, the judicial system treats the Diagnostic and Statistical Manual of Mental Disorders (DSM) as being virtually infallible by accepting such a diagnosis if it is made by a qualified expert. Yet, mental health professionals themselves have many questions about DSM’s overall reliability and its reliability with regard to specific disorders. The old DSM clearly needed to be revised, but the new version (DSM-V) was and continues to be under fire from many quarters in the psychiatric community, as well as from other mental health professionals.
One common criticism is the overly inclusive diagnostic categories that tend to create too many false positives by expanding the types of behaviors that are labeled as disorders. Another criticism is the lack of validation using field testing, particularly for these new categories and changes to the old categories. “[T]here is a `high rate of inconsistency’ … which `often yield false positives…[Also,] social prejudices…negatively affect how …assessments… are made. [N]o group … is more devalued than people with mental disorders, conditions, and aberrations who are deemed to be dangerous… [U]ncritical reliance on DSM-based diagnoses is an unjustifiable…flaw in our legal system. In addition, merely because mental health professionals are qualified does not mean that their diagnoses were rendered in a competent or ethical manner, especially if that expert is hired with the expectation that she or he will reach a particular conclusion.
Predictions of Dangerousness
Even if the diagnosis of a mental disorder is statistically reliable, there are many other reasons why assessments of future dangerousness have been seriously flawed. [A]n imprecise range of possible behaviors… are being assessed… in advocacy-oriented, courtroom proceedings…. Experts are unable to make such predictions with a degree of certainty that should be required in the legal system…no matter what assessment tools they use… The best—or more accurately the least disappointing—results have occurred when clinicians dispense with their subjective judgments and instead rely on structured risk assessment tools. The more highly structured the tool, the less disappointing the results. In making dangerousness assessments, however, there is always going to be unstructured, subjective information that the clinician has gathered…, which is used in the prediction. Also, social scientists and clinicians do not know “which form of structured violence risk assessment has the greatest predictive validity.” In any case, “relatively few practicing professionals employ any form of structured risk assessment,” meaning that most assessments are subjectively-based.
Another statistical anomaly with dangerousness that is nearly impossible to overcome without the use of various legal fictions is what is known as the base-rate problem of predicting statistically rare events …. Actual violence occurs relatively rarely in our society, although images of violence and threats of violence are commonplace. As a result, even if specific individuals are much more likely to be violent than most other people, this does not mean that they are dangerous in the sense of being more likely than not to commit a violent act in the immediate future. If the base-rate for actual violence is 5% in our society, being five times as likely to commit a violent act means that there is only a 25% chance that it will happen. According to the empirical data, however, in America there is no group of people with mental disabilities… that presents even a 10 percent [risk of violence].
Our legal system is supposed to be built on the principle that a person should not be considered dangerous enough to be deprived of their fundamental rights unless the government can provide clear and convincing evidence that the person is currently dangerous. Instead, through various legal fictions, we allow individuals to be deemed dangerous, if the trier-of-fact believes that they are likely to be dangerous where “likely” does not have to be, and rarely is, more likely than not. Moreover, the accuracy of any… risk assessment assumes that the evaluator has all the necessary information…, has enough time to make a proper assessment, has competently considered all the relevant factors, has made no significant errors, and has honestly and objectively reported the results. In an adversarial courtroom environment, each of these assumptions is questionable…. `[T]here is a sharp difference between risk assessments and legal decision-making.”
Heresy Trial in 1530 Using “Modern” Rules of Evidence: An Allegory
Suppose a trial for heresy was held before a judge in a courtroom in 1530 using a version of the “modern” American rules of evidence that are applied in cases involving psychiatric and psychological testimony. It might go something like this.
Prosecutor: “Did you or did you not, Mr. Copernicus, state that the earth revolves around the sun?”
Copernicus: “I did your honor, but I can prove my assertion if you allow me to present my book on that subject entitled De Revolutionibus.”
Prosecutor: “Objection, your honor. All mention of Mr. Copernicus’ book or its contents should be stricken from the record because his theories have not been generally accepted by our astronomy or religious communities.”
Judge: “That is absolutely correct, Mr. Prosecutor. The defendant may not rely on his theory in any manner. Do you have any other witnesses, Mr. Prosecutor?”
Prosecutor: “I do your honor. I would like to call on Hank Ptolemy, an astronomer and the direct descent of Claudius, who will testify that most astronomers in our country believe that the sky is a closed envelop with the earth as its center, which obviously precludes any possibility that Copernicus’ dangerous theory is correct.”
Defense Lawyer: “Objection your honor. As my client can clearly demonstrate based on his book, Ptolemy’s view of the world is outdated, unscientific garbage inspired and enforced by the Catholic Church.”
Prosecutor: Your honor, how long must we endure this nonsense from the defense. My legal argument is ironclad. The law is clear that in order to be admissible scientific evidence must be generally accepted in the scientific community. This means Copernicus’ evidence should be deemed inadmissible because it is novel; and the state’s evidence by definition must be relevant and reliable. Let’s not waste anymore time. I move that we skip to sentencing, so we can send this slime to the gallows. The sun is growing dim and soon it will be dark for the night. We all need to go pray so we can better protect ourselves from these heretics.”
Judge: I agree, Mr. Prosecutor. There can be no defeating your tight logic. The legal proof is unassailable that the defendant is obviously guilty. I am ready to deliver my opinion from the bench.
The religious community generally accepts the proposition that the sun revolves around the earth. Legally, it is heresy to challenge our religion. Mr. Copernicus admits to challenging the religious community based on his highly dangerous theory that the earth revolves around the sun. There can be no reasonable doubt that Mr. Copernicus is guilty of heresy.
Mr. Copernicus, because you have committed heresy involving one of our most fundamental teachings, I sentence you to death and order all copies of your book to be burned.
Legal fictions may seem necessary for our judicial system to function, but inevitably they distort the search for truth. Thus, such fictions should be used judiciously, if at all, and, periodically, they should be reevaluated to determine whether they are doing more harm than good. Nowhere is such scrutiny more needed than with predictions of dangerousness.