THE FUTURE IS NOT NOW
By John Weston Parry, J.D.
Profiling and profilers have become a familiar part of the American landscape, certainly as reflected on television dramas and in movies. Those who make these mostly unwarranted predictions based on the unknowable future are held up us as heroes, intellectual savants, and super humans, rather than carnival barkers. Despite the widely condemned profiling abuses against African-Americans by police, prosecutors, and school systems, this adoration with profilers continues. It is as if the many reported abuses have nothing to do with profiling itself, or it does not matter.
With regard to African-Americans specifically, this national profiling craze has focused on the significant, but unmeasured, percentage of police who make stops, make arrests, inflict harms, and take potentially deadly actions against certain types of Americans based either on profiles that tend to be racist, or racism disguised as profiles. When this kind of racial and ethnic profiling occurs, the legal system is mostly divided between police, prosecutors, and judges who seem to understand the racism or ethnic-discrimination involved, and those officials who feel everything must be done to protect the police, regardless of the consequences to other people. Security comes first no matter what. Furthermore, there appear to be a not insignificant number of law enforcement officers and legal professionals, who are intentionally racist or prejudiced against certain racial and ethnic groups, but use the camouflage of promoting security to justify their prejudices.
Yet, the profound problems with profiling are far wider, deeper, and even more serious than racism or ethnic discrimination alone. As bad as it is for African-Americans or Muslims—and it is bad—the overall impact is far worse for people with mental disabilities. That is because the profiling of mental disorders has been enshrouded in the pseudo-science and mysticism of psychiatric professionals, who predict dangerousness and other harmful behaviors based on their professed expertise. Their flawed predictions are embraced by most of the legal system and, as a result, may be used as a license to control many aspects of the lives of those who are profiled in this manner. It also is based on a false societal belief that people with mental disabilities are likely to be dangerous, or at least substantially more dangerous than everyone else. This stereotype tends to stigmatize everyone who has a severe mental impairment, even more so then being African-American, and no less than being a practicing Muslim.
Making matters worse, increasingly profiles are being used against the interests of almost all Americans in many different ways with few limits or controls. Profiling-based assumptions are made about people not only at airports and throughout the criminal justice system, but whenever they purchase something electronically, buy insurance, apply for financing, or use an online dating or match-making service. Profiling is generally based on so-called “educated” guesses that only are required to meet the arbitrary standards of those who make the guesses or use the results for their own ends. The interests of profilers diverge, sometimes in extreme ways, from the interests of those being profiled. Too often legal expediency trumps justice and controls the legal agenda when profiles are used in courtrooms and to make other legally-based decisions.
At its core, profiling can be divided into two groups, with a third group being formed as a combination of the first two. The first group is made up of profilers who say they base their predictions on their beliefs and impressions guided by their past experiences and training. The more expertise these profilers have in their fields, the more they are likely to believe that they can make expert predictions. In law enforcement, profilers who have trained with the FBI are presumed to be among the best in either identifying potential criminals or those who are most likely to have committed a specific crime that is being investigated. In public health or the criminal justice system, psychiatrists are viewed as being the best in predicting dangerousness and other behavioral harms to self or others. Both profilers and these forensic psychiatrists base their expertise on their abilities to predict human behaviors.
With rare exceptions, profilers do not have to prove their abilities empirically. When magicians are put to the test, for example, they inevitably fail to demonstrate any special extrasensory abilities beyond trickery. The only profilers that have been put to the test as a group, rather than as individuals, are the cream of the crop: psychiatrists and other mental health professionals, who generally have the most training and expertise in human behaviors.
In the 1960's and 1970's, many empirical tests were carried out to measure those presumed predictive abilities. Perhaps the most convincing were carried out by psychologist and social scientist, Professor John Monahan, who for many years has been affiliated with the University of Virginia School of Law. His seminal work was done for the federal government. He convincingly demonstrated, by gathering all the available evidence from all the empirical studies on the subject and his own data, that most predictions of dangerousness by psychiatrists and other mental health professionals were wrong and that psychiatrists and related professionals had no greater predictive abilities than educated lay people. As Bruce Ennis, J.D. and Thomas Litwack, Ph.D., described in their now famous law review article on the subject in 1974, predicting dangerousness in the courtroom was no better than flipping a coin. As it turned out, the empirical results were considerably worse than a coin flip.
Yet, that did not deter much of the legal system, especially the U.S. Supreme Court, from creating legal fictions that would allow the discredited information into evidence in courtrooms around the country. The problem was that because of a number of public interest lawyers and mental health professionals, including Ennis and members of the American Bar Association's Commission on the Mentally Disabled, proof of dangerousness became necessary to involuntarily civilly commit people with mental disorders and to execute most capital offenders, including those with mental disabilities. Unless such evidence continued to be deemed admissible, the use of involuntary civil commitments, executions, and various other social controls would be hampered, substantially.
As a matter of judicial expediency, the law found a way to use such evidence in the courtroom anyway. The threshold concern was establishing that dangerousness was a matter for a jury or judge to decide. That could be accomplished under the Federal Rules of Evidence and many state systems that mimicked the federal rules, if predictions of dangerousness were generally accepted within the psychiatric profession as being an appropriate diagnostic tool. For years these predictions had had been accepted without serious scrutiny. This did not mean that every prediction of dangerousness by a psychiatrist (or other qualified mental health expert) was to be deemed persuasive, but rather that it was a matter for the jury or judge as trier of fact to decide based on their own subjective impressions.
The overwhelming problem and injustice for people with mental disabilities was—and continues to be—that they are devalued as human beings due to stigma and sanism. In particular, people continued to believe that as a group, people with mental disabilities were unreasonably dangerous. Thus, when confronted with a respondent or defendant with a mental disability, the trier of fact was likely to presume that the subject was dangerous. When choosing between two mental health “hired guns” presenting diametrically opposed diagnostic predictions of dangerousness or lack of dangerousness, the triers of fact typically chose the version that was consistent with their own presumption, which is how it tends to work today as well.
Subjective Impressions of “Experts”
By the early 1980's even psychiatrists became convinced that they had no special expertise when it came to making predictions of dangerous. It appeared to lawyers representing people with mental disabilities that the tide had changed direction; that finally profiling based on predictions of dangerousness would be banished from the courtroom. Unfortunately, this would not be the case. In the landmark decision of Barefoot v. Estelle (1983), a majority of the U.S. Supreme Court ruled that even though the American Psychiatric Association had demonstrated in its amicus brief that predictions of dangerousness in death penalty cases were only correct about one-third of the time, that evidence would not be deemed inadmissible as long as it was delivered by a qualified mental health expert. It would be up to the jury to decide what weight to give such testimony.
With that nonsensical ruling, predictions of dangerousness continued to be used to confine people with mental disabilities in civil and quasi-civil detention facilities, to execute them, and to extend their time behind bars and in government custody and supervision. Having a mental disability would no longer mitigate culpability in most criminal cases, but it would substantially increase the possibility of a longer sentence or the death penalty and facilitate various forms of involuntary quasi-civil and civil inpatient and outpatient commitments.
The same year Barefoot v. Estelle was decided, famed psychiatrist, Alan Stone, who would later become President of the American Psychiatric Association, and well-respected lawyer, Cliff Stromberg, wrote their ground-breaking, but largely unsuccessful, Harvard Law Review article, proposing that state legislatures expand involuntary civil commitment criteria to avoid the need to prove dangerousness. Instead of doing away with the more narrow view of dangerousness, the concept was inflated to include a variety of other potential harms to self or others. Thus, profiling based on predictions of dangerousness were greatly expanded and psychiatrists and other mental health professionals became invested in finding ways to get people with mental disabilities more coerced treatment. In order to accomplish this, the obstacle of proving future dangerousness, which the APA had agreed led to more misses than hits, had to be overcome.
Actuarial Risk Assessments by “Experts”
It was around then that the second type of profiling began to emerge and gain legitimacy: risk assessments based on actuarial methodologies. No longer would psychiatrists and other mental health professionals and behaviorists be limited by their impressionistic and inherently biased opinions, which had proven to be so deficient. Risk assessments would be based on clinical expertise informed by objective measurement tools. If the legal system was expedient and gullible enough to embrace impressionistic opinions about dangerousness, it would gladly accept the promises the pseudo-science of risk assessment would bring. Moreover, there was a fairly large and menacing community of potential offenders with mental disorders, who American society wanted to put behind bars before they committed any crimes, or after they had served their time in prison and before they could commit any new crimes: sex offenders and sexual “deviates.” Thus, the risk assessment industry would soon blossom, especially in identifying people who are at high risk of being sex offenders. What could be more politically and governmentally acceptable than protecting Americans, particularly innocent girls and women from being preyed upon by depraved men and boys with mental and sexual disorders. Who could possibly object—other than those presumed commie sympathizers at the American Civil Liberties Union?
The Inherent and Political Limitations of Profiling
What makes profiling so insidious and the purveyor of so many injustices is that it claims beneficence and to be based on science and the scientific method. Unfortunately, to a greater or lesser extent, the false promises of social utility and scientific legitimacy are accepted within the law enforcement and legal communities, which rely on profiling to make their jobs easier and more expedient. For the most part, profilers are simply disingenuous and somewhat delusional. Unfortunately, many of them are liars for hire, assuming that lying means not telling the truth. They personally profit from the mostly misleading and false information that they sell to their employers, or whoever can pay the highest or most lucrative retainers.
If the primary objectives of government is to have the highest percentage of successful arrests and prosecutions or to get as many people who appear to be dangerous or antisocial behind bars or in other types of detention centers, then the purpose of profiling becomes clear: to justify and support those objectives, whether the information or conclusions that are conveyed in the process happen to be true or not. It is no coincidence that our criminal justice system is overflowing with African-Americans, other people of color, and people with mental disabilities. The use of profiles—either explicitly or implicitly—is the major culprit. Fitting a profile should not be evidence, but somehow it has become an acceptable substitute.
The underlying injustice is that so often the profiling that we rely on is statistically unreliable and/or inaccurate. Little effort is made to create standards that coincide with what the law should require in order to be fair. Leaving it up to the police, prosecutors, or triers of fact becomes a license to manufacture injustice. Profiles also tend to camouflage stereotypes, biases, and prejudices, especially among law enforcement, legal professionals, and even educators, who rely on the results to do their jobs more easily. Typically, profiles are based on logical fallacies, which the profilers and users of the profiles tend to ignore or dismiss. Profiling conjures up predictive beliefs, impressions, bigotry, and statistical mirages that are given the imprimatur of social, governmental, and legal legitimacy. Ignorance becomes a legal fiction in which justice is outweighed by the desire to promote convenience, expediency, and political and religious beliefs.
Profiles, as they are currently used, have major deficiencies, which are unlikely to be resolved for years to come. The deficiency that they all share is that they have never been proven to be effective in predicting individual human behaviors, if being correct more often than being incorrect is the threshold standard. More specifically, it has never been demonstrated satisfactorily from a social science point of view that criminal or psychiatric profiling are reasonably accurate, especially when the standards of proof to meet are legal. The only exceptions are if legal fictions are employed to allow such flawed evidence to be used anyway without demonstrating empirical validity.
As noted earlier, there are two types of profiling, and a third which combines the first two. The type of profiles that have come closest to meeting threshold social science standards for accuracy, reliability, and validity, which, along with relevance, are the basic empirical and statistical measures for determining whether a profile methodology is reasonably effective in predicting criminal or otherwise harmful behaviors. The social science data leaves no doubt that law enforcement or psychiatric professionals no matter how much experience and training they may have lack the minimal ability required to predict human behaviors based on their professed expertise alone. When they try to make such predictions they are guided instead by logical fallacies, biases, prejudices, and even bigotry, which make it seem to them that they have such abilities. Much of the time, they know that they have no such abilities, but are able to convince others that they do. Impressionistic profiling when used by itself, or even in combination with risk assessment tools, typically produces a misleading or dishonest profile.
Highly Structured Actuarial Risk Assessments: Insurance, Professional Sports, and Elections
The best type of human behavior profiles with respect to being better able to meet basic social science standards for accuracy, reliability, and validity are highly structured risk assessment actuarial methodologies that try to identify statistically significant human characteristics, which increase the likelihood that the profile is correct and should be used to make objective predictions and related conclusions. Three areas in which risk or similar actuarial assessments appear to produce meaningful results are insurance policies, evaluations of athletes, and elections. Yet, these assessments are inaccurate much of the time, which is why insurance companies lose money, many supposedly talented athletes fail, and Hillary Clinton did not defeat Barack Obama.
There also are fundamental differences between these types of highly structured risk assessments and criminal and psychiatric profiling. Both insurance policies and elections are based on group profiles. It is immeasurably more difficult to predict how a particular human being is going to behave, than how a group with predefined characteristics will tend to behave. Risk assessment is pretty good at measuring group behaviors, but to date has demonstrated very little ability to accurately predict criminal or other dangerous behaviors of individuals.
With respect to athletes, as Nate Silver has pointed out, the statistical data-bases in our major professional sports is far better than in any other business or human endeavor, except perhaps for weather forecasting. Criminal and psychiatric data is nowhere near as complete because that information is so expensive and difficult to collect. The number of athletes playing in our major professional sports is quite manageable. It numbers less than 10,000. It is possible to collect rich data on each of these individuals in terms of how they perform as professionals. Thus, predicting how they will perform in the future and thus how much they may be worth to a team is relatively straightforward as compared to the complexities of predicting how any particular human being will behave criminally or psychiatrically.
The best data criminal and psychiatric profilers have at their disposals is terribly flawed as compared to this type of sports data. It comes from a variety of sources, very little of which is collected for the specific purpose of making criminal and psychiatric profiles. That data is being adapted for a new purpose and thus is inappropriate, to a greater or lesser extent, depending on where it comes from and how it is used. Just think for a moment how flawed our national gun and sexual offender registries are in producing accurate information. The same can be said of the criminal and psychiatric profiling data generally.
The other colossal problem is the base rate distortion of predicting relatively rare events. With insurance, sports, and elections the risk assessors (profilers) are able to limit the number of variables that they need to account for because the results they are interested in depend on a limited number of human variables. In addition, success or failure can be accounted for in group terms, so the likelihood of being wrong a significant percentage of the time is manageable. Insurers try to build in the failures as a cost of doing business, which everyone who buys insurance tends to share, although not always equally. With professional sports the fans, who pay to watch a team play, pay for the failures in evaluating athletic talent. With elections a simple mea culpa often will do or a revision of the survey research model utilizing better data, as Nate Silver has done so successfully in recent national elections.
When it comes to using criminal and psychiatric profiles against individuals, especially those with mental disabilities, there is no shared pain if mistakes are made and various freedoms and rights are taken away. Moreover, these types of profiles tend to be far less accurate and reliable than those used for these aforementioned insurance, sports, and election purposes.
Statistically, the likelihood that an individual in our society will commit a violent crime or otherwise do serious harm to themselves or others is very small, even though the rate is significantly higher than in most—and perhaps all—other industrialized nations. What highly structured risk assessment methodologies attempt to do is isolate various human characteristics or group behaviors that increase the likelihood that someone who has that characteristic, or is part of that group, will commit criminal or other harmful behaviors. Having a mental disability of any sort will rarely if ever increase the risk to such an extent that it can be said clearly and convincingly, much less with proof beyond a reasonable doubt, that the individual is likely to commit a violent crime or otherwise cause serious harm. It cannot even be said with any great reliability that such an individual is more likely than not to act dangerously. What can be said is that for a small percentage of people with certain mental disabilities their disorders make it become more likely that they will act dangerously, assuming all other significant intervening variables can be controlled. This is a big ask.
If one in a hundred individuals will act dangerously in the immediate future, then, even if a person is deemed to be five times as likely to act dangerously due to his or her mental disorder, the statistical probability that the individual will do so is still one in twenty. When trying to predict who will commit a mass homicide, the odds are much more daunting than that. When a profiler concludes that it is likely that a person will commit a crime or otherwise act dangerously, what they really mean is that the risk appears to be substantially greater than for the average American. In almost all cases, that so-called likelihood cannot be demonstrated to meet the minimal threshold of being more likely than not to occur. The dangerousness of those individuals has to be taken on faith based on the profilers professed expertise, which is the nub of the fundamental injustice. In almost every circumstance, there is no meaningful empirical evidence that can support a profiler’s conclusion(s) with any reasonable degree of reliability.
Given the head-scratching gun debate in this country, if one were to take the focus away from people with mental disabilities, African-Americans, or Muslims and begin to focus on individuals who possess certain types of guns, especially if they are illegal, or gangs where members have committed violent crimes, especially with guns, undoubtedly the statistical profiles would be far more enlightening in preventing crimes. Ironically, it also is quite possible—and probably likely—that certain categories of police officers would score highly on dangerousness using risk assessment inventories. Whether any of these risk assessment statistics would be able to meet the empirical thresholds of being right more often than not, remains an unknown because it has never been done. The point is that by only focusing on people with mental disabilities, African-Americans, and Muslims as being potentially dangerous, we not only discriminate against them invidiously with awful consequences, but we ignore groups who are far more likely to be dangerous.
Generally profiling is little better than fortunetelling in terms of its predictive capabilities, but it is far worse with respect to its consequences for the individuals who are the subjects of such profiles. The most respected type of profiling and the one which has the most expertise and science-sounding jargon supporting are the highly structured actuarial forms of risk assessment. If applied judiciously, that type of risk assessment has the potential to be enlightening. Unfortunately, that potential remains far in the future. Moreover, the other types of criminal and psychiatric profiling are even further behind with respect to being reasonably equated with knowledge.
For all types of profiling the rule of thumb should be that unless it can be empirically demonstrated that a specific method in precisely defined circumstances will be correct more often than it is incorrect, it should not be used. If specific legal rights of an individual are involved, the threshold should rise to meet the legal standard that is being applied to that right, which normally would be clear and convincing evidence, or proof beyond a reasonable doubt. Currently, no methods of profiling, including risk assessments by trained mental health professionals, can come close to reliably meeting that threshold.
As I wrote in Mental Disability, Violence, Future Dangerousness: Myths Behind the Presumption of Guilt (Rowman & Littlefield, October 2013) at p.132:
[T]hese predictions should be based on (1) highly structured actuarial methodology;
(2) sound statistical evidence that this particular methodology can validly and reliably satisfy
the specific legal standard of proof that is being employed…; and (3) assurances that the
assessment based on that methodology was done properly…. Furthermore, even if these
basic social science thresholds could be met…there are other extremely serious practical
and systemic deficiencies…that need to be addressed and overcome for any future
dangerousness determinations to be fundamentally fair and just.