The real problem with the scientific limits of analytical technique is not imperfection. Reputable analysts and proficient investigators acknowledge their limits. It is why mature professionals look for corroboration and prefer aggregating evidence from multiple sources instead of depending exclusively on a single, smoking-gun bit of proof. The latter is much too elusive in the real world, no matter how recurrent and dramatic its appearance in fictional crime drama.
No, the real problem is a system that inclines expert analysts to exaggerate certainty under oath, lending an air of infallibility to what should remain open to question in the absence of supporting evidence.
It does not help that this flaw may result from the best of motives. As Spencer Hsu’s Washington Post article points out (available at
http://www.washingtonpost.com/local/crime/convicted-defendants-left-uninformed-of-forensic-flaws-found-by-justice-dept/2012/04/16/gIQAWTcgMT_story.html), microscopic hair analysis from an FBI laboratory was chronically, scientifically suspect yet decisive in leading to convictions of individuals subsequently exonerated by better DNA evidence. These cases involving hair analysis preceded today’s DNA matching techniques.
Hsu’s article strikes enough balance to arm FBI bashers and supporters alike. On the one hand, it cites deficient lab protocols and limited scientific reliability of microscopic hair analysis to conclusively put a given suspect at a crime scene. On the other hand, Hsu includes context and a reasoned explanation from an FBI source who averred that the Bureau was doing the best it could with the tools available at the time.
A larger question, also posed in the article, reflects more negatively on the legal profession than on law enforcement. Specifically, it appears that the FBI came forward with unflattering discoveries about flawed evidence, delivering this information to prosecuting attorneys who had won associated convictions. Some prosecutors acted on this information to initiate reviews of tainted convictions, or at least to advise defense attorneys involved in those cases of this recent turn of events. Others, however, kept the embarrassing information on closer hold, with the inevitable result that some people who appeared to be wrongfully convicted continued to serve prison time past a point where they could have been set free.
Lab protocols and questionable science will no doubt merit painstaking scrutiny in this aftermath. The more systemic folly in the eye of a security practitioner, however, is what appears to be an almost irresistible tendency for experts to magnify the infallibility of their expertise. Consider this example:
• An FBI expert testifying on a hair match that ultimately proved erroneous claimed that his hair matching had been unsuccessful only 8-10 times in thousands of cases that he had worked on over the course of 10 years.
• Another FBI scientist whose case ultimately proved flawed told jurors that he routinely relied on 15 characteristics in matching hair samples to an individual when, in reality, his lab notes revealed he had only measured 3 characteristics of the hair in this particular case.
The problem is two-fold. First, under a full head of steam, the expert exaggerates the validity of his or her expertise by offering dogmatic, convincing opinions in the guise of fact. Second, exaggerated claims of expert infallibility meet with insufficient challenge. Here it is the defense attorney and judge who must share responsibility for resulting injustice in convictions.
Under the circumstances, here follows a prescription for corrective actions on the part of the various principals.
EXPERT: Stick to the facts, and render them into plain language without argot or embroidery to suggest that your analytical tools yield infallible proof.
INVESTIGATOR: Resist the twin forces of confirmation bias and indolence. Corroborate. Investigate fully. Do your entire job, instead of relying solely on the expert to do it for you.
PROSECUTOR: Honor the ethics of your office and profession by not concealing exculpatory evidence. Do not manipulate juries with arcane but scientifically questionable data and testimony that should be open to fair scrutiny rather than presented as incontestable fact.
DEFENSE ATTORNEY: Do your homework in probing validity of evidence and in uncovering fallacies of one-sided pronouncements of experts. Do not let experts get away with inflated claims of success or validity. Probe behind the percentages and success rates. Ask for evidence backing such claims.
JUDGE: Keep the playing field level, compelling prosecution, defense, and expert witnesses to communicate in plain language. Do not let them slide into impenetrable jargon calculated to overwhelm juries and suggest infallibility that does not exist. Keep the burden for making evidence on the advocates, without allowing either side to offer up tomes of incomprehensible data that no mortal should have to decipher.
JURY: Beware the CSI effect, of the increasingly popular trend to make a case exclusively on impressive-looking or impressive-sounding techniques that are not explained to your satisfaction.
FOR ALL: Whether the analytical tool is hair, DNA evidence, or any other forensic advance, remember that a tool has its value and also its limitations. It should be a part of a complete investigation, not a substitute for a full body of evidence. Beware of any claim that over relies on a single tool.
-- Nick Catrantzos
Subscribe Now
AddThis
Nick Catrantzos + Tom Goff
- All Secure
- NICK CATRANTZOS is the lead author of the All Secure website and is the emeritus security director for a large public organization. He has also managed security aspects of corporate takeovers, response to international hostage situations, and finding and keeping secrets. He has been board certified in security management, licensed as a private investigator, and certified as an incident commander. He worked as a case officer for a discreet branch of public service. He protected stealth technology at Lockheed and, later, as a consulting security director for both Kroll and Control Risks, he assisted executives with workplace violence cases and with protecting VIPs and critical assets globally in volatile areas. Nick's partner, TOM GOFF, was a captain in the Military Police Corps, U.S. Army Reserve and has a law degree from the University of California at Davis. He is a former reporter for Fortune Magazine, was senior editor at New York Magazine and Esquire, and has provided crisis and communications counsel to senior executives at Fortune 500 companies, including Lockheed, ARCO, Microsoft, Nissan, and Sony Pictures.