The real problem with the scientific limits of analytical technique is not imperfection. Reputable analysts and proficient investigators acknowledge their limits. It is why mature professionals look for corroboration and prefer aggregating evidence from multiple sources instead of depending exclusively on a single, smoking-gun bit of proof. The latter is much too elusive in the real world, no matter how recurrent and dramatic its appearance in fictional crime drama.
No, the real problem is a system that inclines expert analysts to exaggerate certainty under oath, lending an air of infallibility to what should remain open to question in the absence of supporting evidence.
It does not help that this flaw may result from the best of motives. As Spencer Hsu’s Washington Post article points out (available at
http://www.washingtonpost.com/local/crime/convicted-defendants-left-uninformed-of-forensic-flaws-found-by-justice-dept/2012/04/16/gIQAWTcgMT_story.html), microscopic hair analysis from an FBI laboratory was chronically, scientifically suspect yet decisive in leading to convictions of individuals subsequently exonerated by better DNA evidence. These cases involving hair analysis preceded today’s DNA matching techniques.
Hsu’s article strikes enough balance to arm FBI bashers and supporters alike. On the one hand, it cites deficient lab protocols and limited scientific reliability of microscopic hair analysis to conclusively put a given suspect at a crime scene. On the other hand, Hsu includes context and a reasoned explanation from an FBI source who averred that the Bureau was doing the best it could with the tools available at the time.
A larger question, also posed in the article, reflects more negatively on the legal profession than on law enforcement. Specifically, it appears that the FBI came forward with unflattering discoveries about flawed evidence, delivering this information to prosecuting attorneys who had won associated convictions. Some prosecutors acted on this information to initiate reviews of tainted convictions, or at least to advise defense attorneys involved in those cases of this recent turn of events. Others, however, kept the embarrassing information on closer hold, with the inevitable result that some people who appeared to be wrongfully convicted continued to serve prison time past a point where they could have been set free.
Lab protocols and questionable science will no doubt merit painstaking scrutiny in this aftermath. The more systemic folly in the eye of a security practitioner, however, is what appears to be an almost irresistible tendency for experts to magnify the infallibility of their expertise. Consider this example:
• An FBI expert testifying on a hair match that ultimately proved erroneous claimed that his hair matching had been unsuccessful only 8-10 times in thousands of cases that he had worked on over the course of 10 years.
• Another FBI scientist whose case ultimately proved flawed told jurors that he routinely relied on 15 characteristics in matching hair samples to an individual when, in reality, his lab notes revealed he had only measured 3 characteristics of the hair in this particular case.
The problem is two-fold. First, under a full head of steam, the expert exaggerates the validity of his or her expertise by offering dogmatic, convincing opinions in the guise of fact. Second, exaggerated claims of expert infallibility meet with insufficient challenge. Here it is the defense attorney and judge who must share responsibility for resulting injustice in convictions.
Under the circumstances, here follows a prescription for corrective actions on the part of the various principals.
EXPERT: Stick to the facts, and render them into plain language without argot or embroidery to suggest that your analytical tools yield infallible proof.
INVESTIGATOR: Resist the twin forces of confirmation bias and indolence. Corroborate. Investigate fully. Do your entire job, instead of relying solely on the expert to do it for you.
PROSECUTOR: Honor the ethics of your office and profession by not concealing exculpatory evidence. Do not manipulate juries with arcane but scientifically questionable data and testimony that should be open to fair scrutiny rather than presented as incontestable fact.
DEFENSE ATTORNEY: Do your homework in probing validity of evidence and in uncovering fallacies of one-sided pronouncements of experts. Do not let experts get away with inflated claims of success or validity. Probe behind the percentages and success rates. Ask for evidence backing such claims.
JUDGE: Keep the playing field level, compelling prosecution, defense, and expert witnesses to communicate in plain language. Do not let them slide into impenetrable jargon calculated to overwhelm juries and suggest infallibility that does not exist. Keep the burden for making evidence on the advocates, without allowing either side to offer up tomes of incomprehensible data that no mortal should have to decipher.
JURY: Beware the CSI effect, of the increasingly popular trend to make a case exclusively on impressive-looking or impressive-sounding techniques that are not explained to your satisfaction.
FOR ALL: Whether the analytical tool is hair, DNA evidence, or any other forensic advance, remember that a tool has its value and also its limitations. It should be a part of a complete investigation, not a substitute for a full body of evidence. Beware of any claim that over relies on a single tool.
-- Nick Catrantzos
Wednesday, April 18, 2012
Wednesday, April 11, 2012
Even Crooks Link Timeliness to Intelligence
This is a vivid example of how to apply intelligence usefully, but only if reacting in time. The application is criminal. It is predatory. It is even carried out by villains unlikely to have had the benefit of federal grants or seminars promoting fusion centers and hire of intelligence analysts. Even so, at least the crooks exploiting this intelligence recognize that, unlike brandy, intelligence cheapens with age. Consider the story and the unstated contrast with how bureaucracy would approach the same situation.
What is Happening: Crooks Exploit a Novel Indicator
Burglars who focus on loot with minimal risk know that striking an occupied residence is a bad idea. Risks of apprehension and confrontation skyrocket. Then, if the burglar breaks in armed, the event can easily escalate into a violent crime that spells stiffer penalties and a greater chance of someone getting hurt. Unsurprisingly, proficient burglars prefer to strike when the home or business is unoccupied. To improve their odds, though, Seattle burglars have struck on the tactic of breaking into cars parked at movie theaters, breaking into those cars to grab vehicle registrations, and then burglarizing the homes at addresses reflected on the vehicle registrations. According to the related press report (available at
abcnews.go.com/US/stolen-car-registrations-lead-thieves-empty-homes-owners/story?id=16108396 ), these burglars calculate that they have a good two-hour window to strike unoccupied homes before the victims return from a night out. This tactic gives burglars the advantage of striking during the hours of darkness, when it is easier to remain undetected, while also targeting an unoccupied residence.
What If Tables Turned?
What if a government bureaucracy were contending with trying to take in and act on intelligence like this within two hours? That's right. The time elapsed from obtaining the intelligence that a homeowner is away from home and occupied elsewhere, then acting on that intelligence to go to the unoccupied home and clean it out -- all this has to take place within two hours. Would the bureaucratic organization be able to act so quickly? Let's see. First, there would have to be a special squad with training, equipment, and overtime to set up surveillance on movie theater parking lots. Next, there would have to be special funding to underwrite acquisition of license plate cameras and software, along with connectivity to a special database and a related crime analyst to process that data in order to harvest those residential addresses. Then there would need to be a separate, mobile team specialized in clandestine entry. Naturally, to coordinate the efforts of the surveillance and entry teams, there would need to be a management element, operating out of a specially designed command center. That center would need electronic pin maps to display vehicles and residences, as well as video feeds and wall-sized monitors to show street views of relevant information in real time. Soon, the squad balloons into a platoon, and the platoon into a regiment. With all those people participating in the effort, conditions call for setting up a task force which, owing to the specialization required for optimum performance, ends up becoming one of those temporary activities that turns permanent -- at least as long as funding is available. Unfortunately, though, the abundance of resources and specialists and managerial overseers now makes it impossible to act on any intelligence in only two hours. Consequently, the bureaucracy now needs to deploy a specially trained stall team to engage the targeted movie-goers by staging an accident or contriving some kind of distraction that will delay their return to the unoccupied residence.
Results-Focused Contrast
By contrast, the crooks can do it all with lower staffing, or even just one person. More realistically, the practical skeleton crew would probably involve no fewer than two people: one behind the wheel to serve at lookout and getaway driver, and the other to smash into cars and grab registrations. Then the two drive to the target residence or residences and speed through the burglary. Both probably operate without a budget for exotic electronics and tie-ins to command centers to assist with target selection. Instead, they concentrate on hitting expensive-looking cars that were driven to the movie-theater by people wearing expensive clothes. The more they strike, the more they refine their target-selection protocols. These energetic Davids make up in alacrity and boldness what their more cumbersome Goliaths in bureaucracy only approximate through big budgets, over specialization, and lack of imagination.
Is this comparison exaggerated? Perhaps. Far-fetched? Not necessarily. Sometimes a tighter focus on results trumps the bureaucracy's inherent tendencies to magnify, complicate, and embellish.
- Nick Catrantzos
What is Happening: Crooks Exploit a Novel Indicator
Burglars who focus on loot with minimal risk know that striking an occupied residence is a bad idea. Risks of apprehension and confrontation skyrocket. Then, if the burglar breaks in armed, the event can easily escalate into a violent crime that spells stiffer penalties and a greater chance of someone getting hurt. Unsurprisingly, proficient burglars prefer to strike when the home or business is unoccupied. To improve their odds, though, Seattle burglars have struck on the tactic of breaking into cars parked at movie theaters, breaking into those cars to grab vehicle registrations, and then burglarizing the homes at addresses reflected on the vehicle registrations. According to the related press report (available at
abcnews.go.com/US/stolen-car-registrations-lead-thieves-empty-homes-owners/story?id=16108396 ), these burglars calculate that they have a good two-hour window to strike unoccupied homes before the victims return from a night out. This tactic gives burglars the advantage of striking during the hours of darkness, when it is easier to remain undetected, while also targeting an unoccupied residence.
What If Tables Turned?
What if a government bureaucracy were contending with trying to take in and act on intelligence like this within two hours? That's right. The time elapsed from obtaining the intelligence that a homeowner is away from home and occupied elsewhere, then acting on that intelligence to go to the unoccupied home and clean it out -- all this has to take place within two hours. Would the bureaucratic organization be able to act so quickly? Let's see. First, there would have to be a special squad with training, equipment, and overtime to set up surveillance on movie theater parking lots. Next, there would have to be special funding to underwrite acquisition of license plate cameras and software, along with connectivity to a special database and a related crime analyst to process that data in order to harvest those residential addresses. Then there would need to be a separate, mobile team specialized in clandestine entry. Naturally, to coordinate the efforts of the surveillance and entry teams, there would need to be a management element, operating out of a specially designed command center. That center would need electronic pin maps to display vehicles and residences, as well as video feeds and wall-sized monitors to show street views of relevant information in real time. Soon, the squad balloons into a platoon, and the platoon into a regiment. With all those people participating in the effort, conditions call for setting up a task force which, owing to the specialization required for optimum performance, ends up becoming one of those temporary activities that turns permanent -- at least as long as funding is available. Unfortunately, though, the abundance of resources and specialists and managerial overseers now makes it impossible to act on any intelligence in only two hours. Consequently, the bureaucracy now needs to deploy a specially trained stall team to engage the targeted movie-goers by staging an accident or contriving some kind of distraction that will delay their return to the unoccupied residence.
Results-Focused Contrast
By contrast, the crooks can do it all with lower staffing, or even just one person. More realistically, the practical skeleton crew would probably involve no fewer than two people: one behind the wheel to serve at lookout and getaway driver, and the other to smash into cars and grab registrations. Then the two drive to the target residence or residences and speed through the burglary. Both probably operate without a budget for exotic electronics and tie-ins to command centers to assist with target selection. Instead, they concentrate on hitting expensive-looking cars that were driven to the movie-theater by people wearing expensive clothes. The more they strike, the more they refine their target-selection protocols. These energetic Davids make up in alacrity and boldness what their more cumbersome Goliaths in bureaucracy only approximate through big budgets, over specialization, and lack of imagination.
Is this comparison exaggerated? Perhaps. Far-fetched? Not necessarily. Sometimes a tighter focus on results trumps the bureaucracy's inherent tendencies to magnify, complicate, and embellish.
- Nick Catrantzos
Subscribe to:
Posts (Atom)
