The real problem with the scientific limits of analytical technique is not imperfection. Reputable analysts and proficient investigators acknowledge their limits. It is why mature professionals look for corroboration and prefer aggregating evidence from multiple sources instead of depending exclusively on a single, smoking-gun bit of proof. The latter is much too elusive in the real world, no matter how recurrent and dramatic its appearance in fictional crime drama.
No, the real problem is a system that inclines expert analysts to exaggerate certainty under oath, lending an air of infallibility to what should remain open to question in the absence of supporting evidence.
It does not help that this flaw may result from the best of motives. As Spencer Hsu’s Washington Post article points out (available at
http://www.washingtonpost.com/local/crime/convicted-defendants-left-uninformed-of-forensic-flaws-found-by-justice-dept/2012/04/16/gIQAWTcgMT_story.html), microscopic hair analysis from an FBI laboratory was chronically, scientifically suspect yet decisive in leading to convictions of individuals subsequently exonerated by better DNA evidence. These cases involving hair analysis preceded today’s DNA matching techniques.
Hsu’s article strikes enough balance to arm FBI bashers and supporters alike. On the one hand, it cites deficient lab protocols and limited scientific reliability of microscopic hair analysis to conclusively put a given suspect at a crime scene. On the other hand, Hsu includes context and a reasoned explanation from an FBI source who averred that the Bureau was doing the best it could with the tools available at the time.
A larger question, also posed in the article, reflects more negatively on the legal profession than on law enforcement. Specifically, it appears that the FBI came forward with unflattering discoveries about flawed evidence, delivering this information to prosecuting attorneys who had won associated convictions. Some prosecutors acted on this information to initiate reviews of tainted convictions, or at least to advise defense attorneys involved in those cases of this recent turn of events. Others, however, kept the embarrassing information on closer hold, with the inevitable result that some people who appeared to be wrongfully convicted continued to serve prison time past a point where they could have been set free.
Lab protocols and questionable science will no doubt merit painstaking scrutiny in this aftermath. The more systemic folly in the eye of a security practitioner, however, is what appears to be an almost irresistible tendency for experts to magnify the infallibility of their expertise. Consider this example:
• An FBI expert testifying on a hair match that ultimately proved erroneous claimed that his hair matching had been unsuccessful only 8-10 times in thousands of cases that he had worked on over the course of 10 years.
• Another FBI scientist whose case ultimately proved flawed told jurors that he routinely relied on 15 characteristics in matching hair samples to an individual when, in reality, his lab notes revealed he had only measured 3 characteristics of the hair in this particular case.
The problem is two-fold. First, under a full head of steam, the expert exaggerates the validity of his or her expertise by offering dogmatic, convincing opinions in the guise of fact. Second, exaggerated claims of expert infallibility meet with insufficient challenge. Here it is the defense attorney and judge who must share responsibility for resulting injustice in convictions.
Under the circumstances, here follows a prescription for corrective actions on the part of the various principals.
EXPERT: Stick to the facts, and render them into plain language without argot or embroidery to suggest that your analytical tools yield infallible proof.
INVESTIGATOR: Resist the twin forces of confirmation bias and indolence. Corroborate. Investigate fully. Do your entire job, instead of relying solely on the expert to do it for you.
PROSECUTOR: Honor the ethics of your office and profession by not concealing exculpatory evidence. Do not manipulate juries with arcane but scientifically questionable data and testimony that should be open to fair scrutiny rather than presented as incontestable fact.
DEFENSE ATTORNEY: Do your homework in probing validity of evidence and in uncovering fallacies of one-sided pronouncements of experts. Do not let experts get away with inflated claims of success or validity. Probe behind the percentages and success rates. Ask for evidence backing such claims.
JUDGE: Keep the playing field level, compelling prosecution, defense, and expert witnesses to communicate in plain language. Do not let them slide into impenetrable jargon calculated to overwhelm juries and suggest infallibility that does not exist. Keep the burden for making evidence on the advocates, without allowing either side to offer up tomes of incomprehensible data that no mortal should have to decipher.
JURY: Beware the CSI effect, of the increasingly popular trend to make a case exclusively on impressive-looking or impressive-sounding techniques that are not explained to your satisfaction.
FOR ALL: Whether the analytical tool is hair, DNA evidence, or any other forensic advance, remember that a tool has its value and also its limitations. It should be a part of a complete investigation, not a substitute for a full body of evidence. Beware of any claim that over relies on a single tool.
-- Nick Catrantzos
Wednesday, April 18, 2012
Wednesday, April 11, 2012
Even Crooks Link Timeliness to Intelligence
This is a vivid example of how to apply intelligence usefully, but only if reacting in time. The application is criminal. It is predatory. It is even carried out by villains unlikely to have had the benefit of federal grants or seminars promoting fusion centers and hire of intelligence analysts. Even so, at least the crooks exploiting this intelligence recognize that, unlike brandy, intelligence cheapens with age. Consider the story and the unstated contrast with how bureaucracy would approach the same situation.
What is Happening: Crooks Exploit a Novel Indicator
Burglars who focus on loot with minimal risk know that striking an occupied residence is a bad idea. Risks of apprehension and confrontation skyrocket. Then, if the burglar breaks in armed, the event can easily escalate into a violent crime that spells stiffer penalties and a greater chance of someone getting hurt. Unsurprisingly, proficient burglars prefer to strike when the home or business is unoccupied. To improve their odds, though, Seattle burglars have struck on the tactic of breaking into cars parked at movie theaters, breaking into those cars to grab vehicle registrations, and then burglarizing the homes at addresses reflected on the vehicle registrations. According to the related press report (available at
abcnews.go.com/US/stolen-car-registrations-lead-thieves-empty-homes-owners/story?id=16108396 ), these burglars calculate that they have a good two-hour window to strike unoccupied homes before the victims return from a night out. This tactic gives burglars the advantage of striking during the hours of darkness, when it is easier to remain undetected, while also targeting an unoccupied residence.
What If Tables Turned?
What if a government bureaucracy were contending with trying to take in and act on intelligence like this within two hours? That's right. The time elapsed from obtaining the intelligence that a homeowner is away from home and occupied elsewhere, then acting on that intelligence to go to the unoccupied home and clean it out -- all this has to take place within two hours. Would the bureaucratic organization be able to act so quickly? Let's see. First, there would have to be a special squad with training, equipment, and overtime to set up surveillance on movie theater parking lots. Next, there would have to be special funding to underwrite acquisition of license plate cameras and software, along with connectivity to a special database and a related crime analyst to process that data in order to harvest those residential addresses. Then there would need to be a separate, mobile team specialized in clandestine entry. Naturally, to coordinate the efforts of the surveillance and entry teams, there would need to be a management element, operating out of a specially designed command center. That center would need electronic pin maps to display vehicles and residences, as well as video feeds and wall-sized monitors to show street views of relevant information in real time. Soon, the squad balloons into a platoon, and the platoon into a regiment. With all those people participating in the effort, conditions call for setting up a task force which, owing to the specialization required for optimum performance, ends up becoming one of those temporary activities that turns permanent -- at least as long as funding is available. Unfortunately, though, the abundance of resources and specialists and managerial overseers now makes it impossible to act on any intelligence in only two hours. Consequently, the bureaucracy now needs to deploy a specially trained stall team to engage the targeted movie-goers by staging an accident or contriving some kind of distraction that will delay their return to the unoccupied residence.
Results-Focused Contrast
By contrast, the crooks can do it all with lower staffing, or even just one person. More realistically, the practical skeleton crew would probably involve no fewer than two people: one behind the wheel to serve at lookout and getaway driver, and the other to smash into cars and grab registrations. Then the two drive to the target residence or residences and speed through the burglary. Both probably operate without a budget for exotic electronics and tie-ins to command centers to assist with target selection. Instead, they concentrate on hitting expensive-looking cars that were driven to the movie-theater by people wearing expensive clothes. The more they strike, the more they refine their target-selection protocols. These energetic Davids make up in alacrity and boldness what their more cumbersome Goliaths in bureaucracy only approximate through big budgets, over specialization, and lack of imagination.
Is this comparison exaggerated? Perhaps. Far-fetched? Not necessarily. Sometimes a tighter focus on results trumps the bureaucracy's inherent tendencies to magnify, complicate, and embellish.
- Nick Catrantzos
What is Happening: Crooks Exploit a Novel Indicator
Burglars who focus on loot with minimal risk know that striking an occupied residence is a bad idea. Risks of apprehension and confrontation skyrocket. Then, if the burglar breaks in armed, the event can easily escalate into a violent crime that spells stiffer penalties and a greater chance of someone getting hurt. Unsurprisingly, proficient burglars prefer to strike when the home or business is unoccupied. To improve their odds, though, Seattle burglars have struck on the tactic of breaking into cars parked at movie theaters, breaking into those cars to grab vehicle registrations, and then burglarizing the homes at addresses reflected on the vehicle registrations. According to the related press report (available at
abcnews.go.com/US/stolen-car-registrations-lead-thieves-empty-homes-owners/story?id=16108396 ), these burglars calculate that they have a good two-hour window to strike unoccupied homes before the victims return from a night out. This tactic gives burglars the advantage of striking during the hours of darkness, when it is easier to remain undetected, while also targeting an unoccupied residence.
What If Tables Turned?
What if a government bureaucracy were contending with trying to take in and act on intelligence like this within two hours? That's right. The time elapsed from obtaining the intelligence that a homeowner is away from home and occupied elsewhere, then acting on that intelligence to go to the unoccupied home and clean it out -- all this has to take place within two hours. Would the bureaucratic organization be able to act so quickly? Let's see. First, there would have to be a special squad with training, equipment, and overtime to set up surveillance on movie theater parking lots. Next, there would have to be special funding to underwrite acquisition of license plate cameras and software, along with connectivity to a special database and a related crime analyst to process that data in order to harvest those residential addresses. Then there would need to be a separate, mobile team specialized in clandestine entry. Naturally, to coordinate the efforts of the surveillance and entry teams, there would need to be a management element, operating out of a specially designed command center. That center would need electronic pin maps to display vehicles and residences, as well as video feeds and wall-sized monitors to show street views of relevant information in real time. Soon, the squad balloons into a platoon, and the platoon into a regiment. With all those people participating in the effort, conditions call for setting up a task force which, owing to the specialization required for optimum performance, ends up becoming one of those temporary activities that turns permanent -- at least as long as funding is available. Unfortunately, though, the abundance of resources and specialists and managerial overseers now makes it impossible to act on any intelligence in only two hours. Consequently, the bureaucracy now needs to deploy a specially trained stall team to engage the targeted movie-goers by staging an accident or contriving some kind of distraction that will delay their return to the unoccupied residence.
Results-Focused Contrast
By contrast, the crooks can do it all with lower staffing, or even just one person. More realistically, the practical skeleton crew would probably involve no fewer than two people: one behind the wheel to serve at lookout and getaway driver, and the other to smash into cars and grab registrations. Then the two drive to the target residence or residences and speed through the burglary. Both probably operate without a budget for exotic electronics and tie-ins to command centers to assist with target selection. Instead, they concentrate on hitting expensive-looking cars that were driven to the movie-theater by people wearing expensive clothes. The more they strike, the more they refine their target-selection protocols. These energetic Davids make up in alacrity and boldness what their more cumbersome Goliaths in bureaucracy only approximate through big budgets, over specialization, and lack of imagination.
Is this comparison exaggerated? Perhaps. Far-fetched? Not necessarily. Sometimes a tighter focus on results trumps the bureaucracy's inherent tendencies to magnify, complicate, and embellish.
- Nick Catrantzos
Thursday, April 5, 2012
Good Lesson Lost in Bad Shooting Analysis
The murder of seven students at Oikos University in Oakland by a disgruntled former student triggered pontificating that masked the value of some responses that saved lives and kept the body count from multiplying.
A good representation of the pontificating appeared in the Christian Science Monitor’s article of April 3, 2012, “Oakland school shooting: Is there a lesson to be learned from the tragedy?”
(Available at
http://www.csmonitor.com/USA/2012/0403/Oakland-school-shooting-Is-there-a-lesson-to-be-learned-from-the-tragedy ).
Mention of the useful responses made a cameo appearance in earlier news, as in the Associated Press’s straight reporting on the preceding day, April 2, 2012, in “Cops: 7 dead, 3 hurt in Christian school shooting”
(Available at
http://news.yahoo.com/cops-7-dead-3-hurt-christian-school-shooting-214742437.html ).
Afterwards, however, when the media made its usual calls to self-styled experts on workplace violence to hold forth on the Oakland shootings, the focus shifted to more speculative waxing on themes like school bullying and pre-termination counseling for malcontents, on the unvalidated theory that counselors would (a) soften any harsh blow of bad news that would otherwise send the recipient to his sidearm and thoughts of carnage or (b) magically detect the nefarious intent of such shooters in time to prevent rampage killings. Sadly, there is no proof that the average school or workplace counselor can consistently deliver on either (a) or (b). So, if the speculations of experts appear suspiciously like sales pitches promoting expert wares, it should come as no surprise. What is a little surprising – and disappointing – is the myopic focus on expert offerings and comfort zones at the expense of some timely, life-saving actions that deserve to be celebrated instead of ignored.
These actions epitomize what has become a reflexive response in emergency management: In times of danger, saving lives often comes down to a decision to either evacuate or shelter-in-place. And sometimes, the same event requires opposite decisions, depending on immediate circumstances. So it was with the Oakland school shooting. This was captured vividly in the earlier, AP story, before the expert, Monday morning quarterbacks had their chance to weigh in the next day.
As the AP story revealed, one teacher saved lives by locking her classroom door and having her students shelter in place. The gunman shot out some classroom windows but did not injure any of those students. Another student reported that her English teacher, on hearing the commotion, told students to run, and they did, escaping a chance to expand the casualty count.
What the speculating experts missed in the subsequent article was that some people in the school itself did indeed react well under fire, saving lives by resorting to traditional options to hunker down (shelter-in-place) or flee (evacuate). The old lessons still apply. Extra counseling, while potentially useful, is not nearly so important to saving lives when bullets are flying and the situation calls for action rather than reflection. Before finding fault with school administrators for not instituting a major counseling initiative to observe and mollify dysfunctional students or to overtrain victims about when to rush the shooter instead of standing still while he shoots them one at a time (as the Christian Science Monitor’s article had one expert saying), school leadership would be wise to acknowledge the foregoing life-saving responses that kept the body count from rivaling that of Virginia Tech only a few years ago. There is always room for improvement, but we should not diminish the positive responses in a theoretical search for perfect solutions.
– Nick Catrantzos
A good representation of the pontificating appeared in the Christian Science Monitor’s article of April 3, 2012, “Oakland school shooting: Is there a lesson to be learned from the tragedy?”
(Available at
http://www.csmonitor.com/USA/2012/0403/Oakland-school-shooting-Is-there-a-lesson-to-be-learned-from-the-tragedy ).
Mention of the useful responses made a cameo appearance in earlier news, as in the Associated Press’s straight reporting on the preceding day, April 2, 2012, in “Cops: 7 dead, 3 hurt in Christian school shooting”
(Available at
http://news.yahoo.com/cops-7-dead-3-hurt-christian-school-shooting-214742437.html ).
Afterwards, however, when the media made its usual calls to self-styled experts on workplace violence to hold forth on the Oakland shootings, the focus shifted to more speculative waxing on themes like school bullying and pre-termination counseling for malcontents, on the unvalidated theory that counselors would (a) soften any harsh blow of bad news that would otherwise send the recipient to his sidearm and thoughts of carnage or (b) magically detect the nefarious intent of such shooters in time to prevent rampage killings. Sadly, there is no proof that the average school or workplace counselor can consistently deliver on either (a) or (b). So, if the speculations of experts appear suspiciously like sales pitches promoting expert wares, it should come as no surprise. What is a little surprising – and disappointing – is the myopic focus on expert offerings and comfort zones at the expense of some timely, life-saving actions that deserve to be celebrated instead of ignored.
These actions epitomize what has become a reflexive response in emergency management: In times of danger, saving lives often comes down to a decision to either evacuate or shelter-in-place. And sometimes, the same event requires opposite decisions, depending on immediate circumstances. So it was with the Oakland school shooting. This was captured vividly in the earlier, AP story, before the expert, Monday morning quarterbacks had their chance to weigh in the next day.
As the AP story revealed, one teacher saved lives by locking her classroom door and having her students shelter in place. The gunman shot out some classroom windows but did not injure any of those students. Another student reported that her English teacher, on hearing the commotion, told students to run, and they did, escaping a chance to expand the casualty count.
What the speculating experts missed in the subsequent article was that some people in the school itself did indeed react well under fire, saving lives by resorting to traditional options to hunker down (shelter-in-place) or flee (evacuate). The old lessons still apply. Extra counseling, while potentially useful, is not nearly so important to saving lives when bullets are flying and the situation calls for action rather than reflection. Before finding fault with school administrators for not instituting a major counseling initiative to observe and mollify dysfunctional students or to overtrain victims about when to rush the shooter instead of standing still while he shoots them one at a time (as the Christian Science Monitor’s article had one expert saying), school leadership would be wise to acknowledge the foregoing life-saving responses that kept the body count from rivaling that of Virginia Tech only a few years ago. There is always room for improvement, but we should not diminish the positive responses in a theoretical search for perfect solutions.
– Nick Catrantzos
Subscribe to:
Posts (Atom)
