Debating research, especially when you do not agree with it!

[Outsider]

Pat Wertheim wrote

"Error rate" of the science, or of bias in verifications, cannot be calculated for large numbers of people, then applied across the board.

This is something I have direct experience with in industry. Every process can have two types of error. There can be “steady state” errors which arise out of the capability of the process and these are impossible to eradicate (unless you can find a way to fundamentally improve the process). The other type of error is assignable to a specific cause such as faulty machinery or an unskilled operator. It is vital to be able to distinguish between the two otherwise operators or supervisors get blamed for something which is not their fault, and the errors will return because the real cause has not been dealt with.

The official view is that the method of fingerprint identification is infallible so all misidentifications are assignable to rogue experts who can be rooted out of the system. This has seemed like a reasonable assertion because every time it has been proved that a misidentification has occurred, all subsequent judgement on the match is "exclusion" or "inconclusive".

Now that we have the results of Dr Dror’s research can we be so sure that these misidentifications are so clearly bad work from the original expert? Suppose that Brandon Mayfield had travelled to Europe in the weeks before the Madrid bomb and no better match of the latent print had been found. Can you be sure that every competent fingerprint expert in the world would look at Mayfield’s fingerprint and conclude that it did not match the latent print on the bomb material?

It concerns me that no research had been carried out into the core claim of fingerprinting, which is that the method routinely achieves true individualisation - to the exclusion of all others who have ever lived - from latent prints. Such research would indicate if the method has a “steady state” error rate due to random matches.

Even if all errors are assignable to sub-standard fingerprint analysis juries can only assume an industry-wide error rate unless individual experts are regularly “stress tested” and certified. Airline pilots are routinely tested on simulators with the most difficult conditions that they are expected to handle safely. The equivalent for fingerprinting would involve comparing latent prints with prints selected from large databases which are chosen to be extraordinarily similar, but are known to be from different sources. I cannot see any other way that the capability of individual experts can be separately assessed in a way that would be meaningful to a jury.

[mdavis]

All of which brings me back, full circle, to my original assertion that the vast majority of examiners avoid the potential for error by avoiding making those borderline calls in which they are not absolutely positive will pass independent verification and subsequent comparison with corroborating evidence. Not sure? Don't call it. There isn't a criminal alive that is worth more than my job and my reputation. This is becoming increasingly important when we access huge databases with dozens of highly similar controls. The old "borderline" call may have been relatively "safe" in the old days before AFIS, but no longer.

[mdavis]

I'd like to think this is true -- that the bad apples are quickly spotted and removed, but I fear that the existing mechanism are weak and sporadically applied.

The existing mechanisms for dealing with bad examiners are quite specific and quite adequate for agencies accredited under ASCLD or ISO. These mechanisms are universally applied. If weak and sporadic, then you are talking about agencies that are not accredited and/or do not follow accreditation guidelines formulated by SWGFAST.

I have been working in latent prints for about 15 years. Within that time span, very few cases in the courts have been found in which bad idents were made. You can bet that any bad ident will be plastered all over the media by the defense. Some of these were by non-examiners (a Pennsylvania case with detectives comes to mind), along with a couple of other very high profile cases that have saturated the media for a very long period of time (McKie, Mayfield). Considering that many tens of thousands of latent print comparisons are performed daily, one could expect one or two bad idents from examiners who are either under-qualified or over-confident in their abilities. But we only find one or two a year, not daily. All but a couple of these reported idents are ever challenged because the suspect pleas, or other corroborating evidence supports the ident, thus confirming what is still the single most reliable forensic tool for personal identification, now arguably shared with DNA.

I stand by my statement that bad examiners are "usually" removed from the pool. Other examiners in the agency and the agency itself does not want to be associated with the bad publicity, and I would argue that persons who make their living based on their personal integrity and honesty will seldom if ever try to cover up (Mayfield). If we now have a growing problem of examiners moving from one agency to another to avoid being fired for faulty performance, it is partly due to the almost total lack of ability of any employer, even LE agencies, to do a meaningful background check on an applicant due to recent privacy laws. Prior employers will often tell nothing except dates of employment.

Of all the links in the chain from commission of the crime, to first responders, crime scene technicians, detectives, evidence clerks and couriers, prosecutors and attorneys, judges and juries, the crime labs are the strongest and most accurate link, case for case. If defense wants to probe for weaknesses, there are far more fertile fishing holes and far more errors made at each of the other links.