Forensic Science in Need of Reform

[Ann Horsman]

This article was interesting, any comments on their statistics?

http://www.slate.com/id/2197284/

[Amy Hart]

"A 2006 U.K. study by researchers at the University of Southampton found that the error rate of fingerprint analysts doubled when they were first told the circumstances of the case they were working on."

This is the problem that I have with publication of very preliminary research. It is given significance that it doesn't yet deserve. The above quoted sentence would have a little different impact if it said "the error rate of 6 fingerprint analysts doubled..."

I do not question the validity of the research. I question the publication of results from a study of 6 people. IMHO, that information belongs in a grant application, not in the newspaper. It's just not ready for public consumption yet.

As far as the 1-4 % error rate, I'm not sure where that information comes from. I would be interested to find out where Slate is getting those numbers.

[Gerald Clough]

The issue is not whether or not there are errors. There are errors. There are errors in every discipline in every realm. And, although you may, as every discipline has, introduce practices to reduce the likelihood of error, there will always be errors. In the forensic fields, there are areas where, in nearly every contested case, one or another expert is either wrong, or it must be the case that the possible conclusions are but amorphous labeling that cannot rise to certainty in any way except that the expert is certain that it is his opinion.

Seeking to establish reliable error rates is a fruitless task. (Unless you count academic fruit that can be endlessly produced.) The fact is that one cannot know anything about a non-trivial analytical field's error rate because independent evidence that would absolutely settle the issue is exceedingly rare. To make it even more difficult, any process that pretends to establish an error rate has its own errors. You could never know that the process was absolutely reliable. And, in this, we're really talking about studying analysis done to established standards and under all proper conditions. No study can establish a rate of "dumbass" and "bad attitude" errors.

Okay. So there are errors. What does that mean? Does it mean fingerprint evidence can't be used? Does it mean DNA results and calculations are not evidence? Must all suits in which questions of fact involve technical, scientific, or medical evidence be settled by lay guesswork? Of course it's absurd. Every legal system pretending to fairness recognized from the beginning that absolute certainty is a commodity so rare that it's not worth any expectation. The unavoidable (and, as with all human solutions, flawed) answer is the adversarial system of seeking the most likely truth. Justice is not the delivery of absolute truth. It is the equal opportunity to present different propositions for what the truth may be and an honest decision on whether or not the evidence meets whatever practical degree of proof is required in that matter.

The adversarial advocacy required for justice means not just competent attorneys but competent attorneys who have critical expert advice available. Until the criminal justice system provides for critical expert analysis and review of the conclusions that brought the matter to trial, the focus will continue to be speculative error rates and biases, rather than meaningful examination of the evidence itself. It's not forensic science that's in need of reform. It's how the system applies it.

[Steve Everist]

"
As far as the 1-4 % error rate, I'm not sure where that information comes from. I would be interested to find out where Slate is getting those numbers.

I'm guessing that rate is from New Scientist where they state .8% - 4.4%, which is a cite from Cole (or directly from Cole's article in the Journal of Criminal Law & Criminology (vol 93, p 985)):
http://www.newscientist.com/article.ns? ... 725174.500

This is the same article that states 1900 erroneous ID's in 2002 alone.

It seems like there are a lot of quotes of quotes of quotes lately. The problem being is that once someone's work has been quoted, it somehow becomes fact and continues to be used regardless of inadequacies of the original research or statistic.

[Ann Horsman]

That's why I posted this. Sure seems like no one is identifying their sources but have no problem putting these numbers in print.