Stats Expert to Present Fingerprint Doubts

Discuss, Discover, Learn, and Share. Feel free to share information.

Moderators: orrb, saw22

Post Reply
Dr. Borracho
Posts: 157
Joined: Sun May 03, 2015 11:40 am

Stats Expert to Present Fingerprint Doubts

Post by Dr. Borracho »

A featured speaker at the AAFS casts doubts on fingerprint identification:

http://www.forensicmag.com/news/2017/02 ... type%3dcta

Stats Expert to Present Fingerprint Doubts at AAFS 69th Annual Meeting
Wed, 02/15/2017 - 11:10am

Fingerprints left behind at a crime scene are one of the best possible identifiers of a suspect or perpetrator, rivaled only, in popular opinion, by DNA evidence. However, fingerprint analysis, which has been used in criminal investigations for over a century, may not be as infallible as believed by law enforcement, court officials and the media, according to statistics expert Joseph “Jay” Kadane, a Carnegie Mellon University professor emeritus and upcoming presenter at the 69th Annual Scientific Meeting of the American Academy of Forensic Sciences.

Circumstantial vs. direct

“The idea of identification—that such an expert can say ‘this is the person who left this mark’—is beyond what science can support,” Kadane told Forensic Magazine. According to Kadane, fingerprint evidence should be considered circumstantial, rather than ironclad.

“It’s not that I don’t think that it’s evidence—it is. But it’s not identification evidence,” he says. “It’s circumstantial.”

As Kadane explains, a fingerprint analyst may compare a mark left at a scene with a fingerprint collected from a suspect and point to certain similarities between the two, but just because two prints have similarities does not mean they can only have come from the same person.

“The question is—how many people share those characteristics?” he asks. “At present, with our current scientific knowledge, I don’t think we have a way of estimating that either locally, nationally (or) worldwide.”

The professor emeritus further illustrates his point by comparing fingerprint evidence to another form of circumstantial evidence—vehicle identification.

“It’s similar to ‘I saw a white Chevy drive away from the scene’—well, there are lots of people with white Chevys,” he explains. “But if you add to that a whole bunch of other circumstantial evidence, it gets to the point where people think ‘yeah, this is probably the guy who did it.’ But that judgement is a fact-finder judgement for the jury or the judge and not a matter for a fingerprint analyst to conclude.”

Accuracy called into question

Kadane is not the first to call the validity of fingerprint analysis as direct evidence into question. In 2002, a federal judge in Philadelphia became the first to rule that fingerprint analysis did not meet the Supreme Court’s standard for admissible scientific testimony as set forth by Daubert v. Merrell Dow Pharmaceuticals in 1993, the New York Times reported.

The “Daubert Standard” calls for judges to be the gatekeepers of scientific evidence in the courtroom by weighing several different factors to determine the validity of any scientific theory, method or technique.

Pollack ruled that fingerprint analysis had not been tested enough to meet the standard, saying witnesses could speak to two prints’ similarities, and to the fact that fingerprints are unique to each person, but could not testify that “a particular latent print is in fact the print of a particular person,” according to the New York Times.

Pollack later reversed his opinion, the New York Times subsequently reported, citing, in part, the widespread acceptance of fingerprint evidence in British courts. But since then, fingerprint analysis has continued to fall under some degree of scrutiny by experts in both the legal and scientific communities.

In 2003, law professor Jennifer L. Mnookin published an article in Issues in Science and Technology journal titled “Fingerprints: Not a Gold Standard.” In the article, Mnookin agreed with Pollack’s first ruling on fingerprints, saying “there has been woefully little careful empirical examination of the key claims made by fingerprint examiners.” She also pointed to a 1995 test in which 34 percent of analysts misidentified prints.

More recently, in 2011, Mnookin and a group of 12 other experts in the fields of law and science coauthored an article in the UCLA Law Review that encouraged the development of a “research culture” in forensic science, and which critically examined several different types of forensic evidence, including fingerprint analysis.

“Even if every set of ten prints is different from every other, two specific portions of two prints from different individuals might be extraordinarily similar to one another,” the authors pointed out, among several other potential pitfalls of attempting to “match” prints.

“The claim that friction ridge patterns are highly variable might be a necessary precondition for fingerprint identification, but it does not establish fingerprint analysts’ ability to make a match,” the article states.

The article also mentions the case of Brandon Mayfield, an attorney from Oregon who was falsely tied, due to fingerprint misidentification, to the scene of the 2004 Madrid train bombings, in which over 190 were killed and about 2000 more injured, NBC News reported. The FBI issued an apology to Mayfield, who was detained for two weeks and said he felt targeted for being Muslim, after an error by their fingerprint matching computer system, and subsequent mistakes by FBI examiners, led to his accusation and arrest.

“Three experienced FBI examiners said ‘yes, his fingerprints are a match to this bag we found in Madrid,’” Kadane says. “The defense hired a fourth independent expert who corroborated that. But the fact is that Mayfield had nothing to do with it.

“Those were presumably the best we’ve got for fingerprint analysis,” Kadane adds.

Validity upheld by the courts

Despite criticism of fingerprint evidence, it is widely defended and a frequently utilized form of forensic evidence, with many experts saying fingerprint analysis is valid and can be used to definitively identify a suspect or perpetrator.

In his book Courtroom Testimony for the Fingerprint Expert, retired FBI fingerprint examiner Gary Jones rejects the claim that fingerprint analysis is “junk science” and argues that it passes on every criterion of the Daubert test.

“As of the date of publication of this book (in 2007), the science of fingerprints has prevailed over every challenge made to it by the standards set forth in Daubert,” a passage from the book states.

Additionally, an article published last year in Mitchell Hamline Law Review outlined several cases in recent history where fingerprint evidence admission was challenged by defendants in court but upheld by judges. In one example from 2011, a challenge by accused San Diego courthouse bomber Donny Love was rejected by the judge, who cited a study from May of that year in which only six false positives occurred out of 4,083 fingerprint comparisons.

In a report published by the President’s Council of Advisors on Science and Technology last September, the council called latent fingerprint analysis “a foundationally valid subjective methodology—albeit with a false positive rate that is substantial and is likely to be higher than expected by many jurors based on longstanding claims about the infallibility of fingerprint analysis.”

The report goes on to point out the possibility of confirmation and contextual bias in fingerprint analysis, and suggests latent print analysis should change “from a subjective method to an objective method.”

“It's very hard to know what in the future we may discover,” Kadane says, about any potential advancements in fingerprint identification and technology. “But in terms of right now, it seems to me that identification is simply beyond what, as a science, we can defend.”

Kadane will give his presentation, “Fingerprint Science,” at the 69th Annual Scientific Meeting of the American Academy of Forensic Sciences on Thursday, Feb. 16.
If any visitors to this forum attended this talk, please respond to this post with comments on how the presentation was received by the attendees.
"The times, they are a changin' "
-- Bob Dylan, 1964
josher89
Posts: 509
Joined: Mon Aug 21, 2006 10:32 pm
Location: NE USA

Re: Stats Expert to Present Fingerprint Doubts

Post by josher89 »

I reached out to him when I discovered that he was presenting and asked for his slides. He obliged and I have been preparing a response to him based on the information that he gave me. I have to assume that he presented the same lecture at the NIST meeting last November. The link to his talk is:

https://www.nist.gov/news-events/events ... -nist-2016

You have to select Day 1, Part 3 and scroll over to 35:00 to where he begins.

I would still encourage those that were able to listen to him provide feedback but as soon as I have completed my response to him, I'll share it here as well.

Here is a link to his website: http://www.stat.cmu.edu/~kadane/
"...he wrapped himself in quotations—as a beggar would enfold himself in the purple of emperors." - R. Kipling, 1893
Boyd Baumgartner
Posts: 567
Joined: Sat Aug 06, 2005 11:03 am

Re: Stats Expert to Present Fingerprint Doubts

Post by Boyd Baumgartner »

I think you have to consider a couple different elements.

1) Based on his critique, he's about 10 years behind the times and fails the smell test of someone knowledgeable of philosophy of science or forensics. In that NIST conference you referenced, if you fast forward to the Q&A at the end, I ask the panel what is the best way to study error rates in case work. His response was proficiency tests..... Anyone who references proficiency tests as a model of casework has never worked a case or taken a proficiency test. The strategies involved in testing and casework are diametrically opposed.

2) The last time a stats expert told everyone how wrong they were it ended up in a mea culpa a decade later. I'll take the long view

3) Elham Tabassi nailed it when she said at that same NIST conference (paraphrazed) 'At the point statistical models are most needed, they are the least powerful'. To g.'s point from long ago, the most relevant use of the L/R is the inconclusive which these days tends to be framed around the word 'complex'. From a mere 'John Henry' perspective (man vs machine) people still outperform algorithms. Again, I'll take the long view.

4) USACIL will be putting numbers to 'associations' in the next 90 days if the word on the street has any credibility, but to the previous point, algorithms are just mathematical representations of human values albeit systematized and will fail to tell us how much the L/R calculation is based upon examiner interpretation of features. It just means that we will have systematic error as opposed to randomized error, and if we're to buy into and use the USACIL algorithm, their error will be global and not contained to local municipalities where errors occur. Is this a greater good? I sure don't think so.

Anyway, just my 2 cents.
Dr. Borracho
Posts: 157
Joined: Sun May 03, 2015 11:40 am

Re: Stats Expert to Present Fingerprint Doubts

Post by Dr. Borracho »

Quote for the day:
Not everything that counts can be counted, and not everything that can be counted counts."
Your mission, should you choose to accept it, is to find out who first said that.
"The times, they are a changin' "
-- Bob Dylan, 1964
Boyd Baumgartner
Posts: 567
Joined: Sat Aug 06, 2005 11:03 am

Re: Stats Expert to Present Fingerprint Doubts

Post by Boyd Baumgartner »

I believe it was Terry A Smith, who incorrectly attributed it to Einstein.

I'd say another appropriate quote is Maslow's Hammer which states,
if all you have is a hammer, everything looks like a nail
Bill Schade
Posts: 243
Joined: Mon Jul 11, 2005 1:46 pm
Location: Clearwater, Florida

Re: Famous Quotes

Post by Bill Schade »

It was einstein who said:

"You can't solve a problem at the level at which it was created"
Identify
Posts: 77
Joined: Wed Jan 16, 2008 7:26 pm
Location: USA

Re: Stats Expert to Present Fingerprint Doubts

Post by Identify »

statistics shirt.jpg
You do not have the required permissions to view the files attached to this post.
Boyd Baumgartner
Posts: 567
Joined: Sat Aug 06, 2005 11:03 am

Re: Stats Expert to Present Fingerprint Doubts

Post by Boyd Baumgartner »

I was just looking through a handout one of my coworkers gave me who went to the 66th Annual AAFS conference back in 2014 when it was here in Seattle.

The title of Workshop #16 was Science, Law, and the Inferential Conclusions and Their Role in the Legal Fact-Finding Process (pg 23) which states:

Both science and the criminal justice system strive to discover factual truth; however, in neither can we know what that truth is with complete certainty. Both systems are structured so as to provide information with specified characteristics. This information is relied upon as a basis for drawing necessary inferences. The inferences yield conclusions that may be believed to be correct but never known to be so. The process by which information is obtained and inferences are made determine the confidence we can have in those beliefs. The rigorous framework established by science permits the degree of certitude that can be accorded scientific conclusions to be quantitatively measured and expressed. The legal system provides less rigorous machinery which guides fact-finders in determining whether conclusions can be supported by designated burdens of proof. Even where scientific evidence is relied upon in the courtroom, the inferential processes engaged in are different in these two contexts. Because of this divergence, while forensic evidence often facilitates the determination of factual truth in the courtroom, fact-finders may rely upon it for conclusions it does not support. This workshop will address the epistemological structure supporting legal and scientific conclusions and the interplay between the two when forensic evidence is relied upon.
I think there's an important distinction to be made here. Legal inference and scientific inference are two different enterprises and overstating needs to be tempered on two levels, the legal side and the scientific side. That is why in the Casillas thread, I jokingly point out the difference between what Ken Moses says vs what the attorneys say. Also, in the Articulation Document that I attached in the IAI Position Paper thread, I begin with a discussion on the framing of testimony and conclude with concepts and preferences.

I realize there's a big demand for 'tell me what to say in court' content that consists of phrases and words that are bulletproof against cross examination. However I think it is important to note that words have different meanings in legal, scientific and lay contexts (think: opinion), and come in and out of fashion, especially if they are chasing critique. Additionaly, people use words differently in technical and lay contexts (think: certain), and you have absolutely no control how those words are taken by the juries or how they will be used for or against you in the closing statements. That being said, and to Bill's earlier point about problems not being solved on the level they were created, if the words are the problem, they aren't fixed with more words. They're tempered by transparency, rationality and coherence in the concepts that produce them. Otherwise, you're just creating a contextual testimony dictionary version of the Check-O-Chart.
You do not have the required permissions to view the files attached to this post.
Bill Schade
Posts: 243
Joined: Mon Jul 11, 2005 1:46 pm
Location: Clearwater, Florida

Re: Check O Chart

Post by Bill Schade »

Now there is something that would have been helpful back in 1972 when I was taught the Henry Classification system

:shock:

NOT!!
John Vanderkolk
Posts: 73
Joined: Tue Feb 28, 2006 7:07 am
Location: Washington, DC

Re: Stats Expert to Present Fingerprint Doubts

Post by John Vanderkolk »

One of my favorite pictures! Wonder what Identify thinks!
12038826_923720314362053_238957313915051017_o.jpg
You do not have the required permissions to view the files attached to this post.
Boyd Baumgartner
Posts: 567
Joined: Sat Aug 06, 2005 11:03 am

Re: Stats Expert to Present Fingerprint Doubts

Post by Boyd Baumgartner »

Rereading this thread reminded me of this paper regarding a different perspective on probabilities, namely Baconian Probabilities, also from Carnegie Melon. Here is an associated Powerpoint presentation based on the paper. It's more philosophical than math heavy, and just provides another vantage point from which to view probabilistic thinking.
Post Reply