John's Quote about Confidence and Probabilities

[Strict Scrutiny]

I did a quick search for "scientific certainty" to see if others are using it in court. It looks like it has a distinct legal meaning of 95% sure or greater. In laymen's terms I think that translates to "pretty darn sure". I believe it is a slightly diluted form of positive. Here is a link to a short paper on the term:

http://phys4.harvard.edu/~wilson/sounds ... .Legal.pdf

[Cindy Rennie]

Wouldn't 'quantifying our degrees of uncertainty' be like admitting that we may be 'a little bit pregnant'?

Either you is or you ain't.

I'd hate to leave that door open for the defense.

[Pat A. Wertheim]

Two key concepts in this discussion are PERSONAL BELIEF and SCIENCE.

In PERSONAL BELIEF, I can be absolutely certain. 100%. No doubt whatsoever. No chance of the fingerprint belonging to someone else.

In SCIENCE, there is never absolute, 100% certainty in a conclusion such as a fingerprint identification.

We have to understand that since we cannot (at the current state of the practice of fingerprint identification) quantify the probabilities as is done with DNA, when we testify to absolute certainty, we are testifying to PERSONAL BELIEF, not scientific certainty. Scientifically, the chance of error may be so very, very small that we are willing to disregard it, but technically, a conclusion is not 100% certain. Maybe 99.999999% or greater, but not 100%.

So, "quantifying degrees of uncertainty" is science. Personal belief may result from a valid scientific methodology applied to problem to reach a conclusion, but scientifically, the conclusion still has a probability attached to it. I, for one, would love to see a methodology for calculating scientific probability in fingerprint identifications. I might still testify to PERSONAL BELIEF, but then I could also testify to the scientific numbers to back up my belief.

In the meantime, I will continue to testify that I am 100% certain. Absolutely. 100%. But, of course, I have to also admit that mistakes happen. Oh, uh, but not in THIS case.

[David Fairhurst]

Just to expand a little...

Pat is, of course, referring to the "leap of faith" that Champod and others talk about.
The DNA guys have huge statistical studies of the incidence in the general population of the particular base-pair sequences that their technique uses. They can therefore produce quantitative probabilities and present those to a court.
The Jury, when confronted with a probablilty of 1 in 1 billion, are then entitled or even required to make the "leap of faith" and personally believe that they are 100% certain.

We do not (yet?) have the resources to produce numbers. The data just isn't there. So we as "experts" are given the privilege of being able to express our personal belief in a courtroom.
The jury are not able to make the "Leap" themselves so we have to do it for them (at least for now).

Give me the means to produce reliable probabilities to back up my personal belief/expert opinion/scientific conclusion (whatever you prefer to call it) and you'd better believe I'm going to use it.

[Pat A. Wertheim]

"Leap of Faith" -- there's a phrase that had to have come from the critics. In responding to a defense attorney who used that phrase in the Byron Mitchell case (first Daubert hearing against fingerprints), Bruce Budowle responded that when a fingerprint examiner makes a positive identification, "it is no great leap."

You might add that a "leap of faith" is jumping off a cliff when you don't know how far it is to the bottom, and having faith that God will take care of you. For a qualified fingerprint examiner making a positive identification in the absence of a probabilistic model, it might be more like stepping off a curb. "Absolute certainty" may still be personal belief, but "leap of faith" implies more doubt than we feel.

[David Fairhurst]

I agree Pat, it is no great leap, but it is a small step away from science.

David

[David Fairhurst]

Wouldn't 'quantifying our degrees of uncertainty' be like admitting that we may be 'a little bit pregnant'?

Either you is or you ain't.

I'd hate to leave that door open for the defense.

Quite true Cindy, the state of pregnancy is an absolute.
However, if you get a positive from a home pregnancy test kit can you be 100% certain that you are pregnant?

(I was trying to resisit comments linking "leaving the door open for defense" with "pregnancy"; but I couldn't resist)

David

[g.]

Dave you beat me to the point.

Cindy, you are using the perfect analogy. You are either pregnant or you're not. BUt for you to determine what state exists (pregnant, not pregnant) you have to make an inference from the quantity and quality of information given to you.

Dave's got a good point: the home pregnancy test kit is mostly reliable but not always. Compare that information to being 8 months along, having an extended belly, and some weird "feelings" inside of you. Both have different levels of certitude. And of course, legs in stirrups and a head crowning, guess what, pretty darn sure you're a momma.

But as you put it, it "opens the door for defense". I believe this innate fear of what defense will do is what has slowed us down in pursuing better science and tools. But in our current "jury based-money talks and gets you the Dream Team defense" system, those fears have some basis. I don't know how to resolve it, I just am questioning some of those basic tenets that are taught (and/or parrotted).

And lastly, here's the kicker question, should we be concerning ourselves with the possibility of what defense will do with a slightly open door, or simply presenting the evidence as it is, adding no more personal (as Pat W. notes) or other certainty than is allowed by the actual evidence. Some will say "Keep it simple stupid"; some will say "Just the facts please". An interesting conundrum.


g.

[John Vanderkolk]

Since we have been talking of confidence, probability, knowing, truth, absolute, universal, certainty, etc., check out the 'word of the year' by Merriam-Webster dictionary at http://news.yahoo.com/s/ap/20061208/ap_ ... f_the_year

"Truthiness" was credited to Comedy Central satirist Stephen Colbert, who defined it as "truth that comes from the gut, not books."

"We're at a point where what constitutes truth is a question on a lot of people's minds, and truth has become up for grabs," said Merriam-Webster president John Morse. "`Truthiness' is a playful way for us to think about a very important issue."

Truth is one of the great words of philosophy. If we know, do we believe it as truth or truthiness? Absolutely or universally? Infallibly or fallibly? Virtually unconditioned truth or unconditioned truth? Have all the relevant and appropriate questions been asked? Maybe truthiness occurs without the reflective understanding of the answers to any relevant and appropriate questions. When would truthiness transition to truth? Maybe we should collaborate with Merriam-Webster.

[Dennis Degler]

[g.]

I think (personally) that is a VERY fair and accurate answer, plus as you allude to, there are additional QA/QC things that can be brought up (verification, accreditation, SOPs, audits, etc.).

My testimony sounds similar to that as well.

g.

[g.]

Oh and I forgot to throw in:

That I would also add, that "I believe this to be an accurate identification until I am shown evidence otherwise". e.g. another examiner stating that it is not, someone bringing forth another known to which it could be a match, sufficient disagreement to invalidate the ID, etc. "At which time I would gladly evaluate this new evidence or opinion which challenges mine".

And I sincerely mean those statements. It would be arrogant for me to think that I am incapable of an error and I would welcome that scrutiny.

I think that's where people get so hung up. Defense wants us to acknowledge the "phantom possibility" that an error can be made--fine, depending on the Q&Q, I will, but then they don't offer any proof, evidence, or expert to show that one ACTUALLY occurred. That's where I get frustrated. But I suppose that is all they have to do under the "reasonable doubt" concept, but the phantom possibility in my mind, is NOT sufficient reasonable doubt. A battle of experts (McKie, Camm, etc.) MAY be.

g.

[L.J.Steele]

Be careful of talking about confidence as this can be a buzz word for some judges. There's a great deal of research dealing with the difference between confidence and accuracy in the eyewitness ID field. (Essentially, there's very little relationship between confidence in an eyewitness ID and accuracy.) Some judges will be familiar with it, either because it is being used elsewhere in that trial, or because it has been presented to them in other trials. A few states, like MA and GA have specific jury instructions about charging that confidence and accuracy are not correlated.

Confidence levels can be manipulated by giving suggestive information. They can also be manipulated by post-ID information. The studies show that confidence tends to rise as a court date approaches, merely by virtue of the case going to court. (If folks want cites, let me know.)

I know there's a vast difference between eyewitness ID and expert testimony, but a judge or attorney could attempt to apply the analogy from one to the other. You may want to record on your report your confidence level at the time you made the identification. This will at least insulate you from questions about post-event information artificially inflating your confidence level. You may also want to note any information you received about the crime or suspect before making the ID and/or report. Again, this may insulate you from a claim that the information might have created a confirmation bias or inflated your confidence level. If you did get problematic information, noting it may remind you to think hard about whether it affected your decisions.

[Norberto Rivera]

I have to agree with Pat on his comment regarding "Personal Belief" and "Science". As a new examiner with but a couple of years worth of experience I found it difficult to differentiate between the two. The main reason for this was a perceived lack of consensus among other examiners in terms of semantics mainly. Attorneys are infamous for using word play and, as practitioners, I think we have been drawn into some confusion as to how we describe what we do. Are we certain "to a high degree of scientific certainty"? Are we "absolutely certain"? Are we "100% certain"? Is it "the opinion of this examiner based on accepted comparison methodology and verification"? Everyone has a preferred verbage, mine happens to be the latter. Until someone comes up with some sort of quantifiable method of validating our work we will continue to see Daubert and Frye hearings and defense attorneys downplaying our work and finding loopholes in the articulation of what is probably some great comparison work otherwise.

[Charles Parker]

This was a good thread and since there were a number of presentations on probability in San Diego I thought I would bring it back to the top.

g.---I am a couple of steps closer----starting to get really interested in this thing (the probability of me understanding it has increased).