Absolute conclusions / postitive proof

[Michele]

After the NAS report was released, the President of the IAI wrote a memo to its members. One of the bullet points in this memo was:

Although the IAI does not, at this time, endorse the use of probabilistic models when stating conclusions of identification, members are advised to avoid stating their conclusions in absolute terms when dealing with population issues.

This weekend I read William Leo’s article in the Identification News. He states:

For over 100 years, fingerprint examiners have testified that a latent print identification is ‘Positive proof of identity”. Some examiners use the term “to the exclusion of all others”. Both of these statements are as true today as they have been for over 100 years. The only thing that has changed is that these statements are now supported by the additional research that has taken place and the unprecedented track record of success of this powerful identification tool, making these statements even more powerful today.

I’m wondering, aren't these opposing views? Are fingerprint examiners unaware of the memo from the IAI? Are they aware of it but don’t agree with it? Or maybe the IAI’s memo and William Leo’s article aren’t opposing views, after all, he never uses the world ‘absolute’.

Any thoughts?

[Gerald Clough]

They're not opposing, except in the opposing natures of law and science. I think it's a difference over two related points. One is the relative weight of historical experience, which includes how one analyzes some of the errors, and the weight of strict scientific language and logic. The other is the issue of practical application and formation of professional opinion. The IAI statement recognizes that absolute conclusions that imply that if one could examine all portions of all human skin, only one individual would be found to match. The statement recognizes that there is no scientifically palatable finding that could support such a conclusion, no matter how an individual examiner might be convinced it was correct. Leo's statement includes the ambiguous term "proof." Proof has various meanings, depending on which arena of human endeavor you're operating within. Law cannot be limited to scientific standards of proof. We would would be able to resolve very few contests if that were the standard. You correctly note that he doesn't use "absolute." In most human affairs, we must proceed with lesser proofs. In the old jury charges, before the SCOTUS abandoned formal definition of proof beyond a reasonable doubt, the language ran something like, "...the kind of proof you would apply in making the most important decisions in your life...", such things as marrying or buying a house. If we demanded the kind of proof demanded by the scientific community in support of claimed knowledge, we would be forever single and renting.

Leo is correct that if you speak of conventional conclusions, there is indeed a body of evolving knowledge that supports the likelihood of them being accurate and that that belief is more supported than in the past. At the same time, strict logic dictates that they can never be proven to be absolute. The IAI statement recognizes that in the practical, functional realm of forensics, the test is whether a particular expert has formed an opinion by applying the special knowledge demanded by the law and that this opinion does not constitute in itself evidence to support a scientific certainty. ("Scientific certainty" taking semantic beating in the legal arena where it's often misused.) No matter how eminent the person forming the opinion or conclusion, the fact that that person formed the belief is not proof of anything. (The scientific community itself has suffered from doing so, an example being Skinner whose prominence permitted him to pervert the scientific process in behavioral research for many years.) Conversely, the law must protect itself from attempts to demand scientific standards of proof. It is rightly offensive to science if an analyst states an untenable absolute conclusion as fact. You do not get a pass on truth, just because your realm can make judgments without absolute knowledge.

The conundrum is that the scientist concludes absolutely when supported by scientific certainty and logical consistency. The forensic expert often concludes to a belief that, if absolute truth was known, this is what it would be found to be, a conclusion that need not be bound by scientific certainty and logical consistency, because by applying all that is known in a way that has been found useful in legal fact-finding, such a conclusion has value to the fact-finder. It can be characterized as a guess, but that doesn't make it any less valuable. That is the essence of legal fact-finding, trying to determine as best we can what the truth would likely be if we could know it absolutely. The analyst is an expert guesser, a far more reliable guesser than are our physicians. The analyst, when in the role of practical analyst (rather than conducting formal research), is not a scientist and is not bound by scientific standard.* The analyst is, of course, bound to not speak palpable falsehood. A statement that the analyst has established the one and only possible source of an impression and that it is therefore a fact is a statement contrary to logic, and the IAI recognizes that. A statement that the analyst has formed an opinion that an individual is the source of an impression and that he is certain and confident in that opinion is entirely accurate. Everyone in law understands the nature of expert opinion. It does not, of course, prevent litigants from arguing the lack of scientific proof and inconsistent logic, and but for a very few courts where the judge lamentably cannot grasp the principle, the courts value the expert opinion. (They will also slap down the expert who misstates confident opinion as absolute fact.)

Leo essentially argues the appropriately high value of fingerprint evidence, just as I deal almost daily with the high value of the expert opinions of psychologists whose opinions are in no way amenable, even theoretically, to scientific proof and who routinely offer opposing conclusions in which one of them is inevitably mistaken. The IAI cautions against claiming absolute proof that cannot meet any scientific test and also against making a probablistic statement that is not yet in any practical way supported by scientific knowledge. I think the IAI recognizes the hazard of prematurely opening the door to likelihood argument because, to do so takes the conclusion into the realm of rigorous science at a time when so little is known that it easily bogs down in "what if?" theorizing. It's not really a very intellectually satisfying situation, but as with much in law, it works and is of high value, not matter how much or how little one expects the learn more in the future.


* The ichthyologist is not a scientist when he goes fishing and examines his catch and pronounces that he believes this is a particular species of carp. He's an expert, but he hasn't concluded from rigorous science. It's an interesting analogy, because the scientific truth could be determined in this case, so his opinion at that point would be inadequate to convict of illegal carp fishing, since DNA would reveal the truth, and we need not rely on lesser proof. If he tried to testify to his spot opinion, he would be challenged for not applying the available and highly reliable scientific test. LPE's are sometimes challenged with the assertion that they cannot really know the truth. But no such reliable scientific test is available, so the courts hear the best that is available.

[Charles Parker]

Gerald I was going to post a response but I think in your discussion you nailed down what I thought

I have seen people testify to absolute certainty and it made me cringe when they did it. But they never got challenged on it.

Me----I prefer to say that that the latent print and the inked print came or originated from the same source.

Is that the same as an absolute conclusion? If it is then I guess I am going to have to change it.

[kevin]

Me----I prefer to say that that the latent print and the inked print came or originated from the same source.

Depends on how the attorney is asking the question -i.e. if he askes 'What was your conclusion?' I think it is perfectly acceptable to say you found them to have come from one and the same source as it is a reflection of your interpretation of the data available. It is still your expert opinion that is being presented and I believe the memo was intended to avoid statements like 'I'm 80% sure' or 'there is only a 1 in 10 billion chance this could have been made by another individual...'

More importantly I think Gerald hit on a very good point;

If we demanded the kind of proof demanded by the scientific community in support of claimed knowledge, we would be forever single and renting.

I intend to apply this notion in my personal life rather than the proof beyond a reasonable doubt course as it seems to have worked out for me alot better in the past...

[George Reis]

I wish I could understand what Gerald was talking about.

But, looking at the two quotes, and not knowing the full context, I think that Bill is pretty clear, and that he likely disagrees with the other statement. I'll drop him a note and see if he plans to comment.

I really prefer language that I can understand - and I usually have no problem understanding Bill Leo.

George

[Michele]

Gerald,
If I understand you correctly, I think you’re saying that conclusions aren’t facts but they are a reliable form of identification. We could even extend that to say that conclusions aren’t proven absolutely but it’s proven beyond a reasonable doubt. I understand this but I think that when someone goes into court using the word ‘positive proof’ then the attorneys, judges, and jury could easily think this means that the conclusion is a fact. I would even go so far as to say that using some words may be falsely influencing the courts.

This difference in meanings (with the word proof) could also be misunderstood by practitioners in our own discipline. I just wonder how many practitioners may be reading Bill’s article, not knowing about the other IAI message, and thinking that he’s talking about absolute proof? If practitioners are thinking this then they would easily go into court thinking conclusions are absolute.

I’m just wondering if the message that our discipline is sending is as clear as we think it is.

A statement that the analyst has formed an opinion that an individual is the source of an impression and that he is certain and confident in that opinion is entirely accurate. Everyone in law understands the nature of expert opinion.

Back in February I attended a presentation by Judge Harry Edwards, the co-chair of the NAS committee. He mentioned that during his career he’s heard hundreds of forensic practitioners testify and he always thought that these people were stating facts. After participating in the NAS research, he said he now feels like he was ‘snookered’ into believing this. If such a prominent judge feels this way, I really don’t think that even a small fragment of people in law understand the nature of expert opinion. I’m not faulting them for misunderstanding, I kind of think the different forensic disciplines have helped perpetuate this kind of thinking.

[Gerald Clough]

Gerald,
If I understand you correctly, I think you’re saying that conclusions aren’t facts but they are a reliable form of identification. We could even extend that to say that conclusions aren’t proven absolutely but it’s proven beyond a reasonable doubt. I understand this but I think that when someone goes into court using the word ‘positive proof’ then the attorneys, judges, and jury could easily think this means that the conclusion is a fact. I would even go so far as to say that using some words may be falsely influencing the courts.

The conclusions are the currently most reliable products of fingerprint evidence. Your reference to the conclusions being proof beyond a reasonable doubt is, I think, pretty accurate. I don't think attorneys and judges have any problem with this. They understand exactly the nature of the kinds of testimony presented by experts. There is, of course, some expert testimony that conveys knowledge that is fully "scientific" in that it would be accepted by most scientists as the foundation for the next step in learning about the physical world. An expert who explains the nature of DNA, for instance, is stating current scientific fact, the fact that leads to the applied analysis that also doesn't conclude to a certainty. But the scientific fact testimony is usually merely an explanation made to provide background for other information. To put it into our realm, one might testify to the nature of the development of human friction ridge skin, knowledge gained from definitive research that is generally accepted as fact and something upon which a researcher would depend in forming up a next study. But those things aren't really the issues in lawsuits. The lawsuits are over differences of opinion as to facts. And those are settled by the finders of fact judging the relative credibility of opposing experts by judging which analysis more likely concludes what the truth would be if it were absolutely known.

This difference in meanings (with the word proof) could also be misunderstood by practitioners in our own discipline. I just wonder how many practitioners may be reading Bill’s article, not knowing about the other IAI message, and thinking that he’s talking about absolute proof? If practitioners are thinking this then they would easily go into court thinking conclusions are absolute.

I’m just wondering if the message that our discipline is sending is as clear as we think it is.

It's not clear. I don't even think there's a universal consensus on what the message is supposed to be. (We're not all thinking about it much. This forum represents a tiny subset of examiners.) This is often an issue that arises in appeals when a prosecutor restates and summarizes an expert's testimony. Use of words like "match" in summation when one of the weaker conclusions, such as those properly given in visual analysis of hair and fiber, were presented has caused endless trouble. (The term "consistent with" is wholly evil and should never be used, at least not without a fully developed qualification as to what it meant.) Examiners are supposed to be experts, which means they should understand exactly what their analysis means and what they want to say and how to say it. If someone depends on someone else saying this or that type of conclusion or language is appropriate, they're no expert. But it may be literally true that the examiner deems his analysis to reveal absolute truth absolutely in a scientific sense. I believe he is wrong, not wrong about the ultimate fact at issue but about how absolutely it should be accepted. You may believe he is wrong. Obviously, the IAI believes that's wrong. But, if that his opinion, that's what he may state. That is the nature of expert testimony. On a given issue, one expert may conclude with the belief that his conclusion points strongly to the truth. Another may conclude from his analysis of the same that he has discovered absolute truth. Yet another (and it does happen) states he has concluded absolutely because he is strongly biased toward a particular finding in a case. But it is no bad thing for an analyst to state that he has examined the evidence and that it convinced him that one individual is the source. If someone doesn't agree, they can themselves testify to why they aren't convinced.

Is it fatal when they get it wrong, when they misstate or inadvertently imply that they are stating absolute fact? Probably not. If the analysis was flawed, it's certainly important that it be revealed by appropriate opposition, but we're talking here about what is still the most reliable available method of determining, to whatever certainty, the actual factual truth. The situation of a prosecutor misstating an expert's conclusion is an entirely different matter, since it subverts the communication between expert and jury.

Back in February I attended a presentation by Judge Harry Edwards, the co-chair of the NAS committee. He mentioned that during his career he’s heard hundreds of forensic practitioners testify and he always thought that these people were stating facts. After participating in the NAS research, he said he now feels like he was ‘snookered’ into believing this.

From that statement, standing alone, I would think he's not quite getting it. He's been a long time around courts and lawsuits. I suspect he understands the nature of expert testimony. What I think that statement reflects is one problem with the NAS report. It's an unavoidable problem, but a problem nevertheless. The problem is that the contents seems to come as a revelation to many, the revelation that there's a lot more at issue in fingerprint identification and many other disciplines than they knew. Fingerprint identification occupies a unique position in criminal justice that's worth examining.

It's the most commonly applied forensic analysis.

It's very powerful, in that it points the accusing finger at a specific individual among all others.

There are many practitioners, practicing in many, many small jurisdictions. In short, it's cheap to do.

The education and training of practitioners varies wildly. Granted, a single formal educational prerequisite by no means makes other fields' conclusions any more likely to reflect reality. (in fact, many fields requiring doctoral degrees to qualify are represented in cases in which half the experts are dead wrong. On the other hand, maybe those are the ones that should have the highest requirements.)

It's venerable. It has a long history of conclusions being almost entirely unopposed in court. Any number of defendants, including some innocents, have pleaded Guilty on advice of counsel solely because of a fingerprint identification.


That status has moved us to agonize over what we mean. We know we're largely unopposed. Scientists who are offended by the implication that the conclusions are taken to be absolute fact know we're unopposed, and they're alarmed. I'd think them poor stewards of scientific knowledge if they weren't alarmed. Some in the legal community pretend to be shocked. They're not. They know the game. They do welcome the opportunity to develop some appropriate opposition.

Fingerprint was, until the advent of DNA analysis, the gold standard of forensic identification. Someone who never thought about it, never looked into it, and never heard argument because examiners were rarely opposed, can easily look at the Report's recitation of issues and think they were fooled, that the gold in the standard was really brass. This is not much the fault of the fingerprint field. If there's fault, it lies in the legal machinery. And I happen to think that the essence of the Report is that in the forensic disciplines, we can do better, we can known more, and our tendency has always been to find a comfort level of knowledge and settle into it. I think the Report recognizes the potential vulnerabilities of the legal system and wants to do better.

[NYSLPE]

This discussion is very timely for me. I will be taking an advanced ridgeology course with John Black (Ron Smith and Associates) next week (11/16 -11/23). A co-worker has told me that he disagrees with the whole premise of not being 100%. He disagrees with the way Josh testifyed in Minn. vs. Hull. This is an expert teaching latent print examiners. I would say the message being sent out is clearly wrong or is this the way it has to be taught until the IAI and SWGFAST take more of a stand? I know what I believe but I am not sure what I am supposed to believe at this point.

[g.]

Hi NYSLPE,

I can understand your confusion and conflict. Been there, done that. I have some papers I could share with you that might help. Of course for me it took 3-4 years of struggling with this issue before settling comfortably into my current skin. Testifying confidently, yet avoiding "absolutes" has been very freeing and made my time in court actually easier. For me, it was difficult trying to defend the indefensible. Letting it go became easier with practice and in the end, has not at all diminshed the evidence in the courtroom. You can visit The Detail Archives no. 379 for more info on how I answer questions about "absolute certainty" while avoiding, as Cole has described, "the Examiner Fallacy".

I also can tell you that this is a divided subject, as you clearly are experiencing with your colleague. Many on this board will have sharply opposite opinions on the matter. I respect and understand their position, since I used to share it as well, but I have had a significant change of philosophy on this issue. Much of this influence has come from outside this profession, where you'd be hard pressed to find absolute conclusions and absolute certainty in other areas of science.

Good luck with your struggles. Contact me offline for those papers if you are interested.

glenn.langenburg@state.mn.us

PS-as a further plug for RS&A, we cover this issue quite a bit, especially given its relevance to the NAS report criticism, in the AAA class offered by RS&A. And although we don't offer it as much, but have given some IAI workshops on this issue, it is a big theme in "Statistics Applied to ACE-V" course that Cedric Neumann and I have taught a few times.

g.

[Gerald Clough]

I know what I believe but I am not sure what I am supposed to believe at this point.

That's an interesting response, I think, because it seems to me to be a very common feeling. I believe, though, that if one is genuinely an expert, one is both qualified and bound to represent your own well-formed belief. The reality is that nothing in this universe can be absolutely known. But I have no problem with an expert who expresses a professional opinion, from the expert's assessment of both the particular evidence and the available body of knowledge relevant to the field, of the highest possible confidence in the truth of an absolute statement. Whether it is expressed as a plain fact or whether it is stated as the expert's opinion that it is fact, it always amounts to the same thing, an opinion. Omission of the word "opinion" does not grant the expert omniscience, and no one would read it that way. I do believe that, in the forensic fields because of the stakes, only the highest fully formed and rationalized conclusion should be adopted. If one cannot articulate why one is being more positive, one should limit to the more conservative sustainable opinion.

I expect an expert to be well versed in the various issues within the field and to have evaluated the arguments for themselves. I expect the expert to be prepared to articulate a rationale for their opinion. Very often, expert conviction must be explained through less than rigorous application of scientific knowledge. That explanation must be logical. The logic may include expert estimates and arguments based upon negative results, but it cannot, given the assumption that the estimates and meaning of negative results are valid, violate plain logic. Experts may and very often do differ on their beliefs on if and how certain knowledge and experiences should be used.* But, as experts, each must form their individual opinion. It's not a matter of what they're supposed to believe. If the opinion is a minority opinion, so be it. If the expert can't sustain some credibility for that opinion in the face of opposition, they're either not much of an expert or they just haven't really reached an opinion with sufficient confidence or they lack the fortitude to represent expertise. (Or maybe they just haven't yet managed to articulate their views well.)

* An example might be found among the experts I often deal with on the issue of mental retardation, a common issue in death penalty cases. One scientist (Flynn), himself not a psychologist, describes the results of his study of historical IQ scores that he speculates may show that, for a number of possible reasons, IQ scores using a given test will increase year by year. (The rationale is that, if this weren't so, nearly the entire population in 1900 would have been retarded, an absurdity.) Some experts believe this observation (the "Flynn Effect") means the IQ score threshold for retardation must be increased each year as a particular test continues in use. Others disagree. This, of course, can have a profound effect on whether one or another expert declares an individual to be mentally retarded. Both experts are applying all available knowledge in forming their respective opinions. And both can articulate their rationale. Neither position is inherently weak, except in the opinion of the other side. But neither is adopting the position he is "supposed to." And since both views and middle views are held in academia, presumably both disagree with one or another of their professors. Their ability to do that rationally is part of why they are experts and not mere technicians. It is the burden of expertise, which is not a herd function.