erroneous ident question

[kevin]

Kasey,

Let me break this down so we all understand it better:

An ident which we should not ident is not an ident, per say, unless said ident was subsequently identified independently. Should said subsequent examination include an inquiry as the the initial examiners individualization based on the grounds that the impression is in fact inconclusive or otherwise indeterminate, said identification should be reviewed in accordance with an agency's rules and regualtions. See, simple as that....

Actually the only scenario I can think of that would fit the bill in the case Ashley described is if an examiner calls it an ident, then finds out it is of the wrong pattern type or a different part of the palm (lets say due to cropping of a photo)....that still shows bad judgement by the examiner and I would consider it to be a bad ident because you can demonstrate it did NOT come from the area of FR skin the examiner in question said it did. If you can demonstrate the mark couldn't come from a certain area of friction ridge skin then I would still say bad ident on that examiner.

[Gerald Clough]

The original question did indeed ask it as if there was general agreement that it should have been inconclusive. To call it "erroneous" puts it, because of our conventions, into the category of misidentifications, identifications of a latent as being an impression of a particular source skin portion which it demonstrably was not made by. That is not the case here. You can look at this from the point of view of an examiner who has made an identification and who now accepts that it should not have been made because there is insufficient data and wants to know what to call his original result. I don't much care what it's called. Perhaps the most accurate statement is, "I have been convinced by the demonstrations and arguments of other examiners that there was insufficient detail to justify the identification." But the same examiner may well not be convinced by such demonstration and argument and say, "My opinion remains that it was sufficient and that it is an identification." Now many other examiners must you set against the first to declare that he is wrong? Two? Five? Ten? Or if I believe, after all demonstration, that my opinion that the data is inconclusive, can you declare me wrong? You cannot. You can only say that you believe the data is sufficient. (Well, in reality, you will, if pinned down, declare your belief that I am wrong, but within the practice of examination, you cannot accuse me of error.)

Assuming an examination was objective, free from external influences that significantly influences interpretations and decisions, it cannot ever be anything but a difference of opinion. And since the effect of external influences cannot be determined (except in one's opinion), it is always a difference of opinion. (Keep in mind we are talking here about a strict difference over sufficiency, not frank misidentification.) This is the weakness in the convention of absolute identification. It is a weakness, because it inevitably admits the possibility of difference of opinion as to the conclusion. This is not to say that even a statistically validated probabalistic conclusion is immune, since during analysis, judgments were required as to the character of features that could still be argued. It is not a matter of one must be right and the other wrong, the either-or of whether or not the impression was actually made by a particular portion of individual skin. As 16's contribution points up, unless you can determine just how the detector is calibrated, you can't declare one of the other detector more accurate. And, so far as sufficiency thresholds for absolute identification, you cannot know much about the human detector. It therefore becomes, as do most things in legal affairs, a matter for the finder of fact.

The more serious ethical and legal question is, when there is a difference of opinion (even assuming one or the other isn't overtly or more subtly intimidated and suppresses their opinion) will that descent be reported?

[Kathleen Birnbaum]

Perhaps I am missing something here but if an examiner is convinced of an identification, they saw all the information they needed to call it an ident - then the fact that another examiner looks at it and just doesn't see enough detail to convince them it is an Ident and therefore says Inconclusive .... I am not seeing a problem with that. If you are able to see it and you are convinced, then you call it an Ident. If you do NOT see it and you are not convinced, you do NOT call it an Ident. There is no information given in the original question regarding training, abilities, or anything else. What I might see as an ident in a difficult print, I would not necessarily expect my trainee to see it. Is she wrong? I would consider it wrong for her to make the ident if she can't see it. As long as she isn't saying it is an Erroneous decision (a whole different super serious issue), I would think that an Inconclusive would be very reasonable. Maybe she sees some of what I see but doesn't see all of the details I used such as those in the darker area, whatever. Inconclusive can be you see some but just not enough to make that Ident. So many issues can be involved with that. Maybe Conflict Resolution is needed, maybe just taking the Ident to someone more experienced with difficult latents versus a trainee would be appropriate. Without just a generic question and no background, I am just not seeing a real issue at base level. If anything, I give the Inconclusive person credit for not going along with verifying an Ident they did not see. That is what Verifying is about. They didn't say it was wrong, just that they didn't see it to call it an Ident. Hopefully I said this clearly but perhaps I missed what everyone else was getting out of the discussion.

[Charles Parker]

Kathleen Said

if an examiner is convinced of an identification, they saw all the information they needed to call it an ident - then the fact that another examiner looks at it and just doesn't see enough detail to convince them it is an Ident and therefore says Inconclusive .... I am not seeing a problem with that.

Kathleen, I agree---your statement is closer to the point I was trying to make.

Gerald Said

To call it "erroneous" puts it, because of our conventions, into the category of misidentifications, identifications of a latent as being an impression of a particular source skin portion which it demonstrably was not made by. That is not the case here.

Gerald you are right---it is not erroneous and should not be described as such. I do not believe it is erroneous.

[minu]

Mr. Leo also had a little test at the end his article which goes:
The Fingerprint Examiner’s Subjective Test
1. Is your conclusion based on A or B?
A. An examination and comparison of the features of the prints
B. Your personal perception, feeling, or intentions.
2. Can your conclusion be verified?
A. Yes
B. No
3. Can you show or demonstrate to another examiner what you examined that provided you with the information that allowed you to form a conclusion?
A. Yes
B. No
If you answered “A” to the above three questions, you have proven that the conclusion of a friction ridge comparison is an objective conclusion, not a subjective one. If you answered “B” to any or all of the above three questions, perhaps you should re-evaluate how you are doing friction ridge examinations, or you could become a palm reader.

The problems with fingerprint examination are that different LPE cannot make completely consistent detailed analysis results, detailed comparison results and detailed evaluation results. Detailed analysis results means three level features, confidence of all three level features, and quality map (see CDEFFS document). Detailed comparison results mean correspondence between all three level features. Detailed evaluation results mean the random match probability in some population databases.

For most human behaviors, objective or subjective is just a matter of degree. Calling a technique objective does not mean that it does not make error. Is forensic DNA typing objective or subjective? This does not matter. The key thing is to use a standard, repeatable (in details) and improvable approach. According to these requirements, DNA typing is better than fingerprint examination.

Some critics say that fingerprint examination is more like an art (or Olympic) than a science. One feature to distinguish science with art is that science can be improved, but art cannot be improved (except that an artist can improve his/her skill). The question is: Can tomorrow's LPE do much better than today's LPE? If not, the current visual fingerprint examination approach itself need improvement. This is not an issue about the competency or experience of a LPE.

Jianjiang Feng
MSU

[minu]

If you have a latent print that is said to be an ident and it in fact is not an ident it is erroneous. But, what if you have a latent print that is said to be an ident, but it should have been ruled as inconclusive what is it called then? Since they could technically be correct.

Wow. OK.
The original question provided the assumption that the comparison "should have been ruled as inconclusive". So staying with your premise - that this truly is an inconclusive, and nobody should identify it - I have seen a couple of suggestions.
Call it:
Carl: "Differing opinions"
16 points: "Incorrect identification"
Charles Parker: "Erroneous sufficiency determination"
David Johnson: "Unverified ID"

Let's just talk about the Comparison and Evaluation steps in ACE-V procedure. In the comparison step, LPE checks if two prints match or not. If they match, LPE goes to the evaluation step and evaluates if there are sufficient evidences for an identification.

An erroneous identification can happen due to (i) in the comparison step, unexplainable inconsistencies are not noticed, and/or (ii) the sufficiency of consistent evidences is overestimated.

I think both errors can be termed erroneous identification/individualization. But if you want to distinguish two types of errors, I would suggest erroneous match for the error happening in the comparison step, and erroneous individualization (due to underestimation of random match probability) for the error happening in the evaluation step.

Another thing is that I do not think the term for errors should be dependent on "the fact" (two prints are from the same or different sources). There should be some methods (consensus of LPE or standards) for determining whether a latent examination conclusion in case works is correct or not, without knowing "the fact". For example, the Spain LPE thought of the identification made by the FBI LPE in Madrid bombing as an error before they knew "the fact". Of cause, we need to know "the fact" for gaining experience or building standards. But for real case works, consensus of LPE or standards can be used to detect possible error.

Verification of other LPE has been a main method for controlling quality. But my concern is that better standards are also necessary. This needs a more clear, detailed, and quantitative description of the ACE procedure. Such as,

• What are various ridge events (more detailed, how to compute confidence of ridge event)?

• What is a match and what is not a match (more detailed, what is explainable difference (more detailed, what is explainable difference in distance between two pairs of corresponding minutiae))?

• What is sufficient evidence and what is insufficient evidence (more detailed, how to estimate the random match probability)?

I know it is very difficult to quantify the ACE procedure, especially because there is no sufficient support for this type of work. Without the 1 billion dollar funding, can forensic DNA typing make such a quick progress?

Jianjiang Feng
MSU

[David Johnson]

mimu said

I think both errors can be termed erroneous identification/individualization. But if you want to distinguish two types of errors, I would suggest erroneous match for the error happening in the comparison step, and erroneous individualization (due to underestimation of random match probability) for the error happening in the evaluation step.

You cant really say that. If you have a case where there is detail in agreement but there is not enough to identify you cant really know if the first examiner made a erroneous conclusion or not. You wernt there and you dont know who really made the print.

For example if you have four points (ahhh the point word) around a delta and the first examiner ID'ed it but your agency determines that 4 points around a delta is just not enough to ID a person then the first examiner did make an error. It is just not an erroneous identification in the scientific sence. If you agency has a policy where it is reviewed by another examiner or manager and they determine that it is an error that that would be a policy and peer group determination. However he might be right when he says that that particular person made the print. But if that person is allowed to keep making ID's on such little information then eventually he is going to make a mistake and misidentify someone. That is why we set our standard for ID so high.

Now if you have the lifeboat situation where you have a closed population like Glen often talks about then those same four points might be enough to ID the print.

[minu]

You cant really say that. If you have a case where there is detail in agreement but there is not enough to identify you cant really know if the first examiner made a erroneous conclusion or not. You wernt there and you dont know who really made the print.

Assume there is no option of inconclusive. If you permit a person A, who didn't really see the suspect made the print, to claim an individualization, why cannot another person B say A's conclusion is an erroneous individualization? For example, if A claims an individualization based on ONE minutia in a tiny latent (2mm x 2mm) without Level 3 features and other non-fingerprint evidence, I can claim it as an exclusion (namely A made an erroneous individualization) very confidently.

Jianjiang Feng
MSU

[RL Tavernaro]

... If you permit a person A, who didn't really see the suspect made the print, to claim an individualization, why cannot another person B say A's conclusion is an erroneous individualization? ....

If you call a tail a leg, how many legs has a dog? Five? No... calling a tail a leg does not make it a leg. (Attributed to Abraham Lincoln)

Yes, person B can say A's conclusion is erroneous, but that alone doesn't make it true.

There is a difference in making a call on insufficient evidence, and proving the conclusion wrong. When there are only 2 ground truth choices (one specific individual was, or was not, the source of one specific latent print), both choices have an equal 50% chance of being correct (assuming the latent print source was truly unknown, and no mitigating evidence was available at time of comparison). The weight of evidence in support of an identification (individualization) or exclusion serves to tip the balance in one direction or the other, however a threshold must be determined and reached/passed to form a positive conclusion (whether ID or no ID). Falling short of that threshold leaves a finding of inconclusive.

If one believes that threshold can be well-defined, and will always remain static, the question of 'inconclusive' becomes more black & white. A confident claim of 'erroneous individualization' may become achievable.

However, if one believes that threshold to be variable, including individual examiner variables such as training, experience, and ability, as well as perception of clarity at the time of comparison, then the question becomes more elusive. The standard of proof must then be well-defined, and remain relatively unchanged.
Regards, RLT

[Randall]

If there is error, the error is an error in the application of the evaluation phase in that there may not be sufficient uniqueness present in the latent print. Without knowing the ground truth, there is no erroneous identification but merely a difference of opinion between the two examiners.

The subsequent posts on this subject also piqued my interest.

Minu wrote:
“An erroneous ID can happen due to unexplainable inconsistencies are not noted or sufficiency of consistency is overestimated”

Translated - the previous statement means that discrepancies are overlooked or the examiner is guessing and is in big trouble.

Minu wrote:
“There should be some methods (consensus of LPE or standards) for determining whether a latent examination conclusion in case works is correct or not, without knowing "the fact". For example, the Spain LPE thought of the identification made by the FBI LPE in Madrid bombing as an error before they knew "the fact". . . consensus of LPE or standards can be used to detect possible error."

The Spanish National Police (SNP) knew there was an error because they were certain that the print being identified was not of the person indicated (by the FBI), i.e. the SNP did know “the fact”. The SNP also understood and knew another true fact, i.e. that fingerprints are unique. However, I do agree that “consensus of LPE or standards can be used to detect possible error” (emphasis added).

Minu wrote:
“. . . But my concern is that better standards are also necessary. This needs a more clear, detailed, and quantitative description of the ACE procedure. Such as,
What are various ridge events (more detailed, how to compute confidence of ridge event)?”

The first part of the previous question has plagued the fingerprint discipline for centuries. Purkinje, Galton, the FBI and others have defined fingerprint patterns and ridge events, while Salil Chatterjee described events of ridge edges. However, there is no agreement on the number or types of patterns that exist or on the number and types of ridge events that can occur.

Identifications using ACE are not dependent upon definitions of ridge details, but rather a determination that all ridge details in two prints are in agreement. The fingerprint examination by ACE is not a mathematical calculation, but rather a methodology that is a quantitative-qualitative process. As far as the confidence factor in the examination process, quality refers to the clarity of latent print. Clarity is determined by a variety of infinitely variable factors that affect the deposition and visualization of those latent prints. Clarity is a determining factor for the confidence placed on the reproduction of ridge detail.

ACE -The analytical phase-the print is analyzed relative to distortions, clarity and anatomical factors; the comparison phase (determination that the ridge details are in agreement); then the Evaluation phase in which a decision of whether or not there is sufficient uniqueness to demonstrate identity, i.e., can all other donors be eliminated as the possible source? Of course, this last determination is dependent on the examiner understanding and believing the basic premise of uniqueness.

Minu wrote:
“. . . . What is a match and what is not a match (more detailed, what is explainable difference (more detailed, what is explainable difference in distance between two pairs of corresponding minutiae)?”

A match or individualization (or identification dependent on your opinion of what constitutes either) is defined by SWGFAST, whose members are active practitioners of the fingerprint discipline and that represent many different departments.

A difference in the appearance of two prints is described as either a dissimilarity or a discrepancy. Definitions for dissimilarity and discrepancy are also published by SWGFAST, inasmuch as the term “explainable difference” was determined to be ambiguous.

Minu wrote:
“. . . .“What is sufficient evidence and what is insufficient evidence (more detailed, how to estimate the random match probability)?”

Sufficiency is an issue that latent print examiners grapple with on a daily if not hourly basis. Sufficiency is a integral part of fingerprint examination. The fingerprint examination methodology (ACE) is a quantitative-qualitative process, not merely a quantitative process. John Vanderkolk depicted sufficiency in the quantitative-qualitative process of the fingerprint discipline using a graph and shades of grey. Sufficiency is a grey issue that cannot be reduced to numbers and a mathematical equation nor is it definable in black/white terms.

(By the way, random match probability is a term used in connection with DNA matching and refers to the probability of a particular multiple-locus genotype.)

[minu]

However, if one believes that threshold to be variable, including individual examiner variables such as training, experience, and ability, as well as perception of clarity at the time of comparison, then the question becomes more elusive. The standard of proof must then be well-defined, and remain relatively unchanged.

Ideally there is only one theorem, Bayes theorem, which is followed by both the examiner and the person who evaluates examiner's conclusion. (I am just talking about the evaluation step in ACE protocol).

What I want to say is that the evaluation step itself needs more check. It seems to me that now only the categorized conclusion itself is relevant. As long as the categorized conclusion made by the examiner is consistent with the verifier, we think the examiner is doing good. But, a LPE may be lucky for many years and one day make an erroneous identification on a latent near to his/her gray area. So I think it is more important to check the method and data for drawing the conclusion.

The variation among different examiners in experience and ability does make the problem complicated. But I feel that this is not the most difficult part. In a forensic discipline, several approaches can coexist. Some approaches may be more discriminating; some may be less discriminating. For example, in DNA typing, the markers used in different countries or even in the same country may be different. Examiner A use 10 STR markers, examiner B uses 13 STR markers, examiner C uses 2 RFLP markers. As long as each examiner correctly does the analysis and comparison, and if match is found, he/she reports random match probability estimated using corresponding DNA database, there is not much problem for a lay person to understand quantitatively how strong a DNA match is and compare the relative strength of several DNA evidences which are analyzed using different approaches.

Jianjiang Feng
MSU