detail 233
-
Michele
- Posts: 384
- Joined: Tue Dec 06, 2005 10:40 am
detail 233
When I first started in this industry, fingerprint identifications had become so ingrained in our society that training seemed lacks compared to what it is today (this is just my opinion). I wasn’t fast tracked through training, but the materials I studied were copies of copies and didn’t really give a complete picture of our industry. Believe it or not, the first fingerprint book I read (besides the Science of Fingerprints) was Simon Cole’s book, “Suspect Identities”. I thought the book was well written, easy to read, and had more information than I knew existed. Even though Ashbaugh’s book had been published at this time (as well as several others), I was unaware of its existence. Even after finding out about other books that had valuable information, I always thought Cole’s book was a great resource (especially considering the cost of other fingerprint books).
Over the years many people have criticized Cole. They site errors in his book and state that he isn’t a fingerprint practitioner. Sometimes his arguments overstate the facts or leave out key information but regardless I’ve always found value in his work. Cole’s criticisms have had quite an impact on our industry. In my opinion, Cole’s criticisms (along with other events) have made us better examiners. I think we try harder, have a deeper understanding, and can better articulate what we do.
As noted before, I’ve noticed that Cole has overstated problems in previous articles and left out key information. It’s hard for me to criticize this when I’ve seen the same tendency from people who are fingerprint practitioners. I guess playing the devil’s advocate is sometimes needed to wake people up and make them understand how things could be viewed. If I’m reading this correctly, I think this recent article just goes too far. To sensationalize the situation in the name of justice is sad. This is not playing the devils advocate, but intentionally selling a distorted view for ones own private gain. I’m disappointed in Cole and I’m disappointed in anyone in the scientific community who proof read this paper. Are there people who still believe the end justifies the means? I would hope people value their integrity more than that.
I won’t go into all the issues, but I will mention a few. 1st, Cole uses Imwinkelried’s definition of ‘a validation study’. I have no problem with this definition. I do have a problem with Cole ignoring his own source and the editors failing to notice it. The definition used is:
“A validity study is designed to measure the accuracy of a scientific
technique. The study attempts to identify and quantify the inherent margin
of error in the technique; the researcher inquires how often the technique
will yield inaccurate results even when the analyst strictly follows proper
test procedure.”
Later in the article it’s states:
“A proficiency study is radically different. While a validity study tests a
scientific technique, in a proficiency study the object of the test is a
particular analyst or laboratory. The validity test is designed to ensure to
the extent possible that correct test procedures are used; in a validity
test, the researchers attempt to eliminate any concern about the use of a
proper test procedure because they want to reach the central question of how often the scientific technique itself will produce inaccurate results
despite proper test protocol. In contrast, a proficiency study endeavors to
measure the analyst’s proficiency in the sense of the probability that he or
she will consistently use proper test procedure. Assuming the validity of
the technique, the researcher inquires into the probability that while using
the technique, the analyst will “mistakenly use the wrong materials or make the wrong measurement. . . .” Even when the analyst is utilizing a valid technique, the analyst might commit a “performance–type error”. (1995: 1255, citations omitted) [3]”
Through these definitions, I think it’s obvious that Imwinkelried is saying that a validation study is testing the methodology, not the practitioner. But Cole continues to equate a validation study with the accuracy of the practitioner. Not once does this article mention ACE-V.
I can see that some people might claim “we are the method”. This is slightly true. We are one system using another system to insure accurate results. We use ACE-V much like a computer uses an operating system. The better the operating system, the better the results. The computer itself can’t be measured without knowing what internal system is being used. Ignoring that computers are using an operating system is the same as Cole ignoring that we are coming to a conclusions by using ACE-V.
In the conclusion Cole states, “…many practitioners and defenders of forensic fingerprint identification still do not understand what is meant by the demand for validation studies.” With Cole not using his own source, I’d say he’s adding to the confusion, not clarifying any misunderstandings. He continues, “….the misunderstanding may be deliberate.” Well isn’t that the pot calling the kettle black!!
I also want to mention some quotes to support my claim of sensationalizing or overstating the situation.
“Fingerprint proponents claim that latent print examiners underperform on proficiency test because they take them less seriously than casework.”
Someone may have stated this, but I doubt if fingerprint proponents in general feel this way.
“Indeed, fingerprint proponents have reacted with outrage when others have sought to use proficiency test data as a provisional measure of the accuracy…..”
Really? Outrage? I haven’t ever heard anyone react this way about proficiency tests.
I do know that fingerprint proponents have given reasons why the CTS numbers shouldn’t be used but not proficiency test in general. (for the sake of time I’ll refrain from stating why the CTS numbers would give a false estimate)
“Although uniqueness cannot be proven,
only inferred, if one does infer it, one can then falsely reason that
fingerprint identification is infallible.”
There’s a huge difference between stating that fingerprints are infallible, and fingerprint practitioners are infallible. Is this that hard of a concept to understand? Obviously it’s harder than I would have thought, many critics seems to keep insinuating that these are the same thing.
From this article, it not only looks like we need appropriate validation studies but it appears that Cole is the only person who is educated enough to decide what is and isn’t appropriate. Somewhere I read that one of the most common errors in science is giving too much weight to your own conclusions. This makes you think additional information isn’t needed or valued. This sure seems to apply to this article.
Over the years many people have criticized Cole. They site errors in his book and state that he isn’t a fingerprint practitioner. Sometimes his arguments overstate the facts or leave out key information but regardless I’ve always found value in his work. Cole’s criticisms have had quite an impact on our industry. In my opinion, Cole’s criticisms (along with other events) have made us better examiners. I think we try harder, have a deeper understanding, and can better articulate what we do.
As noted before, I’ve noticed that Cole has overstated problems in previous articles and left out key information. It’s hard for me to criticize this when I’ve seen the same tendency from people who are fingerprint practitioners. I guess playing the devil’s advocate is sometimes needed to wake people up and make them understand how things could be viewed. If I’m reading this correctly, I think this recent article just goes too far. To sensationalize the situation in the name of justice is sad. This is not playing the devils advocate, but intentionally selling a distorted view for ones own private gain. I’m disappointed in Cole and I’m disappointed in anyone in the scientific community who proof read this paper. Are there people who still believe the end justifies the means? I would hope people value their integrity more than that.
I won’t go into all the issues, but I will mention a few. 1st, Cole uses Imwinkelried’s definition of ‘a validation study’. I have no problem with this definition. I do have a problem with Cole ignoring his own source and the editors failing to notice it. The definition used is:
“A validity study is designed to measure the accuracy of a scientific
technique. The study attempts to identify and quantify the inherent margin
of error in the technique; the researcher inquires how often the technique
will yield inaccurate results even when the analyst strictly follows proper
test procedure.”
Later in the article it’s states:
“A proficiency study is radically different. While a validity study tests a
scientific technique, in a proficiency study the object of the test is a
particular analyst or laboratory. The validity test is designed to ensure to
the extent possible that correct test procedures are used; in a validity
test, the researchers attempt to eliminate any concern about the use of a
proper test procedure because they want to reach the central question of how often the scientific technique itself will produce inaccurate results
despite proper test protocol. In contrast, a proficiency study endeavors to
measure the analyst’s proficiency in the sense of the probability that he or
she will consistently use proper test procedure. Assuming the validity of
the technique, the researcher inquires into the probability that while using
the technique, the analyst will “mistakenly use the wrong materials or make the wrong measurement. . . .” Even when the analyst is utilizing a valid technique, the analyst might commit a “performance–type error”. (1995: 1255, citations omitted) [3]”
Through these definitions, I think it’s obvious that Imwinkelried is saying that a validation study is testing the methodology, not the practitioner. But Cole continues to equate a validation study with the accuracy of the practitioner. Not once does this article mention ACE-V.
I can see that some people might claim “we are the method”. This is slightly true. We are one system using another system to insure accurate results. We use ACE-V much like a computer uses an operating system. The better the operating system, the better the results. The computer itself can’t be measured without knowing what internal system is being used. Ignoring that computers are using an operating system is the same as Cole ignoring that we are coming to a conclusions by using ACE-V.
In the conclusion Cole states, “…many practitioners and defenders of forensic fingerprint identification still do not understand what is meant by the demand for validation studies.” With Cole not using his own source, I’d say he’s adding to the confusion, not clarifying any misunderstandings. He continues, “….the misunderstanding may be deliberate.” Well isn’t that the pot calling the kettle black!!
I also want to mention some quotes to support my claim of sensationalizing or overstating the situation.
“Fingerprint proponents claim that latent print examiners underperform on proficiency test because they take them less seriously than casework.”
Someone may have stated this, but I doubt if fingerprint proponents in general feel this way.
“Indeed, fingerprint proponents have reacted with outrage when others have sought to use proficiency test data as a provisional measure of the accuracy…..”
Really? Outrage? I haven’t ever heard anyone react this way about proficiency tests.
I do know that fingerprint proponents have given reasons why the CTS numbers shouldn’t be used but not proficiency test in general. (for the sake of time I’ll refrain from stating why the CTS numbers would give a false estimate)
“Although uniqueness cannot be proven,
only inferred, if one does infer it, one can then falsely reason that
fingerprint identification is infallible.”
There’s a huge difference between stating that fingerprints are infallible, and fingerprint practitioners are infallible. Is this that hard of a concept to understand? Obviously it’s harder than I would have thought, many critics seems to keep insinuating that these are the same thing.
From this article, it not only looks like we need appropriate validation studies but it appears that Cole is the only person who is educated enough to decide what is and isn’t appropriate. Somewhere I read that one of the most common errors in science is giving too much weight to your own conclusions. This makes you think additional information isn’t needed or valued. This sure seems to apply to this article.
Michele
The best way to escape from a problem is to solve it. Alan Saporta
There is nothing so useless as doing efficiently that which should not be done at all. Peter Drucker
(Applies to a full A prior to C and blind verification)
The best way to escape from a problem is to solve it. Alan Saporta
There is nothing so useless as doing efficiently that which should not be done at all. Peter Drucker
(Applies to a full A prior to C and blind verification)
-
L.J.Steele
- Posts: 430
- Joined: Mon Aug 22, 2005 6:26 am
- Location: Massachusetts
- Contact:
Re: detail 233
Partly this comes from some testimony from examiners who assert essentially that: the method is 100% reliable. I, an experienced examiner, have never made a documented mistake. Therefore, I + ACE-V = 100% reliable match.Michele Triplett wrote:There’s a huge difference between stating that fingerprints are infallible, and fingerprint practitioners are infallible. Is this that hard of a concept to understand? Obviously it’s harder than I would have thought, many critics seems to keep insinuating that these are the same thing.
Problem is essentially the one raised by Dr. Dror in some of the post-Mayfield reports. Part of the method involves a black-box comparison in your brain about what's an explainable distortion and what's a dissimiliarity leading to a non-match result. I'd agree that Dror's sample size is small, at least in the current published study (he's got a larger one in the publication process), but there are some serious potential problems with confirmation bias in the ACE part of the analysis and in non-blind Verifications. I just have troubles accepting that any system that depends on the human brain to make judgment calls about what is and isn't a match can every be 100% infallible.
I can see Cole's point in his prior article that you can't seperate the method from the examiner here, as you could if the test were wholly mechanical or chemical and that examiner error is part and parcel of whether latent print matching is 100% reliable. IMHO, in the vast majority of cases is it likely very reliable, tho not 100%. (Heck, almost nothing in life is 100% reliable.)
-
Les Bush
- Posts: 229
- Joined: Tue Jul 05, 2005 4:29 am
- Location: Australia
Thanks Michelle
Thanks Michelle, you've saved me about an hours work in putting together a similar response to the latest from Cole. I agree with your position. I can also see where Lisa is coming from but I disagree with her position on what the human brain is capable of. As a lawyer she has inherited the usual lack of trust in anything but what her own senses can believe, the St Thomas principle. Thankfully we have experts in fields of science who are prepared to be professional and competent in conducting their analysis and being subjected to adversarial testing at court.
My thoughts on Coles latest was he had fallen into the trap of unfair atomistic criticism. Fingerprint science is about ALL aspects of how human fingerprints can be used for identification. Individualisation is part of that outcome. Cole choses to reduce his field of criticism to the issue of difficult latent examinations. But in getting there he attempts to drag the whole of fingerprint science into the same pot. If Simon is reading this or one of his proponents then my message is that you have to be fair. Simon as a social scientist should understand the principle of fairness.
Finally, if Simon is so full of the correct way to conduct a validation study then lets give him the opportunity. Fingerprint science begins with the study of many ten print impressions and historically we have evidence that poorly educated staff can complete ten print examinations if properly instructed. Off you go Simon, use your energies for world good and experience what it is to be able to individualize. When you have got a handle on ten prints then we will introduce easy clear latents, then average latents and finally difficult latents. If your enthusiasm is still boundless we can even throw in a few court cases to see how you explain what you achieved. Simon, take down all those yellow Post It notes about what is wrong with fingerprints and put up new ones about how to help us overcome the curse of criticism. The world would be a better place. Les
My thoughts on Coles latest was he had fallen into the trap of unfair atomistic criticism. Fingerprint science is about ALL aspects of how human fingerprints can be used for identification. Individualisation is part of that outcome. Cole choses to reduce his field of criticism to the issue of difficult latent examinations. But in getting there he attempts to drag the whole of fingerprint science into the same pot. If Simon is reading this or one of his proponents then my message is that you have to be fair. Simon as a social scientist should understand the principle of fairness.
Finally, if Simon is so full of the correct way to conduct a validation study then lets give him the opportunity. Fingerprint science begins with the study of many ten print impressions and historically we have evidence that poorly educated staff can complete ten print examinations if properly instructed. Off you go Simon, use your energies for world good and experience what it is to be able to individualize. When you have got a handle on ten prints then we will introduce easy clear latents, then average latents and finally difficult latents. If your enthusiasm is still boundless we can even throw in a few court cases to see how you explain what you achieved. Simon, take down all those yellow Post It notes about what is wrong with fingerprints and put up new ones about how to help us overcome the curse of criticism. The world would be a better place. Les
-
L.J.Steele
- Posts: 430
- Joined: Mon Aug 22, 2005 6:26 am
- Location: Massachusetts
- Contact:
Re: Thanks Michelle
Actually, Les, just the opposite. Having spent many years reading studies on eyewitness ID, stress induced perception/memory errors in officer-involved shootings/self-defense cases, and a fair amount on heuristics, I find that I tend not to trust what my senses, memory, and instinctive reasoning tell me. I try to double-check what I think I perceive and remember and seek out disconfirming data to challenge my own preconceptions. I try to put trust in well-designed, double-blind, peer reviewed, statistically-valid studies whenever those are available.Les Bush wrote:As a lawyer she has inherited the usual lack of trust in anything but what her own senses can believe, the St Thomas principle.
-
Michele
- Posts: 384
- Joined: Tue Dec 06, 2005 10:40 am
Lisa,
I think we’d all agree that examiners are infallible and bias is a concern. I would say that there was a time when many examiners didn’t understand the difference between 100% reliable and 100% confident, but it’s my belief this has changed. I would hope that examiners are no longer using the terms like ‘absolute’ or ‘infallible’.
Trusting the conclusions of expert and wondering about bias isn’t only a concern with fingerprint examiners in court situations, it’s a concern with all scientific conclusions that people are confronted with everyday. As a quick example, suppose a doctor says you need surgery. If you’ve had reoccurring medical problems you may agree and have the surgery w/o a second opinion. If you haven’t had medical problems you may decide to consult another doctor. In today’s society it’s common for the second doctor to get the x-rays, test results and conclusions of the first doctor, creating the opportunity for bias. Corroboration by the second doctor may not give you the confidence you are looking for, mainly because the conclusion appears to be in a ‘black box’. What makes us confident? When a 2nd, 3rd, or 4th doctor explains the reasons for his conclusions, objectively with the plusses and minuses. This step eliminates the mystery and vagueness, exposing what’s in the ‘black box’, therefore eliminating the ‘black box’ concept altogether. This doesn’t insure correct conclusions but it does give support for the conclusion.
It’s not much different from our industry. As an attorney or a juror, the conclusion has more weight when we know why and how the conclusion was arrived at. In most cases the examiner should be able to show you what information he/she was basing their conclusion on. Generally speaking, we aren’t seeing information that you can’t see. If you can’t see it, this is a good indication you should hire a defense expert (the subjective nature of our conclusions is more so in the sufficiency aspect, not the agreement portion). Of course there are images and contrast issues that we’re more experienced at looking at, but this doesn’t seem to be the typical situation.
It’s tricky to separate the method from the practitioner, but it’s possible to do so at a basic level. I do acknowledge that it’s impossible to separate them completely. Is the method used demonstrating clear, objective, reproducible characteristics with additional predictable characteristics, or did the examiner get a warm fuzzy feeling based on the suspects previous record? Did a practitioner try to support their hypothesis, or did they try to discredit it? Although it might sound logical, that you can’t separate the method from the practitioner, people have done so (at a basic level) for thousands of years, starting with Socrates. This is the history of science and scientific discovery, attempting to reason with objective criteria. You can’t separate practitioner from the method completely, but there are ways to tackle this. If someone doesn’t acknowledge any benefits of dividing the method from the practitioner, then they’ll never know what can be gained by doing so. Blind verification is a valuable tool for diminishing bias but has some draw backs. It’s also only one tool in the arsenal and needs to be understood to be used appropriately. Other tools could be beneficial as well.
I think we’d all agree that examiners are infallible and bias is a concern. I would say that there was a time when many examiners didn’t understand the difference between 100% reliable and 100% confident, but it’s my belief this has changed. I would hope that examiners are no longer using the terms like ‘absolute’ or ‘infallible’.
Trusting the conclusions of expert and wondering about bias isn’t only a concern with fingerprint examiners in court situations, it’s a concern with all scientific conclusions that people are confronted with everyday. As a quick example, suppose a doctor says you need surgery. If you’ve had reoccurring medical problems you may agree and have the surgery w/o a second opinion. If you haven’t had medical problems you may decide to consult another doctor. In today’s society it’s common for the second doctor to get the x-rays, test results and conclusions of the first doctor, creating the opportunity for bias. Corroboration by the second doctor may not give you the confidence you are looking for, mainly because the conclusion appears to be in a ‘black box’. What makes us confident? When a 2nd, 3rd, or 4th doctor explains the reasons for his conclusions, objectively with the plusses and minuses. This step eliminates the mystery and vagueness, exposing what’s in the ‘black box’, therefore eliminating the ‘black box’ concept altogether. This doesn’t insure correct conclusions but it does give support for the conclusion.
It’s not much different from our industry. As an attorney or a juror, the conclusion has more weight when we know why and how the conclusion was arrived at. In most cases the examiner should be able to show you what information he/she was basing their conclusion on. Generally speaking, we aren’t seeing information that you can’t see. If you can’t see it, this is a good indication you should hire a defense expert (the subjective nature of our conclusions is more so in the sufficiency aspect, not the agreement portion). Of course there are images and contrast issues that we’re more experienced at looking at, but this doesn’t seem to be the typical situation.
It’s tricky to separate the method from the practitioner, but it’s possible to do so at a basic level. I do acknowledge that it’s impossible to separate them completely. Is the method used demonstrating clear, objective, reproducible characteristics with additional predictable characteristics, or did the examiner get a warm fuzzy feeling based on the suspects previous record? Did a practitioner try to support their hypothesis, or did they try to discredit it? Although it might sound logical, that you can’t separate the method from the practitioner, people have done so (at a basic level) for thousands of years, starting with Socrates. This is the history of science and scientific discovery, attempting to reason with objective criteria. You can’t separate practitioner from the method completely, but there are ways to tackle this. If someone doesn’t acknowledge any benefits of dividing the method from the practitioner, then they’ll never know what can be gained by doing so. Blind verification is a valuable tool for diminishing bias but has some draw backs. It’s also only one tool in the arsenal and needs to be understood to be used appropriately. Other tools could be beneficial as well.
Michele
The best way to escape from a problem is to solve it. Alan Saporta
There is nothing so useless as doing efficiently that which should not be done at all. Peter Drucker
(Applies to a full A prior to C and blind verification)
The best way to escape from a problem is to solve it. Alan Saporta
There is nothing so useless as doing efficiently that which should not be done at all. Peter Drucker
(Applies to a full A prior to C and blind verification)
-
L.J.Steele
- Posts: 430
- Joined: Mon Aug 22, 2005 6:26 am
- Location: Massachusetts
- Contact:
I've
Agreed. I've actually had these kind of conversations with my doctors about test results for recommended treatment. (Did the specialist order a re-test or just look at the results on the chart, to what extent did the specialist talk with the GP or look at his/her conclusions.) This sort of effort depends on the amount of time one has and the seriousness of a wrong diagnosis. Had a heck of a time with one of them explaining why I wanted an actual independent second opinion....Michele Triplett wrote:As a quick example, suppose a doctor says you need surgery. If you’ve had reoccurring medical problems you may agree and have the surgery w/o a second opinion. If you haven’t had medical problems you may decide to consult another doctor. In today’s society it’s common for the second doctor to get the x-rays, test results and conclusions of the first doctor, creating the opportunity for bias.
One of the problems in life is knowing when you are getting an independent "blind" verification and when the 2nd result is non-blind and potentially influenced by the 1st result. Best answer is to ask lots of questions about the process leading to the results.
Problem here is (1) Mayfield -- I'm sure the three FBI guys and Moses could have given very persuasive answers as to how each reached his conclusion about the match prior to the error being caught; (2) Cowans -- everyone I've talked to who's seen the data since the error was discovered agrees that something was obviously wrong, but two defense experts reviewed it for his original trial and didn't catch the error, again, I'm sure they gave trial counsel a good explanation for thier confirming conclusions; (3) the other 20ish cases of known mis-IDs that Cole cites, several of which involve experienced examiners and review by defense experts who didn't catch the mistake; and (4) the many, many fabrication cases Pat Wertheim described at VT NEDIAI which also have slipped past examiners and defense experts.Michele Triplett wrote:What’s in the ‘black box’, therefore eliminating the ‘black box’ concept altogether. This doesn’t insure correct conclusions but it does give support for the conclusion.
Certainly, the defense bar needs to do a much better job of trying to get inside the black box of the examiner's opinion, but even the best-read of us aren't trained examiners. We're not going to catch some mistakes (or realize when we need to hire an expert). In the main, the judges and juries are far less well-read on these topics. The prospect of an innocent defendant going to jail, and the actual culprit staying on the streets, is the kind of thing that can keep one up at night.
True. But I don't have the experience to know whether the difference I see is an explanable distortion or a dissimilarity. I can look at the Mayfield print, but I don't see it the way that you or the other examiners here do.Michele Triplett wrote:Generally speaking, we aren’t seeing information that you can’t see.
For me, that the examiner knew anything about the suspect's prior record or other confirming evidence in the investigation prior to making the match and writing the report is a warning bell. On the other hand, what the examiner knew is not a routine part of the report I see (mostly just summary results -- match/non-match) and the examiner him/herself may not recall the details of what he/she knew when by the time of a pre-trial hearing or trial.Michele Triplett wrote:did the examiner get a warm fuzzy feeling based on the suspects previous record?
One of the ways that gets sorted out is blind and double-blind testing. As we've discussed in the forum in the past, there are some serious problems with trying to isolate the examiner from the investigation and the suspect's past, and with blind verification. Dror's study where examiners give different results on their own work years later when given suggestive information makes me nervous about repeatability. (Again, sample size is small.)Michele Triplett wrote:Although it might sound logical, that you can’t separate the method from the practitioner, people have done so (at a basic level) for thousands of years, starting with Socrates.
I've got concerns about assumptions that are applied without rigorous testing to confirm them. The new OIG report and FBI reports post-Mayfield seem to be raising questions about the underlying assumptions on Level III detail.
I've got concerns about examiners who testify to things that either aren't true but are likely something they learned in training or based on a logical infernece they haven't tested. FREX rain/immersion in water destroys all prints (false, as y'all have explained to me) so we didn't test the object or the most clear print left on a high-traffic surface must be that of the last person to touch it (logical, but no backup studies to show if true), thus the defendant must have left it while fleeing the scene as opposed to earlier in the day. We've still got a few folks up here testifying about "fresh prints" and age of prints in contrast to various published studies saying you can't "age" a print. (In many of these cases, I'm finding these statements during the appeal -- which means trial counsel didn't know to question the examiner on these assertions.)
I've wandered a bit from your topic.
-
Michele
- Posts: 384
- Joined: Tue Dec 06, 2005 10:40 am
Lisa,
I couldn’t agree more with all your concerns. I am very conscious of the value objective criticisms have brought to this industry. It’s made us reanalyze old dogma that we may have been quoting from our training. In some cases the old dogma is being replaced with more accurate explanations and in other cases perhaps it’s being replaced with new dogma. Objectively questioning ideas is leading us in a positive direction. My problem with Cole’s article is (as Les pointed out) is that I didn’t feel that it was objective or fair. I feel this isn’t bringing problems to light but maybe enhancing a different goal.
Unfortunately, I doubt that we will ever be able to know if a particular examiner or set of examiners is correct or not (as with any industry). Verification by an independent source helps but it’s not fool proof. Sometimes several conclusions are needed in order to find the truth (‘truth’ isn’t the word I’m really looking for, but it’s the closest word I can think of at this time). In the Mayfield case, independent analysis by the Spanish examiners is what it took to arrive at a conclusion the industry (as a whole) would agree with. With these images, if an examiner said “It’s 3 or 4 impressions”, my question would be, “what observations lead you to that conclusion?” Since I’ve never spoke with these examiners, I don’t know what their answer would be. I don’t know if Moses blamed the mistake on 3rd level detail and I don’t want to misquote him but I can recall him saying his mistake was in confirming the FBI’s conclusion instead of trying to disprove it. This has been a valuable lesson to me in the way I conduct verification and my understanding of conducting a scientific analysis. I’ve never met Ken Moses but I’m hoping he realizes what value his mistake and openness has had on many examiners.
Repeatability is important but it shouldn’t over shadow other requirements (justifiable and demonstratable). Dror’s study didn’t ask examiners for this information, nor was it recommended or expected in their previous conclusions. In this study nobody ever tried to open the black box and see how the decisions were made (or what they were based on). As I’ve stated before, it’s my belief that in harder identifications, this would be a good QA procedure.
In regards to ‘the most clear print left on a high traffic surface must be that of the last person to touch it….thus the defendant must have left it fleeing the scene as opposed to earlier in the day”, without doing a rigorous study, my initial thought is that this isn’t always true. It’s dependent on the several factors that I won’t go into for the sake of keeping this post at a reasonable length. If you’d like some of the factors or different scenario’s, I’d be glad to email them to you separately (or someone could start a separate post).
I’m sure you’ve heard this before, but I appreciate your willingness to have an open, level headed discussion on these matters.
I couldn’t agree more with all your concerns. I am very conscious of the value objective criticisms have brought to this industry. It’s made us reanalyze old dogma that we may have been quoting from our training. In some cases the old dogma is being replaced with more accurate explanations and in other cases perhaps it’s being replaced with new dogma. Objectively questioning ideas is leading us in a positive direction. My problem with Cole’s article is (as Les pointed out) is that I didn’t feel that it was objective or fair. I feel this isn’t bringing problems to light but maybe enhancing a different goal.
Unfortunately, I doubt that we will ever be able to know if a particular examiner or set of examiners is correct or not (as with any industry). Verification by an independent source helps but it’s not fool proof. Sometimes several conclusions are needed in order to find the truth (‘truth’ isn’t the word I’m really looking for, but it’s the closest word I can think of at this time). In the Mayfield case, independent analysis by the Spanish examiners is what it took to arrive at a conclusion the industry (as a whole) would agree with. With these images, if an examiner said “It’s 3 or 4 impressions”, my question would be, “what observations lead you to that conclusion?” Since I’ve never spoke with these examiners, I don’t know what their answer would be. I don’t know if Moses blamed the mistake on 3rd level detail and I don’t want to misquote him but I can recall him saying his mistake was in confirming the FBI’s conclusion instead of trying to disprove it. This has been a valuable lesson to me in the way I conduct verification and my understanding of conducting a scientific analysis. I’ve never met Ken Moses but I’m hoping he realizes what value his mistake and openness has had on many examiners.
Repeatability is important but it shouldn’t over shadow other requirements (justifiable and demonstratable). Dror’s study didn’t ask examiners for this information, nor was it recommended or expected in their previous conclusions. In this study nobody ever tried to open the black box and see how the decisions were made (or what they were based on). As I’ve stated before, it’s my belief that in harder identifications, this would be a good QA procedure.
In regards to ‘the most clear print left on a high traffic surface must be that of the last person to touch it….thus the defendant must have left it fleeing the scene as opposed to earlier in the day”, without doing a rigorous study, my initial thought is that this isn’t always true. It’s dependent on the several factors that I won’t go into for the sake of keeping this post at a reasonable length. If you’d like some of the factors or different scenario’s, I’d be glad to email them to you separately (or someone could start a separate post).
I’m sure you’ve heard this before, but I appreciate your willingness to have an open, level headed discussion on these matters.
Michele
The best way to escape from a problem is to solve it. Alan Saporta
There is nothing so useless as doing efficiently that which should not be done at all. Peter Drucker
(Applies to a full A prior to C and blind verification)
The best way to escape from a problem is to solve it. Alan Saporta
There is nothing so useless as doing efficiently that which should not be done at all. Peter Drucker
(Applies to a full A prior to C and blind verification)
-
sscar
- Posts: 3
- Joined: Tue Jan 31, 2006 2:54 pm
- Location: LV, NV
- Contact:
Michele Triplett Detail 233
Good job Michele, very succinct and to the point. I also find it bizarre that this grand leap of logic (lyr leap) is made from making a statement about the uniqueness of Fingerprints –Fingerprints are infallible - to also mean that fingerprint examiners are infallible.
When any other expert testifies about their discipline and makes a statement such as a hydro engineer testifies about how gravity invariably pulls the water down…" They are always talking about hydrology not about if any other engineer can make a mistake when they make their tests. “Gravity makes things fall" doesn’t mean anything else.
Or an astronomer says that "No two asteroids are alike" he is addressing the mineral makeup and creation of that object, not if astronomers can make a mistake in comparing two asteroids.
And we all, like you, should be standing up and shouting that every time this leap of logic is proposed. That is not what that means!
I suppose is does make sense to some if there is an underlying agenda. And speaking of agendas, isn’t LJ Steele, Lisa J. Steel? I have The Defense Challenge to Fingerprints right here on my desk. LJ is commenting on Cole?
“I can see Cole’s point.”
Not that we should restrict any opinions on the BB but of all thousands of FP examiners, the thousands of research scientists working in biometrics & AFIS,( like Prabakar and Jain,) the thousands of research engineers working in Biometrics for security in industry, the thousands of academics doing research in biometrics, the group of anti-Fingerprints people is tiny.
To have people from that tiny group comment on each other’s proposals is kind of like – help me with a good analogy here - two foxes critiquing each other’s attacks on eggs in an industrial size hen house.
Like you I find some of the challenges good for the discipline, it makes us in the discipline think and prepare. But many of them are so far out, like error rates for people (another LLeap) as to defy logic and not provide any useful substance to build upon.
Michele, here is my new mantra.
A Critic can be "a person given to harsh judgments and to finding faults" or " one who expresses a reasoned opinion on a matter" Websters- The substance behind their arguments shows which one. And the underlying agenda -or lack of- dictates where they stand. This should never be forgotten.
[/quote]
When any other expert testifies about their discipline and makes a statement such as a hydro engineer testifies about how gravity invariably pulls the water down…" They are always talking about hydrology not about if any other engineer can make a mistake when they make their tests. “Gravity makes things fall" doesn’t mean anything else.
Or an astronomer says that "No two asteroids are alike" he is addressing the mineral makeup and creation of that object, not if astronomers can make a mistake in comparing two asteroids.
And we all, like you, should be standing up and shouting that every time this leap of logic is proposed. That is not what that means!
I suppose is does make sense to some if there is an underlying agenda. And speaking of agendas, isn’t LJ Steele, Lisa J. Steel? I have The Defense Challenge to Fingerprints right here on my desk. LJ is commenting on Cole?
Not that we should restrict any opinions on the BB but of all thousands of FP examiners, the thousands of research scientists working in biometrics & AFIS,( like Prabakar and Jain,) the thousands of research engineers working in Biometrics for security in industry, the thousands of academics doing research in biometrics, the group of anti-Fingerprints people is tiny.
To have people from that tiny group comment on each other’s proposals is kind of like – help me with a good analogy here - two foxes critiquing each other’s attacks on eggs in an industrial size hen house.
Like you I find some of the challenges good for the discipline, it makes us in the discipline think and prepare. But many of them are so far out, like error rates for people (another LLeap) as to defy logic and not provide any useful substance to build upon.
Michele, here is my new mantra.
A Critic can be "a person given to harsh judgments and to finding faults" or " one who expresses a reasoned opinion on a matter" Websters- The substance behind their arguments shows which one. And the underlying agenda -or lack of- dictates where they stand. This should never be forgotten.
-
L.J.Steele
- Posts: 430
- Joined: Mon Aug 22, 2005 6:26 am
- Location: Massachusetts
- Contact:
Re: Michele Triplett Detail 233
Guilty as charged. I haven't made any secret of my views here. (And it is Steele -- if you've got a misprint on the article, let me know and I'll double -check the source file to find the typo). The version of it from NEDIAI in VT is the latest one, but I need to update it for Patterson at some point soon. (The one on the firm website is out-of-date and needs to be replaced one of these days.)sscar wrote:I suppose is does make sense to some if there is an underlying agenda. And speaking of agendas, isn’t LJ Steele, Lisa J. Steel? I have The Defense Challenge to Fingerprints right here on my desk. LJ is commenting on Cole?
For the most part, my bias in this is (1) getting my colleages in the defense bar to start asking good questions instead of just accepting the examiner's testimony on its face and (2) making sure the right bad guys go to jail on solid evidence. (Doesn't mean that when the bad guy is my client, I'm not going to do less than my ethical duty to zealously represent him or her within the bounds of the law, but that's a different issue.)
Cole and I disagree on some points and agree on others. I think we agree that there are issues about testing and methodology that need to be explored, but both of us would agree that fingerprint matching provides useful information in the vast majority of cases.
You can't blame the critics for that one. Mr. Meagher of the FBI has been testifying to a distinction between error rate of the ACE-V method (zero when done properly) and error rate of individual practitioners (the explanation for every mistaken ID case). Cole's response is that it doesn't work that way in other fields -- the potential for human error is part of the error rate for a procedure or test. My take is that because ACE-V relies on the black-box of the examiner's judgment, human error is part-and-parcel of the method which makes the "error rate" (in the sense that Daubert uses the term) low, but not zero.sscar wrote:error rates for people (another LLeap) as to defy logic and not provide any useful substance to build upon.
[/i]
-
Les Bush
- Posts: 229
- Joined: Tue Jul 05, 2005 4:29 am
- Location: Australia
The Black box analogy
Its interesting that we are using the term black box to express the arena in which a determination is made. As an Australian invention the black box has been a fantastic inovation and inclusion to aircraft. Although made by human means it generally does what is required but there are conditions when it fails to meet expectations One of my pet subjects is the conditional basis for our limited understanding of ourselves and our environment. We will continue to learn if we remain open to sound reasoning and argument. For the critics to say there is a black box associated with how fingerprint identifications are performed then they have failed to articulate what it is about the human condition they suspect is in error. My invitation to Cole has been to apply the system that underpins fingerprint identifications and make his own objective findings. The system I refer to incorporates the fundamental principles of biological uniqueness, the technique of establishing sufficient unique details in sequence, the application of a scientific method and the resulting conclusion that is both verified and demonstrable. To date the critics have focussed their views on the results and generally in relation to examinations that are difficult for the average expert. Much like the medical experts or the air traffic controllers we daily rely upon for their expertise and ability to make valid decisions in high risk situations the same public confidence should be given to competent and professional experts in fingerprint science.
-
Michele
- Posts: 384
- Joined: Tue Dec 06, 2005 10:40 am
Les,
I’m taking what you said out of context but I couldn’t agree more that critics are focusing on the results. Perhaps the same could be said for some fingerprint supporters. Granted the results are important but results are different under different circumstances. Failing to acknowledge this will leave someone with a false understanding of what the results mean. The results can only be understood by looking at all the particular elements.
One element of science is to break things down into individual elements. Once we understand each element then we can apply this knowledge to general ideas.
With regard to error rates, I think it’s important to understand that the actual error rate is the error rate of the methodology plus that of the practitioner. Many processes may be more technical than scientific. These technical processes usually have an inherent error rate. When the practitioner is adequately trained, the practitioner error may be insignificant. Perhaps the actual rate is very close to the theoretical rate and it’s these disciplines that don’t count practitioner error rates (because of the insignificance). In other technical processes where the practitioner isn’t well trained (such as someone taking a home pregnancy tests), the actual error is far from the theoretical error rate, indicating that the practitioners ability to read the directions may be important. In this situation science has found it useful to separate the inherent error rate and the practitioner error rate. I’m speculating that in our industry we can’t say whether or not these numbers should be separated out until we do it and see the significance of each value.
I’d also like to comment on the idea of the error rate of the methodology being zero. I’m speculating that the rational goes something like this, “All prints are unique (or scientifically accepted as such), therefore the inability to individualize is that of the practitioner”. It seems to me that there’s a leap in logic here. The premise doesn’t say anything about how to individualize, what method to use, or what elements are unique. I really don’t think one thing leads to another. I guess ultimately everything in any field can be blamed on the practitioner. If a method fails then the practitioner didn’t use the rigor they should have in double-checking the results with other methods. But in reality, in order to say the error rate of the methodology is zero, we first have to have a comprehensive definition of what the methodology is and what is expected from the practitioner. Someone could argue that scientific results aren’t absolute, they’re always open to further studies and testing. Therefore the scientific methodology may not really have an error rate. Does not having an error rate mean that it’s zero?? This really isn’t meant to be a question that I’m expecting someone to answer, I’m just speculating.
I’m taking what you said out of context but I couldn’t agree more that critics are focusing on the results. Perhaps the same could be said for some fingerprint supporters. Granted the results are important but results are different under different circumstances. Failing to acknowledge this will leave someone with a false understanding of what the results mean. The results can only be understood by looking at all the particular elements.
One element of science is to break things down into individual elements. Once we understand each element then we can apply this knowledge to general ideas.
With regard to error rates, I think it’s important to understand that the actual error rate is the error rate of the methodology plus that of the practitioner. Many processes may be more technical than scientific. These technical processes usually have an inherent error rate. When the practitioner is adequately trained, the practitioner error may be insignificant. Perhaps the actual rate is very close to the theoretical rate and it’s these disciplines that don’t count practitioner error rates (because of the insignificance). In other technical processes where the practitioner isn’t well trained (such as someone taking a home pregnancy tests), the actual error is far from the theoretical error rate, indicating that the practitioners ability to read the directions may be important. In this situation science has found it useful to separate the inherent error rate and the practitioner error rate. I’m speculating that in our industry we can’t say whether or not these numbers should be separated out until we do it and see the significance of each value.
I’d also like to comment on the idea of the error rate of the methodology being zero. I’m speculating that the rational goes something like this, “All prints are unique (or scientifically accepted as such), therefore the inability to individualize is that of the practitioner”. It seems to me that there’s a leap in logic here. The premise doesn’t say anything about how to individualize, what method to use, or what elements are unique. I really don’t think one thing leads to another. I guess ultimately everything in any field can be blamed on the practitioner. If a method fails then the practitioner didn’t use the rigor they should have in double-checking the results with other methods. But in reality, in order to say the error rate of the methodology is zero, we first have to have a comprehensive definition of what the methodology is and what is expected from the practitioner. Someone could argue that scientific results aren’t absolute, they’re always open to further studies and testing. Therefore the scientific methodology may not really have an error rate. Does not having an error rate mean that it’s zero?? This really isn’t meant to be a question that I’m expecting someone to answer, I’m just speculating.
Michele
The best way to escape from a problem is to solve it. Alan Saporta
There is nothing so useless as doing efficiently that which should not be done at all. Peter Drucker
(Applies to a full A prior to C and blind verification)
The best way to escape from a problem is to solve it. Alan Saporta
There is nothing so useless as doing efficiently that which should not be done at all. Peter Drucker
(Applies to a full A prior to C and blind verification)
-
Les Bush
- Posts: 229
- Joined: Tue Jul 05, 2005 4:29 am
- Location: Australia
Are all prints unique?
Hi Michelle,
The context of my last posting was that Cole and his critic proponents are only focussing on the results of examinations about 'difficult latents'. His logic is that the principle of identification science should be applicable across all situations, if it fails in one it fails in all others. As an opponent I would dispute that premise since the variables in difficult latents are not repeated in the other applications of the science. The principles, technique and method are the same but the results for difficult latents will occasionally be affected by the level of capability/competency of the expert. All things are not equal. The practitioner error occurence rate will vary mostly due the lack of quality experience and professionalism.
Coles does not concede fingerprint uniqueness even after he appears to have read the lengthy and comprehensive bibliography attached to his article. What he fails to understand is that the process of biological formation is controlled by genes. The glyphogene is responsible for the patterning of friction ridges. The variables by which areas of friction ridges form constitute a system that is both strategically positioned on hands and feet and highly localised due to its genetic influences. The combination of skin cells, nerve cells, glands and others are each accomodated within the structure of friction skin. The anatomical result and the physiological mechanisms are permanent structures for the life of the organism. What Coles proposes isnt a new idea but it follows the logic of how can the scientific principle of fingerprint uniqueness be confirmed if all areas of friction skin have not been compared. I can defend this by a combination of knowledge, logic and experience. The distal phalange has been the most highly scrutinised area of friction skin over the past hundred years. The Blackburn baby case involving comparisons relating to Peter Griffiths is known to have excluded all but one person as the source of the latent. If biological uniqueness was questioned then the application of its formative processes would apply to the distal phalange as much as to any area of friction skin. In other words if duplication of the anatomical result was to occur the distal phalanges would share equally in that process. Because that has not happened after being highly scrutinised (inked and latent fingerprints) then it proves biological uniqueness. Each expert challenges the principle of biological unqiueness when making comparisons as it forms part of the hypothesis that a source of an unknown can be made to the exclusion of all others. Each expert has a wealth of personal experience that two identical prints from different persons have not been discovered.
Sorry for taking longer than usual but Ill get one more issue out in the open. Coles used the term 'metric ' to focus on what fingerprint experts are not producing with their results. He seems to be advocating a need for a system of measurement by which pure sciences such as mathematics can take over and validate our determination. Once again I would challenge Cole that if this is needed then the ball is also in his court to define such a system for measuring fingerprint results. Our generation has computing software capability that I'm sure would meet those needs. An article I submitted earlier to the IAI covered the design of such a system. Kind regards from down under. Les
The context of my last posting was that Cole and his critic proponents are only focussing on the results of examinations about 'difficult latents'. His logic is that the principle of identification science should be applicable across all situations, if it fails in one it fails in all others. As an opponent I would dispute that premise since the variables in difficult latents are not repeated in the other applications of the science. The principles, technique and method are the same but the results for difficult latents will occasionally be affected by the level of capability/competency of the expert. All things are not equal. The practitioner error occurence rate will vary mostly due the lack of quality experience and professionalism.
Coles does not concede fingerprint uniqueness even after he appears to have read the lengthy and comprehensive bibliography attached to his article. What he fails to understand is that the process of biological formation is controlled by genes. The glyphogene is responsible for the patterning of friction ridges. The variables by which areas of friction ridges form constitute a system that is both strategically positioned on hands and feet and highly localised due to its genetic influences. The combination of skin cells, nerve cells, glands and others are each accomodated within the structure of friction skin. The anatomical result and the physiological mechanisms are permanent structures for the life of the organism. What Coles proposes isnt a new idea but it follows the logic of how can the scientific principle of fingerprint uniqueness be confirmed if all areas of friction skin have not been compared. I can defend this by a combination of knowledge, logic and experience. The distal phalange has been the most highly scrutinised area of friction skin over the past hundred years. The Blackburn baby case involving comparisons relating to Peter Griffiths is known to have excluded all but one person as the source of the latent. If biological uniqueness was questioned then the application of its formative processes would apply to the distal phalange as much as to any area of friction skin. In other words if duplication of the anatomical result was to occur the distal phalanges would share equally in that process. Because that has not happened after being highly scrutinised (inked and latent fingerprints) then it proves biological uniqueness. Each expert challenges the principle of biological unqiueness when making comparisons as it forms part of the hypothesis that a source of an unknown can be made to the exclusion of all others. Each expert has a wealth of personal experience that two identical prints from different persons have not been discovered.
Sorry for taking longer than usual but Ill get one more issue out in the open. Coles used the term 'metric ' to focus on what fingerprint experts are not producing with their results. He seems to be advocating a need for a system of measurement by which pure sciences such as mathematics can take over and validate our determination. Once again I would challenge Cole that if this is needed then the ball is also in his court to define such a system for measuring fingerprint results. Our generation has computing software capability that I'm sure would meet those needs. An article I submitted earlier to the IAI covered the design of such a system. Kind regards from down under. Les