CLPEX Chat Board

How about it folks?

Any ideas and suggestions welcomed

In the Science of Fingerprints how do you carry out a blind verification, let alone a double-blind verification. In a blind verification, do you submit just the one ten-print/major case print, card? Or, do you submit more than one card for the other person to do a truly, blind verification. Next, in that case, what if you have someone with a very rare pattern type such as a cuspal pattern, or a rare form of an accidental whorl? You could then be hard pressed to find another card for the verifyer to be able to complete a true, blind verification; this would also be dependant upon the size of your fingerprint files.
Then there could very easily be very unusual classifications such as a 15/2 primary or even a 2/15 primary; then what. I don't know if I am understanding this correctly, but this is my take on the blind/double-blind verification. How do you truly make it blind in the first place? I have been practicing this science for nearly 36 years total time and nearly 26 years in latent prints alone. And in this part of the state, as far as I am aware, I am the only, qualified by training through the IAI, latent examiner. The nearest one to me would be over an hour's drive in Birmingham. This could also be another problem for many people if they were to conduct a double-blind verification.
Yet, another puzzling aspect--what about a combination of level II and level III friction ridge analysis for a comparison. How do you find enough prints, enlarged, to be similar enough so that a true, blind/double-blind verification could be carried out? Perhaps my questions are moot because I am not truly understanding this type of verification process; however, if I do understand this correctly, then these are very important questions that must be answered before I would make this part of my protocol. Currently, verifications are done; this is SOP in this unit because chances must not be taken with comparisons.

Danny Lamont wrote:In the Science of Fingerprints how do you carry out a blind verification, let alone a double-blind verification. In a blind verification, do you submit just the one ten-print/major case print, card? Or, do you submit more than one card for the other person to do a truly, blind verification. Next, in that case, what if you have someone with a very rare pattern type such as a cuspal pattern, or a rare form of an accidental whorl? You could then be hard pressed to find another card for the verifyer to be able to complete a true, blind verification; this would also be dependant upon the size of your fingerprint files.
Then there could very easily be very unusual classifications such as a 15/2 primary or even a 2/15 primary; then what. I don't know if I am understanding this correctly, but this is my take on the blind/double-blind verification. How do you truly make it blind in the first place? I have been practicing this science for nearly 36 years total time and nearly 26 years in latent prints alone. And in this part of the state, as far as I am aware, I am the only, qualified by training through the IAI, latent examiner. The nearest one to me would be over an hour's drive in Birmingham. This could also be another problem for many people if they were to conduct a double-blind verification.
Yet, another puzzling aspect--what about a combination of level II and level III friction ridge analysis for a comparison. How do you find enough prints, enlarged, to be similar enough so that a true, blind/double-blind verification could be carried out? Perhaps my questions are moot because I am not truly understanding this type of verification process; however, if I do understand this correctly, then these are very important questions that must be answered before I would make this part of my protocol. Currently, verifications are done; this is SOP in this unit because chances must not be taken with comparisons.

I appreciate the logistical and other complexities of doing it correctly and that is why I started this thread, so we can bounce ideas around and maybe get somewhere...

My first thought was to include verifications into regular proficiency testing... as you say it could be a complex set-up to establish but will the long term benefits not outweigh the growing pains? This way we have 2 birds with one stone and a ready batch of prints to include for comparisons...

Then that takes away the double blind factor right there doesn't it

What percentage of your comparisons included cuspals? And why are "rare" pattern combinations an issue? we individualize one finger at a time, surely?


Sorry have to go more later...

That's just it. Not only do I doubt any unit can do double-blind verification and would find genuine blind verification difficult, but either is likely impossible for a very great number of agencies. This group tends to have members from larger agencies. That's to be expected. But when we think about the fact that most law enforcement agencies in the U.S. are very small and likely have one or two examiners who may have other duties, it's next to impossible.

Verifications should always be done. It's no different from other activities where error is possible. We let other people check the work. I think a good bit about some of the dogma in the discipline. One of these issues is that of collaborative examination. By that, I mean that I have heard it preached that a conclusion cannot be a committee conclusion. While I think it's neither desirable or practical for examinations to be done by a group acting together, when we talk about verification in the common way we mean, are we not talking about effectively a sequential group effort, abeit a group of two. A difference in findings will be resolved by some process of reexamination of specific details.

Would most not agree that in most units, verification implies that an identification was made by the first examiner? In my situation, if I sent a latent out to be reexamined, it certainly meant either that I had made an identification or that I believed an identification by a much more experienced examiner might be possible. I would argue that this is in no way different (and really more powerful) than an initial examiner's knowledge that the recorded person is suspect. And I think that sort of knowledge is very common in many, of not most, agencies. We have become deathly afraid of public error. I do not believe there is any process that will effectively eliminate all errors.

My feeling is that the best review is possible when the second examiner has a fully detailed analysis and articulation of the comparison and conducts essentially a close peer review of the examination, more than a peer review of methodology but a critical review of actual decisions. It is very possible that, by striving for something approaching blind verification we may be (1) not achieving any real blind reexamination and (2) restricting the second opinion to a test of whether or not the second reaches the same conclusion, not whether or not the decisions during the examination are validated. I think this is an important point. When we look at the Mayfield incident, we see in the critical report a review and demonstrations of the specific interpretations of details. I think that this kind of review both forces the examiner to articulate the differential interpretation decisions during analysis, which I believe is the truly critical stage, and calls for the reviewing examiner to consider if each of those interpretations are sufficiently sound to use in the comparison.

I think there's a related lesson in the Mayfield matter. The chart prepared for the ETIU is exactly the sort of demonstration that is counterproductive as a demonstration of the validity of an examination. By its nature, the chart is a simple statement that Feature (n) in the latent represents the same defined feature as Detail (n) in the record print. That is in no way a demonstration and justification of the differential decision on the nature of the feature. Yet I suspect that the verbal equivalent, the stating that a <insert name of detail> was found and corresponded to a <name of detail> in the record print, would be many examiner's way of notating the analysis and comparison. What is meaningful and what gives a reviewer the ability to evaluate the decisions is a recording of what it is interpreted to be AND whether the interpretation truly rules out other interpretation. I think that is the kind of documentation that prevents error.

I think it's important to consider the motivation for that we consider doing. Verification issues attempt to address both the accuracy of individual examinations and the overall admissibility of testimony. A number of the government's presentations in court on the issue of admissibility have been very poorly prepared, both on the issues and in the way they have pandered to improperly framed criticism without pursuing a rational view latent print comparison. I think, though, that the admissibility issue will be put to rest, and the challenges will drop to a very few. In the long run, it's the individual examinations that will have to be demonstrated, and the resources are best spent in formulating standard ways to prepare and present the evidence in a way that dispels any suspicion that the conclusion is another error.

There will always be the problem of time and resources. I think we have to think carefully about:

What can reasonably be done with available resources?
Does the problem justify the expenditure? Indeed, what is the problem?
Will what we do really accomplish what we represent?
Does what we propose to do actually improve the reliability of individual examinations, or is it just designed to sound "scientific" in court?

I think that before we set up a protocol for blind or double-blind verification we should first determine if it’s even effective. Otherwise we are putting the cart before the horse. The idea that the verifier should not be aware of the original identification steps outside accepted scientific practice.

I am very curious where the idea of blind verification originated. I have to say I think a critic of the science just pulled this out of his, er-hat one day. Peer review or verification of scientific studies and findings evaluates the creative work or performance by other people in the same field in order to maintain or enhance the quality of the work or performance in that field. In peer review as it is practiced today, reviewers are anonymous but the entire work is examined and critiqued. An expert cannot be expected to evaluate another’s work without reviewing it. In the general scientific community peer review is performed anonymously, but not blindly as is claimed by some. In order for a peer review to be blind a separate study would need to be conducted by someone that had no information regarding the original study or the results. Scientific studies are often repeated by others in an attempt to confirm the results, but this is not done blindly either. What would blind verification achieve for our science? Does it even make sense?

An examiner makes an identification using the ACE-V method, which is a field specific description of the scientific method. The work is then sent to another examiner off site to avoid any possibility of bias. The blind verifier is unable to make the identification for whatever reason (poor quality copies, has less experience, is more conservative, etc.). Does that make the original examiner and his verifier wrong?

In his paper “The ‘Opinionization of Fingerprint Evidence” Simon Cole states that “the fingerprint community has done less to address these ‘difficulties’ [statistical analysis of minutiae rarity] than the DNA community”. I believe this is where we should be concentrating our efforts to instill confidence in our science. We are playing catch-up with DNA and need to find a way to quantify the possibility of error by an examiner. That may not be what Daubert is about but it is what the critics are asking for.

Allow me to post a copy of a document that appeared on this forum a while ago. I keep it handy in case anybody asks me about it when I am testifying:

KEPT - Keeping Examiners Prepared for Testimony - #12 2008-03-25
Blind Verification - Concerns
by Michele Triplett, King County Sheriff's Office

Disclaimer: The intent of this is to provide thought provoking discussion. No claims of accuracy exist.

Question – Concerns over Blind Verification:
What would some concerns be when doing blind verification?

Possible Answers:
a) Since the original evidence has the conclusions written on it, how do we recreate the evidence and not jeopardize the quality of the images.
b) If we use reproduced images then it’s not a true representation of the original evidence.
c) It’s time consuming and we don’t have the time to do this on every case.
d) One concern is that people think blind verification is the best quality assurance measure but it doesn’t look at the reasons behind a conclusion, which is a scientific protocol.
e) Blind verification has a place in looking at reliability but it shouldn’t be used as a replacement for peer review.

Discussion:
This is the perfect opportunity to say that blind verification isn’t the best quality assurance measure. It looks for reliable conclusions but it doesn’t insure the principles and methods used were reliable enough to be used.

Answers a and b: We may not be able to reproduce images that are exactly like the originals but it is possible to duplicate images so they’re an adequate representation of the of the originals.

Answer c: This is a truthful statement but it disregards the fact that blind verification isn’t needed in every case. Blind verification is a tool to protect against bias and bias is only a problem when alternative conclusions are plausible (in complex comparisons).

Answers d and e: I think both of these answers educate the courts about blind verification.

The effectiveness of blind verification in the latent print identification process was questioned in a previous post. The latent print community knows that verification in an integral, necessary and effective part of the examination process. Blind verification is simply another step in an evolving process of quality control. Blind verification was instituted to eliminate some of the possible bias in the ususal latent print identification process. One problem with the verification process has been that the examiner usually picks the person to do the verification. This person is usually a co-worker and knows the first examiner very well. The verifier may be a subordinate (which was brought up as a problem in the Mayfield error). In picking a verifier, the verifier knows that an identification was effected and is being verified.

To rectify these “problems,” blind verification has become the standard operating procedure at a large Federal laboratory. In that unit, a supervisor is involved in the process to pick a verifier. The supervisor also picks a third examiner to be a blind verifier. The blind verifier is not advised of the verification nor of the original conclusion. If there is not a unaminous opinion as to a particular conclusion, the supervisor institutes conflict resolution procedures according to Standard Operating Procedures. In this instance, blind verification involves a third examiner in the ACE-V process and, statistically, has to be more effective that simple verification.

Verification and blind testing what it is and what it is not....

Ok I have heard a lot about blind and double blind or triple quadruple blind testing as the solution to our woes, real or imagined. Some one said well verifications might be flawed in someway so lets do them blind... well how about double blind that has got to be even better. There has been a lot of confusion about what a blind test is and when any why to use them.

First we have to define what exactly blind testing is....
The example most commonly givin is a patient being treated with a drug. That person has no idea whether or not they have received the real thing or the placebo.
In normal blind testing the doctor or test giver is aware of the expected result, ie they know that the patent was given the drug or not.

If we were to translate that into police terms it would be like a photo line up. The cop administering the photo line up knows who the suspect is but the person who is looking at the line up does not.

Single blind testing is useful when there is some kind of placebo or bais effect that will affect the outcome.

Now with latent print examination if we were to "blind" the verifier the only thing that you would need to do is to not tell them what anyone else's conclusions were on the case. This also mean that if you were going to use blind verification it could not be all IDs or the verifier would know that ever time they were given a "blind verification" there was an ID in there somewhere.

Now double blind...

To continue with the drug example, neither the doctor nor the patient knows who has the real drug and who has the placebo. The tester and the subject have no idea what the outcome should be. This is important in cases where the tester can influence the subject. However there are many cases where double blind testing is not useful or is not needed. If the tester has to know what the treatment (placebo v. drug) is then you can not use blind testing. This is often the case with many new medical device treatments or in the cases where the placebo and the real treatment have obvious differences.

In the cop example if you had a double blind lineup then officer would not know who the suspect was when givin the line-up to the witness.

In cases where the test giver (the original examiner) does not have the potential to influence the test taker (verifier) double blind testing is not needed.

This is very much the case with latent print examination. The original examiner will always know the expected result and it is very easy for the original examiner to avoid influencing the verifier by simply removing all conclusion or notes from the case file before having it verified. As long as the original examiner isnt hovering over the verify "So you ID that yet.... Did you see L1, yeah that really looks like the left middle, come on you sure you wanna go inconclusive" there is not any influence in single blind verification

Simple single blind verification is all that would ever be needed in Latent Print Examination.

Since the original examiner would know the result double blind is not even possible.

To continue to use "double blind" is in most cases a misnomer and we should stop using that term in latent prints all together.
We need to examine the real issue:
Number one: Is blind verifications of latent prints a better quality control procedure?
Number two: If it is not needed in all cases when is it and how should it be done and what additional resources are going to be required with this method?

One of the problems in blind testing is in cases where you have a difficult print that the original examiner identifies but the "blind verifier" just fails to find it.
Blind verification may have some benifits when it comes to avoiding bias but it has its draw backs as well.

I found this reference in my fingerprint primer:

On April 4, 2008 The New Hampshire Supreme Court concluded in N.H. vs LANGILL that the use of such procedures as contemporaneous written documentation (“bench notes”) and blind verification “went above and beyond the generally accepted procedures in the field of fingerprint identification.”

The conclusion of the Court stated, ‘Although the enhancement of these factors (bench notes and blind verification) may make the scientific evidence more reliable, the absence of these factors unquestionably does not make the evidence unreliable or inadmissible.’

I think the real problem with verification (or blind-double blind etc) is having “equal ability” among each of the Examiners. Even if we somehow happen to achieve a working blind verification method, the question will always be raised as to “equal ability.” This will be especially true whenever there is a conflict of opinion (disagreement of identity.) Was the 2nd verifier as qualified as the first? Does each Examiner have the same “tolerance level” for detail? And as Pat has always reminded us, did each have a good day at the bench or did outside influences affect the outcome?
Another question is when is verification or blind verification really needed? Do you really get a second verify on a latent palm print with 250 2nd level clear detail? Is each department doing 2nd verify on 10 print identifications? ….Really?
At my previous department we did our best to deal with verification issues. Lack of manpower, funding and a case backlog that grew everyday weighed heavily on our decisions to do verifications. Although 95% of our identifications were verified, there were those “no brainer” >30+ feature< “clear as an inked “ latents that may not have seen a second verification. If that makes you squeamish, sorry, you do the best that you can with what you have. Are we striving for perfection? Is that possible in science?
I like the old 2nd opinion method. Look at my latent and inked, tell me what you think, tell me what you see and let’s compare notes. Now comes the bias issue, well, knowing it can happen is half the battle. I’m not a big proponent of the bias accusations being made. I do believe it’s there, I believe it happens but, I also believe it can be controlled (mostly out of fear of making a mistake.)

I agree with Mr. Reutzal that there are differences in the skill levels of examiners. But that difference is usually exhibited in the proficiency of examiners not in their competency.

Examiners that are competent, i.e. properly trained with sufficient experience, should be able to reach a correct conclusion when comparing a latent print with appropriate inked exemplars. However, there will be disparity in the times that it takes to arrive at the conclusion. Among examiners that are competent, there are surprisingly few differences of opinion about the ultimate question of identity.

There may be, however, differences of opinion about the value of a particular latent print. Since the examination of evidence does entail the opinion of examiners, laboratories should have written procedures in their standard operating procedure for conflict resolution.

Verification is an integral part of the ACE-V process. Verification is necessary to complete the process and the process should be completed regardless of the quality of the prints in question. There is not an acceptable excuse for failing to perform a part of the examination process. Errors occur for a multitude of reasons, hence the need for quality control.

While the issues listed (“Lack of manpower, funding and a case backlog that grew everyday”) are real, those should not be reasons for failing to incorporate appropriate quality control measures in the examination of latent prints in a forensic laboratory. Knowing that bias can occur may be “half the battle”, but that will not prevent a defense attorney from presenting bias as reason for a jury to suppose that an identification may not be reliable or accurate.

I, too, like "the old 2nd opinion method" Mr. Reutzel described. Much knowledge and information about prints and comparison can be obtained and shared by this method. Examiners progress in ability and benefit greatly by the sharing of knowledge. However, I do not believe that "2nd opinion method" is a substitute for the formal verification and blind verification procedures.

We should continue to strive for perfection in the discipline, even though we know that perfection is elusive.

Hello all and thank you for your replies.

I have been out of action for the whole weekend and have lots to catch up on. I promise to read all replies thoroughly and respond later.

A great Monday to all