CLPEX Chat Board

Heidi,

Correct, my assumption was that all ID's would be a part of that process. And it would be a good idea to mandate that every 3rd case (or something like that) you take multiple comparisons for review. Otherwise you run into a situation where examiers may learn that if more than one comparison is being brought to them this indicates that there are definately ID's.

Sharon,

I agree and understand that blind verification isn't required in other diciplines, but I think that a couple of very high profile mistakes have brought prints to the forefront. And in at least one of those mistakes, confirmation bias was determined to be a factor in the error (at least from the second examiner on).

I dont think that it is really a question of IF the LP community adopts a standard of blind verification, I think that its a factor of WHEN we do it.

Sharon,

I LOVE your attitude. I do not work in an ASCLD lab. Actually, I work in a one-person forensic unit, fingerprints being my speciality. When I make individualizations, I take them to the neighboring agency for verification. I am a busy person and that other agency is very busy, too. It would be impossible for me to ask them to do blind verifications, in other words, to do all the comparisons all over again. They do not have the time and they would just refuse to do my verifications. Currently, their two-person lab is now one, the other person just retired. The remaining person will probably bring his verifications to me. It is a courtesy thing that we do. And, neither of us have time to do blind verifications = work the other person's case all over again.

Because I am not an ASCLD lab, I don't do the great amount of documentation that ASCLD labs seem to feel is necessary. As a result, I can turn around a case within a day or two, sometimes, the same day I get it. A few times, I have individualized latent prints within a few hours of the crime and the detectives have been able to find the suspects AND recover the property. How many labs can make that claim? I individualize lots of prints (last month 53), I get lots of AFIS hits (last month 11) and I don't have a huge backlog of cases because I am not strangled with so many excessive rules and can complete work in a timely manner, frequently, with great success.

Kathy Saviers, CLPE

Wonderful! This is a great topic..glad it got started! Its been too long since we had something this good to discuss on here!

I'm the only examiner at my agency. There are several other examiners in other agencies surrounding me who also work by themselves. We verify each others work. When I make an identification I'm giving my opinion and yes the verifier knows what that opinion is. The thing is that they also could give a rat's behind about whether I'm right or not...what they care about is that their signature is going on that verification document and if my opinion is wrong then not only is everything I've worked for going down the drain...so is everything they worked for. We are very fortunate that all of our verifications are done by an outside agency..We have had different opinions and that's the whole reason for the verification. The defense will always be looking for ways to confuse the jury...confirmation bias is just another challenge we'll run into and overcome!

As I understand it "Blind Verification" is a process to overcome "Confirmation Bias".

Surely there are other ways to overcome Confirmation Bias other than blind verification.

Also I would like to see a really good study done on confirmation bias in the latent print discipline. How prevelant is it. I am not talking about a study using 15 non-discipline that were shown photos, or 6 discipline individuals that opted for a false negative to be on the safe side. I am talking about a real large involved study that would involve a number of agencies that are both large and small. With the study directed towards false positives.

How often in a case is bias present? Does bias that is present affect the conclusions reached? What percentage of cases does bias affect the conclusion of a false positive?

For the last question is it 25%, is it 10%, is it 1%.

Just from the way I have been thinking about the subject I believe that bias would affect maybe 1 of 1000 of 1% of the cases. Maybe less.

Do we really want to develop a process that will affect maybe 100% of the cases or 25% of the cases for something that is human driven and only occurs in 1 case for every 100,000 cases.

I do not know what the percentage of cases driven to a false positive by Bias. But I would like to see a good study to tell me the answer.

Once I know how prevalent it is, then develop a process on how to limit it.

Identify correctly the problem and how prevalent it is and then develop the process.

Developing a solution before you even know how to define the problem is like putting the cart before the horse.

I am not opposed to blind verification, but the devil is in the details. But there has been a substantial reaction to very little data, and I am inherently opposed to changing too much without better justification. In Mayfield, we know that the third person admitted a hasty judgment largely based on his knowledge of the other two examiners. As to examiner two, we don't know if it was truly bias, for as a retired examiner working on contract, he declined to talk to the endless inspection and review teams. We assume it was due to some form of bias, or it just could have been the same type of error as committed by examiner one. Other samplings of alleged bias are still quite small. However, there are studies underway, and these findings may shed better illumination on the issue.

One benefit for those who have not been in a cave for the past three years is that Mayfield happened. I recall a conversation with an FBI latent examiner during which he commented that he has not done a comparison since without thinking about Mayfield. Sometimes being forewarned is adequate to forearmed. My mentor was more direct in his instructions concerning verfication. Your job, he said, is to prove the other guy wrong if he is. Don't rely on anyone else's judgment, not even your mother. I loved Mom but I have followed that ever since.

I personally like random selection rather than some other mechanism. We use that and while the percentage of cases reworked is relatively small compared to the examiner's total output, the review encompasses more aspects of casework than just identifications. It works and the time impact is small.

I'm interested to see what other examiner's definition of "blind verification" is. In my opinion, I dont see it as a complete re-examination of all comparisons. To me that is a technical review.

I see blind verification as a process whereby the verifier doesn't know that the comparison they are doing has been already identified by another examiner. I dont see the need for a complete re-examination of all latents in a particular case for fit the 'blind verification' definition.

Do we really want to develop a process that will affect maybe 100% of the cases or 25% of the cases for something that is human driven and only occurs in 1 case for every 100,000 cases.

If there is a chance of putting the wrong person in jail in every 1 of 100000 cases, I'd say that a process is needed and is needed now.

Angie and I are lucky. We can take our individualizations to outside agencies. I think that is one of the best ways to avoid confirmation bias. The outside examiner knows little or nothing about the case so will not be influenced that way. The chances are practically nil that there will be a conspiracy between two examiners in two different departments to knowingly do a bum ident.

If an outside examiner brought me a case and was acting odd about it, saying "it's okay, it was easy, just verify it, trust me," I would look even harder and longer at it. I put my reputation on the line every day and I am not going to be sloppy by putting my name on my own case or a verification for someone else if I am not 100% sure.

I have heard that, in some agencies with several examiners, one or more examiner can be intimidated by more experienced examiners. I have also heard that some older examiners are intimidated by younger examiners with better eyesight. Every examiner has to be trained to have the courage to say "I cannot see this individualization. Can you demonstrate your findings?" And, if that examiner still cannot see it, additional examiners and/or a supervisor should be brought it. Either the first examiner made an erroneous identification or the second examiner needs more training. Either way, no one should be bullied or tricked into making a verification.

Kathy Saviers

Andrew, I agree 110% that one innocent person in jail is an intolerable situation. That is why verification is so important to be done right. Like others I was instructed to find a reason why these two prints did not come from the same source. But I have heard that some times others just use it as a stamp of approval and not even look at it. Bad move. Is that bias, some have said yes it is. I prefer to think of it as sloppy work and bad habit.

As Steve Meagher said the court is our customer. Well that is true but the public is our customer as well. When we start talking about increasing the level of documentation the amount of service is going to go down. If I have 20 cases backlogged and working through the blind verification, and one of the other cases could led to a suspect that is dangerous.

What I am trying to say is that we walk a thin line between being absolutely correct 100% of the time and providing a means of good case work in a timely fashion. It seems we are always trying to balance between the two.

I am not saying to put a innocent person in jail, but can we balance the two and solve crime in a timely manner to stop future crime. It is a high wire act and if we fall either way, we lose or worse someone else loses.

Daves approach is good. Mrs. Saviers approach is good. There has got to be more than one way to solve this problem.

With the number of single person agencies, two person agencies, all the way up to 200 examiners, there is going to have to be a number of different approaches.

If the powers to be decide on only one way then some of these shops are going to have to shut down. That is not helping the public customer base either.

Let us look at all of our customers needs: The Courts, The Public, and Law Enforcement.

Andrew, we interpret a technical review differently. All cases are technically reviewed by another examiner but that does not require comparing latent prints not identified. What we employ is referred to as a random reanalysis in which the case is reworked from start to finish by another examiner who does not know the results of the first. Some restrictions are placed on the nature of the case to ensure there are suitable latents and suspects or elimination prints. Some cases may specify AFIS searches or other parameters, such as substrate type. Technical review, randon reanalysis and proficiency testing (both external and internal) form the core of our quality assurance. In addition, each year, the laboratory is audited by inspectors from outside the lab who will review all aspects of the case, including documented identifications. We have the luxury of six other labs to select outside inspectors, and, on ocassion, we will use someone from another system.

I doubt you will find anyone to disagree with your concerns about the consequences of just one false positive, and I think that is reflected in the seriousness most agencies approach quality assurance. But our former head of the laboratories defined quality as a three legged stool that remains stable only if attention is paid to all three legs. Yes, we want accuracy and thoroghness, for without these legs the stool falls. But timeliness is also part of quality, for the cases we can't get to in a timely manner degrade quality in a less obvious but justly as costly way. We must be judicious in what we do to determine the value of the return to any investment. At last report, the FBI's new procedure to prevent confirmation bias involves blind verification of 70% of the cases. That is a huge investment in resources, and done so on very little hard data that the remedy actually addresses the problem.

If I read Charlie correctly, the information currently available is just inadequate to advocate a drastic alteration to procedures. Alternatives to blind verification may be just as effective and require a smaller allocation of resources. If studies do show the only way to avoid confirmation bias is blind verification, so be it. But let us have more study before we hawk a miracle cure.

Dave and Charles,

I didn't mean to sound harsh in my previous post, and I am definately not advocating that everyone needs to move to blind verification right now. I'm merely throwing out ideas of how it could possibly be done, knowing full well that one approach will not work for everyone.

I also think that there is a wider definition of blind review than I thought.

Andrew, you did not sound harsh to me. You brought up a very good point that I did not put in my post. You got good ideas, keep throwing them out there.

Dave's three stool analogy says it in a nutshell.

On to your question: We do technical reviews of all murders, robberies, and sexual assaults. A technical review is going over the work sheet, determining if the conclusions are proper. If the latents on the cards are marked properly with those that are suitable and non-suitable. Comparison of the non-id's with the suspects. But we do not look at the ID's in TR. That has already been done in verification. Something we may think about and change. All ID's triple checked.

Blind Verification is a little bit harder. I have seen some people say it only refers to the ID's. Others say for it to be truly blind then the non-ids as well. Some say with just the suspects, while others say for it to be truely blind some other non-suspects should be included. I have even seen where all examinations are done blind. I think that solution is really scary.

I suppose someone at sometime will develop the definition for it, but right now, yours works as well as mine.

I like Dave's approach to the problem and maybe we might try to do a random thing as well.

What no one has really touched on yet is the ACE-V Documentation. There are three types that I am familiar with. Written, Visual, and Combination.

The written version scares the PJ's out of everyone in this office. We and the state opt for the visual.

I would like to see more discussion on the different ways of Analysis Documentation.

I like Mr. Ashbaugh but we are certainly not doing it the way he describes in his book.

I didn't think you were being hard, Andrew. I liked what you said. I think this is an excellent topic for discussion and I think it is obvious that we have differing ideas of how to address what may need our attention. But we have drifted from something that does affect eye witnesses to a factor in most verification practices. What we don't know is how large is that factor, how it can be prevent or nullify the consequences if it is significant, and what will work according to agency need. You provided the motivation, for quality work means never abusing fingerprint reliability. You have a strong voice for that priority.

Thanks for the comments, I just wanted to make sure that you all knew that I wasn't trying to be critical.

Charles, as far as the documentation of ACE-V. I haven't addressed it cause that topic gives scares my PJ's as well. Like I said before, it is a slippery slope and if not thought out very carefully could put us under an avalanche.


On Tech. reviews: I have always treated a tech review as a full review of the case, comparisons included. I look at everything Charles mentioned with the addition of looking at the ID's again. At my former agency, usually the tech review was performed by the verifier so it wasn't really more work.

In my opinion, blind verification is an attempt to get a second opinion free from as much bias as possible. I dont think that needs to be a full blind review of all comparisons in a case. I think that presenting ID's to the verifier without any indication of my conclusion is a blind verification. Throw in random non ID's every so often to keep the verifier from getting complacent (automatically thinking that this is a 'verification' as opposed to a 'review') and you have blind verification.

Might I suggest thinking about the time issue in two ways. There's the question of extra time to document and/or blind verify during the ID process.

There's also the extra time taken for preparation and testimony if a defense attorney sees minimal documentation or a lack of blind verification and decides to try a Daubert challenge along the lines of Langill. Or if you get into a longer cross-exam over issues of suggestion and confirmation bias. It gets much worse if defense gets an acquittal for someone who is in fact guilty because of preventable vulnerabilities. (It's my job to represent my clients zealously within the bounds of the law, regardless of my opinion of their guilt or innocence, but that doesn't mean I like seeing someone I think is guilty walk.)

There are some trade-offs here. Good documentation is a good shield -- it will tend to dissuade defense counsel from pushing the fingerprint as an issue (unless he/she has little choice -- the key evidence in the case, FREX).

So I'm curious, why does written documentation scare folks? Is it just the increased work, or are folks worried about creating cross-exam fodder?

Dear Ms. Steele,
I don’t think that anyone is worried about cross-examination fodder. If any untrained person tried to decipher the prose detailing the intellectual process of comparing one fingerprint impression to another, I’m quite sure their head would explode. This may be to the delight of prosecuting counsel but of course would just cause delay after delay. I am sure that Judge Coffey had no idea what she was actually asking for in the way of documentation. The judge offers no guidance in the matter. I suppose she is falling back on that old judicial chestnut, “I know it when I see it.” “I shall not today attempt further to define the kinds of material I understand to be embraced within that shorthand description [hard-core pornography]; and perhaps I could never succeed in intelligibly doing so. But I know it when I see it, and the motion picture involved in this case is not that.” Justice Potter Stewart. Shame that doesn’t work for fingerprint examiners.

Consider the professed methodology of ACE: Analysis, Comparison and Evaluation. This does not occur in a single instance of the comparison process but many, many times. Each instance of an ACE event would have to be described in detail. That may not be too complex when describing Galton points. But when using other features known as third level detail, the descriptive could be a paragraph for a single detail. Once you have described those things that have lead you to your conclusion, you may need to address other issues such as distortion.

Realistically, these descriptions are of no use to competent examiners since each will use the actual prints and not the descriptive to arrive at a conclusion. They may provide insight in to how an erroneous individualization was made but add nothing to the actual examination process. Such an effort, then, needs to be directed to the uninitiated and I imagine would follow along the lines of a courtroom demonstration of a charted comparison. Now, you know, Ms. Steele, that five minutes of testimony in court can easily translate into ten pages of transcript. And that is why there is reluctance to provide a written descriptive of the ACE process. Would the verifier have to provide similar documentation? He would if the effort was truly blind. Better not grumble come April 15th. We’re going to need a lot more fingerprint examiners out there.

Latent print examiners are tasked with providing a comparative analysis of fingerprint impressions. They are not hired to write the next John Grisham best seller.

Ms. Steele, you spend a lot of time informing yourself of fingerprint issues by participating on this forum and elsewhere. How about giving us some direction. Given what you know about fingerprints, which I suspect is considerably more than most attorneys, what sort of information would you like to see in the examiner's documentation? (If you want specifics, please be specific; if not, what general information do you seek?)