Confirmation Bias Study on Experienced Examiners

[Patrick Warrick]

Excuse me, but my testimony is about the techniques I used to process evidence for the possible presence of friction ridge detail and the conclusions I formed during comparisons to known exemplars. If someone goes to prison, or is even executed, based on my findings and testimony...so be it. I do not testify that John X's fingerprint was on the trigger of the gun, therefore he killed Mary Y. I testify that at some point John X touched the trigger of the gun....period. A decade later, DNA exonerates John X from the crime, I will still sleep soundly because my testimony and conclusion was not wrong. The fact remains that at some point John X touched the trigger of the gun....period.

Patrick Warrick

[Pat A. Wertheim]

The question is not how you would feel about having made a correct identification used against an innocent person, but how you would feel about having made an erroneous identification used against an innocent person. Perhaps in a few years after all the Mayfield issues have been resolved, some of the FBI examiners will share with us their personal feelings. But Brandon Mayfield only spent a few weeks in jail. How about a suspect who was erroneously identified who subsequently lost his job, whose wife divorced him and took the kids, who spent ten years in prison being abused by other prisoners? How would you feel about being the examiner who provided the crucial fingerprint identification that turned out to be erroneous? I know every examiner reading this is thinking, "Oh no! Not me! I use ACE-V correctly so I don't have to worry about that. I would NEVER send an innocent person to prison." Well, I suspect the three FBI examiners would have made that statement with sincere conviction the day before they made the Mayfield identification. And as I recall, they and the FBI stuck by the identification even after it was called into question. So I wonder if any of the FBI examiners will discuss their feelings after the litigation is over and the smoke clears. I wonder if anybody else who has made an erroneous identification would care to tell us how it feels to have been responsible for something like that? I suspect there are a few reading this who are in that position.

I'm not saying, "There but for the grace of God go I." But I AM saying nobody should think it could never happen to them. And I am saying we really need to consider this phenomenon called "confirmation bias" and see what ways we can come up with to minimize its effects.

[Michele Triplett]

I have a friend (Cyndi Zeller) who explains bias very well. She articulates it better than I can but I’ll try my best.

Knowing information about the suspect or victim isn’t bias within itself. When you’re using this information to support or justify your conclusion, then bias exists.

How do you know when this information is biasing you? By doing appropriate testing.

When the latent has plenty of individualizing characteristics, testing may only require comparing characteristics in the latent with those in the known. As the amount of individualizing characteristics goes down, additional testing may be required.

When the latent has minimal quality or quantity, blind testing may be needed. Give the latent and the known prints to different examiners without any information. The number of examiners you give it to will depend on the quality and quantity of the latent. Ask the examiners to document why they’ve arrived at their conclusions. You may find that the characteristics you are using to support an identification aren’t reproducible to other examiners. Besides using blind testing, you may want to do additional testing, such as the use of predicting additional characteristics. Set the print aside for a few days. When you go back to it note at what point you think you have confirmed your hypothesis. Then look for additional information in the latent and see if you can predict it’s existence in the known print. If you don’t have additional characteristics to test your hypothesis, you my not want to individualize it. Another tool to use is consultation with other examiners.

After you’ve done what you consider to be adequate testing (I’ve mentioned 4 testing methods above), make your conclusion. It should be based on all the testing methods you’ve used, not the extra information you know about the suspect or the case. The different testing methods should be noted.

After you’ve arrived at your conclusion and finished the case, the case should go through your normal office procedure. In some cases this includes a verification process where the verifier corroborates the conclusion. In other offices the verification process is a complete peer review process where the conclusion isn’t only being corroborated but the verifier is also making sure adequate testing was done, proper procedures were used (like having reproducible data), and the proper amount of documentation was done. I believe (but could be wrong) that some offices refer to this as a technical review.

When conclusions are reached in this manner, the possibility of bias existing is minimized. This may be time consuming, but think of how often you have latents with very minimal quality or quantity. This complete process is only needed in a small percentage of cases so the time involved in doing a correct analysis is minimal and necessary to reduce errors.

Michele
michele.triplett@metrokc.gov

[Steve Everist]

And is there no way that the police can't contact the examiner and say, "could you look at Suspect X on this case" without communicating why they are interested in X? The major bias problem would be the examiner knowing Suspect X's record.

Is it common for examiners to receive this additional information beyond a suspect's name?

In our office, the latent print requests come in to our clerical office. When they are distributed to the examiners, if a suspect is known, the suspect name information and corresponding key number will be listed. It is very rare that we actually have a conversation with the detective regarding any of the suspect(s) listed. More often than not, there is no suspect information made available. When there is, I usually don't know anything about the suspect's history unless I go to the file and find a dozen different 10-Print cards on file to use for my comparison.

Just thinking out loud regarding the issue of blind/double blind testing. When I receive a name to compare, unless noted or obvious as victim information, I automatically assume that the detective has some reason to suspect this person. But as Michele posted earlier, "Knowing information about the suspect or victim isn’t bias within itself. When you’re using this information to support or justify your conclusion, then bias exists."

So to extend the idea of blind/double blind testing, should our clerical office randomly add suspect information to cases so that we won't know if all, some, or none of the named subjects for comparison may actually be a suspect? Since this sort of testing happens during the comparison stage, that seems to be one way to make it blind. It may cause a little confusion when detectives start getting reports including comparisons to subjects that they never requested though.

Personally this doesn't sound very practical.

[David Fairhurst]

So to extend the idea of blind/double blind testing, should our clerical office randomly add suspect information to cases so that we won't know if all, some, or none of the named subjects for comparison may actually be a suspect?

I would say that with caseloads being as they are, most labs don't have the time and resources to start doing all these extra examinations, and if they did they would have a hard time justifying it to the budget managers when they contribute nothing to the investigative process.

Then there's the inevitability of one of your random suspects being identified. How would you explain that in court?
"Oh yes Your Honour. We just picked a name at random and whadayaknow, it turned out to be the right one. Lucky us."

[L.J.S.]

[/quote] In our office, the latent print requests come in to our clerical office. When they are distributed to the examiners, if a suspect is known, the suspect name information and corresponding key number will be listed. It is very rare that we actually have a conversation with the detective regarding any of the suspect(s) listed. More often than not, there is no suspect information made available. When there is, I usually don't know anything about the suspect's history unless I go to the file and find a dozen different 10-Print cards on file to use for my comparison.

Just thinking out loud regarding the issue of blind/double blind testing. When I receive a name to compare, unless noted or obvious as victim information, I automatically assume that the detective has some reason to suspect this person.[/quote]

I don't see a problem with the basic knowledge that the investigators have suggeted a name. If you've got a 10 print card in the files, you obviously know they've been arrested before. The name and the card seem to be minimal necessary knowledge to do your job. I don't think randomly adding distractor/foils adds much to avoid the bias. (Tho one might want to look at the eyewitness ID folks' research on cautionary instructions to folks coming in to view a line-up or an array who can infer that someone in the array is of interest to police.)

Where I'd see a potential problem is if the 10 cards list prejudicial data, such as the substance of the arrests or convictions. (Humm, I'm analyzing a print in a residential burglary case; investigators suggested Suspect X who I see has a dozen 10-print cards for residential burglary arrests.)

Is it common for examiners to be told if the names are suspects, elimination prints from those with normal access, or victims, etc? Or does one just get a list of names. (And might be able to infer status as victim, elimination, or other, if the 10 cards are all recently dated.)

[Norberto Rivera]

Personally, I'm terrified to make an erroneous individualization. This may be due to the fact that I'm a new examiner and don't have the extensive experience that many of you enjoy, however it makes me take a very cautios approach to any comparison I make. I receive history information and investigation updates on a regular basis on cases which I've worked. The nighttime population of the city where I work is 25k. It's hard after several years to not know who my local burglars and crack heads are. It's a bias in that I may pull one of their cards for comparison if I know that a particular person lives close to the crime scene or frequents that area. I don't think I'm biased in my comparison process. I think part of the problem lies with "being an expert" and wanting to maintain that status. The best way I can relate it would be to compare it against a competitive shooter. To be the best shooter you take longer and more difficult shots. To be considered an expert examiner you should be able to make individualizations on latents of less and less quality and clarity. I'm not trying to offend or insult anyone's intelligence here, but I've seen the Mayfield latent online and it was very "poor" (I could find a better descriptor, but not for this format). Actually I think poor is an understatement. I don't think that even 25 or 30 yrs. of experience in the field I would even attempt to compare that latent for individualization. Again, this may just be because I'm inexperienced, but at some point I believe that a minimum criteria for quality and/or clarity should be quantified before a comparison is attempted. A latent either does or does not match a known exemplar, regardless of the skill of the examiner and the process he/she uses to arrive at a conclusion. The difficulties arise when the examiner begins to make "educated guesses" rather than relying strictly on the information in front of him/her. (i.e. looking at the print) I've looked at many a latent lifts that were just plain dirt and I'm sure all of you have. How do you decide when a latent is "of no value for comparison purposes"? Is there a specific number of characteristics that should be visible? Should the contrast be of any specific quantifiable clarity? How do you quantify clarity? To step off my soapbox; Are we driven to push the envelope in order to make individualizations? Would concluding on an individualization of a very bad quality latent lift benefit, help or otherwise bolster the examiner's status or perceived expertise in the field? Does it create unrealistic expectations of newer LPE's to peruse through? (pls pardon my language, It's very late here and i'm tired) [/b]

[Donna]

There is something to be said for confirmation bias. It is what we should strive to prevent. However, some policies suggested by those that conduct ASCLD inspections, may in fact be pushing an examiner toward some bias.

While I don't believe that an ethical analyst would determine someone was guilty simply on the basis of knowledge, it would be easier for the lazy or unethical analyst to confirm another identification, if the identification were already known. For example, if the first examiner looks at the case with no outside information and makes an ID, that is more objective. However, if the by policy, the first examiner now must mark each print that is analyzed and subsequently write the finger (i.e., right ring) next to the latent that was identified, the second and subsequent examiners now know the answer. Rather than looking independently or objectively at the case, the subsequent examiners now have already limited their area of examination and may in fact be trying to confirm the previous identification.

In my opinion, it would be best to make all identifying marks on the latent lift card AFTER the last verifying person has made an examination, not before. While some latent lift cards may contain only one latent, it is not likely. At least the subsequent examiners must now look at all latent prints on the card and make an independent determination as to what finger (if any) should be identified. Less likely for confirmation bias.
I argued that very same point with an ASCLD inspector. Since that was the way they do it, we then had to change our entire procedure. Rather than making the analyses more objective and independant, it was made less.

Also, in regards to taking notes about our examination. While I see no inherent problem with notetaking, in some labs it may in fact add to a case log (in addition to all the additional paperwork that has been required due to ASCLD certification and overzelous administrators). I've taken notes on cases when I knew it would be a while before I could talk to someone and discuss a case, just to refresh my memory and prevent me from having to re-examine the entire case prior to conversation. However, how often have you looked at a latent print and used some focal point on Day 1, then found a different focal point on Day 2? Your notes can refresh your memory, but leads again to overlooking other identifying information that you did not use in your original notes.

How many times have you looked at a latent print at the end of the day and just couldn't ID it or determined it was a no make, only to look at it the next day with fresh eyes and see that latent ID clearly and easily? Would your notes have helped then?

Two things that I may have missed in this conversation and previous conversations is: (1) that each examiner who conducts a comparsion should be trying to disprove that the finger belongs to the subject. The examiner should assume that the fingerprint does NOT belong to the exemplar, and therefore must look for differences, rather than just similarities. Chances are that this theory will be disproved, thus confirming (again) that fingerprints are unique. (2) AFIS is an objective measure of the reliability and uniqueness of fingerprints. How often does the victim get identified with an AFIS hit? The computer (and possibly the analyst) doesn't know the name of the victim. How then could the victim be arbitrarily picked from a database containing millions of prints and thousands of subjects? How often is the detective's suspect identified using AFIS, yet the analysts had no idea that the subject identified was the same person the detective had in minde? After an AFIS hit, the examiner then confirms or denies the information.

Simon Cole and Brent Turvey both often imply nefarious thought patterns in their discussions of confirmation bias. They imply that the examiner is trying to prove what the detective may already suspect. Realistically, how many of us have gotten detective requests that are not supported by our examinations? How many of us have even had the chance to talk to a detective? In large agencies, they have just as much casework as we do. They wait for us to provide them information, in order to work the case...not the other way around.

I think we should focus on our own procedures that may lead to confirmation bias and we should focus on the procedures we use that may in fact help show the objective measure (AFIS) that is used every day in our profession, that also strengthens the point that fingerprints are permanent and unique. I don't think anyone is trying to say we should do less. However, more published studies showing the affirmation of the process need to be done. Current scientific practice, whether it be physics or chemistry, are always an affirmation of a paradigm. You don't publish and re-publish about accepted theories. Journals don't really want to hear about something that has been accepted for centuries. How often do you read new articles about Galileo or Newton? Often only when a NEW process has been utilized that incorporated previously accepted scientific paradigms.

Simon Cole understands that in order to publish, you need to challenge a paradigm. He obviously can't publish a new theory on fingerprints, as he has now knowledge base from which to work. All he has left to challenge is the process or the people. Now he is indirectly challenging the accepted paradigm, and finging a way to make himself notable.

For those of you that are interested and have access to a university library, Simon Cole just published a 95 page article in the Spring 2005 Journal of Criminal Law and Criminology titled "More than zero, accounting for error in latent fingerprint identification". We should also recognize that Simon Cole has found his niche that will garner his 15 minutes of fame, based on something in which he in fact knows very little.

Donna

[Charles Parker]

I wish to thank Ms Triplett and Ms Zeller for the posting last week. The first part of the posting went something like this.

"Having knowledge about something before hand is not BIAS within itself. When you use that prior knowledge to justify your actions, conclusions, or opinions, that is when bias exists."

When I read that statement, it crystalized my thoughts on that particular subject. I was yelling "Way to go girls" in the office. However I did not get to e-mail them right away. The next day I was due to testify in an Armed Robbery case in the next county. After sitting around all day and finding out that the girl that developed the latent print had been on the stand for a little over 3 hours with Daubert like questions, I was told to come back the next day.

Finally before lunch the next day I was called to the stand, and when the defense started in, it was about Mayfield, Error Rate, Methodology, General Acceptance, etc. .

Then the questions turned to "Confirmation Bias". After a couple of questions on that topic the defense then asked me what I thought about Bias (she called it mindset). I then turned to the jury and repeated what I posted at the beginning. The jury was nodding their heads, the prosecutors were smiling, and the defense stopped, turned to her co-counsel and talked briefly, and then finished with a couple of questions about some books she had, and then passed the witness.

Instead of trying to explain the system to the jury and probably lose some of them in the process, I made that statement to KISS, Keep It Short and Simple.

I love that statement. I love this site. What better place to pick up information like that, than with a lot of other Latent Print Examiners (and a few attorneys).

Thank You Ms Triplett, and Ms Zeller--You made my day.