CLPEX Chat Board

Recently there was some good discussion on examiner bias. That thread equated influences on eyewitness testimony to influences on latent print examiners. Personaly I believe this is a poor comparison. HOWEVER, I do think examiners can be biased, especially when they are told up front what conclusion to come to.

Part of the examiner decision making process is subjective, and I think as a profession we must acknowledge dangers, and do our part to keep the ACE-V process clean.

I came across this interesting article for those who have not read it:

http://www.chicagotribune.com/news/spec ... ecials-hed

Any comments?

There are several sources of examiner bias. In the Mayfield case, we generally discuss bias as "confirmation bias," that is, bias generated by the knowledge that another competent expert, perhaps one we know and trust personally, has already made the identification. But there is also a bias generated in some comparisons by the fact that we have already made several easy identifications to the suspect when along comes a marginal latent print, which we are subliminally tempted to identify because we already "know" the guy touched the surface. Then there is the self-induced bias on AFIS when we pull up a candidate list and, on viewing the top candidate, we experience the "eureka!" moment. I believe this happened in the Mayfield case to the first examiner.

As far back as ten years ago when I was teaching my fingerprint comparison class full time, in almost every class a student would bring me copies of images of close non-idents. I have seen dozens of latent prints with inked prints having 8 or 10 or more points in relative position that, minus discrepancies, we might be tempted to call "idents." Now that I have been back on the bench working cases for five years, I have pulled a number of those off AFIS myself.

I remember reading a "letter to the editor" back in the very early 1980's by an old time examiner who was concerned about AFIS for the very reason that it might produce close similarities in otherwise non-suspect persons. I have searched for that letter in vain over the past couple of years. It might have been in "Identification News" or it might have been in the old Texas Division newsletter. I just can't remember (my sister accuses me of "memory leaks"). But the point of the letter was that in the pre-AFIS world, we only compared a latent to inked prints when a viable suspect had been developed, whereas in an AFIS environment we would compare the latent to millions of nonviable persons or, in other words, people who were NOT in the group of people with opportunity to have committed the crime. The writer of that letter cautioned that when using AFIS, we needed a higher standard for identification or a higher level of certainty or a higher degree of caution, before making an identification. That's an application of Baysian philosophy, right?

At the time, I thought to myself, "What a load of crap" (to quote from another concurrent thread). But now, in hindsight, I realize that old timer was right on target. When we pull up an AFIS image and see our target, whether it is a cluster of points, a distinct pattern match, a scar, or whatever, we get that "warm, fuzzy feeling" that Dusty Clark refers to as inadequate basis for an identification. At that moment, we have induced confirmation bias into our own mind for the rest of the comparison.

In the Mayfield case, I believe that self-induced confirmation bias infected the first examiner. I believe he experienced that "eureka!" moment and got that "warm, fuzzy feeling" before he ever seriously dug into the comparison. Subsequent examiners -- the verifiers and even the independent expert -- were infected by the bias that a known, trusted friend had already made the identification. We need to be aware of confirmation bias no matter what the source of the bias and no matter what we are confirming (identification, exclusion, inconclusive). And we need to be on guard to balance that bias somehow.

I don't know the answer, except through awareness of the problem and constant vigilance to protect myself from its effects. Perhaps the researchers investigating the psychology of our comparison and identification process will come up with a solution.

Blind verification may be an answer. But in the case of the "eureka!" moment, what is to prevent a blind verifier from experiencing it the same as the first, unknown examiner did? I would be thrilled if someone could propose an answer to this dilemma.

Strict Scrutiny:

I do think examiners can be biased, especially when they are told up front what conclusion to come to.

This was also something I believed. So I recently conducted a contextual bias study to test this hypothesis.

Anecdotally, I have observed instances where I saw this to be true, but in our experiment where we had a control group, minimal bias, and strong bias groups, we did not see a profound and obvious effect from bias, as critics have conjectured. While I recognize the work of Dror, Chalton, Peron as groundbreaking and important, the bias done in their expert scheme was overexaggeratedly strong, if you will. I was interested more in the typical day-to-day influences one might experience from a colleague or trusted examiner.

In any event, I will be presenting the results of the study at the IAI this year and hopefully have drafts for publication at that time too. We are still working through the data to assess its signficance, but at first blush it is clear that individuals simply did not "bowl" over and acquiesce to the prompts and bias they were exposed to. That is encouraging.

Dror (I believe in Evidence Technology Magazine 2006, vol 4 (3)) discussed the fact that while his group has demonstrated examples of bias, he readily accepts that not all people will react to bias in the same way, some can be resistant to it, and in fact, training and awareness may increase resistance (not eradicate, but reduce). These are important aspects of bias. I would also submit that the Spanish Natl Police in the Mayfield case are perfect examples of this. Given the strong influence of the FBI in that case and their insistence of a match, the SNP still resisted and did not succumb to the contextual information that was provided. To me, that is a shining example of being able to resist strong bias in favor of objectively assessing the evidence.

Lastly, we can discuss all we wish about the potential for bias. As a community what we really need is the research to highlight the true weaknesses of our discipline so that we can address THOSE, and not make radical changes, submitting to wishes of critics merely because they suggest the "PHANTOM" possibility of bias error, without any data or APPLICABLE studies to demonstrate that bias is as rampant as they suggest.

That's my take and I look forward to sharing our data with the community. (I have to give credit to Pat Wertheim who generously donated his time, expertise, and assistance to the experiment and to Christophe Champod, who has been a great critic, friend, and mentor in conducting these experiments).

g.

In a somewhat related conversation my friend Lisa Steele and I have been having today via email, Lisa said,

"I think the Courts should be looking very hard at each and every case, but as I understand Dror’s work, the suggestion bias problems won’t override a clear non-match or clear match, it is the hard cases where there’s a problem. What I’m most unhappy with is judges who say “Geez, counsel, it’s been good since the 1920s, I’m going to admit no questions asked”, or my (defense) colleagues who say “Geez, I’m not going to win this Daubert thing, so I’m not going to do any cross.”

Lisa has a point. The majority of identifications made in our business involve latent prints that are unambiguous. But we should be able to answer questions under an aggressive cross examination. It is the "borderline" latents, smudged, double-taps, etc., where the greates potential for error lies. How do we distinguish? One examiner's difficult ident may be easy to another examiner. Some are hard for the best of us.

I'm looking forward to Glenn's published research, which should help answer a number of these questions, and the research of others that is also underway.

Just mulling over one of my earlier posts on this thread in which I postulated three forms of confirmation bias, when a fourth one poked me in the eyeball (actually, a slight variation on one of the earlier biases I discussed). I was sitting here at my bench doing comparisons this afternoon when a particularly difficult second identification popped up. The first identification was an easy #9 finger on an AFIS hit. Adjacent to it was a smear with a pattern matching the #8 finger of the suspect I had just identified. Simultaneous impressions? After an hour, I claimed the second identification, too. There were some artifacts in the smudge that were open to interpretation. Was I making subjective evaluations of those artifacts and resolving them in favor of the second identification based on a bias from the first identification? I have seen other examiners make erroneous identifications of a second fingerprint on the "assumption" that it was simultaneous with an adjacent easy identification. Isn't that also "confirmation bias" of a sort?

Pat A. Wertheim wrote:I have seen other examiners make erroneous identifications of a second fingerprint on the "assumption" that it was simultaneous with an adjacent easy identification. Isn't that also "confirmation bias" of a sort?

I see this too often with my new students, just starting latent print comparison exercises. They make an ID on one print, then 'assume' that a similar looking pattern in close proximity must be an adjacent finger. When questioned, it becomes apparent they were impatient and took shortcuts in (not) applying proper methodology.

I think this may be as much an example of a type of suggestion, as confirmation bias. The 'knowledge' of an ID of a specific fingerprint 'suggests' the latent print next to it likely was made by an adjacent finger, the likelihood further 'confirmed' by similarity of pattern type. This double whammy seems to create a synergistic effect accounting for an increased error rate for my new students in this scenario.

A positive effect is that once bitten by the error bug, and shown the error of their ways, they typically learn more patience and become more adept at practicing proper methodology.
Regards, RLT

Ladies and Gentlemen,

I keep silent about many issues because, as I have stated, I am no longer a practicing practitioner in latent prints, but this subject of bias in opinions associated with identifications is troubling. Sorry about using ‘opinion’ and ‘identification’, but I am an old ‘toad’, ‘dinosaur’, etc., whatever!!

First, I do not believe I am singular, unique or unusual in this position, but I refuse to believe I was ever bias in rendering any of my reports over many years of conducting examinations. I also include the reports and decisions of examiners whose cases I reviewed. First of all, in the venues in which I operated, the majority of cases in the initial examination did not have clear-cut ‘this is the guy/gal’ indicators that would generate a bias to make a specific identification in pursue of justice. I can remember instances, some involving horrendous, disgusting (hated those child cases!) and highly publicized cases, in which I REALLY wanted to associate an individual to the crime, but I did not alter/modify/adjust my comparison values to make that happen. I even remember times in which I had a marginal latent print and in my ‘heart-of-hearts’ knew it was made by their person-of-interest, but I would not call the print.

I am not sure where my attitude/aptitude came from and I am sure it exists in many examiners that do not agree with the bias influenced decision studies. If a study does reveal this exists in an individual/agency/organization, it should be immediately and aggressively addressed. If you believe this is something to which you are susceptible (or proven) to doing, please find another line of work, NOW. I thoroughly agree with the thoughts of Brouardel: “… You have no victim to avenge, no guilty or innocent person to ruin or save…”.

Glenn,

I strongly agree that the issue needs to be studied prior to just accepting confirmation bias as a reality. I would also like to caution that the Hawthorne effect could skew your results on this particular problem. I think if people know that their work will ultimately be checked with the correct answer they may be more conservative than if the correct answer cannot be proven. That said, I agree in principle with your post.

Pat,

I always admire your unflinching look at this profession. I particularly like your statements about AFIS and suspect Information. You have brought that up before. The old-timer you mentioned WAS onto something. You also posed this question a couple of years ago (on this forum) in a slightly different manner. You asked if a badly distorted match came from suspect information rather than AFIS, would the suspect information add more weight to the match? I believe the forum response was a resounding NO! We don’t do it that way! Every print stands on it’s own merit! No outside information is used!

But I know from outside scientific training that bias is an insidious thing. It can creep in when we are unaware. We will often not even realize we are relying on outside information if we are not diligent in recognizing it. I fear this happened in the Mayfield case and we should study the phenomenon and be open with our critics.

Here is an example. I have often seen an examiner bring another examiner a troubling comparison for advice. After some time struggling with the “E” phase of ACE-V, examiner #2 will often say: “Is this from AFIS or a suspect?” My response was always “Why does it matter?”

We all know why it matters. You are the only one I have ever seen handle the issue honestly. Most of the rest of the responses I have received have been strident denials that they would ever let anything bias them.

Our field has a long-standing word of mouth policy that outside influences don’t happen. (Or don’t happen often enough to raise concern).

Ashbaugh addressed the danger of bias slightly, but did not take it too far.

The FBI has always maintained the ID process is 100% objective (subtext = bias can’t happen if we remain 100% objective). I’m sure Charles Parker or Dave Grieve will produce a J. Edgar Hoover memo to prove me wrong, but I’ve looked in the “Science of Fingerprints” and this issue of subjectivity and bias is not there.

I’m going to go back and re-read your examples of biases and try to wrap my mind around the whole subject; including Glenn’s study, all the examples listed (both you and Tavernaro), the Hawthorne effect, and even the strident denials. I think this would make for a good JFI article.

Also suggest reading:

Dror.I.E ;Charlton.D(2006).
Why Experts Make Errors
Journal of Forensic Identification, 56, (4), 600-616

Dror, I.E., Charlton, D., & Peron A. (2006). Contextual information renders
experts vulnerable to making erroneous identifications. Forensic Science
International, 156 (1), 74-78.


Dror, I.E., Peron, A., Hind, S., & Charlton, D. (2005). When emotions get the better of us: The effect of contextual top-down processing on matching fingerprints. Applied Cognitive Psychology, 19(6), 799-809.

Several of these posts have triggered a memory of the case of the “Prime Time Rapist.” A rapist in the mid-1980’s who hit his targets in the early evening hours was given that nickname by the press. After a series of rapes accompanied by high profile publicity, the police were feeling heavy pressure to solve the case. The detective heading up the investigation received a “tip” one evening from a woman who, on her way home from work, was glared at by another motorist, a man wearing camouflage clothing and driving a muscle truck. She got the license number of the truck and called the detective, who she happened to know. He checked and found a minor criminal record on the registered owner of the truck. After a little more investigation, he was convinced the man was the rapist. The detective went into the ID unit late that night. The only person there was Tim O’Sullivan, a photo technician with basic fingerprint comparison training under his belt at the time and not yet assigned to latent print comparisons. The detective demanded an identification while standing over Tim (not prohibited in that department in those days) while hammering Tim with all the reasons why this guy was the Prime Time Rapist. Tim saw what he interpreted as basic pattern type, plus I guess a few points, and succumbed to the pressure. He called it an ident. The detective made the arrest and got his headlines. Twenty four hours later, the completely innocent “suspect” was released with apologies after the internal verifier had second thoughts (yes, she had originally verified the ident, also under pressure, but woke up the next morning with an uneasy feeling) and an examiner from another department declared it an erroneous identification. The innocent guy settled his lawsuit for what I believe was a trifling amount. (The real rapist was correctly identified a short time later and, as I recall, was killed in a shootout with police trying to arrest him.)

Did Tim undergo a form of “confirmation bias?” Arguably, I think you could say so. He was bombarded with all of the positive bias “proving” this was the right guy, pressure to make the identification was applied, and he confirmed the detective’s belief. Isn’t that a variation on what a verifier normally does, confirm the belief of another person? I met Tim a number of years after the Prime Time Rapist case and got to know him well. He had learned his lesson. I never met a better but more conservative examiner. His shame over that case and his internal fortitude forced him to salvage his career and his reputation. He went on to become a certified examiner and he never repeated the mistake. By the time he died a few years ago, I would say he had thoroughly redeemed himself. But Tim’s case sums up many of the points made by Bob Tavernaro -- 1) a beginner; 2) extraneous information suggesting the identification; 3) failure to apply the proper methodology; 4) as a result of their error, he learned more patience and became adept at the methodology. In addition, I believe Tim gained a tremendous amount of strength over the affair, which made him able to resist future efforts to confirm an identification because somebody else thought it was correct.

I worked with another examiner in the 1980’s who once gave a bullying detective two identifications in an armed robbery on my days off. When I came back and looked at the prints, I immediately raised the cry to halt the process and release the “suspects.” Of the two “identifications,” one was an easy exclusion to both “suspects” and the other latent was a worthless smear. When I confronted the other examiner in private, he was shamed over the incident and explained it by saying he “just didn’t have his priorities straight.” In my opinion, his problem was one of ethics. He was not a very talented examiner so he tried to please the detectives by giving them what they wanted. A week later the correct robbers were arrested and they immediately confessed. None of the latents from the crime scene were ever identified to either of them.

It is important to understand that in our methodology the process is objective, but the final decision (identification, exclusion, inconclusive) is made in the mind of the examiner and is therefore, by its very definition, subjective. Being subjective, the decision may be subject to external influences, to a greater or lesser degree. Not all people respond to those external influences the same. Dave Charlton’s work with Dr. Dror indicated that some examiners resist external pressure better than others. I believe that may be a self confidence or self image related characteristic.

Ernie Hamm stresses that, in effect, the identification process should be all about the print and nothing about any other factors. Ernie is absolutely correct. Ernie has one of the strongest personalities and highest sense of ethics of anyone in our business. New examiners in the field would be well advised to meet Ernie and other "dinosaurs" like him. There is much to be learned from the old timers. If all examiners were as strong as Ernie and had the high integrity and ethics Ernie has, we would not be having this discussion (we also would not be having some of the other discussions on this website.) Unfortunately, we have people in the business who lack Ernie’s strength and integrity. We have the beginners Bob Tavernaro refers to, and the under qualified people like Tim O’Sullivan when he made the erroneous identification in the “Prime Time Rapist” case. And we have people who lack ethics. Obviously, we have more to work on than “confirmation bias” to make our profession better, but understanding this source of influence and pressure can help each of us individually to resist it.

I think this is an interesting example of what you chaps are discussing.

[urlhttp://www.scottish.parliament.uk/business/com ... eilPDF.pdf][/url]

Trying again for the link...

http://www.scottish.parliament.uk/busin ... eilPDF.pdf

Ernie, thanks for jumping in. As usual, your contribution is appropriate and sensible. I am a little concerned this is something else in which sweeping generalizations are getting out of hand. It seems to me we are individuals first and foremost, and our occupation, career or whatever is as an examiner, not the other way around. Bias invades human existence, and some are more prone to the influence of bias than others. I don't believe this is an old school versus new school, science or non-science issue.

I am never opposed to research but I am concerned that the inference this is a separate pitfall reserved for latent print examiners is misleading. Some people maintain the discipline to reach conclusions based upon what is before them in the images while others may be adversely swayed by extraneous information.

Before the bias train gathers too much momentum, I think the post mortem assignment of an alleged confirmation bias to the Mayfield case is pure speculation and reflects a bias in the part of those conducting the autopsy. The first guy screwed up. It happens. Yes, the warning issued a long time ago about AFIS doing exactly what it is supposed to do is warranted, but how much that influenced examiner one is something he cannot answer, nor can anyone else. Guy two screwed up, but since he would not talk to the panel or investigators, we will never know whether he was influenced by bias, staying up too late the night before, had the ball game on the radio, or just planning his next vacation when he should have been paying attention. Mike, guy three, admitted he screwed up and gave the supposed identification only a cursory look. Was he biased? Maybe, but the main thing is, he didn't do his job. Confirmation bias, AFIS bias, terrorism case bias, FBI bias or some unknown bias MAY be part of the answer, but, bottom line, the system failed. These were not green rookies but experienced examiners who all knew about external influences, warned others about external influences and still screwed up.

My main concern is that pitfalls exist that are as old as the profession, and being aware of them helps. But when these external influences are presented as some form of affliction, then we have shifted responsibility aware from the examiner to the influence. Nope, I won't go there.

David L. Grieve wrote:Ernie, thanks for jumping in. As usual, your contribution is appropriate and sensible. I am a little concerned this is something else in which sweeping generalizations are getting out of hand…. Bias invades human existence, and some are more prone to the influence of bias than others. I don't believe this is an old school versus new school, science or non-science issue….

I am never opposed to research but I am concerned that the inference this is a separate pitfall reserved for latent print examiners is misleading….
Before the bias train gathers too much momentum, I think the post mortem assignment of an alleged confirmation bias to the Mayfield case is pure speculation and reflects a bias in the part of those conducting the autopsy.…

My main concern is that pitfalls exist that are as old as the profession, and being aware of them helps. But when these external influences are presented as some form of affliction, then we have shifted responsibility aware from the examiner to the influence. Nope, I won't go there.

I can’t say I agree with any of this except that the danger of bias invades all of human existence and that the risk has been there since the beginning of the profession. I think most sciences are more receptive to criticism than the forensic sciences--because it is from criticism that advancement is stimulated, and bias is identified and eradicated.

If we accept that forensic examiners are more resistant to the acknowledgement of bias, we are then left to answer the question, “why?”

I doubt there will be many inside the fingerprint profession to make this acknowledgement so the subsequent question “why?” is probably a moot point.

Interestingly I found the following article on examiner bias:

http://www.scientific.org/case-in-point ... mpson.html

The article examines subjectivity and bias in DNA work. I think it would be appropriate for all forensic scientists: DNA, tool mark, fingerprint, toxicology, etc… to encourage a running debate on the issue, since it has always been with us, and it appears that it will never go away.

I wish I had the source of the following and, for those that know me; I am really into knowing the source of information. However, I do not know this citation because I read it many, many years ago and, even though I have tried, I can not locate the reference. However, the article essentially stated that when you are on the stand as an expert witness, there will eventually be the ultimate question: “Will you state your opinion.” At that time, you are ALONE!! There is no one with you in that witness stand. There are no back-ups, no reference material, no documentation, no graphs, no anything; just you and you alone and you must state your opinion. I encountered this situation hundreds of times in my career involving various types of comparative examinations, but mostly in the area of latent prints. I have stated my opinion with conviction and with personal assurance that I am correct in the opinion based on the evidence as established through comparative examinations ONLY.

Now, Strict Scrutiny, have you? This is the unknown quality in many of these postings. Are these cloaked individuals in a position to judge or evaluate the profession? Have they been there? Have they had to answer the above question in a court of law? Have they actually made forensic examinations and render reports of examinations? Or are they self-proclaimed ‘forensic scientists’ that never do these things, but yet sit in judgment of them? There may be competent examiners that disguise their identity for political protection, but there are others that are no more than lurkers, pretenders, researchers and ‘others’. The webmaster may have the sources to establish the legitimacy of these individuals through technical sources and permit their posts, but I do not have this knowledge. I consider anonymous postings as a ‘moot point’ and of no particular consequence without some authenticity.