More Thoughts On Bias

[Charles Parker]

Recently I have been reading the book “Speaking as an Expert: A Guide for the Identification Sciences from the Laboratory to the Courtroom” by Stephen C. McKasson and Carol Richards, Charles C. Thomas Publisher, 1998, LCCCN: 97-32099. They have some interesting things to say about bias and the following is an excerpt from pp 9-10.

“For example, there are some examiners who say that they want to know as few details about a case as possible before starting the examination process to avoid bias. This is a very scary statement because it seems to say that a scientist cannot sort out the facts by observation if he or she is given outside information. It had better be the case that nothing could be further from the truth! What is bias but a negative word for a working hypothesis? A scientist works with bias constantly, but since he or she is trained to test and prove a position, an expert opinion; bias should not be a problem. A good scientist should not have to be unbiased, but rather, should be observant of those things that will either convert a bias into a well reasoned and well supported conclusion or expose the bias as wrong (Hively, 1996). “The better equipped he (or she) is to understand his (her) own biases and prejudices the better forensic scientist he (she) will be” (Turner in Davies, 1974, p.7).”

“It is possible that the examiner is more concerned with the appearance of bias to a judge or jury rather than with actual undue influence. Here again, we reject the argument because it is far more appropriate to demonstrate how bias is dealt with by the scientist, especially since it is impossible to avoid. Remember, a complete lack of bias would mean the inability to make assumptions, of any kind, and that would throw all scientific endeavors out of the realm of observation and into the world of guessing.”

“Our solution is a simple one. ……Know your discipline. Know its theory and its practice. Know its hypotheses and its assumptions. Know what you know. Observe well and that includes observing your observations.”

Both McKasson and Richards have more to say on the subject of bias and I believe it is certainly worth reading.

I have also recently been talking to persons who are involved in Photographic Analysis (Interpretation if you wish). Bias is also a concern in their discipline. This is countered by teaching the subject of bias in their training program. Also they do not have a system of verification as the latent print discipline does, but they do have peer review, which they call ‘presentation’. When conclusions or decisions are made they must present their reasoning and justification behind the conclusion to a team (3 or more). I do not mean just handing some people some photographs and saying this is my conclusion, look for yourself. They must present it: the good, the bad and the ugly. The pros and cons. They must account for what they observe, the conclusion and how they arrived at that conclusion.

From my POV “blind verification” is an approach to deal with “confirmation bias”. Is it the only approach to deal with confirmation bias? I do not believe so. I also believe that blind verification is not the best or even a reasonable approach to deal with “Bias”. My reasoning for that is:
1. BV is not used in other disciplines of scientific inquiry!
2. BV has not been tested sufficiently as a means for dealing with biases!
3. BV replaces “Peer Review” as means of checking conclusions!
4. BV will confuse those not in the discipline when small differences are noted that do not effect the overall conclusion.
5. BV will decrease operational efficiency.
6. BV will foster a system of second guessing instead of real scientific process of Peer Review.
7. BV will disable the learning process in how other examiners reach conclusions.

So if I reject Blind Verification as the only means of overcoming biases or if I reject it as the “best” means of overcoming biases, then I must have something to fall into its place.

The first step is to increase awareness. If you do not have a section on “Biases” in your training manual, put it in there. If you already have training block on what is Biases and dealing with them, then strengthen it.

The second step is those single latent prints with poor quality and reliability or from high profile cases these should be “Presented”. I do not mean just verify but the Examiner (Analyst) must present to a group. They must be able to point out each distortion effect. Each ridge feature. Each feature relationship. They must be able to discuss the Substrate or Matrix when necessary. They must be able to present each factor that was related to their conclusion. I would bet that if someone made a presentation on a particular latent print that all features were clear and sharp and any discrepancies could be accounted for by a 66 degree twist, some eyebrows would have been raised. Each presentation must follow an outline on all aspects of a latent print analysis. If a particular aspect does not apply then they must say so. Will this work with smaller agencies that only have 1-4 examiners. It would if there was an agreement between some agencies. It would be like Ms Saviers said in an earlier post. It probably is better.

They say if you want to learn something very well, then teach it. If LPE’s have to present their conclusions on difficult latent prints it will improve their communication skills and their craft far better than two examiners doing it blind and writing it on a sheet of paper. Full Presentation is a high form of “Peer Review”.

Your comments are welcome.

[David L. Grieve]

You have offered much food for thought. As I expressed earlier, I think random reanalysis is more effective than some blind verfication scheme, and I have concerns about known criteria related to what cases will be subjected to blind verification as having the potential to be detrimental to overall quality. One adopts a different attitude when knowing in advance the case will be reworked by another examiner. I see that in trainees during casework under my supervision, that awareness that a safety net is below the high wire. Once you know that, you cannot "not know" what is going to happen. Conditions are changed and with that comes difference. I explain that to my students by relating an Outdoor Sportsman program I saw many years ago featuring William Shatner who was hunting Kodiak bear with a bow and arrow. When asked if he were apprehensive or down right fightened in facing a charging Kodiak while armed with only a bow and arrow, Shatner replied he was not. When the interviewer asked how he stayed so calm in those conditions, Shatner said having a guy with a high powered rifle standing right behind him made all the difference.

Blind verification was tossed out years ago as a solution to a problem poorly understood, and like a snowball rolling down hill, has gained in both size and momentum since. I'm not ready to abandon further exploration into the concept, but it has been losing appeal for many of the reasons you list. And I am still not convinced this is addressing the real problem, if a problem does exist. A long time ago, one of my sons was caught playing with matches at school. Everyone involved, including my ex-wife, was sure this was a symptom of some serious problem, and within a week, we had our first appointment with a child psychologist. The man listened as to why we were there, then asked, "Did anyone ever tell the child not to play with matches?" We all shook our head and said no. He then turned to my son and said in a serious tone, "Don't play with matches. They can be dangerous." Amazing, the patient was immediately cured.

I like your idea of presentation, although we have not used this method once an individual is out of training. In training, we use oral boards during which the trainee must be prepared to explain any topic and answer all follow up questions. Defending one's position to thorough questioning in such an environment is a valuable exercise and excellent preparation. I want to mull it over some but I do like the idea.

Your quotes from Steve MacKasson's book may give some insight into why it was so much fun working with the guy. I learned a great deal in such discussions with Steve that occurred almost daily, and I do miss that opportunity since he retired. Carol Richards is Steve's sister, and I find her perhaps even more fascinating. Her grasp of logic and her insight into understanding problem, cause and effect are incredible. It is a good book and well worth the read.

[Charles Parker]

McKasson and Richards book is good. I have already taken some of it and incorported it into my Court Training and Introduction to Science Modules. Why I did not pick up on their book earlier is eluding me (My Bad).

You stated "And I am still not convinced this is addressing the real problem, if a problem does exist" which I agree 100%, but the damn thing has gotten into the lexicon of the courts and I believe it will be here for awhile. I am prepared that if I am asked the question "is blind verification the best form of peer review" my answer will be "NO" and I will list some, all, or more of the responses that I have presented.

I honestly belive that BV in response to CB is a "Poor Response To An Idea That Has Not Been Very Well Thought Out".

My discussion with the Photo Analyst was very interesting on the topic of "Presentation" and how they do it. Your comments about using it in an increasing training situation even spurs me on even more to work with it and fine tune it. At this time I see it as increasing Communication Skills, Reasoning Skills, and truely understanding the "Identification Process" and being able to "ARTICULATE IT". From my POV the inability to articulate the complete "Identification Process" is the Archilles Heel of this discipline.

When I mean articulate I do mean on the level that promotes understanding from the courts and juries and not from the 16-18 grade level (academia) that seems to be popping up.

Training and Presentation was a posting from an initial idea that needs to be thought out further and discussed further so as not to become a "poor response" as well.

The other book I wanted that dealt with CB is not a real bias but more like a error or "Information Shortcut" has not gone to print yet, so I will have to wait a while longer for it.

I want to be able to respond to that statement that I read recently that "Blind Verification is the only solution for Latent Print Analysis to become truely scientific". That statement pushed a red button.

Time to go and fly back to Texas. Thanks for the information Dave.

[Shane Turnidge]

Bias has been around as long as the science. The counterweight has always been integrity. The recent discussions regarding bias and bench notes, for the most part, all seem to target optics rather than substance.

When it comes to the issues of addressing bias and improving upon bench notes, the question in my mind is; Does the court require these modifications to our business practises?

Ultimately, we serve the Court. If the Court perceives that there is a problem, we should and must address the issues.

The other point that probably needs to be discussed is the type of change, if necessary, we should undertake. The science is well established in virtually every country on the globe. If change is deemed necessary, we should endeavour to make it as incremental as possible. To that end, if it is illustrated that BV is prefferable, perhaps Blind-er verification is the way to go. If bench notes need to be improved, we should also take a similar approach. Doing so reinforces the validity of our profession and keeps the Court's confidence in us and what we do.


Shane

[Charles Parker]

Bias has been around as long as the science. The counterweight has always been integrity.

According to some bias is in all aspects of life.

I would add to the counterweight besides integrity, knowledge and training.

Ultimately, we serve the Court. If the Court perceives that there is a problem, we should and must address the issues.

Yes we serve the courts but they make decisions based upon presentations by witnesses and arguments by attorneys. Some attorneys argue better than others. Some witnesses do not present the information very well. Courts on occassion make incorrect decisions as they along with eveyone else have good and bad beliefs (biases). But if the courts tell me I must do blind verification then I will. If my management tells me I must do blind verification, then I will. If my wife tells me to get up early saturday morning and mow the yard I will do it. I may not agree with them but I will do it.

And if they require it of fingerprints are they going to require it of firearms examiners, document examiners, drug chemists, DNA analysts, medical examiners, investigators, psychologist, accident reconstructionst, blood alcohol chemist, and all the other types of expert witnesses.

If they do that then I think they will have to rewrite all the rules on expert testimony.


PS-I wrote this post to see if I could figure out how to make the little blocks with the peoples names above it. I did it.

[Shane Turnidge]

Absolutely Charles.

Poor expert fingerprint witnesses are the bane of a good fingerprint pratitioners existence. This issue alone is why I'm a proponent of proficiency testing for all LPEs.

In retrospect, the point I made was simple. Let's see how the court handles the need for BV and more comprehensive bench notes.

The people that really need to be paying attention to this debate are Human Resources departments. If BV and comprehensive bench notes become part of an agency's SOP there will be a need to hire significantly more people everywhere.

Shane Turnidge