CLPEX Chat Board

Can anyone point me to research done in whether or not bias exists when experts are taking tests versus working cases (some sort of placebo effect in test-taking)? The PCAST report points out in their findings (specifically Finding #5) that the false positive rate reported via the two black-box studies are likely higher in casework.

Conclusions of a proposed identification may be scientifically valid, provided that they are accompanied
by accurate information about limitations on the reliability of the conclusion—specifically, that (1) only
two properly designed studies of the foundational validity and accuracy of latent fingerprint analysis have
been conducted, (2) these studies found false positive rates that could be as high as 1 error in 306 cases in
one study and 1 error in 18 cases in the other, and (3) because the examiners were aware they were being
tested, the actual false positive rate in casework may be higher. At present, claims of higher accuracy are
not warranted or scientifically justified. Additional black-box studies are needed to clarify the reliability of
the method.

So, is this a call for research into that single aspect (meaning none exists now)? If it is, I'd say it's a pretty bold claim if that research doesn't exist with which to base that finding.

Can we be sure that examiners aren't being less careful when they know they're being tested? It's possible that some examiners are more relaxed about reporting idents/exclusions precisely because it's not real casework. No one's going to jail as a result. If that's the case I'd expect errors in real casework to be less common, not more.

Alan C wrote:It's possible that some examiners are more relaxed about reporting idents/exclusions precisely because it's not real casework.

It is equally possible that a latent print examiner with testing phobia would be more nervous when it is known to be a test and, as a result, be doubly cautious.

What is the probability of an LPE taking the test being more cautious compared to when they do casework?

To answer your specific question, the only forensic/fingerprint related critique that I could find which addresses it, is from a 2002 volume of the California Law Review

That being said, it's long been known that social science research suffers from a host of bias problems like self selection, self reporting, design and publication bias. It should come as no surprise that social science (read qualitative science) has some familiar critiques. As a matter of fact the Cracked podcast I was just listening to the other day addressed some of these very things. It's well established that social science studies suffer from what's called the replication crisis in which the findings of the peer reviewed studies were not able to be replicated. So, I'd say there's a real concern about the difference between study and the application and the best way to overcome some of the problems of social science research is to perform them via natural experiment, which in our case would be to make up a fake case and send it through the normal workflow.