CLPEX Chat Board

Stumbled across a nifty paper in research on some eyewitness ID issues which has bearing on the confirmation bias problem and ability (or not) to train people to be aware of hindsight bias.

Take a look
Harley EM, Carlsen KA, Loftus GR. The “saw-it-all-along” effect: demonstrations of visual hindsight bias. Journal of Experimental Psychology: Learning, Memory, and Cognition 2004; 30(5):960–968

http://faculty.washington.edu/gloftus/D ... Loftus.pdf

This seems to have been a well-known effect in radiology. I recall back in the 60's or 70's reading a physician's stories from his early hospital practice where something wasn't recognized on an earlier x-ray, and the patient died after it enlarged enough to be readily seen. I remember his comment that, "Now, everyone can see it." We do the same thing. Put up a latent from a case where an erroneous identification has been revealed. Now, everyone can see it. You absolutely cannot judge how likely or not it would have been for anyone else to have made the error, once you know it's an error. It's a real problem in something like a civil suit. No matter how little you say, any expert is going to know there's something about the x-ray that's been questioned. In criminal cases, it's not so much a problem in exactly the same way when the evidence is being re-examined. At that stage, it's likely to be more of a confirmation bias, the bias being toward the outcome the client wants. You can't say, "Oh, no. I'm always neutral." It's not a matter of the examiner being neutral. It's a matter of the data that's presented. That data often includes explicit or implicit metadata. If you know it's a verification, the data is loaded. If you're hired by a client who's questioning a conclusion, the data is loaded. You can make every effort to view the evidence as it is, but the sum of your input data is NOT what it would have been without the additional information. Even if you recognize the possible influence of metadata, and you become just a bit more conservative, you've frankly applied a bias. And when we already know the correct outcome, as we do when we view revealed error latents, we can't possibly examine the evidence under the same conditions as the original examiners, because we can't escape the very powerful metadata.

It's not something we have to be defensive about or obsess over. It's good, though, to be constantly reminded that there are various sources of biasing information that we can't always filter out. And even when we're conscious of them, we can't entirely switch off their effects. It's built into the same facilities of mind that allow us to do observational analysis that can't be done by machines. Brains integrate and use information in wonderful ways, but they don't filter out part of it just because we want them to. It's a good thing, too. If we had to consciously specify every bit of information the brain would use, we couldn't do what we do.