Testimony Over Time

[Boyd Baumgartner]

An interesting article came out a couple days ago: https://www.propublica.org/article/elkh ... conviction

It involves a conviction based on pattern matching and is framed in such a way that seems common these days, to question whether a conclusion is overstating. The examiner in question in this instance was John Vanderkolk and while it wasn't fingerprint comparisons it was fracture pattern matching of a trash bag in the case. Keep in mind this conclusion was made in the 80s. I reached out to John, just because I was curious if he had read the article and to get his insight on it from a now vs then perspective, he hadn't read it yet but encouraged a conversation about here in the forums.

So, my question is: Given that forensic disciplines change over time, what do you think contributes to the most stable testimony (holds up over time)?

[Dr. Borracho]

Thomas Kuhn, a historian of science in the mid Twentieth Century, wrote in 1962 that in any field of science, new ideas may eventually come in with new scientists. The old scientists in the field will likely resist the new ideas. But gradually, as the old scientists retire or die off, the ideas of the new scientists come to dominate and the old ideas fade into disuse. Kuhn coined the phrase "Paradigm Shift" to designate this transition. Historically, most, if not all, fields of science have gone through paradigm shifts, some fields multiple times.

Henry Swofford postulated in 2015 that fingerprint identification is going through just such a paradigm shift. Given all that has happened in our science in the seven years since Swofford made that pronouncement, I think we have to concede that he is correct. If that is so, then I believe we have to ask whether a scientific conclusion arrived at under one paradigm should be accepted as correct under the new paradigm that has taken over. Or should we go back and rework every old scientific conclusion in all branches of science every time there is a paradigm shift?

If a scientist reaches a conclusion and is going to present it as valid under the scientific theory du jour, that scientist should not have to try and forecast what theory will dominate twenty years down the road. He should present his opinion based on accepted theories and best practices at the time his conclusion is rendered.

[Boyd Baumgartner]

I think we have to concede that he is correct.

I'm not sure I would concede that. Kuhn's whole point was that scientific progress was dialectical and therefore competing ideas were incommensurable. Translation: Scientific communities have two competing paradigms that don't integrate and one wins over time and the adherents of the old way die out.

If you glance at David Kaye's blog on the topic he quotes Champod's vision of statistical models. Swofford cites Champod and Evett as initiating said paradigmatic shift.

I foresee the introduction in court of probability-based fingerprint evidence. This is not to say that fingerprint experts will be replaced by a statistical tool. The human will continue to outperform machines for a wide range of tasks such as assessing the features on a mark, judging its level of distortion, putting the elements into its context, communicating the findings and applying critical thinking. But statistical models will bring assistance in an assessment that is very prone to bias: probability assignment. What is aimed at here is to find an appropriate distribution of tasks between the human and the machine. The call for transparency from the NRC report will not be satisfied merely with the move towards opinions, but also require offering a systematic and case-specific measure of the probability of random association that is at stake. It is the only way to bring the fingerprint area within the ethos of good scientific practice.

This is a synthesis of the human/statistical model not a rejection of one for the other. Therefore in Kuhn's own paradigmatic definition, this isn't the case.

If you take 'paradigm shift' to mean something less Kuhnian and more dictionary-esque, then I'm still not sure I'd concede the point. Swofford's paper is titled The Emerging Paradigm Shift in the Epistemology of Fingerprint Conclusions, the key word being Epistemology (read: the justification of a knowledge claim). Fingerprint conclusions should be based on the objective data in the print (shape/direction/sequence/spatiality) in agreement or disagreement not the agency/method/uniqueness/error rate/title/ certification/confidence.


What sounds most accurate?

I came to my conclusion based on:

• ACE-V
• Uniqueness
• Confidence
• The fact that I work for X-PD
• I'm certified
• I hold a (insert educational achievement here)
  
• the objective data in the print

Swofford's own model used Examiner input to calculate the final output, so it's just as susceptible to bias because it doesn't differentiate between clear data and highly interpretive data. This notion flies in the face of the perceived benefit as outlined by Champod. Namely, a reduction in bias. The anecdotal evidence of our own foray into 'plot the data' from the Show Me the Print thread should illustrate my point.

[Shane Turnidge]

I think we should also consider that identifications were once supported by dogmatic thresholds instead of scientific ones. Fingerprint conclusions were once considered binary in nature. Policing and to some extent the Courts, liked that because it was simple. Now fingerprint conclusions are less dogmatic and are supported both to the extent the examiner has the ability to defend their position and/or within the boundaries of the agency's accreditation and SOPs. The process used to support today's fingerprint decisions is far more complicated than it ever was back in the early 90's let alone the 80's.

The most ridiculous thing I ever heard in my career was that, "fingerprints have been around since 1911 or so and they haven't changed since then and they aren't about to change now". As I reflect on it now, I saw nothing but change my entire career.

It can be tempting to measure the past by the tools we use today but we should resist doing so. Identification to some extent is a mental exercise and I think it would be unreasonable to try to ascertain undocumented mental processes and organizational formalities from such a long time ago.

Shane Turnidge

[NRivera]

Unless you can see into the future (in which case, just go ahead and retire now and pass me some winning lotto numbers while you're at it ), I don't think there is a reasonable way to "future-proof" your testimony. All we can do is adhere to best practices and standards that apply at the present time. Scientific knowledge evolves and technology keeps moving forward, if a conclusion happens to be debunked in the future so be it. So long as there was no nefarious intent in reaching that original conclusion we just amend it as appropriate and move on.

[Michele]

At the risk of sounding like a broken record, I think you can “future proof” your testimony. A small amount of knowledge on scientific protocols can help your testimony stand the test of time.

For all of the pattern evidence disciplines, I don’t think “the science” has changed or evolved; I think we’re now just starting to see the benefits of using scientific protocols (and understanding what that means). Using scientific protocols is more transparent, helps diminish bias, helps protect against overstatements, helps protects against errors and helps produce stronger results.

So, what are scientific protocols? To keep it simple, I’ll just discuss one protocol. Just understanding and applying this one rule can help quite a bit.

RELY ON ACCEPTABLE DATA AND VALIDATED METHODS (NOT ON THE NOTORIETY OR BELIEFS OF THE PERSON/GROUP STATING AN IDEA OR FINDING).

It sounds simple but it instantly produces an array of issues (or a needs assessment for the pattern evidence disciplines):

-What data is allowed to be used?
-Are creases, scars or pores allowed to be used as the basis for a conclusion?
-Where is the ‘allowable data’ researched and published?
-What is the ‘method’?
-Where is the validation of the method?
-Where are the rules to follow within the method?
-Do I have to fully analyze the unknown print prior to looking at the known? Where is the validation to support that?
-How much data is needed to arrive at each conclusion?
-If an ID is based on the scientific method of induction, is that also true for exclusions as well (is induction a valid method for exclusions in science?)
-Was saying the error rate was zero ever scientifically supported?
-If we are going to say ‘the science is evolving’, what research made the science evolve? Was it the error rate studies that made us aware that the error rate was not zero?

A scientific method would state the parameters of each conclusion instead of saying ‘it’s based on the tolerance level of the practitioner’. If we say that conclusions are based on the tolerance level of the practitioner, aren’t we saying that the conclusion is our opinion (not a scientific conclusion)?