Latent Priorities

[ER]

I wanted to start a discussion on the priorities that we set for our field. The following goals are all important, but working hard to fulfill one goal might make another goal less likely to be completely achievable. We should then prioritize which goals are most important. Here's my list. I'm curious if other LPE's have them in the same order.

1) Never make a bad ID
2) Never miss an ID or miss the chance to call an ID
3) Never make a bad EX
4) Never miss an EX or miss the chance to call an EX

I think our discipline chooses to not call some ID's (and not fulfill #2) in order to ensure that #1 is virtually always true. But what about the rest of the list? Are we willing to accept some bad EX's if it means that we miss fewer ID's? Are we willing to accept some missed EX's if it means making fewer bad EX's?

[Bill Schade]

I'll play

1) Never make a bad identification
2) Try not to miss any identifications
3) Provide timely information on cases (no backlog)
4) Provide relevent information to investigators and the court

[timbo]

100% agree, with absolute certainty, with Bill.

[Tazman]

I'll go along with Bill, too, with an addendum.

3) Provide timely information on cases (no backlog)

Some labs subscribe to the philosophy of "Thorough and Complete" on all cases. But in underfunded, understaffed labs, that is impossible if one is to keep up with backlog. Throughout most of my career, I had the advantage of exercising discretion in how I worked cases and after a short time I adopted the philosophy of "The Greatest Good for the Largest Number of Users." Using that idea, I prioritized my cases by Persons Crimes: Top Priority; Property Crimes: Second Priority; Status (Victimless) Crimes: Low Priority. For most Persons Crimes, I would pull the case from P&E as soon as I could and I used sequential processing to the extent practicable in my lab. When no Persons crimes were in the queue, I would work Property Crimes, but usually only one processing technique that I deemed most probable for developing prints on the items in that case. For Status Crimes, on a slow day when there was not much else to do, I would look at any of those for which the statute of limitations had not expired. On occasion, I was known to call investigators and tell them point blank that prints were not necessary in their case, or not possible on a surface, or irrelevant because of handling or chain of custody issues, and I would simply refuse to do the case. I was fortunate that we did not have accreditation policies to prevent me from doing that. I realize the standard way of doing business thirty or forty years ago would probably get me fired today, which is the main reason I'm glad I'm off the bench.

On another point ER brings up,

I think our discipline chooses to not call some ID's (and not fulfill #2) in order to ensure that #1 is virtually always true. But what about the rest of the list? Are we willing to accept some bad EX's if it means that we miss fewer ID's? Are we willing to accept some missed EX's if it means making fewer bad EX's?

I think it is not only unreasonable, but impossible to demand an examiner make every ID and every Exclusion with a "zero error rate." ER gets it right in his implications. If you want minimal erroneous idents, you have to accept some erroneous exclusions. If you wanted minimal erroneous exclusions, you would have to accept some erroneous idents. In "the old days," we demanded zero erroneous identifications, but we accepted erroneous exclusions as a reasonable cost for that demand. Nobody ever got punished for missing one, but your career was literally over if you made a bad ID. Having been raised in the business that way, I think we have gone overboard on the idea of identifying or excluding every single latent print of value in every case.

You have to accept reasonable compromises. With limited budgets and limited manpower, some of those compromises may be considerable.

[Tazman]

On a very related topic, check out the "Commentary" by H. J. Swofford, "The Emerging Paradigm Shift in the Epistemology of Fingerprint Conclusions," in the current issue of the Journal of Forensic Identification. Mr. Swofford tells us where the profession of latent print examination is headed. I have no doubt he is correct. Old timers won't like it, new kids will have to live by it.

[Boyd Baumgartner]

I just finished reading Mr. Swofford's commentary, and while I commend him for wading into that realm as I think the philosophy of science is where the virtues and vices of scientific theories are fought, I think he misreads Kuhn on some key elements.

1) Kuhn himself argues that the probabilism that bridges the observation/theory gap has to be assumed in order to even be considered which makes it part of the current paradigm. Therefore, referencing Kuhn to cite that there's a coming paradigm shift and that paradigm shift is probabilistic in nature, even though probabilism by Kuhn's own admission doesn't count as a paradigm shift is paradoxical.

In their most usual forms, however, probabilistic verification theories all have recourse to one or another of the pure or neutral observation-languages discussed in Section X. One probabilistic theory asks that we compare the given scientific theory with all others that might be imagined to fit the same collection of observed data. Another demands the construction in imagination of all the tests that the given scientific theory might conceivably be asked to pass. Apparently some such construction is necessary for the computation of specific probabilities, absolute or relative, and it is hard to see how such a construction can possibly be achieved. If, as I have already urged, there can be no scientifically or empirically neutral system of language or concepts, then the proposed construction of alternate tests and theories must proceed from within one or another paradigm-based tradition. Thus restricted it would have no access to all possible experiences or to all possible theories. As a result, probabilistic theories disguise the verification situation as much as they illuminate it. Though that situation does, as they insist, depend upon the comparison of theories and of much widespread evidence, the theories and observations at issue are always closely related to ones already in existence. Verification is like natural selection: it picks out the most viable among the actual alternatives in a particular historical situation. Whether that choice is the best that could have been made if still other alternatives had been available or if the data had been of another sort is not a question that can usefully be asked. There are no tools to employ in seeking answers to it. (pg 144-145)

Link to Structure of Scientific Revolutions

2) In Kuhn's model of the Phases of Science, the second phase involves the accumulation of anomalies, which expose weaknesses of the old paradigm. Instead of recognizing anomalies as mere problems with the theory, Popper hung his hat on falsificationism as the solution, to which Kuhn replies that falsification falls prey to the same objections as probabilistic verification.

A very different approach to this whole network of problems has been developed by Karl R. Popper who denies the existence of any verification procedures at all.2 Instead, he emphasizes the importance of falsification, i.e., of the test that, because its outcome is negative, necessitates the rejection of an established theory. Clearly, the role thus attributed to falsification is much like the one this essay assigns to anomalous experiences, i.e., to experiences that, by evoking crisis, prepare the way for a new theory. Nevertheless, anomalous experiences may not be identified with falsifying ones. Indeed, I doubt that the latter exist. As has repeatedly been emphasized before, no theory ever solves all the puzzles with which it is confronted at a given time; nor are the solutions already achieved often perfect. On the contrary, it is just the incompleteness and imperfection of the existing data-theory fit that, at any time, define many of the puzzles that characterize normal science. If any and every failure to fit were ground for theory rejection, all theories ought to be rejected at all times. On the other hand, if only severe failure to fit justifies theory rejection, then the Popperians will require some criterion of “improbability” or of “degree of falsification.” In developing one they will almost certainly encounter the same network of difficulties that has haunted the advocates of the various probabilistic verification theories.(pg 145-146)

This is a bit schizophrenic, because in our discipline the same guy who wrote this in the Fingerprint Sourcebook, is also leading the charge in the 'new paradigm' which is characterized as really just the old paradigm by Kuhn who is being used to incorrectly justify the new paradigm by Mr. Swofford.

Sir Karl Popper (1902–1994) recognized the difficulty of defining science. Popper, perhaps one of the most respected and widely known philosophers of science, separated science from nonscience with one simple principle:falsifiability. Separation, or demarcation, could be done if a theory or law could possibly be falsified or proven
wrong (Popper, 1959, 1972). A theory or law would fail this litmus test if there was no test or experiment that could be performed to prove the theory or law incorrect. Popper believed that a theory or law can never be proven conclusively,no matter the extent of testing, data, or experimentation. However, testing that provides results which contradict a theory or law can conclusively refute the theory or law, or in some instances, give cause to alter the theory or law. Thus, a scientific law or theory is conclusively falsifiable although it is not conclusively verifiable (Carroll, 2003).
Fingerprint Sourcebook, Chp 14

3) Kuhn's work has been remarkably analyzed and he wrote a follow up thesis explaining the virtues of later theories that came to replace earlier theories (read paradigm shift) in a paper called Objectivity, Value Judgment, and Theory Choice. They are:

• Accurate - empirically adequate with experimentation and observation
• Consistent - internally consistent, but also externally consistent with other theories
• Broad Scope - a theory's consequences should extend beyond that which it was initially designed to explain
• Simple - the simplest explanation, principally similar to Occam's razor
• Fruitful - a theory should disclose new phenomena or new relationships among phenomena

While the probability model as a theory lacks some aspects of these, as it relates to being fruitful, I'd say probability models have yet to demonstrate such fruitfulness. When a probability model can say something fruitful with regards to the Mayfield, Lana Canen, Shirley McKie and the Zero Point Identification prints, I'd be happy to listen. I think technology in the field (AFIS) has actually made the most impact on the field and has actually driven the biggest paradigm shift. The problem is, it was also cited in the OIG Report as the main contributor to the first cause of several in producing the error. Now, the latest round of probability models is based on an AFIS algorithm. Something doesn't add up.

One benefit we get with being 'behind the times' is that we can see that the new paradigm has some serious problems of it's own as they are on display with the D.C. DNA Crime lab. I'd like to see a commentary on how policies or IAI Probability Evaluation committees would prevent/handle such instances. Since ER is on the IAI committee, perhaps he can tell us how such concerns are being considered.

[Tazman]

I think he misreads Kuhn on some key elements..

I agree with you that Mr. Swofford missed the meaning of Kuhn's work, but from a different perspective.

As I understand Kuhn, he was contesting the idea that science is a steady progression and refinement of theories and ideas. Kuhn coined the term "paradigm shift" to explain a major revolution in thinking. For example, the shift from a Ptolemaic model of the universe to a Copernican model was a paradigm shift. The revolutionary ideas of the Theory of Relativity set the long accepted theories of Newtonian physics as erroneous, except for within the parameters of normal experience. Those were what Kuhn meant by "paradigm shifts." Those were cases in which models long accepted as correct were proven to be wrong.

In fingerprints, the fundamental theory is three pronged: 1. Friction ridges are unique; 2. Friction ridges are persistent; and 3. Given sufficient quantity and quality of detail in two prints, identification is possible. Nothing Mr. Swofford proposes changes those propositions in the least. Instead, the statistical modeling, calculation of likelihood ratios, determination of uncertainty, and error rates only serves to validate the three fundamental principles. There is no "paradigm shift" in the Kuhnian sense. Instead, the progress in refinement of fingerprint science is merely an example of the more traditional concept that science undergoes a steady progression and refinement of theories.

[Boyd Baumgartner]

I think we're dancing around the same issue, namely that given the premise that Swofford makes that latent print examination is merely examiner intuition, probability models can at best only attempt to justify our intuitions and that's not a new paradigm, that's just propping up the established one that is criticized as being bad. The problem is, the models are using less information (they're non continuous and have no sequential data to the best of my understanding.)

Going back to Kuhn for a minute in your example, the difference between the examples Kuhn references and fingerprint examination, each example cited revealed a more fundamental/foundational reality of what was being studied. The progress of science has been reductionist in nature (eg. Charles Law reduces to Boyle's Law which reduces to the Kinetic Theory of temperature). Fingerprints run up against the fact that an individualization is a status placed on a comparison and not a property of the print, which means that it's not solely the physical properties of the print but the attitudes that people have about those properties. This means that ultimately, the more foundational aspect of fingerprints evaluations should include the brain and those studies should be driving probabilistic models. The problem is that consciousness is not reductionist, it's a higher level function and no one knows how it works.

However, we do know some things:

Individuation by experts is already established and linked to a certain region in the brain.

And considering that suitability is a value judgment means it's a moral decision (bad vs worse vs good vs better), the study of which has been dubbed 'trolley-ology' because of the Trolley Problem. The moral decision making processes and regions of the brain are also established.

Also, Dr. Busey and John Vanderkolk have already established that fingerprint experts take advantage of information that non experts do not. (experts choose better information than non experts).

For the record, if someone wants to stick me in an fMRI machine for research, I'm in!

As it relates to our discipline, you can see that these issues are rearing their head in the field. For instance, what do you think the whole overstating our conclusions is about if not recognizing there are a moral gradient of conclusions (worse vs bad vs good vs better) or what do you think the whole AFIS inconclusive dilemma is about if not a question of intuition. Especially since intuition is considered nothing more than the ability to recognize patterns of experience which drives learning.

The larger question, is with all this research poking at the underlying issues of our dilemmas why is our science so isolated as to miss collaborating with already peer reviewed and accepted science? I would offer up that the future of our science looks like artificial intelligence that is the result of eye tracking data, human AFIS encodings and tracings (to ensure that the data is continuous not just discreet) and brain mapping that 'teach' a system what is good information. The result will be demonstrable, both able to perform better in a white box study and beat Ken Jennings in Jeopardy.

[John Vanderkolk]

Boyd > Tom and IU Psychology has an fMRI. He put me and some other LP friends into the IU fMRI about five-ten years ago. Fascinating experience. Spent almost an hour in it with my head in a plastic brace looking at images via a mirror and pressing buttons for the decisions I made. Maybe, we can get Tom to resurrect the research with fMRI. Another research professor is responsible for fMRI studies.

Tom and I will be at IAI. Hope we can visit.

When can you come to Bloomington? We will have to make sure Tom is ready for us.

[Boyd Baumgartner]

Hi John,
Thanks so much for the invite, I'd love to come down to Bloomington! I can accommodate whatever you're able to set up. So let me know and I'll be on the next plane out.

Also, I'm signed up for your class at the IAI, so we can discuss it then.

Looking forward to it,

Boyd