CLPEX Chat Board

Numbers provide people with a comfort level in decision making but statistics should be evaluated with care, as probabilities are always conditional; i.e., you can make statistics say whatever you want them to say, it’s just a matter of plugging in the right numbers to support your opinion.

I agree that providing a likelihood ratio along with typical testimony will impress jurors, but how do we make the information relative and comprehensible? The last thing any examiner wants to do is mislead the court with information that they don't understand and that they can't provide significance to.

A statistical figure applied to a latent print individualization could provide data similar to DNA; specifically, how likely it is that a particular grouping of minutiae would appear in a fingerprint from a different subject. But a number will not tell us how valid a conclusion is or how likely it is that an error has occurred.

These discussions all seem to illustrate that the future of friction ridge identification is a hybrid model that combines both the purist's view of friction ridge science and the issues being raised by academia.
I appreciate that there are people on this discussion board that can bridge those two hemispheres to assist both sides of the relevance and importance of both bodies of knowledge. The closer we get to a shared understanding the better off we'll all be. Perhaps then will we be in a better position to endorse and accept true universal standards.

Shane

Back to the SWGFAST Draft for Comment

While there are some very good sections in this newest draft, there are still some glaring problems as well.

First and foremost, the "Approach #1" as described in the Analysis section seems designed to produce erroneous exclusions. Never mind the fact that this approach discourages examiners from even attempting to compare low q/q prints. If an ID can't be made, the examiner must either exclude a print or ask for better exemplars. How is this "Approach" in a SWGFAST document with all the recent research on the high rate of erroneous exclusions? I know that certain agencies dance around the issue by 're-analyzing' the print and reporting that no comparison was done. But that's patently dishonest, and we all know it.

I understand documenting the reasons for an inconclusive result in the notes, but why would SWGFAST mandate that the reasons be included in the report. The officers reading our reports don't care why it's inconclusive. If someone really wanted to know, they could request the notes. The report should be a condensed summary of the results. The reasons belong in the notes.

I really wish SWGFAST would back off from the word 'shall'. I know that they want this to be the new national standard, but come on. "Table 1 SHALL be used for categorizing the levels of quality..." Really? SHALL? How about "Table 1 describes levels of quality" or "Table 1 is an example of how levels of quality can be categorized". This is the Table that SHALL be used for every comparison from here on? It's not even a very good table.

And please don't say that "quantity is the number of ridge endings, bifurcations, and dots". It's not. Quantity is the "amount of features and area" of the print. SWGFAST should really review sections 4.1.4.1 and 5.2 and make them agree.

The most frustrating part of this new SWGFAST document is that members seem to be constantly asking for comments from the rest of us. These comments have already made to SWGFAST on the first draft of this document, but all were ignored.

I really do understand the tough position that SWGFAST members are in. They feel the need to write national standards for a discipline that doesn't want national standards. I wish that I had the time and resources to join SWGFAST, but I don't. All I can do is comment. I applaud SWGFAST for trying to create standards but hope that they refrain from writing mandates.

-Unfortunately Anonymous

PS
(And why is there an N coming out of box 400? That bugs the hell out of me.)

One problem I see with the Draft for Comment is the limited number of categories of errors. When I reviewed the lists of documented errors floating around I came up with five general categories:

Misidentifications – erroneous individualization
Ethics violations – intentional forgery/fabrication, questionable testimony, destruction of evidence, etc.
Erroneous exclusions
Clerical errors or insufficient – mismarked cards, incorrect finger number or person, unjustifiable insufficient finding
Computer errors – AFIS computer miss, identity theft, computer discrepancy

We shouldn’t place misconduct/ethical violations in the same category as misidentifications, or lump computer and clerical errors together. Of course these additional categories add some gray area to the nice black and white formula presented, but more accurately reflect real-life IMHO.

g. wrote:PS

I will never get up on the stand and say I 'mated' two prints.

And no one is asking you to. As your next post shows, mated refers to two images from the same source as DETERMINED BY GROUND TRUTH.

...knowledge or consensus determination

This does not apply to case work since ground truth does not exist in case work.

Since the full definition does include consensus determination - does this then apply to casework?

In case work, you would still "Identify" or "Exclude", these are the standard conclusions you would provide.

Identify isn't a standard conlcusion as per SWGFAST.
It is in parentheses next to Individualization in Ver 1.0 (9/11/03) of Standards for Conclusions http://www.swgfast.org/documents/conclu ... ns_1.0.pdf

But in the Standards for Examining Friction Ridge Impressions and Resulting Conclusions Revised Draft for Comment Ver 1.1 (2/11/11) it no longers exists. I'm wondering if it will be removed in future versions of Standards for Conclusions.
http://www.swgfast.org/documents/conclu ... FT_1.1.pdf

And in Standard Terminology of Friction Ridge Identification Ver 3 (2/11/11) it references individualization and is only defined independently as:

In some forensic disciplines, this term denotes the similarity of class characteristics.

So there is some conflict in SWGFAST documents between the use of "identify" and it may not be the best term to use or refer to as a standard conclusion.

[quote][Yet, I could get an intern and in 2 days probably manually go through 1000 cards (10,000 fingers) maybe 10,000 cards in a week or two....looking for any examples of "?" in the core. And I could walk into court and say "Based on a survey of randomly selected whorl patterned fingers from this database, the occurrence of such a feature was observed less than 1/100,000 times. These data comport with my personal observations, based on the hundreds of thousands of comparisons I have performed (i.e. training and experience) in my career."

Which answer is better in court? Data/Statistics enhancing training and experience...or training and experience alone. I'll take data any day over "I'm an expert". Data tends to shut down most arguments.quote]

I know it appears that providing the data in this example strengthens the examiner's personal observations (experience), but does it really? In my experience of examining millions of friction ridge patterns over my career, seeing a core pattern in the shape of a question mark is an EXTREMELY rare occurrence; in fact the image in this post thread is the only one that I can recall seeing in my career. Just so you can say that an intern (not even yourself, but an intern), with little or no training, went through a file/database to find patterns with a core in the shape of a question mark and didn't find any, doesn't truely add anything.

As far as the statistical modeling goes, I am cautiously optimistic that it may be useable some day...but we are nowhere near that stage yet. The major flaw is minutea/data selection by users is too subjective. Examiners in the same laboratory cannot even agree on minutea placement on anything other than pristine ridge detail. Latents, or knowns for that matter, that are low quality is where some kind of modeling would be most useful. But anytime the ridge detail is low quality, the minuteau selection and placement would vary among users, making the numbers vary greatly between who was doing the selection/placement...thus lessening and real impact of having statistics support any kind of conclusion.

Patrick

Although I agree with your premise about statistics and how they will save the day for fingerprint identification I think your reference to minutia "placement" being subjective is not the issue. Minutia placement is more a function of an AFIS search, not the actual identification.

I think minutia "existence" and sufficiency is the problem. Two examiners might not agree on what detail is present in a print or even whether it is "of value". This seems to be the issue defense attorneys hammer us on in court and it is the reason I resist having two examiners testify on the same case. The chance that explainations might sound different can be very confusing to the court.


I too am "cautiously optimistic" that statistical modeling will help, but I don't think for a minute that it will answer the challenges we face on cross examination. There is always another question.

I know I am breaking my own rule (replying more than 3 times in a thread...but I just couldn't resist any longer...)

So I will respond to the one thing that I have to ask myself, "How can scientists continue to put training and experience on such a pedestal?". It seems as non-sensical as putting statistics on a pedestal. I am fairly constant in my repeated message, statistics should be used to enhance and inform one's subjective (training and experience) views. To your point, Pat:

I know it appears that providing the data in this example strengthens the examiner's personal observations (experience), but does it really? In my experience of examining millions of friction ridge patterns over my career, seeing a core pattern in the shape of a question mark is an EXTREMELY rare occurrence; in fact the image in this post thread is the only one that I can recall seeing in my career. Just so you can say that an intern (not even yourself, but an intern), with little or no training, went through a file/database to find patterns with a core in the shape of a question mark and didn't find any, doesn't truely add anything.

Millions. This is why I love the training and experience answer.

I pulled 150 cards from our lektrievers. Keep in mind that we don't organize them by Henry, rather, we do it by SIDs. So these aren't even sampling just the whorl patterns. In 15 minutes, I found 2 question marks. The first one at #37, the second at #125. I guess I just hit the 1/millions lottery. Twice today. I will take data any day (..and use it to shape my opinions).

And for the record I think I can teach a college graduate/masters student to look for a question mark. Yes. I believe I can do that. And would trust their data. Research scientists use "grunts" all the time to collect data. No problems there.

g. (last one, I swear).

Isn't it amazing how these discussions can diverge from the original topic

As someone who probably would have agreed with Patricks "testimony" that "in all my years of looking at prints, I can't recall ever seeing a question mark like the one in this print." I have to give Glenn credit for skillful "cross examination" and rebuttal testimony


For my next witness I would call upon another expert on statistics to debate the size of Glenns sample group. It should be much larger than 125 so that the fractional value becomes consistent.......yada yada yada

Then maybe an expert on Henry classification to discuss whether the QM1 sample you presented is really a question mark (a recurve) or two ridge endings in the core.....snooze snore

If the OJ trial taught me anything, it is that a "battle of the experts" does not always help the trier of fact arrive at a conclusion that "helps them sleep at night"



I guess my "comfort zone" is being challenged and I know I'm not alone in that feeling.

I want to adapt to the changing world, really I do! But I was raised in a "faith based" environment and attended the church of the bureau under Pope RH and Monsignior SM.

I'm still not sure that modeling will be my salvation but I will keep paying attention and try to keep up with the debate

You're right Glenn, naive appeal to training and experience is less than helpful. It's basically acknowledging the inability to articulate the processes involved in the examination. Limitations of training can include Verification as confirmation and Comparison as purely quantitative point counting. Limitations of experience can include falling for the representativeness heuristic http://www.sfb504.uni-mannheim.de/glossary/heurist.htm and underdetermination of theory by evidence http://en.wikipedia.org/wiki/Underdetermination(Cliff Notes) http://plato.stanford.edu/entries/scien ... rmination/ (More complete). However, patterns in experience are the basis for prediction; predictive reliability a basis for Evaluation; consistent Evaluation a basis for consensus and consensus a basis for testimony. So to deny it can have equally devistating consequences as to accept it without constraint.


As for why it's put on a pedestal, that is a question of history. Cowger publishes and attributes the notion to T. Dickerson Cooke on pages 146-147 of his book "Friction Ridge Skin" originally published in 1983.

Pronouncing that two friction skin impressions, whether from fingers, palms toes or soles, were or were not made by the same area of friction skin is an art, not an exact science. It is entirely a matter of judgment based on training and experience. (Cooke 1973)

Considering the influence of early books such as Cowger's, Olsen's and The Science of Fingerprints, you had somewhat of a conceptual reliance on published material. This is somewhat compounded by a similarly timed resolution by the IAI that removed the a point standard. The vaccuum of leadership between the IAI Resolution and Ashbaugh's more elegant articulation is the era that many Examiners with seniority were brought up in. Like so many other tensions in science, there has been a tension in the 'leadership' of the discipline as well. On one end we have had the 'it's my opinion' model and on the other we have an absolute certainty model. Very rarely have we seen the critically rational aspect of leadership in the discipline. So the short answer to your question is that this is the paradigm modern day Examiners have had available to them. A paradigm shift is no doubt occuring especially in light of all the publications that exist, but it's not quite clear where the shift will end up resting.

I believe this is why you're seeing a backlash of skepticism by some Examiners to some of the trends in the discipline, most notably statistics. The flip side to your perceived irrationality of training and experience is a perceived irrational reliance on statistics which falls prey to the same pitfalls as training and experience such as base rate fallacy http://www.sfb504.uni-mannheim.de/glossary/baserate.htm.

To sum it up, training and experience at it's worst can be thought of as reducing to 'Because I said so' and statistics to 'That's what she said'. Neither are where we want to be.

Millions. This is why I love the training and experience answer.

I pulled 150 cards from our lektrievers. Keep in mind that we don't organize them by Henry, rather, we do it by SIDs. So these aren't even sampling just the whorl patterns. In 15 minutes, I found 2 question marks. The first one at #37, the second at #125. I guess I just hit the 1/millions lottery. Twice today. I will take data any day (..and use it to shape my opinions).

QM1.jpg

QM2.jpg

And for the record I think I can teach a college graduate/masters student to look for a question mark. Yes. I believe I can do that. And would trust their data. Research scientists use "grunts" all the time to collect data. No problems there.

g. (last one, I swear).

1 in 75 chance....That is why I love the statistic model answer.

It also is a great example of the subjectivity of user defined characteristics and how it skews perception. I clearly see the difference of the first image versus your two images. And how you may see question marks because you want to see question marks. Possibly just like an intern would.

The point I was making about my "testimony" of the mark is that because of my training and experience (Holy Crap! I said that phrase and I'm still standing!) I know the significance of seeing ridge flow in the core that actually looks like "?", not "u" or "n" or ">" or "/\"...(hey this is kinda fun playing with my keyboard).

FYI--
Lets check the math...52 weeks/year, but to be conservative lets say I worked 45 weeks/year. 45 weeks @ 5 days a week for 26 years (WOW, I'm old). And since my bosses always say I only work half the time, 5850 days @ 4 hours/day. Then using your STATISTIC of 600 cards/hour. And oh yeah, 10 fingers per card....

45x5=225
225x26=5850
5850x4=23400
23400x600=14040000
14040000x10=140400000


Don't get me wrong, I am not a hardcore "training and experience" examiner, but I do admit it has much more influence on me than "statistics".

In "Error and the Growth of Experimental Knowledge", Deborah Mayo suggests Kuhn (testing) and Popper (falsifiability) both have it right:

“Kuhn’s attitude toward the exemplars of normal tests is analogous to Popper’s treating the decisions required for testing as mere conventions. They simply report the standards the discipline decides to use to declare a problem solved or not. By Kuhn’s own lights, however, before normal practitioners may take a puzzle [scientific problem/question] as solved the hypothesized solution must have passed stringent enough tests. The arsenal needed for normal testing, then, is a host of tools for detecting whether and how conjectured hypotheses (of a given type) can fail. They call for methods capable not only of determining whether a hypothesis correctly solves a problem, but also of doing so reliably.” Mayo

“When we adopt a certain hypothesis, it is not alone because it will explain the observed facts, but also because the contrary hypothesis would probably lead to results contrary to those observed. So, when we make an induction, it is drawn not only because it explains the distribution of characters in the sample, but also because a different rule would probably have led to the sample being other than it is.” Peirce

Mayo quotes Laudan when defining qualitative elements as “hypothetico-deductive inference,” and then goes on to discuss the value of quantitative data: “… if we press the normative ‘why’ question about what makes quantitative effects so special and quantitative knowledge so robust, we see that what is desirable is not quantitative accuracy in and of itself. What is desirable is the strength and severity of the argument that is afforded by a special kind of experimental knowledge. As such, it makes sense to call all cases that admit of a specifiably severe or reliable argument ‘quantitative’, so long as this special meaning is understood. Quantitative knowledge teaches not only about the existence of certain entities but also about the properties of the process causing the effect.”

If we examine Mayo’s analysis of error statistics, she purports that what we want to know are the error probabilities associated with particular methods of reaching conclusions about this world. Establishing a probability error rate or, as termed in contemporary polls, a 'margin of error' will lead us to the promised land of experimental knowledge. By merging these different philosophies perhaps we will finally bridge the gap between ‘forensic science’ and ‘normal science.’

"printlady,"

Thank you for a most informative post!

-- Pat

Pat
To be informative it needs to be understandable, I think I am more likely to see the Bibical promised land than the forensic promised land, what ever that is.

Patrick
Thanks for some wise input.

Reminds me of that thing about 1 year of training and the next 20 years of repeating the same mistakes!

But then after 40 years of wandering in the desert the forensic promised lands will be before us, well not me actually, I will be dead.

The point I was trying to make was that a combination of different scientific philosophies and techniques will provide forensic science with the foundation demanded by the NAS report; sorry if I was too cryptic. These philosophies are outlined in various scientific texts, but we must determine which specific concepts are applicable to our needs. Kuhn suggests rigorous testing is the basis for good science while Popper says falsifiability proves theories. Mayo’s opinion supports hypothesis testing which leads to deductive conclusions, and also proposes that we can learn from the process we use to evaluate the quantitative, qualitative data provided through testing. She also suggests that Kuhn and Popper’s general philosophies aren’t necessarily in opposition. Throwing error statistics into the mix will provide others with an idea of how reliable the conclusion reached may be. This is the general direction we are headed and scientific literature supports it.

In ‘The Structure of Scientific Revolutions’ Kuhn differentiates normal science from revolutionary science, suggesting that scientific progress is achieved in leaps of logic rather than every day, routine scientific practice and that normal science is a cumulative endeavor. I know some of these concepts seem intuitive, but I think it is imperative that we understand why we are changing what we do and how we do it, not just spewing out dogma and calling it good enough for government work (I know what you're thinking... why didn't she just say so!).