NAS Report & ACE-V

[izatso]

Hi MDavies,

Sorry not to have been able to get back to you sooner, but I was out of town yesteday and not near a computer with internet access. Your last response was rather lengthy. Unfortunately, at least with regard to what I have been discussing it is, how should I say, misdirected. I think it does serve the purpose of underscoring that we have been talking at crosspurposes. I have to say emphatically that no, you do not understand my position. Although, I thought I was making myself clear, perhaps I was not.

Anyway, you go into a lot of detail conscerning latent print comparison. I do not have a problem with what you say nor the problems you delineate therein. I can only say, "apples & oranges." I am speaking about apples, the underlying science which make all identifications possible whereas you are speaking about oranges, latent print comparison. Obviously, both tenprint and latent comparisons are dependent upon the underlying theory of the uniqueness of friction skin.

In another sense, you have placed the "horse before the cart." Yes, we will have to do studies on latent comparisons which do in fact take into consideration all of the factors which affect latent print individualizations. But first things first. Before we can even go there, we must insure that the basic theory of uniqueness which undelies all individualizations regardless of whether we are talking about rolled or latents, is sound.

Be that as it may, we do need to know if minutia configurations do replicate and if they do, at what "number of Munitia" and how frequently. Obviously, and something I believe with which you would agree, we should not consider minutia independent of the friction skin upon which it appears and we must consider distortion, spatial relationship of all the minutia in any particular group as well as it location in the larger print. In any case, if we can answer the above question then we actually do put ourselves into position to begin the research that your last email suggests.

I am not saying that research on latents should wait for this, but I am stating that all individualizations are dependent on what is discovered regarding uniqueness of friction skin. All results in other facets of tenprint, palm or latent is contingent upon this first question.

Again, distortion is an area in which we need to do a lot more research. I applaud people like Alice Maceo who are doing this research already. As a field, I believe we will all benefit from that. We hosted Alice here a couple of years ago and I frankly believe that in my career, that is one of the best classes I have ever had. One of the concerns you expressed in your post above is that we might return to what you called "backsliding" as you said,

Another question would be how many level two details are in a grouping? This smacks of a backslide to the old ridge counting thresholds which we (the IAI, SWGFAST, etc.) have so desperately tried to abandon.

As a chemist and hopefully as a scientist, you should not go into any scientific testing with the view to impose your presuppositions upon the phenomena you are studying. What should drive the direction and process of making individualizations is not personal opinion, not anecdotal information nor your preferences, but what such tests reveal. Research should drive the process, our previous understandings may be of help in formulating tests, but the results need to speak for themselves and if they prescribe that we change our currently accepted practices, the we change our currently accepted practices.

At this point, it is premature to say that we should or even must change the way we form our conclusions nor am I advocating this. Lets wait and see what the research reveals. As I mentioned previously, if say there is replication in minutia groupings, then I believe that is something you as a examiner would not only want to know, but something which would be given weight by you in forming a conclusion which consisted of only this as yet unknown lower threshold number in an area of the print where such groupings have a higher percentage of occurring.

Roy Marzioli, Manager
Central Identification Services
Contra Costa County Sheriff's Office
Martinez, Ca.

[L.J.Steele]

Just an aside on Gerald Clough's point about legal admissibility v. credibility (of the result? method?), another attorney just sent me this abstract, which raises yet more questions about the weight vs. admissibility issues in expert testimony.

The gatekeeper effect: The impact of judges' admissibility decisions on the persuasiveness of expert testimony.
By Schweitzer, N. J.; Saks, Michael J.
Psychology, Public Policy, and Law. Vol 15(1), Feb 2009, 1-18.
Abstract
In a pair of mock-trial studies of a possible “gatekeeper” effect, our participants were presented with a summary of a trial that included a piece of expert scientific evidence. The judge's decision was manipulated to admit the scientific evidence, as well as the quality of the evidence and the credibility of the expert. Participants were found to be less critical of and more persuaded by expert evidence when it was presented within a trial, compared with the same evidence presented outside of a courtroom context. These findings suggest that, when judges allow expert testimony to reach the jury although the evidence is of low quality, they imbue it with undeserved credibility. Furthermore, no changes in participants' perceptions of the evidence were found if the mock jurors were explicitly informed that the judge had evaluated the evidence, suggesting that the participants assumed that judges normally review evidence before allowing it to reach the jury. In addition, implications for basic research are discussed, as the moderating effects of a gatekeeper have not previously been considered by established models of persuasion. (PsycINFO Database Record (c) 2009 APA, all rights reserved)

[mdavis]

I am all in favor of scientific testing and validation. My point is that there are so many variables even within multiple impressions of the same source, that designing unbiased tests is essentially impossible. That's the reason we've never been able to come up with the requested statistical data that the DNA people so richly enjoy. I hope someday that may be overcome, but as an experienced examiner, I am not optimistic. It has been decades since much of the statistical studies were published. Nothing has come forth. When we rely on three levels of variable detail, we should be designing computer recognition scoring that eliminates human bias and incorporates all three levels. But we aren't there yet, and the simplicity of our best AFIS systems underscores that fact. AFIS is using level two almost exclusively, yet, as I mentioned above, the AFIS reliability rate in scoring the exact candidate is far below that of the average, ethical, trained, experienced examiner. That's not what we want, it's not what the NSF wants, it's not what the court system wants, but that's what we're stuck with until we find a way out.

...we must insure that the basic theory of uniqueness which undelies all individualizations regardless of whether we are talking about rolled or latents, is sound.

In the meantime, our level two AFIS systems continue, in their own simplified way, to underscore the belief held for a hundred years now, that friction ridge skin is both permanent and unique. They are doing that by comparing millions of latent prints daily all over the world. To the best of my knowledge, no two have ever been found to be the same, searching latent after latent against databases of millions every day. We believed that when we switched to fingerprints for individualization, and AFIS is reinforcing that belief every day, true to not. What everyone wants is "proof." Well, so do we. What more can we do than to upgrade our computer systems? That leads to the point I made about the difficulty if not he impossibility (a harsh word, yes) of using a "blind", unbiased sorting algorithm that is both accurate, yet compliant enough to accommodate flexibility. Those two are at cross purposes, even limited to level 2 alone. I don't think it will be done with any degree of scientific rigor. We must allow a hundred years of history and millions of searches to continue to underscore the reliability of fingerprint science (or use your own word if you don't like "science") because I don't see any other answer on the horizon. I support and encourage those who would try. I just don't think it will happen. If that means the courts choose to throw fingerprints out as "junk science", they do so at their own peril. If fingerprints ever fail to pass the test, then what other forensic science other than DNA can possibly survive -- firearms, cartridge casings, bullets, hair, fiber, trace, tires, shoes, drug field testing, microscopy, soil? How about eye witnesses? Officer reports? Attorney claims for their clients? Photographs? Video cameras? How many times must we drop an apple to "prove" the theory of gravity while waiting for one to go up instead of down?

...we do need to know if minutia configurations do replicate and if they do, at what "number of Munitia" and how frequently.

Let's just go back to counting points after we define what a "minutia configuration" is, and then set a hard and fast standard number. If the print doesn't have a "minutae configuration" defined in the handbook and it fails the point count test, it must not be an individualization? Throw out level one and level three. That's about the only way we'll ever get hard numbers for the statisticians (if we can also agree on the degree of variation or flexibility allowed in our measurements.) Somehow I think we've been there and done that, and we got beyond it. It would provide numbers. What would they mean? As a scientist, show me proof that there is a better way, and I'm all for it. I've also done this long enough that I don't believe we're even close. But I'll always listen. I've been listening for many years and I haven't heard anything better than our own history. Nothing is 100% sure when humans are involved. But the discipline is being discredited over a very small number of high profile cases that become high profile by their scarcity. Would that the burden of proof could be upon those who discredit the discipline to show it to be unworthy of legal consideration. Unfortunately it only works that way for the lawyers. I'll put my bet in the hands of a trained, competent, ethical latent print examiner any day, absent so far of the desperately desired proof of uniqueness.

[L.J.Steele]

First, the NAS report is being discussed on NPR's Science Friday show today.

http://www.sciencefriday.com/program/archives/200902275

Listed guests include:

Barry A.J. Fisher
Crime Laboratory Director
Los Angeles Sheriff's Department
Los Angeles, California

Constantine Gatsonis
Co-Chair, Committee on Identifying the Needs of the Forensic Science Community, National Academy of Sciences
Professor of Biostatistics and Director, Center for Statistical Sciences
Brown University
Providence, Rhode Island

As to MDavis' point on uniqueness and permanence. I haven't read the full report yet, but my understanding from the summary and the press conference was the issue is reliable discernability those unique features in crime scene evidence. I don't think courts are going to toss out most forensics, but judges may get much, much more interested in training, certification, methodology, and limits on how a conclusion is phrased.

[Gerald Clough]

I don't think courts are going to toss out most forensics, but judges may get much, much more interested in training, certification, methodology, and limits on how a conclusion is phrased.

Of course the courts are not going to reject fingerprint evidence. They have repeated pointed out that it has shown itself to have great value for many, many years. The courts are not bound by a pure science viewpoint and are quite prepared to properly accept a great weight of human experience in forming their views of evidence. If the justice system were operated by scientists, few issues would be resolved. Most witnesses would not be allowed to testify at all, because there would be no scientific proof that they were giving an accurate account of events or even that they weren't lying. So, where might we expect courts to go with fingerprint evidence?

We might profitably look at eyewitness identification. It is not so different. The most obvious is that eyewitness facial identification a matter of human observation and comparison. The comparison is done against an image in memory, and the "analysts" don't use an expert terminology, but research suggests they subconsciously apply a structured analysis. Not a perfect analogy, but not so foreign to fingerprint comparison as we may think. But for our attempt to project the future legal view, the important similarity is that both have similar cultural traits. There is a powerful belief that both human faces and fingerprints can be differentiated sufficiently to have great identification value. Facial identification is seen as an inherent human ability. Fingerprint identification, while seen as not nearly such a natural ability, has one-hundred years of utility that moves people to see trained fingerprint examiners to have abilities in comparing fingerprints similar to that of humans identifying faces. Juries were imagined to be able to reliably judge the credibility of facial identification as a matter of common sense. Fingerprint identification was seen as similarly amenable to common sense, the technique being viewed as something straightforward, albeit requiring training and experience that, in the case of facial identification, was taken for granted as developing naturally as a human life skill. All these views have flaws, but I think fairly represent the general attitude toward the respective types of testimony.

Judges traditionally have rejected attempts to put forth experts to educate juries about the nature of eyewitness identification, specifically citing the "common sense" abilities of juries who share the experiences of identifying fellow humans' faces. The legal basis for rejecting such expert testimony about eyewitness identification in general was that there was no body of published research and therefore no general acceptance of what those experts would present. As serious research proliferated, mostly in academia, the federal circuits began more frequently holding that it was error to exclude expert testimony on the nature of eyewitness identification. The rulings are far from consistent across the circuits. Some have not adopted it at all. Some have conditioned it as being required when no other substantial evidence supported the identification. But the trend is unmistakable.

Such experts are, of course, most valuable to jurors when their testimony addresses specific issues peculiar to the circumstances of the eyewitness experience in the case at trial. The reason that they are increasingly allowed to testify is that eyewitness identification is not so reliable as was once assumed and is subject to a number of internal and external influences and sources of error. Let me change change the subject of that statement:

Fingerprint identification is not so reliable as was once assumed and is subject to a number of internal and external influences and sources of error

Neither sort of identification is inherently unreliable. But no individual identification of either type should be blindly accepted as never subject to critique or question. It is the nature of adversarial justice that it seeks to apply all relevant available evidence and to hear it in such a way that its credibility can be judged by the triers of fact. Any credible expertise that can assist the tried of fact is welcome. The same standards of admissibility apply to testimony reflecting on other evidence as to the evidence it addresses. What the NAS report calls for, in large part, is more work on the nature of fingerprint evidence. We have seen some offerings of critical expert testimony on the nature of fingerprint identification rejected as "junk." I dislike that term. It seems to imply that anything labeled "junk" is badly motivated or incompetent. It does not mean that. it merely means, in some instances, that there's just not sufficient research product to admit it. But we should remember that the germ theory of disease was once derided as "junk," as was the mosquito vector theory for malaria and yellow fever.

I think the NAS report brings to the attention of the courts that they should be prepared to consider testimony on the nature of fingerprint identification. It both raises questions that are worthy of investigation and begins to shape the judicial considerations in the future. Expect that future to include testimony based on published and accepted research. Like all research, it will not immediately settle every issue. It will not wring every drop of insight from the material. It will always raise more questions than it answers. It is, in fact, these questions that are the most valuable product of research. And as questions, they will challenge assumptions and practices. And, as scientifically acceptable knowledge, it will be welcome in court. As this legal issue evolves, it will be more and more insufficient to appeal to jurors on the basis of mere experience. The universal experience of facial identification is insufficient to answer what has been learned about how we do it. I, for one, welcome the exploration of every part of our subject. I think it will increasingly bring our discipline to legal maturity and, in the end, make us better.

[Ducky]

[quote]MDavis stated “Let's just go back to counting points after we define what a "minutia configuration" is, and then set a hard and fast standard number.” [/quote]
Point counting is what the scientific community used to accept fingerprint individuality as reliable. In the last few decades or so, many practitioners of holistic non numeric examinations from America threw out the foundation of the original scientific acceptance. Relying primarily on level 3 friction ridge structures for conclusions, which has been proven unreliable and not durable in transfer, practitioners who deny the existence of statistical validity have thrown the baby out with the bath water.

[quote]“As a scientist, show me proof that there is a better way, and I'm all for it. I've also done this long enough that I don't believe we're even close. But I'll always listen. I've been listening for many years and I haven't heard anything better than our own history.”[/quote]
Our history started with a statistically supported foundation of randomly formed events created by the folding friction ridge skin and formation of the primary and secondary ridges. This is what is truly unique, not a bump on a ridge or a shape of a ridge edge. Biological uniqueness applies to the total entity based up random individual parts. Biological uniqueness is not based upon an individual part.

Point counting level 2 features to a statistically validated level of predictable reliability is scientifically defensible. To remove subjectivity quality assessments in the Q-Q equation, a quantifiable definition is needed before using level 3 to validate statistical models. Only then can level 3 take the conclusion to sole source biological attribution beyond the statistical model. This is scientifically supported and proper. It is the yardstick that is needed to measure sufficiency during the evaluation phase of ACE.

[quote]“But the discipline is being discredited over a very small number of high profile cases that become high profile by their scarcity. “[/quote]
These cases have merely exposed there is no money in the bank to pay the checks that the fingerprint community have been writing after throwing out the validation of the uniqueness premise……The NAS report merely points out the need to make a scientific deposit into the fingerprint account.
The king has no clothes, by many refuse to see this…..
We would be better off understanding our shortcomings and not prefer to hide behind the green curtain in OZ.

[izatso]

It has been decades since much of the statistical studies were published. Nothing has come forth. When we rely on three levels of variable detail, we should be designing computer recognition scoring that eliminates human bias and incorporates all three levels.

Actually, nothing has come forth because the studies have not been done. I note you put a lot of emphasis on 1st and 3rd level detail. I find this to be very ironic with regard to latents since both of these are unreliable levels of detail in the majority of latent prints. That is unless your department only accepts latents with 1st and 3rd level detail. We accept what they give us and evaluate each latent on its own merits. Its all well and good to hope for the greatest amount of clarity, but seldom is that hope realized with regard to latents.

In a former post, you mentioned marginal or borderline prints. What makes them marginal or borderline. There is generally a scrap so there is no level 1 detail and a lot of distortion so there is no dependable level 3 detail. In such situations, what do you depend on? Level 2 detail! The irony here is that you talk about all the problems associated with latent print deposition, but you seem to be ignoring the lack of 1st & 3rd level detail in most latent prints. Your vision of using level 1 & 3 seems to be based upon an idealized latent print because it has sufficient level 1 and level 3 detail to individualize. In my expereince, that is not the norm, that is the exception. So basically your standard is based upon some ideal which is very seldon realized. That is my experience. I believe that a poll of most examiners would reveal the same thing.

When we rely on three levels of variable detail, we should be designing computer recognition scoring that eliminates human bias and incorporates all three levels. But we aren't there yet, and the simplicity of our best AFIS systems underscores that fact.

Is it simplicity or is it because of consistency? I think it is the latter. I think that most examiners who are honest with themselves can see the level 2 detail is the work horse of the profession regardless of whether you claim to quantify it or not. You create this either or situation because of what I perceive to be a commitment to a method rather than to the physical phenomena with which we work. Most good examiners that I know will use all available information in a print. You falsely imply that those who use 2nd level detail ignore level 1 & 3. This is absurd. I am not obligated by my adherence ot ACE-V to ignore any level of detail and on the contrary, I must take all of it in to consideration when I draw my conclusion. I have and will continue to reject the false dichotomy of ridgeologist and point counters. I believe that point counters as understood by you is merely a myth. More likely, it is a straw man which is easy to set up and criticize. The only problem is that it doesn't exist. I hope you realize that straw men arguments are logical fallacies.

When designing systems such as AFIS you want to base it on the information which is the most reliable. That just so happens to be 2nd level detail. As it just so happens, all of the AFIS systems that I know work very well because they use the same level 2 detail. For latent searching, level 1 and level 3 are undependable. I have and will continue to advise against the use of 3rd level detail as the main source of individualizing information and I would expect you to do the same. Why? Because your experience should demonstrate that 3rd level detail is unreliable. Why is it unreliable? Because in most latent prints, it isn't there. Computers don't do magic. They can only do what we program them to do and they can only be as effective as what we tell them to search. Again, 3rd level detail is inconsistent and unreliable. Why would I make a computer that can search it? Will it make 3rd level detail more reliable? I don't think so. Remember, you can't make a silk purse out of a sow's ear.

Even if you program a computer to search for it, it isn't going to make it any more reliable or dependable. Unless it is given negligible weight much like we weigh an incipient ridge, 3rd level detail will increase your percentages of misses because it is either missing or distorted in your latent. If you are depending what you perceive as 3rd level detail in your latent, because of distortion, it is highly likely that what you see in the latent looks nothing like the less distorted exemplar print you would like to hit in an AFIS. End result, miss in AFIS.

[mdavis]

Excellent! You just saved me a heck of a lot of hard work. I can now go back to (what was it, 12?) level two details and forget all those years of education, training and experience. I can forget pattern types, scars, incipient ridges, contours and pore locations because they don't mean anything because they cannot be statistically validated? If they cannot be validated, then we shouldn't even be using them?

Point counting level 2 features to a statistically validated level of predictable reliability is scientifically defensible

Not unless we can create machines that can independently agree 100% of the time on what a level 2 detail is, and within what limits they will allow for friction ridge skin flexibility. Using level 2 AFIS results as statictically defensible is pretty lame, dont' you think? I have watched several different AFIS input operations. It is always a necessity for operators to "clean up" what the computer mistakes for level 2 details -- add missed markers and delete incorrect markers. This is human interference and human subjectivity. Left alone, AFIS systems are often pretty sloppy and even inaccurate far beyond the point where even a lay person could recognize errors. So what do you do? Let the computers fend for themselves and obtain statistical results from algorithmically flawed data? Or do we inject operator subjectivity, thus "tainting" the database? Either way, statistical relevance is lost.

...checks that the fingerprint community have been writing...

Let's talk about individual practitioners, can we? Most of us have stayed back from the bleeding edge of controversy by remaining highly conservative in our conclusions and realistic in our opinions on reports and in the court. Then one or two "experts" who are either poorly educated or overly zealous, claim individualization where they have (later) clearly crossed the boundaries of good judgement and common sense, and the entire "fingerprint community" is suddenly without a valid basis until we come up with a statistical model for uniqueness and probability. The "fingerprint community" may be what you hear about, but there are hundreds of us who have never reported a bad ident because we don't claim anything but what we can see. That doesn't imply we haven't made mistakes, but that we have never been challenged or shown to be in error. I ask myself on every ident, "if this goes to court and the defense obtains a legitimate, trained, unbiased third party to re-examine my work, will it stand up?" How many thousands of perfectly valid idents are made every day and how many bad ones? No one knows absolutely because there is no "ground truth." We cannot prove statistical validity the same way DNA can with "blind" mechanization, scoring and comparisons. If someday that becomes possible (I remain skeptical), then we will only be needed to capture latent prints for the AFIS computers, never again using humans for manual comparisons.

I have and will continue to advise against the use of 3rd level detail as the main source of individualizing information and I would expect you to do the same. Why? Because your experience should demonstrate that 3rd level detail is unreliable. Why is it unreliable? Because in most latent prints, it isn't there.

There has only been one case to my knowledge that has been made based solely on level 3 detail. I advocate, as you do that one uses everything that is available during a comparison. Obviously, if level 3 is not there, it is unreliable by definition. When it is, it adds immensely to the mental "score" (far beyond the ability of a mere level 2 computer search). Yes, level 2 is is the "workhorse" of comparisons. LIke level 1, it is quick and exclusionary, which is how we perform comparisons. But level 3 is unreliable? Not if it's there. It's just far too complex to codify for a blind mechanical system such as our AFIS computers, so we use it but we don't "score" it for the purposes of "validity?" Why not? Because we can't find a way to score it due to the complexity and infinite potential variations, which I think was your point as well.

My point remains, there is currently no practical way, basic "point" counting included, to statistically validate the uniqueness of friction ridge skin detail. It is an analog feature, and to digitize it to a numerical entity in an attempt to gain statistical relevance is to grossly oversimplify the process of comparison. One only has to look at the relatively poor results of AFIS systems to score level two latent print comparisons when generating candidate lists to see that a non-human approach is inadequate and too far in the future to help us. The whole process lies with the ability of the individual examiner to "know when to hold 'em, know when to fold 'em, know when to walk away, know when to run."

I don't think we're apart on the basic ACE-V system (if you will) of comparisons. I don't think we disagree on the pressure from outside groups to "validate" our conclusions. Where I do think we disagree is on the probability of being able to do so in a scientifically, statisically valid way either by limiting (severely) our data to "simple" level 2 details, or by attempting to incorporate level 1 and level 3 details in our data. How ya gonna keep 'em down on the farm after they've seen DNA? Well, they've brought some of the farm with them on their shoes, thanks to a very few examiners who have forgotten the very basics of latent print comparison and the enormous responsibilities we have to the accused, the public and the court system, and yes, even to our own careers.

[Ducky]

MDavis,
Your view of discounting AFIS as not being valid because it cannot supply the correct candidate every time is myoptic and ignores the utility AFIS provides to supply the closest non match of durable transferance of level 2 features, however limited to feature extraction and matching algorithms, which allows human evaluation exponentially far beyond the limited empirical observations made in the 100 years you cite for fingerprint experience. Inconsistencies you point to are not human, but occurs from what attempts to emulate a human comparison by the feature extraction algorithms. These can perform perfectly or innacurately based upon the AFIS vendor extraction and matching algorithms, as well as the calibration of the input device.

AFIS deals solely with persistent level 2. Level 3 concurence cannot occur if level 2does not. The closest level 2 non match validates a level of reliability to the comparison at hand and the amount of agreement of level 2, and what level 3 supplements the level 2 data. To date AFIS has provided many close non matches based on level 2, but has not been able to falsify statistical models on level 2 individuality sufficiency, indeterminate level 2 with supplemental level 3, nor the tripartite rules of Locard which is followed throughout most the world as a standard.

The level 3 individualization you cite as the only one you know of was the one published in JFI on the rifle scope. If you look at the details used in this individualization, you will note that it was solely incipient ridges, which by definition is level 2, an inmature permanent ridge. Do you have another example of solely level 3? Do you have any citation of studies that promote level 3 as reliable vs. all those that support the opposite?

If friction ridge analysis is scientifically reliable, and humans are one species, then there can only be one valid scientifically defensible standard for individualizaiton regardless of where on this planet we practice our profession. It is our responsibility to promote what is scientifically supportable in both the basis of, and the conclusion of friction ridge individualizaion.

[mdavis]

Some good points have been made here. However I am not discounting the utility of AFIS, just the opposite. I use it every day. What I do discount is using current non-human, blind computer technology to form the basis of trying to validate the scientific basis of latent print uniqueness. Where we (the latent print community) went wrong was in trying to push our craft from the researched, biological underpinnings of the formation of friction ridge skin, beyond biology and into the realm of human subjectivity. Now we (the community at large) are having to "back up" from claiming total scientific validity of an "absolute" identification using ACE-V and either admit to the last link of subjectivity, or we struggle to find a statistically valid way to confirm that last link. It is that quest which I fear is insurmountable. I truly hope I'm wrong.

Inconsistencies you point to are not human, but occurs from what attempts to emulate a human comparison by the feature extraction algorithms. These can perform perfectly or innacurately based upon the AFIS vendor extraction and matching algorithms, as well as the calibration of the input device.

Computers are essentially "perfect" in that they do exactly what we program them to do. They can be "perfect" while still being inaccurate. But the link between the "absolute perfection" of the unbiased computer is influenced by the imperfect programmer and the uncontrollable variations of friction ridge skin impressions and all the myriad of influences that cause those impressions to vary. What the computer cannot do is provide a reliable analysis of repetitive impressions from a common source through the haze of natural phenomena. The AFIS vendors are constantly trying to improve their algorithms to increase accuracy in comparing two variable impressions made from a common source. They are good, but far from perfect. We have nothing better. And if we cannot come up with a statistic for the variability and inherent inaccuracy of those algorithms, we cannot use those algorithms to provide statistical validity for themselves, can we? If all impressions were "perfect", we could easily demonstrate identity by overlaying the latent with the control. Any examiner knows this is impossible. Yet isn't that what we are asking our algorithms to do? If we account for the natural variations in multiple impressions from a common source, we must allow for tolerance or "error" in the algorithm so as not to overlook distortion. If we lock the algorithm to "perfection" then we miss many comparisons. Either way we lose.

Level 3 concurence cannot occur if level 2does not.

I see this as a common flaw in our reasoning. Friction ridge skin detail is a continuum. It begins with the primate form, and runs to incredibly minute detail -- from arms and legs to hands and feet (and some prehensile tails) to palms and fingers to ridge patterns to Galton details to however small you want to go. We have arbitrarily classified "levels" in an attempt to explain why we can use varying levels of detail in our comparisons. A lot depends on your definitions. Incipient ridges are commonly grouped with level 3 observations along with creases, warts, cuts, scars, etc. I'll agree that incipient ridges are a manifestation of level 2 details, yet they are commonly excluded from our "point counting" because they, like other level 3 details, may or may not appear depending on skin condition, inking, pressure -- you know the drill. Does it matter? Aren't level 2 details made up of level 3 details just as level 1 is made up of level 2 and 3? These are all as interdependent as zooming in on a fractal. Our "levels" are arbitrary simplifications for the purpose of trying to explain our conclusions to the lay jury and more recently to help support the ACE-V methodology.

I am skeptical of our being able to scientifically validate latent print comparisons to the point of removing human subjectivity somewhere in the chain or at least being able to provide the statistical validity that is being requested. We are subjective when we define our "levels." We are subjective when we design the algorithms for our computer comparisons for our "unbiased" comparisons. We are subjective when we decide on how much variation to allow when writing the code for the "objective" computer to compare millions of impressions against one another. And are not many "bad idents" the result of examiners relying too heavily on computer-generated candidate lists from massive databases? I use AFIS as an example because that is the best we can do with regard to "unbiased" non-humanly subjective comparison. If the code is flawed, we cannot use it to generate scientifically valid results.

What I am not saying is that latent print comparisons, performed by experienced, trained, ethical professionals are not scientifically valid and highly accurate (despite critical allegations to the contrary). What I am saying is that there are too many uncontrolled variables in the equation to be able to come up with a valid set of numbers, absent human subjectivity, that will satisfy the NSF study for statistical validity. What I think I'm seeing is that the community looks good in trying to accommodate the request of the NSF. What I'm afraid of is that it will ultimately look foolish when we can't come up with a valid result for all the reasons given. We are treading on the same ground here that muddied our shoes when we pushed the entire concept of ACE-V as being totally scientific rather than acknowledging the inevitable subjectivity factor.

[Ducky]

Mdavis states,
"I see this as a common flaw in our reasoning. Friction ridge skin detail is a continuum."
Contiuuum does not faciltate class charateristics and their required link to indidividual charteristics. It is scientifically and inevitably locked. There is a break point where continuum ceases to exist..... except in your vision.


""I am skeptical of our being able to scientifically validate latent print comparisons to the point of removing human subjectivity."
Subjectivity is only how you measure the sufficiency yardstick to a Q/Q equation that is quatifiable to both number of level 2 and clarity of level 3.
By listening to your perspective, you seem to believe in the perspective of the inverse triangle which cannot stand.[attachment=0]levels.jpg[/attachment]

What ever you belive...gods speed
Whatever you need to subsitantiate in your conclusions, gods speed

[skflynn]

I have some quick thoughts.

For over 20 years we as a profession have been using AFIS systems in one way or another and I have yet to hear of a case of two people having the same individualized ridge detail (no two people having the same print). That being said I believe that the foundation on which we rest is solid and the technological changes that have been brought forth have proven it out.

Pat W. and many others have pointed out in the past, ACE-V is a methodology to describe the process that we use to of how we individualize. If a chemist or criminalist improperly applies the steps or method in their analysis and reaches a wrong conclusion in lets say in a drug analysis or DNA testing does that in fact make the science wrong or just their methodology wrong in that test? In my mind It also goes without saying that properly trained examiners are the foundation of the ACE-V process and verification the most improtant step.

As for Mr.Scheck, his position seems to go the direction that his client of the day needs. If the DNA helps his client the science is sound; if not attack it as flawed, contaminated or improperly applied. He is an attorney and a very good one at that. Mr. Starrs positions appear to fit his need to keep his name out there in the publish or perish world of academia. You have to give him credit he has been successful to that end.

The focus that the NAS reoprt will bring on our profession is not bad and in my mind and will only cause the dicipline to continue look at the way we do business as well as our methodology. That in turn will keep us sharp.

This is just my opinion and your mileage may very.

[mdavis]

Contiuuum does not faciltate class charateristics and their required link to indidividual charteristics. It is scientifically and inevitably locked. There is a break point where continuum ceases to exist..... except in your vision.

The term "class characteristics" is an arbitrary definition, well accepted within the profession. If there is a break point in a continuum, then it isn't a continuum except at the ends. What that sentence has to do with the discussion I'm not sure, nor do I understand what is meant by "it is ... locked" and "individual" characteristics.

Using the triangle illustration, we all begin from large detail, progressing to small as we look at a latent print. The process of comparison is one of elimination if for no other reason than efficiency and time, and the basic way we all compare everything from human faces to latent prints. No one would begin with fine details without first eliminating pattern and Galton details, which is a stable triangle. What details we have left when we can no further eliminate, we use, I would hope -- all of them. Most of us use "ridgeology" in our analysis where it is clearly seen. This, however does not make one a "ridgeologist" at odds with a "point counter." The real question is how good and how many? The "point counters" want rules and numbers and statistically validated acceptance of them. Not all comparisons fit that restriction without being passed over and/or falsely identified. If we count points, then how many do we need, what is a "point", and if the comparison falls one short of the "magic number", do we throw it out? What do we do with the rare possibility of having a "magic number" of, say 12, and our latent has 8 trifurcations? Not likely, but the "rules" say we would call it inconclusive. Where do creases and scars belong? Does a palm crease "cause" ending ridges on either side? Can you count them? Can you count the crease as a "point?"

This has turned into a "40-point type" shouting match over semantics. The point of discussion, I thought, was in response and challenge to attack from outside the profession, not to create one from within.

[Ducky]

MDavis
You looked at the triangles as a comparison model. The triangles are for demonstration of the ability to substantiate conclusions relying on features that are neither consistently repeatable nor durable in transfer.

The questions you ask of how good and how many are not an insurmountable task to attain. The variables you point to if defined, measured and used with established values would help to eliminate the contentious subjective state we are now in both legally and scientifically. The sufficiency and clarity yardstick needs to be created.

There needs to be an acceptance of the shortcomings of the foundation for our conclusions which has been seen by the scientific community for decades. Rather than shouting down our naysayers as having no standing to judge our practices as they are not friction ridge analysts, our community needs to accept the criticisms that Mayfield/Jackson/Cowans etal have exposed in our shortcomings.

The attack from outside our community is scientifically valid and they are here to stay. Far too many F/P examiners are again circling the wagons in defense by using the same logic that temporarily staved off the legal challenges. That logic cannot stave off this scientific challenge.

The arguments you disdain within our community have been unresolved for years and there is still not a large amount of agreement in the USA as to sufficiency. Many do not agree on the method of application (Quantification or Holistic) of ACE and our country is still divided. I would estimate quantification examiners to be proportionally in orders of magnitude for the majority. The holistic enthusiasts have pointed to Great Britain as now being non numeric since 2001. However during a 2007 IAI statistical presentation the examiner from England stated that there is a national written fingerprint norm and a personal fingerprint norm. This examiner stated that they quote the written norm but no one in England individualizes on less than 8 Galton.

Many state publically that they don’t quantify due to peer, agency, or other associative pressures. In reality many of these do quantify privately. Those who quantify also use level 3. They do not put on level 3 blinders and count to a magic number and stop. So there is not much difference between the two friction ridge camps. How we explain features, their use, and utility, and how much weight we place on them is where each of the two camps differs on validating their conclusions.

From a prior unanswered question: Have you or anyone you know found a real example of a level 3 only individualization or a study published which supports the level 3 feature reliability in transfer?

Thanks for the discussion

[izatso]

Hi MDavis,

Using the triangle illustration, we all begin from large detail, progressing to small as we look at a latent print. The process of comparison is one of elimination if for no other reason than efficiency and time, and the basic way we all compare everything from human faces to latent prints. No one would begin with fine details without first eliminating pattern and Galton details, which is a stable triangle.

It appears to me that you are referring to an idealized latent print comparison. I say idealized because many, if not most of the latents we receive for comparsion, lack any level 1 detail and with considerable distortion, have very little discernible or level 3 detail. My own experience is that even with the best of prints, you cannot trust level three detail in either the latent or the exemplar. If 3rd level detail is there, we use it, if not, we don't automatically move to exclude. If we did, we would be eliminating a lot of latent prints. As for how we proceed, we start with the latent which informs us of the quality threshold we have to deal with between the latent and the exemplars. It doesn't matter how good your examplar is, any comparison to any level of detail will be dictated by what appears in the latent print.

In moving from latent to exemplar, we are of course looking for exclusionary factors, but in their absence, we will look for sufficient similarities which will allow us to individualize. You mention creases and ridge endings. Has it been so long that you would mistake a ridge crossed by a crease for an ending ridge? I don't think so and if you were in doubt on this matter then that would form part of your conclusion. As for magic numbers their are none. Why? Because no studies have been done to see at what level of minutiae grouping replication occurs. This brings us back to the NAS Report and our short comings. The report is asking that we do studies both on our comparison method but also one the physical phenomena we use for analysis:

To properly underpin the process of friction ridge identification, additional research is also needed into ridge flow and crease pattern distributions on the hands and feet. This information could be used to limit the possible donor population of a particular print in a statistical approach and could provide examiners with a more robust understanding of the prevalence of different ridge flows and crease patterns. Additionally, more research is needed regarding the discriminating value of the various ridge formations and clusters of ridge formations.

The recommendation is for research is not just on the reliability of our comparison methods but on the natural phenomena upon which our analysis depend. We need research on ridge flow and crease pattern distribution and also on "discriminating value of the various ridge formations and clusters of ridge formations." I realize that you have grave doubts as to whether or not this can be achieved. This is duly noted. However, I think we ought to withhold judgment pending what such studies encounter and possibly reveal.

Roy Marzioli, Manager
Central Identification Services
Contra Costa County Sheriff's Office
Martinez, Ca.