CLPEX Chat Board

Dear mdavis, thank you especially for that last post. You said the things I was trying to say in a much more concise, articulate, and comprehensible manner. You are a good spokesperson for our science.

Dave Grieve and I were in a discussion similar to this one during the early impact of the Mitchell case. Dave commented to me that science in the laboratory is not necessarily the same as science in the courtroom. I think that is basically what you meant in your second paragraph in your last post. I think that is one of the primary points on which this debate thrives. Some believe that in order to be good science, a field must meet the same high standards as a theoretical science in a research laboratory. That misses the fact that applied sciences may never live up to the same standards used in theoretical or laboratory or exact or pure sciences.

Consider a spectrum from white to black. On the white end are the exact sciences. Arbitrarily, let's peg mathematics as pure white. On the black end of the spectrum are the non-sciences that claim to be science such as graphology, phrenology, astrology, etc. What I'm saying is that to define whether fingerprints fits into the definition of good science relies upon agreement as to where in that spectrum we need to draw the line. Exactly what shade of gray is still good science, and which slightly darker shade of gray is not?

The research scientists would probably set the limit up very near the white end of the spectrum. The lawyers would set the line somewhere else, depending on their needs in any particular trial I suspect. I believe Daubert allows the judge in any given case to take a number of factors into consideration in determining where he or she needs to set the line in the courtroom. This is what Dave Grieve meant when he said science in the laboratory is not necessarily the same as science in the courtroom. The research scientist in the laboratory will probably set the line much closer to the white end of the spectrum than the judge who is trying to reach a verdict "beyond a reasonable doubt" in the court room. So maybe the criteria should be "reasonably" scientific.

Some sciences lend themselves to the calculation of a precise error rate. Fingerprint identification does not. Erroneous identifications may go undetected or worse, covered up. Still, I believe anyone practicing in the field and most people outside the field accept that fingerprint identification is still highly reliable. It may take a precisely calculated error rate to rank all the way to the white end of the spectrum, but it should not take a precisely calculable error rate to still qualify as science with an acceptable level of accuracy. And fingerprint science does have that acceptable level of accuracy without being able to precisely calculate it.

Thanks again, mdavis. Great post.

While I was writing, Pat and mdavis got their licks in, but I'll go ahead.

The great thing about the stir induced by the Report to make one focus on what's really at issue. I think a good way to think about this is to imagine latent print evidence as having just entered the criminal justice arena. In this imagined introduction of a "new" forensic discipline, we can take as available the body of pure science research on friction ridge skin, the work focused on the phenomenon of apparent association between skin and its impressions, and the body of experience in practical efforts to associate potential sources to recovered impressions (viewing it as something done for some unspecified reason). What do we then begin with?

I don't think any rational person interested in the question would hesitate to conclude that comparison of latent impressions to controlled impressions can provide information of use to anyone charged with considering whether some individual made the impression and that consideration is valid and should be used in investigations and trials to determine criminal responsibility.

The issue then breaks down to two prongs:

One is that, since comparison depends on accurate interpretation of various features of an impression, some degree of certainty must obtain for the interpretation of each feature that is to be used in the interpretation. As a corollary, some degree of certainty must obtain with regard to areas of an impression that cannot be interpreted to determine if they contain features that eliminate potential sources.

Because one aspect of this collateral issue involves a determination that unusable areas are presumed to contain no contradictory features, it must then lead to the second prong.

That is the question of what can be concluded from various degrees of agreement. Were this a new forensic field, we would be asking:

• Can a source be absolutely identified?
  Can a numerical probability be assigned to the association of source and latent?
  Can a verbal expression of likelihood of identification be stated in a way that provides consistent value to a trier of fact?
  Can a demonstration of the analysis and association of features provide value to a trier of fact?
  Can any conclusion by one expert be stated and demonstrated in such a way that it can be subjected to review, critique, and possible contradiction by
  another expert?

We should always make an effort to view our own discipline with what a Zen practitioner might call "beginner mind." We cannot extend the utility of our discipline without also looking back and refining the whole. If we look critically at our foundations and find them solid and our assumptions and practices reliable, so be it. But if we find that something must be revisited, even if it rewrites the catechism, we must do that, too.

Throughout, we must remember that while generally accepted scientific knowledge is part of what is being applied, we do not operate and are not entirely tested in the environment of Science. The tests of admissibility of criminal evidence are not those of science. They are the tests of law and the conduct of human affairs. While the legal discussions often use the category of "scientific" or "technical" evidence, it is perfectly clear that opinions such as in Daubert could have adopted the criteria of the scientific community for acceptance but did not, and instead provided a general framework within which courts should position themselves to consider admissibility. This is not to say that a purely scientific view of fingerprint identification is not an entirely valid point of view for the scientist to use in critique of practices and use and in deciding what will constitute accepted scientific fact upon which to base further scientific inquiry. But there is absolutely no requirement that legal authority rely on the scientific community to pass on what is useful in deciding questions of fact in law.

A great deal of the criticism of latent print identification in application is the objection to any status conveyed on account of longevity. It is an obvious and valid objection for a scientist to make. Science is nothing, if it is not the rejection of long-held opinion as a measure of validity. (It wold be an interesting exercise for a science historian to try to determine just how much scientific effort results in extending accepted knowledge and how much results in significant revision of knowledge.) It is not such a thing to be avoided in law. Longevity of substantially useful practice is a perfectly appropriate consideration in law, a sort of "proof in the pudding" argument that often serves very well.

And, because we operate in the arena of adversarial law, we must be prepared to both answer challenges of both science and law. We are simply not going to satisfy scientists with arguments appropriate to law but not to science. Scientists cannot compel courts to accept only what meets the standards of science. Both arguments, legal and scientific, have weight in our forensic world. While the term "forensic" science is bandied about, it is really forensic (pertaining to, connected with, or used in courts of law or public discussion and debate) evidence that is at issue.

Hi MDavis,

mdavis wrote:The question is, are the deficiencies "correctable" or are we tilting at windmills when we try to push comparison psychology into pure science in order to verify what we've essentially "proven" over 100 years? Isn't this what got us into trouble in the first place -- pushing the envelope on our definition of science clear to the point of decision? There is little or no science in any courtroom unless we introduce what little we have. Still, it is seldom understood or appreciated.

I think you are very much off target. "Comparison Psychology" is what we all use in our everyday cognitive lives. It happens that in our profession and in many other sciences besides, that the task of making comparisons requires the honing of comparison skills to a higher level of sharpness based upon its application to a defined area of nature we study which is clearly beyond what average person would have looking at looking at the same phenomena. Don't you realize that this is something that goes on in every science?

In my opinion, "comparison psychology” as a synonym for subjectivity is a side issue of what the NAS report is addressing. However, it is my opinion that this is being over emphasized and marked as a stumbling block and justification for digging our heels in and saying that we can’t do what this report is recommending.

Clearly, subjectivity is not a basis for a scientific conclusion. Of course, in recent years, subjectivity has gained myth status in our field. This is a myth I would like to dispel since it is clearly being put forth as a stumbling block to accepting the recommendations of the NAS Report.

First of all, lets get clear on what we mean by subjective.

Subjective – taking place within the mind and modified by individual bias.

How about this one:

Subjective - of a mental act performed entirely within the mind

Our comparison is not performed entirely in the mind. It is in fact mediate by the phenomena which are being examined with an understanding of potential bias and accepted practices which minimize this. Now looking at the first definition, is this what you do when you make a comparison? I don’t think so. Don’t you look out into the real world at real world objects and draw from those real world objects information upon which you are going to make your conclusion. Notice how what we really do does line up with being objective:

Objective - involving or deriving from sense perception or experience with actual objects, conditions, or phenomena

This ought to be clear based upon what you do in comparing friction skin. Your opinion is not based upon subjectivity but what you take to be objective fact. What is the objective fact? You have two prints in front of you which are going to be the basis for some conclusion you are going to draw. You are going to look at each to compare them in order to ascertain whether there are any similarities or dissimilarities which would lead you form an opinion to individualize or to exclude. It isn't that you subjectively believe they are from the same donor, it is the fact that you have compared ridge flow and minutia in sequence, which is objective evidence upon which you base your conclusion also being guided by your training, experience.

Do you think this is different than any other sciences? Do you think that DNA analysts, toxicologists or firearms examiners are not doing something similar? Of course they are looking at a different set of natural data, but they are drawing scientific conclusions based upon the natural phenomena which are presented to them. There is a subjective element, but that does not make the conclusion subjective unless, like the definitions state above, you are performing a mental act entirely in your mind and modifying it with bias. This means that you are in no way taking into consideration objective facts which are being presented to you through your senses.

So if this is the case, why is our science labeled subjective? The NAS Report quotes Judge Souder states,

In October 2007, Baltimore County Circuit Judge Susan M. Souder refused to allow a fingerprint analyst to testify that a latent print was made by the defendant in a death penalty trial. In her ruling, Judge Souder found the traditional method of fingerprint analysis to be “a subjective, untested, unverifiable identification procedure that purports to be infallible.”

NAS Report 1-7

Notice that there is a difference in use of subjective in this sense than in what you associate with “comparison psychology.” We need to stop confusing the use in these two situations. If our methodology was truly subjective in the senses described above, we could not consistently make good identifications. It is because we are objective in the sense I describe above that we have and continually are accepted as “experts” in our subject matter. Of course Souder’s statement is about our procedure, but its not the performance of the procedure which is being criticized, it is the basis for assuming that the procedure can produce the results which are claimed for it. The question is, how can we know that it can do what is asserted that it can do? If we don’t have scientific studies based upon objective evidence to support this claim, then all we have is an assertion which in essence “[took] place within the mind and [was] modified by individual bias. Why? It is subjective. There are no scientific studies which support this. Our knowledge of natural phenomena and science in general should be sufficient to inform us that no amount of scientific research can ever support a conclusion that any scientific process is infallible. I think it is nonsensical to say that our method is infallible, but the examiners who use it are not. Really? That’s like saying a jelly donut has not calories until you eat it. This is absurd.

So how do we demonstrate reliability? Pat and I discussed above the problems that area associated with Latent prints based upon deposition and distortion. In science the first thing you do is examine phenomena under ideal or close to idea circumstances. Once you know that you can begin to consider the same phenomena in less than ideal circumstances.

This is where our reference collections come in to play. Given prints taken in an ideal or controlled environment, what can we expect with regard to the replication of minutia groupings? Above Pat mentioned the 50/50 study. Even though it was a flawed and was never written up in any of our journals, it does give us an idea of what we should be looking for. I have to know what I can expect and trust in an ideal setting before I can begin to apply it to something like a latent print. You mentioned the "ground truth" in your last posting. This is what it is. If I know the answer to this, even if it is a statistic, I have a basis to approach all friction skin comparison and I have a guide for making conclusions for individualizations.

In there report, those who represented us before the committee claimed we cannot do population studies because of the fact that in latent prints minutia cannot be anticipated from one latent deposition to another. The report states it like this:

By contrast, before examining two fingerprints, one cannot say a priori which features should be compared. Features are selected during the comparison phase of ACE-V, when a fingerprint examiner identifies which features are common to the two impressions and are clear enough to be evaluated. Because a feature that was helpful during a previous comparison might not exist on these prints or might not have been captured in the latent impression, the process does not allow one to stipulate specific measurements in advance, as is done for a DNA analysis. Moreover, a small stretching of distance between two fingerprint features, or a twisting of angles, can result from either a difference between the fingers that left the prints or from distortions from the impression process. For these reasons, population statistics for fingerprints have not been developed, and friction ridge analysis relies on subjective judgments by the examiner. Little research has been directed toward developing population statistics, although more would be feasible.

[/quote] NAS Report 5-10

The above was based upon a comparison between DNA and friction skin analysis. Obviously it is based upon what the committee was told by representatives from our field. Honestly, it is nothing more than smoke and mirrors. We can certainly do these kinds of studies using our reference databases. Of course latent prints do pose a different challenge, but what we discover in the reference sets will have tremendous implications for latent prints.

The reality of friction skin comparison is that uniqueness is actually determined using level two detail. Anyone who disagreeing with this, please, feel free to jump in and argue that. Yes we do consider the condition of the whole print, ridge flow, morphology etc., but the reality is, if you don’t find sufficient minutia in sequence, you don’t identify regardless of whether you have some number in mind or not. The above does note the feasibility of developing population statistics. I have already suggested how this would be done.

It just so happens that at this stage in the development of technology supporting our science that we have the means to carry this out. I am of course speaking of AFIS. Everyone marvels at the wonders of DNA databases, but few seem to think about the fact that the largest biometric databases in the world are populated, not with DNA, but with fingerprints and palms. What do they search? Go ahead say it… minutia! To say that we can’t do population studies now is to close our eyes, go into denial and claim that AFIS doesn’t exist. DNA based its model on far smaller population databases than we have currently available to our community. Not only are population statistics possible, these systems give us the ability to check any results of such a study in systems all over the world. My own suggestion is that we get busy.

Roy Marzioli, Manager
Central Identification Services
Contra Costa County Sheriff’s Office
Martinez, Ca.

Roy, I wonder if your idea regarding a population study of minutiae in relative position would really yield the basis for "measurement" that NAS is looking for, but I would love to see a study that proves you correct. Christope Champod presented a paper in 1995 at the Israel Fingerprint Symposium in which he looked at the locations of various level 2 minutiae in the overall pattern. I think what you are talking about would be an extension of that concept. But I wonder if distortion wouldn't vary the relative positions enough to make it questionable to apply such a "measurement" to any specific comparison to prove its reliability. I think what you are proposing would translate to an AFIS system in which a correct identification, if it were in the data base, would always show up in the #1 candidate position. Wouldn't that be the result of what you are talking about? I believe there are several studies being done now on probabalistic models and on error rate. Should we cease making identifications and testifying to conclusions until after those studies are completed, peer reviewed, and published? I would argue no. I am confident those studies will support and help us understand what we are presently doing. But I don't think we need to quit making examinations until we can support our work with published studies.

On the difference between objective and subjective, I agree with Asbaugh's description of the relationship between the two, as I recall it: The process is objective, the identification is subjective. From your comments above, I suspect that you have heard Dr. Thomas Busey's presentation with John Vanderkolk. Dr. Busey, a cognative psychologist, has proven that an expert's brain works differently than a layperson's brain when doing a comparison. That is the result of training and experience, and helps define an expert. I think most laypersons could follow an objective procedure to do a comparison, but the expert is more reliable in his/her subjective conclusion because of training and experience. If the whole thing were both purely objective, then the layperson's conclusion would be just as reliable as the expert's.

I hope you understand, Roy, that I support all of the research projects the NAS recommends. I also support the idea of a Department of Forensic Science Services removed and separate from the police. I have been saying for years that would be the best way to establish crime laboratories. The sad fact, however, is that most labs now are divisions of police or prosecution agencies, and they would be loathe to give up the power and control over the laboratory function.

What I disagree with is the idea that fingerprint identification and other fields of criminalistics are something less than science just because we don't fit the model of "Real Science" taught in Universities. We do need to continue to progress into better understanding and more scientific validation of our processes and procedures than we now have. If we find better ways of doing things, then that's the best result of all. But if our current process is the bathwater and identification is the baby, then let's don't throw the baby out with the bathwater.

My use of terms was not vetted through Funk & Wagnals, and my informal usage was intended to include common acceptance. We all contrast "objective" with "subjective" in our discussions, just as we all commonly accept "identification" for the newly fashionable "individualization."

...nothing more than smoke and mirrors. We can certainly do these kinds of studies using our reference databases.

"Reference databases" are small. Is it fair to make the leap of faith from manually contrived controls and "unknowns" to the totally unknown "ground truth" of the real world of case work? What is the statistical reliability of this transition when "ground truth" is never known in reality? A statistic is not an answer, it is a mathematical probability model with percentages of error expressed in the result.

The reality of friction skin comparison is that uniqueness is actually determined using level two detail.

Quite often this is used as a quick yardstick by examiners while performing the elimination process of comparison. But level 1 detail is usually also used before that for culling unlikely candidates. Most of us are aware of the case in (if memory serves me correctly) Illinios in which an ident was performed using only level 3 detail. Yet most every examiner who has seen that case will agree on the validity of the ident. AFIS would never consider it.

It just so happens that at this stage in the development of technology supporting our science that we have the means to carry this out. I am of course speaking of AFIS.

Our AFIS systems are "dumbed down" to work essentially with level 2 detail. Although the exact algorithms are "proprietary" and trade secrets, we know enough to recognize that they work primarily on "scatter plots" created from level 2 details, some even ignoring level 1 completely unless entered by the operator, and never considering level 3 at all in their infinite detail. I work with AFIS nearly every day. It is not uncommon to have a candidate turn out to be a "bad ident". Likewise, I have received a candidate search listing in which the actual identification was the 40th candidate on the list (a list of 40 by the way). I don't see how we can possibly call upon this technology in it infancy to generate meaningful statistical results that "second guess" experienced examiners when their own error rates are so out of control.

I am in no way advocating that what we do lacks credibility or even statistical significance. What I am saying is that we currently lack the technical ability to bridge the gap from the human conclusion to a statistical, AFIS (or other)-validated "peer review" of our discipline. We deal with far too many "borderline" impressions in the real world. If we were ever to develop such computer capability, there would no longer be a need for our profession.

I think we have strayed on two fronts: first, we have somewhere lost the internal peer pressure of the first hundred years that helped maintain our professional integrity. Second, we have been inundated by the sheer number of very close candidates presented by our AFIS databases that, by their absence, would never have tempted examiners to consider. In short, there are more of us who are inadequately trained and overstimulated with likely candidates for our conclusions. As I mentioned earlier, we must individually recognize the limitations of our craft as well as the possibilities, and stay away from the precipice where the bold proclaim their "superiority."

mdavis wrote:I think we have strayed on two fronts: first, we have somewhere lost the internal peer pressure of the first hundred years that helped maintain our professional integrity. Second, we have been inundated by the sheer number of very close candidates presented by our AFIS databases that, by their absence, would never have tempted examiners to consider. In short, there are more of us who are inadequately trained and overstimulated with likely candidates for our conclusions. As I mentioned earlier, we must individually recognize the limitations of our craft as well as the possibilities, and stay away from the precipice where the bold proclaim their "superiority."

Good points. It also seems apparent that some examiners have projected their imagined ability to reliably conclude from complex analysis beyond what can be justified. These seeming advances in ability to conclude from more and more complex impressions or lesser quality impressions are not based in new knowledge. They are not grounded in newly available information. They are supported purely by an accumulation of experience, and it's experience that can too easily be falsely bolstered by confirmation of actual source. The fact that your concluded source happens to turn out to be the source, does not validate the examination. And I agree that the candidate pool growing beyond the local known criminals increases the likelihood that a process that would once almost certainly identify the true source (no matter how flawed the process) can now lead to revealed error. I don't think it's mere coincidence that the current issues in latent print examination have become heated at this time. Certainly, a greater focus on forensic disciplines and their recognition that there's academic hay to be made are factors. But the revealed errors definitely encourage attention.

I say "revealed error," because we simply do not know how many poorly concluded examinations are not discovered to be flawed when they happen to hit the target. I believe that there are only two courses that can effectively provide a high degree of protection against erroneous conclusion. One is to retreat to a lower tolerance in quality and quantity. The other is for the justice system to routinely provide for adversarial review. And the latter tends to influence the former, and while it will more often contest the conclusions of the more complex examinations, that is no bad thing and may make examiners more critical of their own work. I would rather forgo some arguable conclusions than insist that all conclusions are equally certain to be correct.

(Okay. I'm feeling prickly tonight.)

Hi MDavis,

There may be some misunderstanding between our positions, but I believe there is more agreement than disagreement when we clear up the misunderstandings.

mdavis wrote:"Reference databases" are small. Is it fair to make the leap of faith from manually contrived controls and "unknowns" to the totally unknown "ground truth" of the real world of case work? What is the statistical reliability of this transition when "ground truth" is never known in reality? A statistic is not an answer, it is a mathematical probability model with percentages of error expressed in the result.

Forgive me for being sloppy. I am actually referring to Individual Characteristic Databases, which are our reference material in friction skin comparison. In my oWn agency we have two AFIS databases one with 800,000 subjects, the other with about 350,000. Both have been designated as reference material by us for ASCLD. As for real world, if you search latent prints in AFIS, you can't get any more real than that. I think can see that you do need to know if the particular minutia grouping you are weighing your conclusion upon is likely to be reproduced in the population with you are dealing and the likelihood that the hit you are considering is merely a replication of the subject you are actually hoping to hit. I don’t know about you but I have seen minutia groups replicated in other prints. What I don’t know is how often. In the cases I have observed, fortunately there have been sufficient clarity to exclude on other grounds. But what is of concern to me with regard to latent prints is a lack of clarity and information and the effect that distortion can have on a section of friction skin so as to make a latent print appear to look very much like a section of friction skin in our AFIS. I have to believe that you would want to know the answer to this question yourself.

mdavis wrote:Quite often this is used as a quick yardstick by examiners while performing the elimination process of comparison. But level 1 detail is usually also used before that for culling unlikely candidates. Most of us are aware of the case in (if memory serves me correctly) Illinios in which an ident was performed using only level 3 detail. Yet most every examiner who has seen that case will agree on the validity of the ident. AFIS would never consider it.

I take it that you only use latents with sufficient level one detail for searching or comparing. My shop is not so lucky. Quite often we are faced with scraps which have sufficient detail to individualize, but no level one detail. As for level 3, good luck with that! It is my experience that level 3 detail is notoriously undependable. If you have had occasion to compare a number of rolled impressions for the same subject, you will find that with regard to edges, ridge morphology and pore arrangements, that they are different in every roll. This is with rolled impressions. It only gets worse with latent prints. Hopefully in the Illinois case to which you refer, there was other incriminating evidence than Level 3 detail in order to make that case. If that was all they had, I feel very sorry for the suspect. By the way, I would very much like to see both the latent print and the exemplar/reference print used in that case.

Gerald Clough wrote:Our AFIS systems are "dumbed down" to workessentially with level 2 detail. Although the exact algorithms are "proprietary" and trade secrets, we know enough to recognize that they work primarily on "scatter plots" created from level 2 details, some even ignoring level 1 completely unless entered by the operator, and never considering level 3 at all in their infinite detail. I work with AFIS nearly every day. It is not uncommon to have a candidate turn out to be a "bad ident". Likewise, I have received a candidate search listing in which the actual identification was the 40th candidate on the list (a list of 40 by the way). I don't see how we can possibly call upon this technology in it infancy to generate meaningful statistical results that "second guess" experienced examiners when their own error rates are so out of control.

Isn’t that more of a personal opinion? I think you are going to find that in AFIS systems are not dumbed down, they
are using the only reliable source of information which can consistently be used for individualization. I know in ridgeology, the holy grail of individualization is 3rd level detail. For ever becoming the sole means of individualization, I would suggest that it is every bit as real as the holy grail. As mentioned above it is unrealiable. At a Cogent users group
meeting about three years ago, they announced that they were going to develop algorithms for searching 3rd level detail. I suggested that they should treat 3rd level detail the way we now treat incipient ridges. If it is there use it, but don't depend on it alone. It should be obvious why. It has proven to be inconsistent in rolled impressions it has even less consistency in latents. Seriously, I really do want to see the 3rd level detail on its own which led to a criminal conviction.

At this point, I am not interested in where minutia groupings place on a candidate list. I am interested in their occurrence and the potential risk they pose to my examiners. Basically, I am interested in finding them. I don't care if the come out as the number one candidate or are found in candidate 250. If they are there, perhaps they come out in the 40th position this time. The next time they are in your #1 candidate. I am fairly sure you would want to know that and you would want to know how often such a phenomenon occurs in a given population. Hence the need for studies!

mdavis wrote:I am in no way advocating that what we do lacks credibility or even statistical significance. What I am saying is that we currently lack the technical ability to bridge the gap from the human conclusion to a statistical, AFIS (or other)-validated "peer review" of our discipline. We deal with far too many "borderline" impressions in the real world. If we were ever to develop such computer capability, there would no longer be a need for our profession.

I am not sure I understand what you mean by a "AFIS (or other)-validated "peer review". I am more concerned with the reliability of the individual characteristic databases (aka. reference prints) we use for latent comparisons. Everything we do with latents depends on the reliability of these prints. It is for this reason we need to know how often minutia grouping replication can occur and at what level. You note the borderline impressions with which we have to work. We have our own share of borderline prints as well. I can't imagine you would say this has no bearing on borderline impressions in the real world. I do believe that AFIS systems can be used to find the probability in minutia replication which would form a basis for establishing the lower limits of reliability in friction skin. The math has already been worked out by Christophe Champod. We need to test it in the real world.

Roy Marzioli, Manager
Central Identification Services
Contra Costa County Sheriff's Office
Martinez, Ca.

Hi Pat,

Pat A. Wertheim wrote:Roy, I wonder if your idea regarding a population study of minutiae in relative position would really yield the basis for "measurement" that NAS is looking for, but I would love to see a study that proves you correct. Christope Champod presented a paper in 1995 at the Israel Fingerprint Symposium in which he looked at the locations of various level 2 minutiae in the overall pattern. I think what you are talking about would be an extension of that concept.

I am not thinking of that article, but I am thinking of two other articles written by Champod, "Computation of Likelihood Ratios in Fingerprint Identification for Configurations of Three Minutiæ" and "Computation of Likelihood Ratios in Fingerprint Identification for Configurations of Any Number of Minutiæ" printed in the JFS in 2006 and 2007 respectively. Obviously in any test, distortion is a factr, but I think it is less of a factor in our reference collections than in latent prints. I am not concerned at this point in such configurations showing up as AFIS hits. I am more concerned with finding them. Of importance to us and to our field is that such configurations exist at all. If they exists, and depending on how many can be found in any particular grouping at a particular statisical rate in a given population, this would go a long way in addressing our need for standards. To quote Champod's 2006 article,

Recent challenges of fingerprint evidence combined with recent cases of false identification have strengthened the need for statistical research to underpin the fingerprint identification process.

J Forensic Sci, November 2006, Vol. 51, No. 6. Christophe Champod, "Computation of Likelihood Ratios in Fingerprint Identification for Configurations of Three Minutiæ"

This is exactly what the NAS Report is recommending. As for the stoppage of comparisons, I don't see anything in the NAS Report recommending this and I certainly don't advocate it nor do I think it necessary. Hopefully we have the objective/subjective issue worked out. No analyst or scientist for that matter can be totally objective, but by the same token, “relative” objectivity can be achieved only by incorporating the data received from natural phenomena in your conclusion. If there is no observation of the real world phenomena and no inductions drawn from that data, then conclusions would be subjective.

Pat A. Wertheim wrote:I hope you understand, Roy, that I support all of the research projects the NAS recommends. I also support the idea of a Department of Forensic Science Services removed and separate from the police. I have been saying for years that would be the best way to establish crime laboratories. The sad fact, however, is that most labs now are divisions of police or prosecution agencies, and they would be loathe to give up the power and control over the laboratory function.

I agree. I think we are in more agreement than it may have first appeared. I am fairly certain that we both believe that what we do is "science." Strictly speaking, I don’t believe that we aren’t a science; I believe that we are being challenged by this report because in ways, as described in this report, we are not scientific. Think about this, if we weren’t a science, I believe we would have been put out of business long before this report came out. I believe this is affirmed by the report. As the report states,

Some scientific evidence supports the presumption that friction ridge patterns are unique to each person and persist unchanged throughout a lifetime.

But the report is also saying, now it is time to “prove” it. Basically, the report is saying we have to do the research that supports our science. By doing the research and implementing the other recommendations the NAS report, our science becomes scientific.

Roy Marzioli, Manager
Central Identification Services
Contra Costa County Sheriff's Office
Martinez, Ca.

Roy's view, if I understand it, is that we should only rely on level two detail, and we create or use existing algorithms to build databases based on frequencies of level two groupings for the purposes of creating statistical reliability in latent print comparison accuracy. I feel that this is gross oversimplification. Level two "groupings" can vary tremendously between impression capture of the same source. A tapered ridge end can appear to end several millimeters long or short of another impression depending on the pressure, time interval, substrate, matrix and other factors. Ridge spacing can vary by a factor of two or more depending upon lateral pressure. Rotational distortion can be worse. Just as level three detail is enormously variable, so is the attempt to categorize groupings of level two details.

Another question would be how many level two details are in a grouping? This smacks of a backslide to the old ridge counting thresholds which we (the IAI, SWGFAST, etc.) have so desperately tried to abandon.

I am not opposed to creating vast statistical studies of fingerprint data. Nor am I suggesting that what we do is not scientific up to the point of making a stated conclusion. My skepticism is that we do not have the necessary human manpower (excuse me, person power), time and money, nor do we have the computer algorithm sophistication to be able to deal with the inevitable variables that flexible friction ridge skin gives us.

Level one is often not available, as we all know, or may look basically the same (finger tips), and level three is too variable to be used (Roy's admission). What's left is level two. Dealing with the inherent flexibility of friction ridge skin and transfer of three dimensional impressions to two or three dimensional surfaces, forces the computer programmer to build in a "slop factor" or variable range within which a given "point" or level two detail is allowed to exist to be counted as within a group. Without this "slop factor", AFIS would miss a much higher percentage of "hits". But adding this "slop factor" to accommodate inevitable variations of multiple impressions from the same source greatly reduces the validity of any such computer scoring. Damned if you do and damned if you don't. Limiting a computer to level two detail for the purpose of creating statistics in latent print comparison, which is what virtually all modern AFIS systems do, is little more than ridge counting. As a scientist, I don't think it can be statistically accurate enough to be of much value. But then we do a lot to be politically correct and satisfy the uninitiated. If that satisfies the bean counters, I say go for it.

Mr. Wertheim,

Any journal validation articles for me to review in support of your position that ACE-V has been validated? You indicated that you enjoyed the back and forth but we cannot continue to do so until you post the journal articles. If you have none, then simply say so.


LPE never make mistakes wrote:Perhaps Mr. Wertheim can point out the reference articles validating ace-v involving latents from easy to hard latents, new to experienced examiners working on those latents? How do you know it is good science when there is not one true journal article as to validation of ACE-V? Science becomes science why one applies the dictates of science. Teh party line is clear, we are already doing the research requested (where is it), report is not critical (it is scathing), lip service to validation, documentation, reform etc.


Sorry, "LPE never make mistakes," but aren’t you lashing out a little bit here? Let me try to get away with just addressing one of the charges you make – “not one true journal article” – Is a “true” article one published in a peer reviewed article? Or do you mean the journal of, perhaps, IAFS but not that of IAI?

LPE,

Roy Huber first articulated ACE-V. He explained that he was simply using the Acronym as a different way of explaining scientific methodology, he was describing hypothesis testing. How do I know this?? 1st, because his explanations are the same as hypothesis testing, and 2nd, because of email conversations I had with him prior to his death. Hypothesis Testing was formulated thousands of years ago by people like Aristotle and Thales, and has been improved upon throughout the centuries up to Popper. It doesn't claim to perfect but it is accepted as a valid scientific method.

This isn't to say that ACE-V is a valid scientific method... it is only a valid scientific method if it's used in accordance to accepted scientific ideas. I'm not going to describe every mis-use here but one example could be that if people are using V as a confirmation process then it wouldn't be considered as scientific as if they used V as the untimate scrutiny that Huber defined it as.

If ACE-V is used as hypothesis testing (as initially described) then wouldn't any writings on the validity of hypothesis testing apply? Since that list would literally be thousands of books and articles, I won't attempt to try to list some here.

Michele wrote:This isn't to say that ACE-V is a valid scientific method... it is only a valid scientific method if it's used in accordance to accepted scientific ideas.

I quite escapes me why we or anyone else worries about the "validity" of ACE-V. ACE-V was an insightful way to express what should be happening. Its value is in the clear articulation of sequence of data-collecting operations and the association with hypothesis testing to make the point that an examiner must share the mindset of the honest scientist, the mindset that is always ready to accept the evidence, even if it contradicts a cherished idea, and the acceptance that a result is not going to be accepted if it cannot be independently reproduced. It simply reflects the proper discipline and control of what is declared to be reliable knowledge. It is an expression of how we embed this simple principle in our own discipline. It also serves as a framework to more clearly discuss issues of retroactive contamination of one phase of examination by another. As an expression of observational collection and analysis of data and general hypothesis testing, it is always valid by out conventions of scientific inquiry and needs no proof specific to the process labeled "ACE-V." More importantly, it does not, in itself, invalidate any other sequence. It merely identifies one sequence that militates against error.

We have, I believe, fallen into an error of endowing ACE-V with far too much weight on the side of validating the discipline. As an expression of hypothesis testing, it is fundamental, but it is the least of issues in arguing for the value of the conclusion of latent print examination. We are, as a discipline, going to have to be more grown up than that. It is also a mistake to equate the status of fingerprint identification research with the legal admissibility status. Daubert does not demand that individual conclusions be universally or generally accepted. Nor does it demand that a technique have a specified error rate, and it does not demand that the argued error rate be generally accepted. It merely demands that the technique the sort that can be constructed in such a way that it is amenable to rational and articulable review and study, that it be of the sort that is subject to meaningful testing, and that it be practiced to a known standard. The technique must be deemed by the court to satisfy those concerns to some reasonable degree. It does not declare that only correct conclusions may be admitted, nor does it demand that all analysts conclude similarly. The correctness of any conclusion is outside the admissibility argument and is a proper matter for the trier of fact. ACE-V speaks only to a portion of the standards of practice test. It is in no way related to error rate (except by implication if it is violated) or the review of assumptions applied during Analysis and Evaluation. Those are the "working" areas in arguing the value of fingerprint conclusions. Those are the both the critical operations subject to error and the operations that, when properly applied, reveal apparent violation of the order of operations labeled ACE-V.

It is further a mistake to fail to make a clear distinction between admissibility and credibility. Admissibility does not imply credibility. One is whether it will be heard. The other is whether it will be admitted to the trier of fact's accumulation of proof or not, according to their acceptance. We have become heavily embroiled in admissibility on account of a lack of challenge to credibility. A legal anomaly has been allowed to exist, a class of evidence for which credibility has been de facto presumed. If such a class of evidence were to exist, it's admissibility standard would have to be so impossibly high as to trespass on the fact-trier's territory. We are in the unfortunate position of facing hard examinations of admissibility by the same system that uncritically accepted our claims of inherent credibility. We had better start taking a hard and serious look at how we represent our evidentiary value. If we don't, we can expect it to be attempted by those working from an inappropriate point of view or from limited knowledge and experience in our field. As we tell our kids: Grow up. Get real.

Really what can I add to this email that so clearly shows an absolute lack of any understanding of validation.

I quite escapes me why we or anyone else worries about the "validity" of ACE-V. ACE-V was an insightful way to express what should be happening.

Sad but predictable that the NAS report which just happens to disagree with everything in this post is so clearly falling on deaf ears.

Here is the classic head in the sand perspective. Only fingerprint examiners know what they do and they alone even with all evidence to the contrary know that all is just dandy. Validation is universal regardless of the science that is being validated. NAS is neither limited as to knowledge or experience. To suggest that anyone on this listerve is in a better position to assess the state of fingerprints than NAS is just absurd. NAS listened and rejected your tendered experience and knowledge. It is sad that there appear to be only two posters with an interest in moving forward but they are swimming against an insurmountable tide.

If we don't, we can expect it to be attempted by those working from an inappropriate point of view or from limited knowledge and experience in our field. As we tell our kids: Grow up. Get real.

Finally, not one journal article has been tossed into the ring by Mr. Wertheim or anyone for that matter as to the validation as to fingerprints. So yes it is time to grow up and get real isn't it?

LPE,

I could give the same complaint to you as you've given to others about not answering questions. I asked you if ACE-V is used as hypothesis testing (as initially described) then wouldn't any writings on the validity of hypothesis testing apply? Instead of answering the question you just changed your question from looking for articles on the validity of ACE-V to be looking for validation on fingerprints. What specifically are you wanting? It's hard to supply what someone wants when they're changing the request in midstream.

LPE,

I get the impression that you expect the entire fingerprint community to stand up and either applaud the NAS and adopt their recommendations immediately and wholeheartedly, or you expect us to scream and shout that the report is completely wrong. Instead there has been a measured response and thoughtful discussion of what the report means for the science and its Practioners. I think the majority of examiners are reviewing the report and trying to decide how to use the information in a positive and practical manner. I assure you that none of us have ‘a bunker down mentality’ or ‘our head in the sand’. I am not sure why your postings are so hostile, but I assure you we are listening to what has been said. I suggest that it is you that needs to ‘grow up’ and ‘get real’.