NAS Report & ACE-V

[izatso]

I am sure that many of you have read the NAS report "Strengthening Forensic Science in the United States: A Path Forward" (NAS Report) and are aware of following statements made in the report with regard to ACE-V:

Unreliability:

In Maryland v. Rose, a Maryland State trial court judge found that the Analysis, Comparison, Evaluation, and Verification (ACE-V) process (see Chapter 5) of latent print identification does not rest on a reliable factual foundation

NAS Report 3-16

Does not specify measurements or a standard test protocol:

Note that the ACE-V method does not specify particular measurements or a standard test protocol, and examiners must make subjective assessments throughout. In the United States, the threshold for making a source identification is deliberately kept subjective, so that the examiner can take into account both the quantity and quality of comparable details. As a result, the outcome of a friction ridge analysis is not necessarily repeatable from examiner to examiner.

NAS Report 5-9

Can be improved by the introduction of measurement criteria:

Clearly, the reliability of the ACE-V process could beimproved if specific measurement criteria were defined.

NAS Report 5-10

The conclusion:

ACE-V provides a broadly stated framework for conducting friction ridge analyses.
However, this framework is not specific enough to qualify as a validated method for this type of
analysis. ACE-V does not guard against bias; is too broad to ensure repeatability and
transparency; and does not guarantee that two analysts following it will obtain the same results.
For these reasons, merely following the steps of ACE-V does not imply that one is proceeding in
a scientific manner or producing reliable results. A recent paper by Haber and Haber
Summary Assessment
Historically, friction ridge analysis has served as a valuable tool, both to identify the
guilty and to exclude the innocent. Because of the amount of detail available in friction ridges, it
seems plausible that a careful comparison of two impressions can accurately discern whether or
not they had a common source. Although there is limited information about the accuracy and
reliability of friction ridge analyses, claims that these analyses have zero error rates are not
scientifically plausible.
A new paper by Haber & Haber presents a thorough analysis of the ACE-V method and its scientific validity. Their conclusion is
unambiguous: “We have reviewed available scientific evidence of the validity of the ACE-V method and found none.”30

[W]e report a range of existing evidence that suggests that examiners differ at
each stage of the method in the conclusions they reach. To the extent that they
differ, some conclusions are invalid. We have analysed the ACE-V method itself,
as it is described in the literature. We found that these descriptions differ, no
single protocol has been officially accepted by the profession and the standards
upon which the method’s conclusions rest have not been specified quantitatively.
As a consequence, at this time the validity of the ACE-V method cannot be tested.

NAS Report 5-12

Better documentation of the process needed:

Recent legal challenges, New Hampshire vs. Richard Langill32 and Maryland vs. Bryan
Rose,33
Some scientific evidence supports the presumption that friction ridge patterns are unique
to each person and persist unchanged throughout a lifetime.
have also highlighted two important issues for the latent print community:
documentation and error rate. Better documentation is needed of each step in the ACE-V process
or its equivalent.

NAS Report 5-13

Regardless of what we might think of the sources, I think this is a fairly good assessment with where we stand with the method and what we need to do in order to improve it and make it scientific. What do you think?

[Pat A. Wertheim]

Regardless of what we might think of the sources, I think this is a fairly good assessment with where we stand with the method and what we need to do in order to improve it and make it scientific. What do you think?

My biggest disagreement is with the underlying presumption that there is one and only one definition of "science" and "scientific." Is what we do in fingerprints pure science? Exact science? Hardly! Does that mean a thing is not science? Hardly! In the overall universe of "science," there is a subset of "applied science." In that subset, there is a smaller subset of "forensic science." In that smaller subset are still smaller subsets, including drug analysis, DNA, firearms, document examination, fingerprint comparison, blood alcohol, etc. But even among those small subsets of drugs, DNA, fingerprints, etc., there is no one-size-fits-all definition of science.

I think the claim a decade ago of zero error rate was ill-advised, misguided, and basically wrong. But I don't think the fact that some in our field claimed that for a while removes us from the subsets in "forensic science," nor is forensic science removed from applied science, nor is applied science removed from "Science." What we do in fingerprint comparison/identification is science. It is reliable science and it is good science. Our critics are, for the most part, our adversaries in court. Talk about bias!!! National Public Radio did a spot in their evening news yesterday and as an interview resource, they allowed Barry Scheck to hold forth on why forensic science is bad science. Now there is a reliable, authoritative source! NOT!!!

That does not mean I automatically disagree with our critics' conclusions. Dr. Simon Cole, for example, makes some very valid claims about what we should be doing to improve our science. I may disagree with some of the things he says, but not with everything he proposes by any means. That aside, I have not read and digested the entire NAS report. Having seen summaries, the ASCLD response, and the IAI response, I have to agree with you for the most part, Mr. Izatso. We need to constantly continue to strive and continue to improve what we do. What field of science can say otherwise? What we are doing is not wrong and it is not bad science, but many of us in fingerprints are plagued with the inability to adequately articulate what we do and why it is scientific. No science can be said to be 100% accurate and reliable. The fact that there is a Brandon Mayfield Case does not automatically invalidate millions of correct cases over a century of performance. The fact that one judge found the Bryan Rose case did not meet the test of science does not make that ruling right in other cases in other courts.

Given time to digest the lengthy NAS report, I'm sure some of the recommendations will be adopted. I suspect some will fall by the wayside. But we in the science need time to read and study the report fully. Organizations such as ASCLD and IAI, while having issued position papers, have had no time to respond in detail nor to question some of the conclusions, much less to act on them. We'll get there.

[izatso]

Hi Pat,

It is my hope the as a community we will embrace the findings of this report and move to put our science on a solid foundation. This would include doing the research which we have known we needed to do, if not before Byron Mitchell certainly afterward.

You state,

My biggest disagreement is with the underlying presumption that there is one and only one definition of "science" and "scientific." Is what we do in fingerprints pure science? Exact science? Hardly! Does that mean a thing is not science? Hardly! In the overall universe of "science," there is a subset of "applied science." In that subset, there is a smaller subset of "forensic science." In that smaller subset are still smaller subsets, including drug analysis, DNA, firearms, document examination, fingerprint comparison, blood alcohol, etc. But even among those small subsets of drugs, DNA, fingerprints, etc., there is no one-size-fits-all definition of science.

I think with regard to science, at least as it is practiced and applied broadly to all investigations of natural phenomena the following NAS description is rather appropos:

The scientific method presumes that events occur in consistent patterns that can be understood through careful comparison and systematic study. Knowledge is produced through a series of steps during which data are accumulated methodically, strengths and weaknesses of information are assessed, and knowledge about causal relationships is inferred. In the process, scientists also develop an understanding of the limits of that knowledge (such as the precision of the observations), the inferred nature of relationships, and key assumptions behind the inferences. Hypotheses are developed, are measured against the data, and are either supported or refuted.

NAS Report 4-1

One of the things that is sadly missing with the current application of ACE-V is the lack of what we could call "measure" and in the method itself, a lack of true verification.

Scientists continually observe, test, and modify the body of knowledge. Rather than claiming absolute truth, science approaches truth either through breakthrough discoveries or incrementally, by testing theories repeatedly. Evidence is obtained through observations and measurements conducted in the natural setting or in the laboratory.

NAS Report 4-1

If you disagree with the above, I can point you to a number of works which are from different sources which say pretty much the same thing.
In contrast,

Note that the ACE-V method does not specify particular measurements or a standard test protocol, and examiners must make subjective assessments throughout.

NAS Report 5-9

Based upon the kind of conclusions we have been taught to make with the use of ACE-V there is no objective measure by which our results can be appraised. Our verification is not really verifying what one examiner has done, rather, it is as we well know a re-identification. In scientific testing, one would do an experiment and another would follow the same steps using the same data to see if they could obtain the same result. In essence you agree that the data used to form the conclusion does in fact support that conclusion. That someone gets their own identification says nothing about whether or not the method as practiced by either examiner was appropriate.

You state,

What we do in fingerprint comparison/identification is science. It is reliable science and it is good science. Our critics are, for the most part, our adversaries in court.

I strongly disagree with this statement. We may have adversaries in court, but I am not so sure that it isn't in some measure adversarial relationships of our own making. I personally do not think this is an adversarial report and we would do it a tremendous disservice to call it that. There were many members from the scientific community on the NAS committee who I believe are more interested in practicing good science than setting up adversarial relationships.

I think that this this is the crux of the disagreement between our official stance and this report. What the report is saying is that there are practices currently being done in our science which can and do promote unreliability, and that these practices are out of sync with what could be called good science. The report is not saying we cannot be a "good" forensic science but it is saying that as we are practicing it today, it is not good science. As you read through the report pay particular attention to Chaptor 4 and Chaptor 5. Chapter 4 describes the scientific method. Chapter 5 describes what we are currently doing and in contrast shows how out of sync we are with standard scientific methodology. Keep in mind that what the report is recommending is well within our collective reach to achieve.

Roy Marzioli, Manager
Central Identification Services
Contra Costa County Sheriff's Office

[Danny Lamont]

Good evening Pat,

As you know, I do not necessarily expound on a subject, unless, I have read something that irks me; then I will speak up using my knowledge, expertise, training, and experiences. I concur with much of what you have said on this subject. Most especially, I agree with the fact that fingerprints is a science and, as such, it is a good science. However, as we both know, it is not a perfect science. You have also mentioned this, not only in the past in your classes, but, just about any time I have heard you speak about the science. It is no more perfect than our form of democracy is perfect. Yes, it had a good start, based on what was going on in history at that time; however, as things have changed in this country over the past two hundred plus years, it too, has changed. Sometimes for the bad such as brought about by laws that allowed segregation; that allowed citizens of this country to be hanged unjustly. But eventually, cooler, and much wiser heads, determined that this was not to be. That by all of our beliefs, these things were wrong; they did not abide by the Constitution of the United States. Finally, to make a very long story short, the Civil Rights Act was passed, and slowly, things have been for the better. Our country is still trying to find its way, and it is continually changing. We just hope that people learn from the past and make the U.S.'s future bright and right.

No, I do not digress; by the same point the forensic Science of Fingerprints is the same. It started out as a great idea, and by the way it started out was according to what was going on in history in its infancy. Back then, something other than branding, tattooing, or measuring parts of the anatomy must be found to actually tell one person from another; one innocent person from a criminal person. Over the past one hundred years, it too, has changed. Again, many of the changes have come about by laws that were passed. Again, just like democracy in this country, it is changing for the better. Perhaps the changes are too slow for some, but when dealing with peoples lives as this science does, slow is a good thing. By rushing our thoughts or studies on this matter, those same peoples' lives may become at stake. This would truly be bad for fingerprins. Unlike Mr. Marzoli, I have not read this report completely, but as you pointed out, Pat, if cooler and wiser heads pay attention to it, just as they paid attention to friction ridge analysis, rather than points; that just as they paid attention to being able to explain what and how we do our work in a court of law rather than just falling back on statements from others, then this science will advance even further; it will become even better. Now, I do disagree with Mr. Marzoli on one point; we (I am referring to those of us who practice the science and testify to our findings in court) we are not in some measure responsible for the adversarial relationships. ( I am not making any assumptions on who the we are). Our form of law in court is "Adversarial" just as in the Central Criminal Court of Iraq, the system is, instead, Inquisitorial. I happen to be trained for testimony in both systems. We, those of us testifying to our findings, did not add to nor subtract from that system. As someone who has testified in federal, state, and local court in my state, I do not become an adversary of lawyers. I testify to my findings; how I came to those findings; what system I used to come to those findings. And it does not matter if I am called on behalf of the state or on behalf of the defense, I answer all questions fully and honestly. I do not allow either side to put words into my mouth. I am, just as my science dictates, neutral. One other thing, Mr. Marzoli, Pat pointed out that when an organization wishes to know about the Science of Fingerprints or any other type of forensic science, just as the NPR did, they will call in someone with no knowledge on how these sciences truly work, just as Mr. Scheck. I had the same thing happen in court many years ago in an auto theft case. The judge, who as an ADA worked in family court and as a Circuit Court Judge, she was a family court judge with very little understanding on how a criminal court worked. When wanting to understand how fingerprint comparisons were effected, and to explain it to the jury, she called on the defense attorney to do so because he was formerly an ADA who worked fingerprint cases. He did so, but up to a point. In other words, this is not unheard of in my case. Finally, to all, let us see if those who should be paying attention to this and other pertinent studies, do so, and if we are lucky, they will make decisions that will carry our science further into the 21st century, so that we may be able to do our jobs properly using proper methods. I do believe strongly though, that this subject not be limited to this chat board. It is important enough that it should be put out where more of the practioners in fingerprints can know what is going on. Let's bring all of this out to the Journal of Forensic Science.

[izatso]

Hi Danny,

Finally, to all, let us see if those who should be paying attention to this and other pertinent studies, do so, and if we are lucky, they will make decisions that will carry our science further into the 21st century, so that we may be able to do our jobs properly using proper methods. I do believe strongly though, that this subject not be limited to this chat board. It is important enough that it should be put out where more of the practioners in fingerprints can know what is going on. Let's bring all of this out to the Journal of Forensic Science.

Excellent point! I hope that in future months that we well see the recommendations carried out and articles detailing what is being done published in the JFI as well as other scientifc journals.

Roy Marzioli, Manager
Central Identification Services
Contra Costa County Sheriff's Office

[Pat A. Wertheim]

Hi Danny. Thanks for the historical perspective you instilled here. You are absolutely correct. Fingerprint science started in a form much more primitive than that we practice today. I would like to temper that, though, by stating that the minds and thought processes of our fingerprint pioneers were no different than ours. In other words, when they did comparisons and identifications in the US a century ago, they were doing exactly the same thing we do at our workbenches today. Two things have evolved more than anything else: 1) Our understanding of what we do, and 2) Our ability to put it into words so others can understand. As I have said in other threads (and Daktari loves to misquote me on this) the state of fingerprint training when I got into the business was abysmal. To be sure, there was excellent training available. But there was no mandate to train new fingerprint examiners correctly. Even today, in the US we lack any such mandate at the Federal level. Some states have mandates, some local agencies have mandates, but there is still nothing to prevent a sheriff or police chief from taking a substandard street officer or an underqualified civilian and proclaiming him or her the department's fingerprint expert. Nothing except our adversarial system of justice, in which the defense attorney can challenge the credentials of the unqualified person. If some of the standards, training requirements, and funding proposed in the NAS summaries I have seen come to pass, it will be a major leap forward.

Roy, one of the "points" (please excuse the pun) of the NAS report with which I have trouble is that of "measurement." DNA is the forensic science perhaps best suited to statistical analysis because of the nature of a DNA molecule and the ease of doing population studies on DNA. While we in fingerprints can do population studies and compile statistics on pattern types, that helps us little with a specific identification. The problem with "measurement" in fingerprints is that there is an almost infinite number of variables in the appearance of features we use to identify a fingerprint. We know that "counting points" is not a proper method because it fails to take into consideration a vast amount of information. If we could follow Galton's model, or Locard's, or any of the other statistical models, then point counting would yield the measured results our critics want to see. But to follow those models would itself be unscientific in a sense because it would require us to ignore so much other valuable information. And there is no guarantee it would yield more reliable conclusions.

I have had conversations with those in our field who are trying to design statistical probability formulas for fingerprint identification. I strongly encourage their efforts, but I sincerely doubt they will come up with such a formula or model. I hope I am wrong! What a relief it would be to go into court and be able to present a formula, plug in the numbers, and satisfy the critics who deride us for not having "measurements." But you know as well as I that if we suddenly had that magic formula, the Barry Schecks of this world would find something else to attack us on and would still be every bit as critical and outspoken as they are today. Perhaps the other scientists whose fields use measurements would be willing to more readily accept fingerprints as science, but the adversaries in court would not.

The other issue I take is with the word "subjective." But maybe that is not a separate issue at all. Maybe it is related because if we had measurements, we could dispense with the term "subjective." Our critics would have the world believe that "subjective" translates to "unreliable." Nothing could be further from the truth. "Subjective" is an adjective that means, basically, a decision made in the mind of the practitioner. Yes our conclusions are subjective! There is nothing wrong with that. And subjectivity alone does not make a fingerprint examiner's conclusion unrealiable. The key there is training. A trained, experienced, proficiency tested examiner with a track record of reliability is perfectly able to reach correct conclusions, regardless of how subjective the process of making an identification is.

That is my complaint about the "one size fits all" definition of science. Other disciplines may use measurements more than we do, and they may be more objective than we are. But even in those other disciplines, there is a degree of subjectivity. In a recent drug proficiency, one of our drug analysts identified a small spike on graph as meth (if I recall correctly) when the anticipated drug in the proficiency included no meth. The peer group met and decided that there must have been some small degree of contamination of the sample, so this analyst was not "wrong." Meanwhile, the other analysts who had taken the proficiency had failed to identify that small spike. Were they wrong? The peer group, as I understand it, accepted that both were acceptable answers. Without using the word "subjectivity," they said in essence that it was within the analyst's perogatives to identify the meth spike, even though it was very small in comparison to the main drug.

My "point" is that we as fingerprint examiners may rely more on subjectivity than criminalists in other disciplines, but that does not mean we are less scientific. It means that the "one size fits all" definition of "science," as it relates to forensic science, needs to be more flexible.

I agree with you, Roy, that we need to continue moving forward. We need to listen to our critics, but not automatically conform to everything they demand. We need most of all to police ourselves and assure that what we do is reliable and that we can explain it well to others, whether in court or in front of a committee of other scientists.

[Michele]

Pat,

It’s easy to say what we do is good science but that’s not good enough. People need to understand what good science is before they can determine if their application is good science. If we expect people to take these statements at face value then aren’t we promoting bad science?

Are our training classes and training programs teaching good science or are they promoting mis-information? Is it good science to be adamant one day that conclusions are “to the exclusion of all others” but then change our views the next day without an explanation of why?

Here are some questions I think people not only need yes or no answers to but they need complete explanations of why they are answering a certain way. What someone does is only good science if they understand these concepts.

1) Are absolute conclusions scientific? Why or why not?
2) Are statements of fact or proof scientific? Why or why not?
3) Is the statement “to the exclusion of all others” scientific? Why or why not?
4) Is a zero error rate for a methodology scientific?
5) Are conclusions that are based on your training and experience scientific (instead of being based on objective characteristics)? If characteristics are only seen by me because of my training and experience then aren’t those characteristics subjective and less scientific?
6) Is excluding from 1st level detail scientific?
7-8) Are these statements (from the Daubert card) scientific?
“if there is a difference in two impressions that cannot be accounted for, then the impressions do not match”
“it is understood that suggesting a dissimilarity outside that available area is to propose an impossible situation”
9) Is using blind verification on single conclusions scientific?
10) Is using one person to blindly verify something scientific?
11) Is using blind verification to check a conclusion scientific (or should it be used as a test prior to arriving at a conclusion)? Even the Pepsi tests understand this principle.
12-13) What about statements that we expect the critics to refute our claims (instead of us validating them), are those scientific?
(2 statements from the recent IAI letter)
“None of those projects refuted the scientific principle that fingerprints are unique and permanent.”
“There is no research to suggest that properly trained and professionally guided examiners cannot reliably identify whole or partial fingerprint impressions to the person from whom they originated.”
14) Last one (now that I see the post on the Y7 committee conclusion), is it scientific to publish results without publishing the justification behind the results? Isn’t this biasing people?


Of course this is too much to respond to. I’m not looking for answers, I just think we need to clearly understand the problem before we can address it.

Michele

[izatso]

Hi Pat,

Roy, one of the "points" (please excuse the pun) of the NAS report with which I have trouble is that of "measurement." DNA is the forensic science perhaps best suited to statistical analysis because of the nature of a DNA molecule and the ease of doing population studies on DNA. While we in fingerprints can do population studies and compile statistics on pattern types, that helps us little with a specific identification. The problem with "measurement" in fingerprints is that there is an almost infinite number of variables in the appearance of features we use to identify a fingerprint. We know that "counting points" is not a proper method because it fails to take into consideration a vast amount of information. If we could follow Galton's model, or Locard's, or any of the other statistical models, then point counting would yield the measured results our critics want to see. But to follow those models would itself be unscientific in a sense because it would require us to ignore so much other valuable information. And there is no guarantee it would yield more reliable conclusions.

I think it is a false dichotomy to divide Fingerprint Examiners into point counters and ridgeologists. I would hope no one in our profession is a strict point counter. If people actually do fall into both groups, then I would suggest there is yet a third group that combines both the merits of "point counting" and ridgeology into their comparison methodology. I personally have met very few strict ridgeologists who didn't in some "measure" quantify the number of minutia they found in sequence and use that as a basis for their conclusion even if they do not state it. At conferences on this issue I have informally surveyed many who say they are strict ridgeologists, but when discussing minimum thresholds for making an identification, I have found that most if not all have a minimum number of eight. This is very interesting given the fact that as a professional position neither IAI nor ridgeology as a discipline admit this. It appears to be the relative that no one wants to acknowledge. But in this we digress.

You mention that there are an infinite amount of variables and with regard to latent prints, I agree. There is very little that we can control regarding the deposition of latent prints. There is an attendant set of problems associated with this that no simple tests are going to be able to address. However, this is only part the comparison process. We take latent prints and we compare them to rolled prints found in our reference collections. But just how reliable are those prints that we all depend upon so much? It doesn't matter how well we follow ACE-V methodology, any conclusions we make are only as good as the reference print (Exemplar) to which we compare. It is my opinion that if we can establish the level of reliability we can expect to find in our reference collections, then we will go a long ways forward in establishing an error rate and putting our discipline on a solid foundation. We understand the human element with regard to rolled or collected prints, but there is a natural part that is being overlooked. This has to do with minutia groupings and the likelihood of replicating them in more than one subject. The question we need to ask is does this occur and secondly, at what level.

As I mentioned above, there seems to be a de facto standard of eight. I would ask the question why eight? I don't know the answer to this for certain, but I suspect it has to do with the number of prints we encounter where we in fact do see minutia groupings that we have seen previously in other subjects. Also, I am not suggesting that the number of minutia found in such grouping is large, but we do need to see if they exists and how often they repeat within a given population. For anyone having experience with AFIS hits we are in fact encountering some replication of minutia at some level and more often than we like, we are discovering what I would call approximating prints. This is such an issue for us that we are now requiring a 2nd verification on any case which has a single latent print that was found in AFIS.

Given the effect that distortion has on all prints whether latent or rolled, this should be of great concern as distortion can and does make prints look more like those of other donors. Obviously this is less an issue for anyone who is comparing rolled images and who has a complete set, but for all examiners who are comparing mere scraps with greater degrees of distortion, this can be a serious problem. I don't think there is any disagreement that reliable individualization can be obtained using ACE-V methodology. The issue is at what level. Establishing this would increase our knowledge of the natural phenomena with which we work and provide a greater degree of confidence in the conclusions we make.

I have had conversations with those in our field who are trying to design statistical probability formulas for fingerprint identification. I strongly encourage their efforts, but I sincerely doubt they will come up with such a formula or model. I hope I am wrong! What a relief it would be to go into court and be able to present a formula, plug in the numbers, and satisfy the critics who deride us for not having "measurements." But you know as well as I that if we suddenly had that magic formula, the Barry Schecks of this world would find something else to attack us on and would still be every bit as critical and outspoken as they are today. Perhaps the other scientists whose fields use measurements would be willing to more readily accept fingerprints as science, but the adversaries in court would not.

I am currently doing something similar to what I describe above myself. I think there is a great deal of merit in carrying these kinds of studies forward. I think this does address our critics with regard to error rates. I don't believe the result is going to be a magical formula. We will still have to approach each comparison as we do today but knowing this will increase our confidence in the conclusions we make. Also, I am not saying that anything we do is going to satisfy all of our critics, but it addresses a gap in our own theoretical understanding which I believe needs to be filled. I think it will provide answers which may not be accepted by the Barry Schecks of the world, but will strengthen our scientific underpinnings.

The other issue I take is with the word "subjective." But maybe that is not a separate issue at all. Maybe it is related because if we had measurements, we could dispense with the term "subjective." Our critics would have the world believe that "subjective" translates to "unreliable." Nothing could be further from the truth. "Subjective" is an adjective that means, basically, a decision made in the mind of the practitioner. Yes our conclusions are subjective! There is nothing wrong with that. And subjectivity alone does not make a fingerprint examiner's conclusion unreliable. The key there is training. A trained, experienced, proficiency tested examiner with a track record of reliability is perfectly able to reach correct conclusions, regardless of how subjective the process of making an identification is.

I know sometime ago that you and I had a conversation about subjectivity. I don't believe that we are being as subjective as some are making us out to be. We are using a natural phenomena which produces consistent reproducible results based upon a careful analysis of that natural phenomena. Subjectivity comes into the picture more than it should because we don't know the lower limits of individualization. It is because of this that I believe our approach is more subjective than it need be. Interestingly, our method continues to work because of the soundness of our underlying theories. This is our experience. Now we need to test them in order to add scientific credence to what it is we already experience and take ourselves out of that subjectivist stereo-type. Of course, anywhere there is human involvement,there is always going to be subjectivity at some level regardless of what science you are talking about. But that doesn't mean we can't make our process more objective. I think that if we can establish a lower threshold of error in our reference collections then we do put our own science on an objective footing. In this we do see a similarity between the loci of DNA and the minutia of friction skin. You will note that as DNA has progressed, it has had to to raise the number of Loci used in order to reduce the probability of replication in other donors. I think we need to establish similar limits with regard to our own reference collections.

What ever we want to call ourselves, I can say that if we found groupings of four minutia replicated in different subjects, that we would all be very cautious in drawing a conclusion to individualize with a grouping of four minutia. This is of course extremely funny, since most of us are already cautious about individualizing with a four minutia grouping anyway and we don't know at this time what the probability is of such a group replicating in a particular population.

Roy Marzioli, Manager
Central Identification Services
Contra Costa County Sheriff's Office

[LPE never make mistakes]

Sad, the most important report to come out as forensics and seven posts? The NAS report is not hard to digest, in fact the prior posters have laid out the essence of the critique. It us a scathing report as to fingerprints and a surprise to no one who has looked at fingerprints with an open mind. I give tremendous credit to the two posts listed below but to the other posters you cannot continue to place your heads deeper into the sand. The biggest problem is the present leaders cannot lead because they refuse to see the fundamental problems:

What we do in fingerprint comparison/identification is science. It is reliable science and it is good science. Our critics are, for the most part, our adversaries in court. Talk about bias!!! National Public Radio did a spot in their evening news yesterday and as an interview resource, they allowed Barry Scheck to hold forth on why forensic science is bad science. Now there is a reliable, authoritative source! NOT!!!

Perhaps Mr. Wertheim can articulate why Barry Sheck cannot opine as to foreniscs? Perhaps Mr. Wertheim can explain why 60% of wrongful convictions involved bad foreniscs. Perhaps Mr. Wertheim can point out the reference articles validating ace-v involving latents from easy to hard latents, new to experienced examiners working on those latents? How do you know it is good science when there is not one true journal article as to validation of ACE-V? Science becomes science why one applies the dictates of science. Teh party line is clear, we are already doing the research requested (where is it), report is not critical (it is scathing), lip service to validation, documentation, reform etc

Until you can answer the questions posed by Michele, then it is not a science and no one can quantify the reliability of fingerprints. Kudos to those who will ask the hard questions and seek reform, shame to the others.





Pat,

It’s easy to say what we do is good science but that’s not good enough. People need to understand what good science is before they can determine if their application is good science. If we expect people to take these statements at face value then aren’t we promoting bad science?

Are our training classes and training programs teaching good science or are they promoting mis-information? Is it good science to be adamant one day that conclusions are “to the exclusion of all others” but then change our views the next day without an explanation of why?

Here are some questions I think people not only need yes or no answers to but they need complete explanations of why they are answering a certain way. What someone does is only good science if they understand these concepts.

1) Are absolute conclusions scientific? Why or why not?
2) Are statements of fact or proof scientific? Why or why not?
3) Is the statement “to the exclusion of all others” scientific? Why or why not?
4) Is a zero error rate for a methodology scientific?
5) Are conclusions that are based on your training and experience scientific (instead of being based on objective characteristics)? If characteristics are only seen by me because of my training and experience then aren’t those characteristics subjective and less scientific?
6) Is excluding from 1st level detail scientific?
7-8) Are these statements (from the Daubert card) scientific?
“if there is a difference in two impressions that cannot be accounted for, then the impressions do not match”
“it is understood that suggesting a dissimilarity outside that available area is to propose an impossible situation”
9) Is using blind verification on single conclusions scientific?
10) Is using one person to blindly verify something scientific?
11) Is using blind verification to check a conclusion scientific (or should it be used as a test prior to arriving at a conclusion)? Even the Pepsi tests understand this principle.
12-13) What about statements that we expect the critics to refute our claims (instead of us validating them), are those scientific?
(2 statements from the recent IAI letter)
“None of those projects refuted the scientific principle that fingerprints are unique and permanent.”
“There is no research to suggest that properly trained and professionally guided examiners cannot reliably identify whole or partial fingerprint impressions to the person from whom they originated.”
14) Last one (now that I see the post on the Y7 committee conclusion), is it scientific to publish results without publishing the justification behind the results? Isn’t this biasing people?


Of course this is too much to respond to. I’m not looking for answers, I just think we need to clearly understand the problem before we can address it.

Michele





NAS Report & ACE-V

New postby izatso on Fri Feb 20, 2009 9:33 pm
I am sure that many of you have read the NAS report "Strengthening Forensic Science in the United States: A Path Forward" (NAS Report) and are aware of following statements made in the report with regard to ACE-V:

Unreliability:

In Maryland v. Rose, a Maryland State trial court judge found that the Analysis, Comparison, Evaluation, and Verification (ACE-V) process (see Chapter 5) of latent print identification does not rest on a reliable factual foundation

NAS Report 3-16

Does not specify measurements or a standard test protocol:

Note that the ACE-V method does not specify particular measurements or a standard test protocol, and examiners must make subjective assessments throughout. In the United States, the threshold for making a source identification is deliberately kept subjective, so that the examiner can take into account both the quantity and quality of comparable details. As a result, the outcome of a friction ridge analysis is not necessarily repeatable from examiner to examiner.

NAS Report 5-9

Can be improved by the introduction of measurement criteria:

Clearly, the reliability of the ACE-V process could beimproved if specific measurement criteria were defined.

NAS Report 5-10

The conclusion:

ACE-V provides a broadly stated framework for conducting friction ridge analyses.
However, this framework is not specific enough to qualify as a validated method for this type of
analysis. ACE-V does not guard against bias; is too broad to ensure repeatability and
transparency; and does not guarantee that two analysts following it will obtain the same results.
For these reasons, merely following the steps of ACE-V does not imply that one is proceeding in
a scientific manner or producing reliable results. A recent paper by Haber and Haber
Summary Assessment
Historically, friction ridge analysis has served as a valuable tool, both to identify the
guilty and to exclude the innocent. Because of the amount of detail available in friction ridges, it
seems plausible that a careful comparison of two impressions can accurately discern whether or
not they had a common source. Although there is limited information about the accuracy and
reliability of friction ridge analyses, claims that these analyses have zero error rates are not
scientifically plausible.
A new paper by Haber & Haber presents a thorough analysis of the ACE-V method and its scientific validity. Their conclusion is
unambiguous: “We have reviewed available scientific evidence of the validity of the ACE-V method and found none.”30



[W]e report a range of existing evidence that suggests that examiners differ at
each stage of the method in the conclusions they reach. To the extent that they
differ, some conclusions are invalid. We have analysed the ACE-V method itself,
as it is described in the literature. We found that these descriptions differ, no
single protocol has been officially accepted by the profession and the standards
upon which the method’s conclusions rest have not been specified quantitatively.
As a consequence, at this time the validity of the ACE-V method cannot be tested.

NAS Report 5-12

Better documentation of the process needed:

Recent legal challenges, New Hampshire vs. Richard Langill32 and Maryland vs. Bryan
Rose,33
Some scientific evidence supports the presumption that friction ridge patterns are unique
to each person and persist unchanged throughout a lifetime.
have also highlighted two important issues for the latent print community:
documentation and error rate. Better documentation is needed of each step in the ACE-V process
or its equivalent.

NAS Report 5-13

Regardless of what we might think of the sources, I think this is a fairly good assessment with where we stand with the method and what we need to do in order to improve it and make it scientific. What do you think?

[Pat A. Wertheim]

Pat,

It’s easy to say what we do is good science but that’s not good enough. People need to understand what good science is before they can determine if their application is good science. If we expect people to take these statements at face value then aren’t we promoting bad science?

Are our training classes and training programs teaching good science or are they promoting mis-information? Is it good science to be adamant one day that conclusions are “to the exclusion of all others” but then change our views the next day without an explanation of why?

Here are some questions I think people not only need yes or no answers to but they need complete explanations of why they are answering a certain way. What someone does is only good science if they understand these concepts.

1) Are absolute conclusions scientific? Why or why not?
2) Are statements of fact or proof scientific? Why or why not?
3) Is the statement “to the exclusion of all others” scientific? Why or why not?
4) Is a zero error rate for a methodology scientific?
5) Are conclusions that are based on your training and experience scientific (instead of being based on objective characteristics)? If characteristics are only seen by me because of my training and experience then aren’t those characteristics subjective and less scientific?
6) Is excluding from 1st level detail scientific?
7-8) Are these statements (from the Daubert card) scientific?
“if there is a difference in two impressions that cannot be accounted for, then the impressions do not match”
“it is understood that suggesting a dissimilarity outside that available area is to propose an impossible situation”
9) Is using blind verification on single conclusions scientific?
10) Is using one person to blindly verify something scientific?
11) Is using blind verification to check a conclusion scientific (or should it be used as a test prior to arriving at a conclusion)? Even the Pepsi tests understand this principle.
12-13) What about statements that we expect the critics to refute our claims (instead of us validating them), are those scientific?
(2 statements from the recent IAI letter)
“None of those projects refuted the scientific principle that fingerprints are unique and permanent.”
“There is no research to suggest that properly trained and professionally guided examiners cannot reliably identify whole or partial fingerprint impressions to the person from whom they originated.”
14) Last one (now that I see the post on the Y7 committee conclusion), is it scientific to publish results without publishing the justification behind the results? Isn’t this biasing people?


Of course this is too much to respond to. I’m not looking for answers, I just think we need to clearly understand the problem before we can address it.

Michele

Okay, Michele, let me give this a shot.

1) Are absolute conclusions scientific? Why or why not? – Not if you accept that in true science, there are no absolutes.
2) Are statements of fact or proof scientific? Why or why not? – Define “fact.” Define “proof.” I think maybe, maybe not. Depends.
3) Is the statement “to the exclusion of all others” scientific? Why or why not? – It is scientific if you have compared “all others.”
4) Is a zero error rate for a methodology scientific? – “Zero error rate” is a fiction begun during preparation for the Mitchell hearing. We all know it is not scientific.
5) Are conclusions that are based on your training and experience scientific (instead of being based on objective characteristics)? – No, but training and experience lend credibility in court as well as other forums. If characteristics are only seen by me because of my training and experience then aren’t those characteristics subjective and less scientific? – Of course they are subjective. But if you are following an objective methodology, a subjective conclusion is not necessarily unreliable.
6) Is excluding from 1st level detail scientific? – If the images are clear, the comparison is objective, and the practitioner is proficient, the conclusion should be reliable. “Scientific,” yes, I believe it is. Go ahead and disagree, and tell me why.
7-8) Are these statements (from the Daubert card) scientific? – those statements fall into the realm of dogma. That, in itself, does not make them unreliable statements. Dogma, like science, can be reliable sometimes and it can yield erroneous conclusions on occasion.
“if there is a difference in two impressions that cannot be accounted for, then the impressions do not match”
“it is understood that suggesting a dissimilarity outside that available area is to propose an impossible situation”
9) Is using blind verification on single conclusions scientific? – No, of course not. It’s a quality assurance measure written into some policies and procedure.
10) Is using one person to blindly verify something scientific? – Ditto.
11) Is using blind verification to check a conclusion scientific (or should it be used as a test prior to arriving at a conclusion)? – Again, ditto. Even the Pepsi tests understand this principle.
12-13) What about statements that we expect the critics to refute our claims (instead of us validating them), are those scientific? – Of course not, but when you go to court, you had better be prepared for the opposing attorney to refute your claims no matter how scientific they are.
(2 statements from the recent IAI letter)
“None of those projects refuted the scientific principle that fingerprints are unique and permanent.”
“There is no research to suggest that properly trained and professionally guided examiners cannot reliably identify whole or partial fingerprint impressions to the person from whom they originated.”
14) Last one (now that I see the post on the Y7 committee conclusion), is it scientific to publish results without publishing the justification behind the results? Isn’t this biasing people? – Does science itself require that you publish justifications? Are the conclusions less scientific if you publish a conclusion without publishing a lengthy justification? The process is science, the conclusions are scientific. The justification may be a tradition in science, but isn’t it dogma to say that without justification, a conclusion is unscientific?

Roy, you said

I think it is a false dichotomy to divide Fingerprint Examiners into point counters and ridgeologists. I would hope no one in our profession is a strict point counter. If people actually do fall into both groups, then I would suggest there is yet a third group that combines both the merits of "point counting" and ridgeology into their comparison methodology. I personally have met very few strict ridgeologists who didn't in some "measure" quantify the number of minutia they found in sequence and use that as a basis for their conclusion even if they do not state it. At conferences on this issue I have informally surveyed many who say they are strict ridgeologists, but when discussing minimum thresholds for making an identification, I have found that most if not all have a minimum number of eight. This is very interesting given the fact that as a professional position neither IAI nor ridgeology as a discipline admit this.”

I agree completely that it is a false dichotomy. As I said earlier, although in slightly different terms, in fingerprint comparisons trained human brains work the same. The early pioneers’ brains made identifications the same as today’s fingerprint examiners' brains. Point counters’ brains work the same as ridgeologists’ brains. “Point counting” and “Ridgeology” are more protocols to explain what we do than a statement of "the scientific process." They are imperfect explanations. We cannot fully understand the complexities of the human brain. In fingerprints, maybe more subjective decisions are made in the brain than in other forensic sciences. That does not make fingerprint identification bad science, nor does it make an identification unreliable.

As I mentioned above, there seems to be a de facto standard of eight. I would ask the question why eight? I don't know the answer to this for certain, but I suspect it has to do with the number of prints we encounter where we in fact do see minutia groupings that we have seen previously in other subjects.

I think the number eight represents both objective and subjective evaluations of large numbers of comparisons. Galton said 12 points, but Locard said 8 to 12, depending on what else appears in the print. Many examiners since then stuck with eight. Could it be a generational dogma accepted as science, or was there science behind Galton’s and Locard’s calculations that makes the number 8 scientific? I don’t know, but I know that I’ve seen AFIS close hits in which distortion prevented accurate discrimination amongst 8 points on different areas of friction skin. Ask Brandon Mayfield about that concept.

What ever we want to call ourselves, I can say that if we found groupings of four minutia replicated in different subjects, that we would all be very cautious in drawing a conclusion to individualize with a grouping of four minutia. This is of course extremely funny, since most of us are already cautious about individualizing with a four minutia grouping anyway and we don't know at this time what the probability is of such a group replicating in a particular population.

Stephen Meager boldly announced (or others in the FBI in conjunction with Stephen’s talks) that the odds of finding four points the same from two areas of friction skin was what, one in ten to the 27th? Some ridiculous number like that. Every fingerprint examiner in the business for a few months has seen four points in one fingerprint that appear to “match” four points in another fingerprint, but none of us have ever compared 10 to the 27th number of fingerprints. Try entering only four points in AFIS and see how many hits you get. This is where training and experience come into the equation rather than raw statistics. Stephen was relying on raw statistics.

Perhaps Mr. Wertheim can articulate why Barry Sheck cannot opine as to foreniscs? .

Of course Mr. Scheck can opine on forensics. He seems to do so daily and without restraint. Let me tell you about a case I examined some years ago for Mr. Scheck and the Innocence Project. The police had identified a bloody palm print at a murder scene as having been made by the victim, whose body was lying in close proximity to the bloody print on the wall. Initially, the police had failed to make the identification, but had then exhumed the victim’s body for better exemplars and had made the identification. It was not an “easy” identification, but it was a correct one, “in my opinion.” I advised Mr. Scheck of that. Unfortunately for the client, who was, in fact, innocent of the murder for which he had been convicted, that opinion did not give him the grounds he needed for a new appeal. So Mr. Scheck relied instead on the opinion of a full-time document examiner who dabbled in latent prints on the side. This other “expert” could not verify the identification and reported the bloody print did NOT match the victim. Mr. Scheck went with that conclusion and completely ignored mine. Read all about it. The book is “The Innocent Man” by John Grisham. Mr. Scheck can opine all he wants. But an authority on what is scientific? I think not!

Perhaps Mr. Wertheim can explain why 60% of wrongful convictions involved bad foreniscs.

Uh, I thought we were talking about fingerprints. Did 60% of wrongful convictions involve incorrect latent print identifications? I thought eyewitness identifications were the pet blame for most wrongful convictions.

Perhaps Mr. Wertheim can point out the reference articles validating ace-v involving latents from easy to hard latents, new to experienced examiners working on those latents? How do you know it is good science when there is not one true journal article as to validation of ACE-V? Science becomes science why one applies the dictates of science. Teh party line is clear, we are already doing the research requested (where is it), report is not critical (it is scathing), lip service to validation, documentation, reform etc.

Sorry, "LPE never make mistakes," but aren’t you lashing out a little bit here? Let me try to get away with just addressing one of the charges you make – “not one true journal article” – Is a “true” article one published in a peer reviewed article? Or do you mean the journal of, perhaps, IAFS but not that of IAI?

Until you can answer the questions posed by Michele, then it is not a science and no one can quantify the reliability of fingerprints. Kudos to those who will ask the hard questions and seek reform, shame to the others.

Ah, I get it. If I can answer Michele’s questions, it is science; If I cannot, it is not. Help me out here, what’s the passing score? 70%? Can I get a “D” with 60% correct answers? If I score, say, 50%, is fingerprint comparison unscientific for all people and for all time? Or can I take a makeup exam? Gosh, the weight of all this responsibility is making me nervous. Does the future of fingerprints actually hinge on the score I make on this test Michele proposes?

Okay, come on back and lets spar some more. This is fun.

Roy, Michele, if you have any more serious comments, I enjoy the debate.

[LPE never make mistakes]

Re: NAS Report & ACE-V

Postby Pat A. Wertheim on Mon Feb 23, 2009 1:21 am

Okay, Michele, let me give this a shot.

3) Is the statement “to the exclusion of all others” scientific? Why or why not? – It is scientific if you have compared “all others.”

Since by your answer you acknowledge the obvious, that it is not scientific to state individualization, then why do so? At the very least, it is the end of individualization.

4) Is a zero error rate for a methodology scientific? – “Zero error rate” is a fiction begun during preparation for the Mitchell hearing. We all know it is not scientific.

If not zero than what is it? You do know know, no one knows because ace-v has never been validated. Blanket assertions of "it is low" has no basis because there is no research as to error rates. The honest answer is "no one has a clue as to an error rate

5) Are conclusions that are based on your training and experience scientific (instead of being based on objective characteristics)? – No, but training and experience lend credibility in court as well as other forums. If characteristics are only seen by me because of my training and experience then aren’t those characteristics subjective and less scientific? – Of course they are subjective. But if you are following an objective methodology, a subjective conclusion is not necessarily unreliable.

As NAS points out 5-12 "ACE-V provides a broadly stated framework for conducting friction ridge analysis. However, this framework is not specific enough to qualify as a validated method fo r this type of analysis ... does not guard against bias ... too broad to assure repeatabilility ... merely following the steps of ace-v does no imply that one is proceeding in a scientific manner.

(2 statements from the recent IAI letter)
“None of those projects refuted the scientific principle that fingerprints are unique and permanent.”
“There is no research to suggest that properly trained and professionally guided examiners cannot reliably identify whole or partial fingerprint impressions to the person from whom they originated.”

This is classic, there is no research saying examiners can or cannot reliably identify, you cannot pick glass half empty

14) Last one (now that I see the post on the Y7 committee conclusion), is it scientific to publish results without publishing the justification behind the results? Isn’t this biasing people? – Does science itself require that you publish justifications? Are the conclusions less scientific if you publish a conclusion without publishing a lengthy justification? The process is science, the conclusions are scientific. The justification may be a tradition in science, but isn’t it dogma to say that without justification, a conclusion is unscientific?

Wow dogma to demand justification? No dogma is to say all is right regardless of an NAS report to the contrary. What a scary response , science depends on rigor, full data sets and yes science demands that conclusions be justified, geez. This response again demonstrates the lack of understanding of the most basic tenets of validation


LPE never make mistakes wrote:Perhaps Mr. Wertheim can articulate why Barry Sheck cannot opine as to foreniscs? .

Of course Mr. Scheck can opine on forensics. He seems to do so daily and without restraint. Let me tell you about a case I examined some years ago for Mr. Scheck and the Innocence Project. The police had identified a bloody palm print at a murder scene as having been made by the victim, whose body was lying in close proximity to the bloody print on the wall. Initially, the police had failed to make the identification, but had then exhumed the victim’s body for better exemplars and had made the identification. It was not an “easy” identification, but it was a correct one, “in my opinion.” I advised Mr. Scheck of that. Unfortunately for the client, who was, in fact, innocent of the murder for which he had been convicted, that opinion did not give him the grounds he needed for a new appeal. So Mr. Scheck relied instead on the opinion of a full-time document examiner who dabbled in latent prints on the side. This other “expert” could not verify the identification and reported the bloody print did NOT match the victim. Mr. Scheck went with that conclusion and completely ignored mine. Read all about it. The book is “The Innocent Man” by John Grisham. Mr. Scheck can opine all he wants. But an authority on what is scientific? I think not!

You fail to differentiate between Mr. Scheck advocating for clients and advocating for reform in forensics. Please direct me to one public statement of Mr. Scheck critical of forensics science that you find is not defensible. Just one, I am easy


LPE never make mistakes wrote:Perhaps Mr. Wertheim can explain why 60% of wrongful convictions involved bad foreniscs.


Uh, I thought we were talking about fingerprints. Did 60% of wrongful convictions involve incorrect latent print identifications? I thought eyewitness identifications were the pet blame for most wrongful convictions.

Perhaps you need to remember that you referred to forensics science and Mr. Scheck . Tsk, tsk, Cowan, Mayfield see Cole as to the hall of shame ids

Our critics are, for the most part, our adversaries in court. Talk about bias!!! National Public Radio did a spot in their evening news yesterday and as an interview resource, they allowed Barry Scheck to hold forth on why forensic science is bad science. Now there is a reliable, authoritative source! NOT!!!

LPE never make mistakes wrote:Perhaps Mr. Wertheim can point out the reference articles validating ace-v involving latents from easy to hard latents, new to experienced examiners working on those latents? How do you know it is good science when there is not one true journal article as to validation of ACE-V? Science becomes science why one applies the dictates of science. Teh party line is clear, we are already doing the research requested (where is it), report is not critical (it is scathing), lip service to validation, documentation, reform etc.

Sorry, "LPE never make mistakes," but aren’t you lashing out a little bit here? Let me try to get away with just addressing one of the charges you make – “not one true journal article” – Is a “true” article one published in a peer reviewed article? Or do you mean the journal of, perhaps, IAFS but not that of IAI?

I will make it easy for you, one journal article - peer reviewed - science accepts noting less than peer review. So go ahead and paper me with a hundred years worth of validation studies but I am easy I will take one. Please provide the journal citation and what you assert the article represents and I will happy to pull it and respond. We both know that there is not one article that has validated ace-v but surprise me. Oh and please nothing about uniqueness, twin studies etc, enough of that duck and weave. Let us just accept the notion that full prints are unique but lets talk about partial latents comprable to those from a crime scene. So just one article but surely you have dozens to win your arguement.

LPE never make mistakes wrote:Until you can answer the questions posed by Michele, then it is not a science and no one can quantify the reliability of fingerprints. Kudos to those who will ask the hard questions and seek reform, shame to the others.

Ah, I get it. If I can answer Michele’s questions, it is science; If I cannot, it is not. Help me out here, what’s the passing score? 70%? Can I get a “D” with 60% correct answers? If I score, say, 50%, is fingerprint comparison unscientific for all people and for all time? Or can I take a makeup exam? Gosh, the weight of all this responsibility is making me nervous. Does the future of fingerprints actually hinge on the score I make on this test Michele proposes?

This response says it all, belittle the hard questions as though that minimizes the importance. Here is the kicker, sciences demands validation, metrics, hard/easy metrics as to latents, a procedure that truly is specific as to sequence, documentation, controls bias, takes examiner skill into consideration, error rates, statistics probabilitlies. So the passing score is not we tried hard, it is conducting validation. So here is the kicker, yes you actually have to do validation studies. Fingerprints examiners have no one to blame but themselves. The same questions since Mitchell and the same bunker down mentality hoping it will go away. That will not happen, if you cannot address the hard questions or cannot appreciate their significance than cede the road to those who can.

Okay, come on back and lets spar some more. This is fun.

Roy, Michele, if you have any more serious comments, I enjoy the debate.
Pat A. Wertheim
Tucson, AZ

Pat A. Wertheim

Posts: 526
Joined: Thu Jul 07, 2005 1:48 pm
Location: Tucson, AZ

* Private message

[mdavis]

I suspect this is all a dead end for several reasons.

First, the vast majority of latent print cases never go to court because the suspect is truly guilty and admits so when confronted by the evidence.
Second, there are thousands (perhaps tens of thousands) of latent print identifications (sorry, individualizations) made every working day that are admittedly accurate and not contested, either because of my first point above, or because corroborating evidence coexists in support, and defense realizes the futility of challenge.
Third, we can never truly know "ground truth" in any comparison in case work, thus effectively eliminating statistical accuracy reporting. Such statistics can only be estimated from my second point. We simply cannot equate the statistical basis of DNA testing (where large databases are being compiled with known controls) with fingerprint comparison for these reasons.
Fourth, I don't believe the criminal justice system can refuse to accept the single most (or now with DNA) or perhaps the second most accurate forensic analysis being done on any evidence. Compare the accuracy of fingerprint analysis or DNA with any other form of "evidence" presented at trial -- the allegations of the accused as put forth by his attorney or the prosecution, the accuracy of an "eye witness", the testimony of the officer who made the arrest or submitted the case for prosecution, the ability of the judge and jury to understand and evaluate expert testimony, the accuracy of jury decisions based on days in court and allegations from both sides, often where no note taking is allowed! A juror is on an emotional roller coaster, and many jurors are elderly retirees with failing memory and comprehension skills.

The latent print community has hurt itself immensely by over reacting to challenges. The NAS report largely criticizes technicians for overstating the value of their findings. The court is guilty of accepting expert testimony at face value (yes, from both the initial examiner and the hired assassin bought and paid for by the defense). We need to stay well within our own conservative limits, perform true, ethically honest blind peer reviews of identifications, and quit pushing the envelope on identifications that are not clear to a majority of qualified examiners. Let's also throw out the "bad idents" that are the result of unqualified persons (police detectives in Pennsylvania come to mind), or from biased persons intimately involved in the outcome of the case when attempting to list individualization "errors" so as not to skew the results.

We all become overly defensive when the news media pushes headlines out of context. If the court system chooses to follow the lead of the media and discredit the immense success over a period of 100 years of latent print comparisons, there is little we can do to save them from themselves. I will continue to be as accurate as humanly possible and shamelessly refuse to call borderline comparisons. Many years ago I had an FBI instructor claim (s)he could make identifications that most examiners could not because of her/his experience. I wondered in recent years if that examiner might have been involved in the Mayfield case and either pushed the edge of the envelope too far, or rubber stamped a peer's ident.

[Michele]

I never understood when the critics would state that we don’t know if past identification were correct just because they weren’t contested. I actually thought this was a ridiculous argument but recently I’ve had the opportunity to review several past ID’s that weren’t contested. The suspect was found guilty and convicted, but the ID’s were erroneous. I think these weren’t initially contested because the person probably was really guilty, but that didn’t mean the ID’s were correct. What I’m trying to say is that I’m starting to understand the critics point. I’d tend to think that this is rare but honestly, how are we to know? Especially when you consider that most cases aren’t prosecuted on fingerprint evidence alone (I’m speculating about this from my own experiences).

Pat – The reason I believe that excluding on 1st level detail or by pattern type is unscientific is because the examiner has a plausible conclusion or a hypothesis that the prints don’t match (and is right the majority of the time). Even if the conclusion is right, that doesn’t mean it’s scientific unless the hypothesis is tested. Testing can be done by checking to see if characteristics match (which is excluding by 2nd level detail). Most people call for rigorous testing for something to be considered scientific; this could mean that several areas of level 2 details should be tested. Fig’s 2, 21, 22, and 23 could be examples of where people might falsely exclude if they only look at the ridge flow and don’t look at some 2nd level details. Excluding on 1st level detail may be very accurate but that doesn’t mean it’s scientific.

[izatso]

Hi MDavis,

I suspect this is all a dead end for several reasons.

I think you remark here is premature. If this were indeed the case,then all of us should affirm that what we have been calling "fingerprint science" is not a science and all of us should be tendering our resignations and going into a different field of work. However, our experience contradicts this. Keep in mind that we are dealing with natural phenomena. I don't no about you, but I have never found any natural phenomena which were beyond scientific study or explanation. Also, any regularity of nature is subject to probabilistic studies.

Second, there are thousands (perhaps tens of thousands) of latent print identifications (sorry, individualizations) made every working day that are admittedly accurate and not contested, either because of my first point above, or because corroborating evidence coexists in support, and defense realizes the futility of challenge.

Actually, this is not the point. This does of course describe current events and 100 plus years of history. The problem is that it is nothing more than 100 years of anecdotal information about natural phenomena, but, to repeat, it is not science. Because we have done business like this for over a hundred years is not going to correct the deficiencies exposed in this report. This report is about moving us away from subjectivity and off of our anecdotal history and putting us on to a scientific base. (I am not saying that we will eliminate all subjectivity; It is, after all, human beings who are involved. Many of the recommendations of this report actually address the human element and give us ways to reduce subjectivity and human error.)

We do want corroborating evidence. I am not sure of what you mean when you say corroborating evidence coexists in support. I would be interested to know what evidence you are referring to? With regard to the success of our methods and the support for our underlying theories, we do not yet have what could be called scientific evidence but it isn't as hard to find as you might think. Unfortunately anecdotal information will never provide that. We muct approach this in a rigorous scientific manner.

Third, we can never truly know "ground truth" in any comparison in case work, thus effectively eliminating statistical accuracy reporting. Such statistics can only be estimated from my second point. We simply cannot equate the statistical basis of DNA testing (where large databases are being compiled with known controls) with fingerprint comparison for these reasons.

I strongly disagree with you here. I would also point out that there is tremendous irony in this statement. Here we have examiners making "absolute" individualizations "to the exclusion of all others?" Yet you state "we can never truly know "ground truth?' There is something terribly incongruous with these two statements. What the heck are we doing? How do you reconcile these two statements?

No offense, but as for statistical accuracy, I don't believe you know what it is. I say this because if you knew what it was you would see that this remark doesn't make any sense. In any case our data set is not the same as the DNA data set and no one working with statistics would say they are the same. What we do need to know is what the probability is of replicating minutia groupings in our reference collections. The problem has been in the past is that we have been perhaps blissfully blind to this very key component of the comparison process. You have been trusting that the reference set is completely trustworthy and then dealing with all of the problems associated with latent deposition and distortion without a second thought as to what you are comparing with. Don't you think that knowing how reliable your reference collection is has some bearing on the accuracy of your latent comparison?

Fourth, I don't believe the criminal justice system can refuse to accept the single most (or now with DNA) or perhaps the second most accurate forensic analysis being done on any evidence.

I don't think judicial system will either. But I do believe they are going to force us comply with the recommendations of this report. This becomes a win/win situation as we become scientifically based and the courts have more trust in the conclusions we present. (This is an aside: I don't think we are the second most accurate forensic analysis. We know that there is more uniqueness to be found in friction skin than in DNA. Why would you think we are the second most accurate?)

The latent print community has hurt itself immensely by over reacting to challenges. The NAS report largely criticizes technicians for overstating the value of their findings. The court is guilty of accepting expert testimony at face value (yes, from both the initial examiner and the hired assassin bought and paid for by the defense). We need to stay well within our own conservative limits, perform true, ethically honest blind peer reviews of identifications, and quit pushing the envelope on identifications that are not clear to a majority of qualified examiners. Let's also throw out the "bad idents" that are the result of unqualified persons (police detectives in Pennsylvania come to mind), or from biased persons intimately involved in the outcome of the case when attempting to list individualization "errors" so as not to skew the results.

I agree with much of your statement here. I believe that by embracing the recommendations of the NAS report that much of what you state here will be corrected.

Roy Marzioli, Manager
Central Identification Services
Contra Costa County Sheriff's Office
Martinez, Ca.

[mdavis]

I believe it is generally accepted that the basis of fingerprint science is sound -- biologically speaking. I do not see expert examination and pronouncement as a science, and apparently neither does the NAS until we "prove" it to be so.

The question is, are the deficiencies "correctable" or are we tilting at windmills when we try to push comparison psychology into pure science in order to verify what we've essentially "proven" over 100 years? Isn't this what got us into trouble in the first place -- pushing the envelope on our definition of science clear to the point of decision? There is little or no science in any courtroom unless we introduce what little we have. Still, it is seldom understood or appreciated.

The reference to corroborating evidence was intended to include any other evidence presented in an individual case that supported the allegation that a person touched an item and that the fingerprint identification is further supported. That might include eye witnesses, cell phone data, video cameras, location records or job attendence history that would provide a person's access to an area during the crime.

"Ground truth" is simply knowing with absolute 100% certainty that an identification is valid. To me, that means not that we cannot ever be "100% positive" that our ident is absolute, but that we can never know for absolute certainty that the latent print was made by a certain person unless we personally witnessed the deposition of the latent print during the crime. This is a "no brainer" for most of our identifications, but it rears its ugly head on "borderline" latents in which examiners are biased based on other "corroborating" evidence or knowledge of the situation. I can say with "virtual" certainty that an ident is valid, but I can never know with 100% statistical, scientific validity that a given latent was in fact proven to have been left by a given person at a crime scene.

There have been studies made over the century attempting to "quantify" and "qualify" latent print comparison. They have failed, largely due to the fact that we do not have quantifiable means of describing the infinite detail found among individual pores, ridges, scars, incipient ridges, creases, Galton details and patterns. It is my belief that once we attempt to standardize our descriptions, mathematically or digitally, in an effort to "rate" or "score" a latent against its exemplar, placing a statistically valid number on the "comparison" in order to "scientifically validate" the process, then we move away from the individual examiner and into the world of super-AFIS. Such comparison would have to do a better job than the human brain both in recognizing salient features but also in being able to recognize and describe variations and their origins. We simply cannot code an infinite number of possibilities into a computer and have it recognize them and score them accurately.

I am well aware of science, having been educated as a chemist. I also recognize the current state of AFIS comparison and search algorithms. Such descriptive standardization and accuracy will never happen in my lifetime. I think we agree here for the most part. I am simply skeptical that the current state of the art can even begin to approach the suggested solution to the perceived problem. We are being placed at the mercy of over-zealous defense attorneys who do not understand the vast differences between comparing DNA printout charts and comparing friction ridge skin impressions. The NAS has some valid concerns for the extremely rare case of a mis-ident. They and we have no solutions except to keep a better eye on our peers and to pull back if there is any doubt on a borderline comparison. We can continue to work toward statistical validity, but I don't think it will happen.

Good discussion. Thanks for the feedback.