KEPT - Keeping Examiners Prepared for Testimony

[clpexco]

Question – Documentation - Levels of Details:
What levels of details did you use to make your identification?

Possible Answers:
a) I used all the levels of details; I assess everything prior to arriving at a conclusion.

b) I couldn’t say because we don’t document that information.

c) Simple conclusions are arrived at by analyzing the quantity, the quality, the rarity, and the spatial placement of the level two details. I may notice the level 1 and level 3 details but I don’t document it in my notes unless I specifically used it to make my individualization. Since I didn’t note it, it wasn’t needed to make this individualization.

d) I documented in my notes that the level 1 detail was consistent, the level 2 detail was consistent and sufficient to establish an individualization, and very clear level 3 detail (pores and ridges edges) was also visible but not needed to make the individualization.

e) The definitions for the levels of details are slightly vague. One person may label an short incipient ridge as level 3 detail because it may not always be visible and another person may label it as level 2 detail because it’s a ridge ending. Regardless of what people call these characteristics, I noted that the ridge flow, the ridge events, and the ridge shapes were within the tolerance that we normally see.

Discussion:

One topic for discussion regarding this question is the use of the term ‘identification’ vs using the term ‘individualization’ (but that’s a topic for another discussion).

Answer a: Although this may be true, it sounds suspicious when people testify to small details that happened 3 years ago without these details being documented. Big details are usually memorable but remembering small details, like whether or not you used all the levels of details to make an individualization, is hard for most people to believe.

Answer b: This may be a correct answer but it would sound better if the courts knew whether noting this information is needed or not. Without stating that it isn’t needed, many people assume it wasn’t noted but it should have been.

Answers c, d, and e: These are all good answers but I like c the best because d and e both may include excess documentation that isn’t needed (the necessity of such notes should be based on the complexity of the comparison). Stating that documentation isn’t always needed could be important for different reasons; primarily so agencies that don’t have the time for excess documentation don’t feel pressured into adding a process that isn’t scientifically recommended, and when testifying in a trial with other experts we don’t want it to appear that there are different standards.

[L.J.Steele]

c) Simple conclusions are arrived at by analyzing the quantity, the quality, the rarity, and the spatial placement of the level two details. I may notice the level 1 and level 3 details but I don’t document it in my notes unless I specifically used it to make my individualization. Since I didn’t note it, it wasn’t needed to make this individualization.

This question could be a prelude to inquires based on the Mayfield IG reports about the existing research on Levlel 3 detail.

Answer (c) may open up inquires into the existence and extent of research on rarity and spacial placement as a factor in IDs. If you are relying on your own memory for what's rare and what's not, without keeping any sort of records, you may have some additional questions to field.

[clpexco]

Disclaimer: The intent of this is to provide thought provoking discussion. No claims of accuracy exist.

Question – Verification – Explanation:
Why does the verifier know the results of the first examiner?

Possible Answers:
a) It’s a time saving measure.
b) You have to know the previous conclusion so you can try to prove that conclusion wrong. Trying to prove something wrong helps diminish bias.
c) Without knowing the results you can’t review if the information is correct. This includes checking whether the conclusions being reported are accurate, whether proper procedures were followed, and if the conclusions were drawn from the data that was available.

Discussion:
Answer a: This is the most common answer and even though knowing someone else’s conclusion does save time, it’s not a concern of the verifier. Confirming a previous conclusion is scientifically acceptable with simple conclusions but with complex situations then a conclusion needs more weight behind it than merely confirming the conclusion. When complex situations arise, the conclusion is given more weight if the verifier corroborates the conclusion and insures that it was arrived at using appropriate principles. The only way to scrutinize a conclusion in this way is by knowing the first examiners conclusion.
Answers b and c: These answers are both good answers but answer c is a little more comprehensive.

[clpexco]

Disclaimer: The intent of this is to provide thought provoking discussion. No claims of accuracy exist.

Question – Proficiency Testing:
What are proficiency tests testing for?

Possible Answers:
a) Proficiency tests are a critical tool for evaluating a laboratory’s overall performance.
b) Proficiency tests are testing to see if an examiner can arrive at accurate conclusions.
c) Proficiency test are something that is done to help you comply with accreditation.
d) Proficiency tests are testing to see if an examiner can make individualizations.
e) Proficiency test can help get advanced warning of potential performance problems.
f) Proficiency tests are testing to see if examiners can orient latent prints, locate a potential source of the latent print, and individualize the latent print.
g) Proficiency tests are testing to see if a practitioners or laboratories can find answers that others would arrive at (consensus answers).

Discussion:
The CTS tests can be administered in several different ways. Individual offices can require justification behind conclusions and agencies can require that examiners state the reason for a non-individualization. A non-individualization could be due to an exclusion, due to an insufficient amount of information, or due to an inability to orient and locate the latent print. The answers below are not considering how individual offices are using the CTS test but only looking at how CTS is using the test.

Answer a: This is stated on the CTS website. While it is a critical tool for evaluating the ability to make individualizations, it doesn’t test other job functions of an examiner. Stating it evaluates the overall performance may be a slight exaggeration depending on the job functions of the examiners in your office.

Answer b: The CTS latent print test doesn’t only insure that conclusions are accurate, the conclusions that are considered correct are both accurate (ground truth) and conclusions that others would arrive at (consensus answer).

Answer c: Proficiency tests do help you comply with accreditation but this answer doesn’t answer the question that was asked.

Answers d, e, and f: These answers are correct but if someone doesn’t individualize a latent print then these tests don’t determine where the problem lies (orienting a latent print, locating potential candidates, individualizing who left the latent print, or perhaps differing tolerance levels).

Answer g: This may be the best answer for two reasons. First, some agencies use the CTS tests for individuals while other agencies take the test as a group. And second, the conclusions that CTS gives are not simply the ground truth answers but the answers that at least 75% of accredited labs arrived at.

[clpexco]

Disclaimer: The intent of this is to provide thought provoking discussion. No claims of accuracy exist.

Question – Terminology:
(This question would be to a verifier or a supervisor) The results for KL3 indicate that no identification was effected. Therefore it has been established that my client did not leave this latent print. Isn’t that true?

Possible Answers:
a) Yes.
b) No.
c) I’m not sure if the original examiner excluded your client or not.

Discussion:
ASCLD/LAB and SWGFAST both recommend technical reviews. SWGFAST specifically states that The reported results are clear, concise, accurate, and complete. Phrases such as ‘No Identification Effected’ may have different meanings to different people. If agencies are using their own phrases in their reports then these statements should be defined somewhere.

[clpexco]

KEPT - Keeping Examiners Prepared for Testimony - #17
Terminology - No Identification Effected
by Michele Triplett, King County Sheriff's Office

Disclaimer: The intent of this is to provide thought provoking discussion. No claims of accuracy exist.

Question – Error Rates:
What is the error rate for fingerprint individualizations?

 Possible Answers:

a) The error rate of the ACE-V methodology is zero. All errors found have been caused by practitioners not using the methodology as diligently as they should have.

b) Some rates have been estimated by using the CTS proficiency tests, the IAI certification tests, studies on bias, and errors in training classes. These estimates may be artificially high because of the parameters of the data for each event. Empirical data shows the error rate is so extremely low that it’s almost zero.

c) Our office has standards and controls in place and the error rate of our office is zero.

d) In the last 100 years of using fingerprints as a form of identification, between 20 and 30 errors have been found world wide. When compared to how many comparisons have been done and how many identifications have been made, statistically, the error rate is very close to zero.

Discussion:

One important aspect of this question is that it’s asking for the error rate of individualizations. If the questions were about fingerprint comparisons then the answers may be very different (we’d have to consider type 1 errors, type 2 errors, errors after the ACE process, errors after the ACE-V process, errors of simple conclusions, errors in complex conclusions, etc).

Answer a: This is a common answer but it may not be the best answer. There’s never been any research to support this answer. Without asking examiners how they arrived at a conclusion, it’s impossible to blame an error on their improper use of ACE-V. Out of all the errors ever found, the Mayfield erroneous individualization has been the only error researched and had the results published. While it’s easy to blame errors on examiners, it could be just as easy to blame the agency for not having proper procedures in place. Besides blaming the examiners involved or the agency, others claim the Mayfield error was due to the extreme similarity between the latent print and Mayfield’s known print. We may never know the true cause of an error but we can state that errors are very rare. They may not be zero but they are very close to zero.

Answer b: This is a good answer because it not only talks about the actual error rate but it acknowledges theoretical error rates that have been published.

Answer c: Judges have accepted the error rate of an agency since an exact error rate of our profession hasn’t been established.
Answer d: Even though an exact error rate hasn’t been established, an estimated value is usually acceptable as long you state how the value was determined.

[L.J.Steele]

c) Our office has standards and controls in place and the error rate of our office is zero.

d) In the last 100 years of using fingerprints as a form of identification, between 20 and 30 errors have been found world wide. When compared to how many comparisons have been done and how many identifications have been made, statistically, the error rate is very close to zero.

The problem with answer (c) is how do you know the error rate is zero. You can say that, to date, no one has discovered an error. (Let's assume that error is defined as a mis-ID after verification is complete.) That doesn't mean that undiscovered errors don't exist -- the BPD had a zero error rate prior to Cowans, yet post-Cowans review led to some major changes in that office.

One way to to shore up this answer would be a small amount of additional record-keeping. If your agency's fingerprint unit is isolated from its DNA unit, you can cross-check your fingerprint IDs against DNA matches and be able to say that in the past X years, there were Y number of cases where DNA evidence was also available and that your DNA unit has independently found DNA evidence inclupating the same suspect in Z of those cases. (This is not saying that DNA is more or less reliable than prints, only that it can provide an independent means of inculpating the same suspect.)

Note the independent and isolated above -- if the DNA unit is aware of your print unit's results, or vice-versa (depending one who's results came in first), then you have a potential confirmation bais issue and a less reliable check.

You may have to explain why Z is a smaller number than Y -- the DNA sample may have been contaminated, or degraded, or matched to a co-offender, etc.

Your DNA unit could also use the same results for its testimony, in that case it would have to talk about the Z cases where there was potential print evidence that yielded inconclusive results.

[Gerald Clough]

I think (c.) is the one we can most realistically use, but I think it needs some elaboration if it is not to sound like wishful thinking.

"No precise error rate is known, and because it is very difficult to design a process to test error rate that uses a realistic model of working latent print examination, any overall error rate is largely an informed guess, but I can speak to the error rate in our unit. If we use all known errors in actual cases, it is very close to zero, and in those cases we know the primary factors were human factors, and not the result of any lack of uniqueness. In some 100 years, no known error of identification has been produced on account of two fingerprints having been so similar as to mislead competent examiners following proper procedures. Because we recognize that all the known errors are human, rather than inability to identify fingerprints with certainty, we impose specific procedures, verification of any identifications, and review of our adherence to those procedures to insure that, in the cases under our control, there are no errors."

By stating that I can speak to the local error rate, I am avoiding being cut off as having answered the question with, "I don't know." Since the question did not specify all examinations everywhere, I can keep the meaningful answer where I want it, within the limits of my direct knowledge and specific to what was done in this case.

There is no requirement that any field have a known or even estimated error rate. It's merely one factor the court may consider. I think the above gives the court something meaningful, without claiming more than we can back up. When one considers that there are areas of expert evidence in which error rate cannot be known or is obviously either high or meaningless, we are rather well off.

[Kasey Wertheim]

KEPT - Keeping Examiners Prepared for Testimony - #18
Research Studies - Reliability Studies
by Michele Triplett, King County Sheriff's Office

Disclaimer: The intent of this is to provide thought provoking discussion. No claims of accuracy exist.

Question – Reliability Studies:
Have there been any reliability studies on ACE-V?

Possible Answers:
a) I’m not aware of any studies that are published.

b) There are a number of error rate studies such as…..

c) There are a number of statistical studies on the probability of two prints having the same characteristics. Some of the more well known studies are those done by Galton, Balthazard, Cummins, Wilder, Champod, etc.

d) I know that different people are studying the validity of ACE-V but I’m not sure if any of these studies have been published yet.

e) Are you speaking about the reliability of the conclusion or the reliability of the method itself (the principles within ACE-V)? The reliability of a conclusion may sounds impressive but it may not mean you have a good conclusion. Reliable conclusions just show reproducible results but it doesn’t show if the results are good. For example, my blender goes the same speed on every setting. This isn’t good but it happens all the time….it’s very reliable.

f) Specifically what kind of reliability studies are you looking for?

g) ACE-V is synonymous with the scientific method of hypothesis testing. Hypothesis Testing has been used and tested for hundreds of years and found to produce reliable results.
 
Discussion:
This is an interesting question because it implies that we should have some reliability studies. If someone answers that none have been done then it looks like we’re using a method that hasn’t been tested. My experience has taught me that most people asking this question don’t really know what they are looking for. They are commonly using words that they don’t understand. For example, reliability indicates that something is able to be reproduced. It doesn’t indicate that the results are good results. My toaster is a very reliable instrument; it burns the toast every time I use it. I can try it over and over again and it always gives the same results. I can rely on the results my toaster gives even though those are not my desired results.

When we are asked about reliability tests in court we should ask exactly what are they referring to.

Answer a: This may be accurate but it implies that ACE-V hasn’t been tested as a viable method and it leave the jury and the court with the opinion that ACE-V isn’t a good way to arrive at conclusions.

Answer b: This answer assumes the question is about the reliability of our conclusions and this may not be what they are asking about.

Answer c: This answer is assuming that they are asking about the possibility of patterns duplicating and it ignores the error rate of our interpretations.

Answers d, e, and f: These are all good answers but answer e might be giving them more information than they are looking for.

Answer g: This may be true but it doesn’t answer the question, it just states that ACE-V is accepted but doesn’t show that it’s been tested.

My personal opinion is that d and f are the best answers because they don’t assume the meaning behind the question.

[Gerald Clough]

This is where we find ourselves when we elevate a simple acronym for for best-practice sequence to the status of "method." ACE-V is, like many procedural acronyms, merely a representation of the principle of completing each operation before moving to the next, with the goal of eliminating improperly revising judgments to fit a satisfying result. It is merely an expression of a protective practice common to every kind of analysis of the sort. It is not amenable to "testing for reliability," because it is not a complete description of method, nor can it insure accuracy.

In emergency medicine, SAMPLE is a reminder of a logical sequence of inquiries in soliciting patient histories. It does not insure that the operator solicits properly. There are too many other factors to be applied. PEMDAS prompts the correct order of operations in simplifying mathematical expressions. It in no way insures that the individual operations are correctly applied. There is no "test" of any of these sequences. They exist because the sequences they represent are imposed by conditions of various strength. SAMPLE is rather weakly imposed as a logical and agreed upon sequence. ACE-V is more strongly imposed by the known and suspected hazards of improperly biasing judgment. PEMDAS is absolutely imposed by the definitions of the mathematical operations and their notations.

We're still off-target and so, quite understandably, the courts are drawn off-target. I take issue with d), because it joins the implication that ACE-V is "the method." f) is, I think, the only usable short answer and has the virtue of likely shutting down that line of questioning as meaningless and, if they persist, it sets up for the, "Oh. I see where you're confused." response.

There are four meaningful factors in the testimonial value of latent print conclusions.

1) Logical presumption of this particular aspect of biological uniqueness, grounded in specific knowledge of skin formation and an enormous amount of observation.

2) A sequence of operations insulating judgments from biasing potentials within that particular examination and executed by a person qualified by education and experience to make observations and judgments leading to a conclusion.

3) Credible justification of every judgment made in the course of the various stages of examination.

4) Verification of the conclusion through an examination in which the various judgments and conclusions are not contradicted.

!) and 2) are within Daubert considerations. They show that the testimony is grounded in scientific knowledge and that it assists the trier of fact. 3) and 4) are credibility issues (although documentation of 3 gets drawn into admissibility arguments). So long as 1) and 2) are found to exist, we err when we or the court attempts to test the individual conclusion by some combination of Daubert tests. I would point out that 4) is presented simply because latent print conclusions are not often reviewed by defense experts. This "privilege" to have not only the final word but the only word is, I think, why the discipline is so vigorously challenged. Areas of expert testimony in which opposing experts are offered are not attacked in this way. To do so would undercut the opposing expert's testimony, too. Some of these fields would be fiercely contested if their evidence was generally presented by only State's witnesses. Because we generally go in without expert opposition, we tend to shortcut 3). The legal system we approach with our conclusions does not routinely provide opposing expert review of those conclusions. Therefore, whatever method we use to reach them is easily viewed as secretive and subjective when detailed documentation is not built in as part of the method.

If we step back from the shock and awe of Judge Souder's decision and look at the State's approach to attempting admissibility there, we see the effect of viewing ACE-V as "The Method." We see largely a discussion of observations and conclusions without the scientific foundations. We also see the misuse of "subjective" and a lack of documentation standards that aggravates the "subjective" argument. Souder latched onto the failure of ACE-V in the Mayfield case to, in part, discredit latent print comparison conclusions and was able to do so because the documentation didn't exist to attempt to show how decisions were made during differential analysis of characteristics.

We would do well to consider how we would proceed if we were routinely confronted by opposing experts questioning our observations and interpretations. While it's true that most conclusions of identification are relatively inarguable, the discipline will be tested against the most difficult comparisons. Those are the ones against which the standard will be tested.

[Charles Parker]

Gerald Said

This is where we find ourselves when we elevate a simple acronym for for best-practice sequence to the status of "method." ACE-V is, like many procedural acronyms, merely a representation of the principle of completing each operation before moving to the next, with the goal of eliminating improperly revising judgments to fit a satisfying result. It is merely an expression of a protective practice common to every kind of analysis of the sort. It is not amenable to "testing for reliability," because it is not a complete description of method, nor can it insure accuracy.

Gerald just when I was considering the Popper thing your answer to this post is excellent. Great. Matter of fact I have copied it and downloaded it and I am going to plagerize the h$!! out of it.

It is an acronym. That is all it is, nothing more nothing less. Easy to remember and easy to teach. But it is what is inbetween that matters and that is not so simple of an explanation.

I loved your examples and I do not think I have seen what I believe to be so correct as to be explained so well. Thanks.

[Gerald Clough]

And I would point out that the point was not to make light of the value of the ACE-V paradigm. It was an vitally important articulation of something I think is still not taken as seriously as it should be, and an acronym is the classic way to focus attention on something. Maybe there's a principle of some sort that would say that if you can reduce something to an acronym, it's easy to define and explain. The work of Analysis is not so easily defined. It's a job of what we might call differential "diagnosis." Here's a feature. What sort of structure does it represent? Can I say with certainty? I can't just explain Analysis to someone and set them to doing it.

In other words, once I explain ACE-V to someone, and they agree to do follow it, they will get the sequence right every time. They may be a rank beginner who will not be able to identify with certainty what structures some features represent, and will therefore not have the benefit of those features later as they compare, but they will inevitably get ACE right. I can explain Analysis to someone, but they will only get it right according to their experience and talent. For that matter, in my mind, the Analysis was correct if the examiner's declarations of which features they could interpret with certainty matched their ability. It is a given, or should be, that you will sometimes have the use of some features, and I will not, because you are vastly more experienced. That you can be certain and will conclude identification where I cannot and will call it inconclusive can in no way detract from your conclusion. The point is that we are equal in our abilities to apply ACE-V. We are not equal in our abilities to mine a latent for features interpreted to certainty.

<Begin Soapbox> This is kind of related to Popper. One might say that it's not a genuine expert field if two qualified experts cannot disagree on a conclusion. We have not wanted to think about that, I think because we are not often challenged. Particularly in civil practice, if it weren't for differing conclusion among experts, there wouldn't be nearly so many of them being hired. The real questions are not whether experts disagree. They can't both be right, but the real questions are about just why they hold the conclusions they report. That one of them must be wrong does not render their field of expertise inadmissible. When we set up infallibility as the basis for a claim of admissibility, we set up for failure. The test is one of ability, not infallibility. <End Soapbox>

[clpexco]

KEPT - Keeping Examiners Prepared for Testimony - #19
Other - Scientific Basis for Individualization
by Michele Triplett, King County Sheriff's Office

Disclaimer: The intent of this is to provide thought provoking discussion. No claims of accuracy exist.

Question – Scientific Basis:

How do you know that fingerprints are unique and permanent?

Possible Answers:

a) The FBI has been keeping records since the early 1900’s and no two fingerprints have ever found to be the same. There are also hundreds of thousands of records showing that fingerprints of individuals remain the same.

b) Several statistical studies have been done to show the possibility of duplication is virtually impossible.

c) By looking at how the ridge configurations are initially formed. Generally speaking, the interpretation of ridge configurations can be grouped into two categories. The first considers ridge configurations or patterns to be “primary foci of evolution” and therefore directly determined by genetic factors. The second interpretation, proposed by Cummins in 1926, considers ridge configuration as a product of epigenetic processes involving response to growth forces. This second interpretation suggests that ridge configuration is not directly determined by genes but rather is indirectly determined by adjacent tissues with which the developing glandular folds of the basal epidermal layer interact. These adjacent interactive tissues may include the volar skin itself, the soft tissue of the dermis, bone, and/or other epidermal appendages like the fingernails.

d) Several researchers have done embryonic studies of friction ridge growth and it’s been determined that friction skin patterns are formed by pressures present during the growth process. Since it’s impossible to duplicate the pressures on any finger or any person, no two fingerprints will ever be the same. The friction ridges stop forming during a certain time during fetal growth. Once the growth process stops the friction ridge arrangement is persistent until death and decomposition.

Discussion:

Answer a: Although this answer is a correct statement, this is empirical evidence. It shows what we’ve seen to be true but it doesn’t tell us why this event happens (it supports the scientific law but doesn’t give enough data to establish a scientific theory).

Answer b: The statistical studies done also support uniqueness but they don’t show why it’s impossible for fingerprint arrangements to ever be duplicated.

Answer c: It’s important to use explanations your audience can understand. This answer may be adequate for those in our discipline but it may be too much for a jury (an explanation quoted by W. Babler). Once jurors tune out what you’re saying, it may be very difficult to ever get them back. Be careful not to talk above the jurors level and be careful not to make something so simple that it appears that you’re talking down to them.

Answer d: This answer doesn’t mention the particulars of the friction ridge growth process but it does give accurate information that’s easy for a jury to understand. Answers a and b support the scientific theories established by research scientists but this answers states the research done to establish the scientific theories.

[Charles Parker]

Michele, I would have to say I agree with this one. C is too complicated for in front of a jury.

I like D.

[L.J.Steele]

Are you still getting legal challenges to uniqueness and permanence?

Wasn't there some question in the Mayfield OIG report about the studies underlying permanence for Lvl 3? Has that been shored up?

I had thought that the challenges were moving to a slightly different question -- the ability to reliably determine uniqueness.

One can confidently say that prints are unique when one has a plethora of good data -- a well done rolled 10 print from nail to nail, for example. But if one is working with a very small partial latent of poor quality, then I'd assume there may not be enough data to match or exclude.