CLPEX Chat Board

Does anyone else use a point standard?

~~~~~~~~~~~~~

January 21, 2010 http://detnews.com/article/20100121/METRO01/1210477

Detroit Police: Fingerprints on carjack victim's door match suspect's

STEVE PARDO
The Detroit News

Detroit -- A fingerprint expert testified prints pulled from the car door handle of a carjack victim matched those of De Marco Harris, a 13-year-old boy accused of murdering a 24-year-old Flint woman.

Marcia McCleary, with the Detroit Police Department, said the fingerprints pulled from the driver's door of Trisha Babcock's car were a 23 point match to Harris' fingerprints. A 9 point is the standard Detroit police use in determining a match.

"Having a 23 point match, kind of nails it," said McCleary. "The print I entered (into evidence) was a clean print, a good print."

Prosecutors allege Harris, who was 12 years old at the time of the Aug. 1 incident, robbed and fatally shot Babcock on the city's northwest side. Harris has been charged with the robbery and death with a provision that allows Wayne County Circuit Court Judge Sheila Ann Gibson leeway to charge Harris as a juvenile, as an adult or give him a blended sentence.

Police say Harris shot Babcock while she sat in a parked car at West Outer Drive near Minock. Police said fingerprints found on the window of the silver Buick led them to Harris.

At the time, Harris told police he shot Babcock during a robbery attempt.

"I ran up to the car and told her to give me her money," Harris told police, according to earlier court testimony by officers who questioned him. "She rolled up the window and grabbed my arm, and I shot her."

Harris later told police that Babcock had grabbed his .45-caliber pistol, not his arm, adding that he didn't mean to shoot her.


--------------------------------------------------------------------------------

Ann we have one for sufficiency (suitability) which in my PPOV we have a big problem with. It goes something like this:

Suitability Criteria
1. The sufficiency of a latent print will be determined by the quality and quantity of the friction ridge detail present.
2. Generally speaking, a latent print with 8 or more level 2 detail (L2D) should be considered suitable for identification when the quality of the features are fair and their presence can be demonstrated.
3. As quality of detail diminishes, the quantity needed for suitability will increase.
4. The presence of level 3 detail (L3D) can decrease the number of L2D needed.

We have to go with suitable for identification because not all the LPE agree with the philosophy of suitable for comparison (exclusion at a different threshold than identification).

We have no set criteria for Identification of a latent print, but if it can pass mustard for sufficiency and we have the correct and proper exemplar, it should be able to be identified. So I guess you could argue the position that they are linked.

That sounds very much like an 8-point identification standard. It seems to say that, if there are (disregarding L3 here) eight clear points, it is presumed that, if a record of the source individual is available, it will be identified, and that, if there are less than eight points, it won't be attempted, because seven is not enough for identification. If it said eight points was sufficient to compare, I wouldn't so readily read it as an identification standard and more just a cautionary minimum. But I can think of no other reasonable meaning of "suitable for identification" than "sufficient for identification." I could imagine that it really meant something like, "suitable to proceed with a hypothesis of identification," except that L3D is thrown in there to drop the number of points. That means to me that it's defining a vital tipping point, a critical threshold, and for us, the critical threshold is that for identification. And it seems to set that point without regard to such things as distortion, which if you're talking about some widely spaced details and a highly distorted latent with the relative positions way out of wack, eight might well be not nearly enough for comfort. I know. I know. They were trying to convey the idea of what's enough to seriously explore the prospect of identification and not intending to declare that eight is enough, and I doubt they'd slam an examiner who said he found eight points but wasn't convinced of the identification. But the only item on the list that was needed was: "1. The sufficiency of a latent print will be determined by the quality and quantity of the friction ridge detail present." If you don't figure your examiners are sufficiently expert or have sufficient judgment to determine what's worth exploring, you probably need something different in the way of examiners.

We can get so balled up in writing rules that there's little room for expertise beyond merely collecting data by observation, which seem to be pretty much where a simply point standard put things.* So some object to "suitable for comparison" on the ground that it takes only the observation of a small cluster of data that's just not present in the record impressions to exclude, and therefore they don't want to forgo a possible exclusion. Are we so bound up in what "comparison" and "examination" mean that we can't operate without dumbing it down? Maybe we walked ourselves into much of the contempt of critics when we said we're expert but felt we had to tediously instruct each other in just what it takes to form an opinion on anything and in the same breath said it's whatever the examiner says it is. And it's not like we can even agree on even a point standard, even as a point count below which we won't opine.

Maybe the proponents of a higher standard of nationally modeled school and a period of being deemed competent but needing close monitoring are right. Maybe "trained to competence" just isn't the same as being an expert.

* I've never quite understood the position that a point identification standard is invalid on account of not being scientifically supported, while an everyone to conjure their own standard on the fly is proper. If you argue that the apparent community conglomerated non-point standard is valid for having never been shown to falsely identify when proper interpretations are made, how can you not settle on a point standard that's subject to the same test. At the very least, the point standard is far less subject to poor guessing?

Most of Europe and Asia....

g.

(In the U.S. I encounter this "unofficially" all the time. I would be shocked if DPD has it written down anywhere...it is unofficially what is taught to examiners and that's what they learn.)

[Incidentally, I was under the impression DPD lab was shut down and all the work transferred to Mich State Police. The two events are unrelated...but you do the math.]

[Double incidentally, what of it? As long as the sole decision is not based exclusively on the counting of X points, as I can tell you that is not the case in Europe, they do consider L3D and they do allow rare circumstances to go below the threshold, but for your garden-variety run of the mill prints, is it so bad/evil/unenlightened to have a general threshold above which one is comfortable. I, of course, am being a bit provocative and teasing in stating it this way, of course. But Arie Zeelenberg (Holland) has made some great arguments over the years about using a standard threshold for decision making and for using point standards in general.]

Gerald, it is not so much about identification as it is trying to get a case through the review process. As part of accreditation we had to visually indicate those latent prints that a LPE felt were sufficient or suitable to work with. (Some LPE have informed us that we have misapplied that ASCLD-LAB criteria).

“The work with” part is a confusing since part of the unit has one standard and the other part has a different standard which is based upon where and by whom they were trained and introduced to latent prints. It has nothing to do with logic and reasoning. It is a current belief in what is right and what is wrong. Hard to defeat and get around.

Let me give you an example. If I created a “sufficiency proficiency” of 20 of the closet borderline latent prints and then sent them out to the LPE to mark those ridge characteristics that they could observe. Better yet ask them to mark those they could observe easily with hi-confidence and those with some reliability issue with low-confidence and when I get them back start counting up the markings. (Some LPE believe this is wrong as all points have equal weight---if you see it then it counts).

If I counted up the ones in a particular unit and the differences were maybe 1-2 ridge characteristics then I would say that is a pretty tight unit and they are all working on the same concepts and observations.

If I counted up a different unit and the differences on some of the latent prints were 3-6 ridge characteristics in difference then I have a problem with sufficiency. I have seen as much as 7-8 point spread in a latent print between two LPE with 20 years of experience.

When you have that much spread then you have review clog up because the reviewer believes one thing and the primary believes in another and one sees it one way and the other sees it another way.

So our little sufficiency rule has really nothing to do with identifications as the ridge characteristics and relationships necessary for identification are present while the sufficiency rule it just a way we are trying to resolve an issue with a single latent print that basically is unidentified but must be accounted for accreditation. However I do see where someone could argue that in theory it does effect and identification, but in practical application in the section it does not.

Just trying to get everyone on the same page when it comes to declaring a latent print sufficient or non-sufficient and I can tell you right now it still does not work very well.

I am open to any suggestions from anyone on how to improve a LPS or Examiners ability to determine what is good and what is not.

g. like you I have encountered a number of agencies that have the unwritten standard. Back in 80-90’s you had this FBI and Texas DPS Flyer come out where there were no set number of points needed for an identification. I am sure everyone has seen one. These were passed out at training schools. In 1999 when I started with the state lab the Latent Supervisor at that time called me into his office and told me up front that he would never allow any latent print identification out of this office with less than 7 points. I responded by pointing out the training paper he had been passing out for years and he stated again that nothing less than 7 points. The unwritten rule.

I believe if you take 3LD out of the mix, take out the rarity of ridge formation and relationship and only stick with ending ridges and bifurcation every LPE has a threshold. You just do not see any identification of 3-6 related ridge characteristics that are composed of just L2D of ending ridges and bifurcations.

There is just no set threshold because you cannot tell up front just what the mix is going to be made up of. You cannot tell scientifically what something is going to consist of before you have even had a chance to look at it.

I know. Less than entirely comfortable things sometimes have to be done in the real world. Sure, it can affect identification, but the effect is to maybe, once in a great while, keep an identifiable one out of the process, rather than tending towards frank error. Just like in other things. Investigative units have criteria for assigning a case with the intention that it go beyond a follow-up to see if the reporting officer neglected something. You might, from time to time, actually be able to clear one of those sparse cases, but the overall effort would suffer.

I can understand the state lab supervisor's position. I can see it as a statement reflecting his experience and judgment that he's not prepared to accept that anything less then seven points can be used to identify. (And being the guy ultimately responsible, he gets to say.) While that may deny an idea of what may be believed to be rare formations, if we consider rarity, we're adding something into the data that we really don't know enough about to apply with much confidence. The significant thing about it is that it's an "unwritten" rule. Perhaps it's fear that it will drag you into conflict with some other examiner who presents an identification with six, becoming an unwilling witness against that examiner. (He obviously considered that some examiner might identify with six points, or he wouldn't have taken the position. Six was acceptable, not advocated, to be sure, but acceptable, in the course instructed by the same agency when I went through it. Perhaps an avoidance of conflict sort of acceptance, come to think of it.)

If there's anything to the fear of dispute being a concern, it's running a bit close to withholding an expert opinion in favor of the state, since very nearly every identification offered in a case goes to weight against a defendant. (Real world, again.) Just about everything you can say with regard to what has to be overcome to support an identification is really a limit on what can be used to accuse. If your expert opinion is that six is just not enough, you can't mean anything but that it's not enough for anyone. If you say it only applies to your unit, you can only mean one of two things, that you're willing to maybe pass up some identifications, or you don't mind other units presenting identifications you think are unsupportable. Or maybe he was just aware that, especially with the more elite units, the threshold for identification can shift ever lower over time, and he felt like that was a pretty safe lower limit. (Although I think the threshold drift has more to do with interpreting individual details and drawing imaginative relationships among details than the actual quantity used.)

We're a lot less independent than we like to think. We want to think we operate without much of a bias load and form completely independent opinions. But we also like the comfort of the herd. We need the herd to validate, in a circular way, that we conform to the herd standard and that it's a good standard because everyone in the herd more or less conforms to it and agrees that if you're reasonably close to the center of the herd standard, it always produces the correct result. (It does, in fact, seem to do that, according to all our negative evidence.) And if you, on your own best judgment, don't conform, for instance maybe being much more conservative than the rest of your herd, you're not going to be allowed to remain in the herd very long. And if National Geographic teaches us anything, it's what becomes of a herd animal without a herd. None of that's necessarily a bad thing at all. It tends to clarify our view of who is competent and it makes use of everything we presently know. And it keeps the herd functioning and the members from dashing off in different directions to be eaten. Physicians do the same thing. The overly timidly conservatives and the fringe renegades aren't welcome. Of course, we work within a system where someone willing to say the herd is wrong can always find some work. But those with the credentials to take those positions and be heard did themselves begin in the herd and drifted away or just never liked the rest of the herd, anyway.

Edit

Charles Parker wrote: Let me give you an example. If I created a “sufficiency proficiency” of 20 of the closet borderline latent prints and then sent them out to the LPE to mark those ridge characteristics that they could observe. Better yet ask them to mark those they could observe easily with hi-confidence and those with some reliability issue with low-confidence and when I get them back start counting up the markings. (Some LPE believe this is wrong as all points have equal weight---if you see it then it counts).

If I counted up the ones in a particular unit and the differences were maybe 1-2 ridge characteristics then I would say that is a pretty tight unit and they are all working on the same concepts and observations.

If I counted up a different unit and the differences on some of the latent prints were 3-6 ridge characteristics in difference then I have a problem with sufficiency. I have seen as much as 7-8 point spread in a latent print between two LPE with 20 years of experience.

When you have that much spread then you have review clog up because the reviewer believes one thing and the primary believes in another and one sees it one way and the other sees it another way.

I recall a study from g that did something along these lines, asking examiners to mark minutiae on a set of latent prints. I also recall that some latent prints had a broad range of minutiae marked say from a low of 7 to a high of 14/15 for the same print. Wow, what's going on with that? However, when asked to mark minutiae with a confidence level (high, medium, low), g found that the examiners on the low end (those marking around 7) were only marking high confidence minutiae (i.e. very crisp/clear). Those at the high end 14/15 would mark high, medium, and low confidence minutiae. If you consider the minutiae marked by confidence, then the examiners showed some consensus. g, please clarify/correct me if needed.

The point being, if examiners aren't agreeing on the minutiae count, then maybe there is a degree of information missing. Otherwise, I find minutiae counts/numerical thresholds quite useful. Just don't let the examination process degrade into arithmetic.

To ER, I would say that numerical thresholds are quite appropriate. In fact, they are based on the same empirical foundation currently espoused by SWGFAST and other authoritative bodies, namely experience. Think about it, our current standards for conclusions are based on some intangible idea of "sufficient" quantity and quality of detail which we determine based on our training and experience. Suppose then you work in a laboratory where IDs are routinely charted as part of the case notes. You then go back and sample the charted IDs for the last 5yrs and discover that the average identification has a range of 8-12 minutiae. Then you have an empirically sound basis for stating that, "Generally, eight minutiae (or more) are needed to make an ID."

If it's based on your experience (what you are already doing), then I don't see the problem. I suspect the issue becomes how this finding is articulated. For example, "I called this an identification because the two impressions have eight minutiae in common." That is the number of minutiae itself should not justify the conclusion (I only say this because there is no rigorous data to show the numerical error rate for a comparison based on "X" number of minutiae).

Any how, I like minutiae. The more I have, the better I like it.

16 points:

I recall a study from g that did something along these lines, asking examiners to mark minutiae on a set of latent prints. I also recall that some latent prints had a broad range of minutiae marked say from a low of 7 to a high of 14/15 for the same print. .......If you consider the minutiae marked by confidence, then the examiners showed some consensus. g, please clarify/correct me if needed.

Correct you are.

I did that study and now have drafted it for review. Hope to be submitting for publication this year. It's really interesting. I have been sitting on it for some time because we wanted to see if we can offer some solutions to reduce variation. I think we are beginning to crack this nut.

To give some numbers to you for the latent print you are referring to, keeping in mind analysts were marking up just a latent print with no exemplar during the Analysis Phase.

The ranges of total minutiae marked (by trained casework working examiners, N = 24) were
High confidence = min 2, max 14, avg. 6.3
High + Medium confidence = min 3, max 18, avg. 10.3
High + medium + low confidence = min 4, max 23, avg. 12.4

There was more consensus for the high confidence minutiae and a lower error rate too for the number of falsely marked minutiae (they don't exist in the ground truth source friction ridge skin).

Also please recognize we have not published and peer reviewed these data...so....treat appropriately until we have submitted.

g.

But don't we all have a point standard? Not a point threshold defining identification, but a number below which we would not be convinced. I think if I started to count down from, say, twenty, asking if, all unit policies or legal standards aside, you could ever comfortably determine the source of a print with fewer than ___, wouldn't the answers likely begin with "yes" for twenty, and at some point, you would say "no, never"? (Saying "maybe" is a "yes," since you could conceive of being conclusive with that number.) That's a very different thing from asking if the last number that got a "yes" was always enough. I suspect the range of smallest numbers would be pretty narrow, but it would be interesting to know how much effect training and policy has on that number. For instance, where examiners' official reporting is constrained by some higher point count, do they nevertheless find themselves confident of an identification they can't report out? Do examiners adopt their instructor's standard, even if the number is not stated but tends to become the actual minimum in practice? Since we almost inevitably do count, even if we don't depend on the number, does experience produce a number that becomes a comfortable minimum? And how much is individual experience, and how much is communal experience?

I think I'm interested in this, because I'm not entirely comfortable with the nature of the quantity/quality combination. Quality means to me how confidently an examiner can determine the nature of a feature or a relationship among features. When we imply that quantity can compensate for quality, we risk encouraging reaching for count. The danger is on account of there being no practical standard for determining what physical feature is represented by a possible impression of that feature. Of course, there is an articulable standard that can be offered. That standard is that, among all the physical configurations that could have made that impression, including the possibility that it's not an impression of a physical feature, only a genuine feature of a particular character could have been the source. That is very like the standard for identification. We don't claim to know to an absolute certainty that only that individual could possibly have been the source, but we claim that the source skin is exactly of that character. The actual association of an individual is simply drawn from expert experience that convinces us sufficiently to name the source.

That last part is a pretty well-founded assumption behind our confidence in the identification. We have plenty of experience with very clear latents to draw upon in judging how much is enough. But accumulating features that might be one of two or more different characters of feature is not something where our experience can guide us. Almost all the experience would be based upon our identifications. Our confidence in using ambiguously impressed features would be grounded in conclusions drawn from using ambiguously impressed features. This is, in fact, one of the most common sources of error, the misinterpretation of individual features. It is one of the most insidious modes of error, because it can appear in the work of very experienced examiners who very naturally are considered to be more adept at working with the "hard ones." The examiner who makes such an error is appropriately demanding with regard to the amount of data accumulated during Analysis. The community standard for the threshold amount of data required is pretty consistent and obviously sufficiently high. But, as the cited study suggests, there is no similarly consistent community standard for "identifying" impressions of source details.

If we consider this in the realm of scientific inquiry, I think it has to be considered that ambiguous data can only accumulate so as to increase the ambiguity of the result. Tolerances accumulate to wider, not more narrow, tolerances. No amount of ambiguous data, no matter how many data points, can combine to determine a precise result. If I'm going to determine which possible results might fit the data, every data point that could be something inconsistent with a particular result must be excluded from the data used to produce that result. And no amount of clear data that does not accumulate to satisfy the threshold of identification can determine which alternative interpretation to use for an ambiguous point. If using the clear data left us short of identification, how can we justify taking the identification as concluded and using that to decide what we're seeing? If we are, to throw out a number, 80% confident of our interpretation of an apparent impressed detail, how can we use it to accumulate to a 100% conclusion?

Now, most of the time when we do that, the result turns out to be correct. Sometimes it does not, though. Much grief has come from those. How, then, do we justify a threshold of very high confidence with regard to how much data presumes an identification while admitting a lesser confidence in the production of data? Perhaps we don't really accept a lesser confidence in interpretation. Perhaps the problem lies more with how we become confident with an interpretation. But I think there's some serious thinking to be done about what goes into choosing among alternative interpretations of an impressed detail and how often it is influenced by the improperly circular use of latent-record agreement in the sub-threshold clear details. Maybe more importantly, how often the ACE sequence is broken to do it.

One thing that I wonder about the cited study is just what the subjects meant by "low confidence." I could imagine it referring to confidence that it's a detail at all. ( I suspect that's what it was, because there apparently was a significant incidence of marking false points.) I could also imagine it meaning it was unquestionably a detail, but they couldn't confidently characterize it. And I could imagine it meaning they could characterize it as to nature, but maybe couldn't say, for instance, if it was a bifurcation or a curving ending ridge. It may be that some might call one "low confidence" when they could actually confidently determine enough to use it, just maybe not being able to say if the curving ending ridge in that feature in the source skin quite joined up to become a exactly proper bifurcation. And I presume they weren't asked which minutiae they would actual use toward a conclusion. That's quite important, and may by a good example of how the value of any study is generally how it directs the next one. And it matters more because they were marking some false ones. What may be most significant is the degree of consensus in high-confidence marks. We do indeed want to have as consistent a community standard as possible, and if the data scatters a lot as confidence decreases, we learn something about how to propose the standard of interpretation.

Charles Parker wrote:
Ann we have one for sufficiency (suitability) which in my PPOV we have a big problem with. It goes something like this:

Suitability Criteria
1. The sufficiency of a latent print will be determined by the quality and quantity of the friction ridge detail present.
2. Generally speaking, a latent print with 8 or more level 2 detail (L2D) should be considered suitable for identification when the quality of the features are fair and their presence can be demonstrated.
3. As quality of detail diminishes, the quantity needed for suitability will increase.
4. The presence of level 3 detail (L3D) can decrease the number of L2D needed.

I wrote that policy. At the time it was written, there was the problem within our section of some LPE marking latent prints unsuitable (for anything) that had quite a bit of features of high confidence present. They were passing on them, I suspect, for some of these reasons:

1. They may have already identified the subject to another latent print in the case.
2. Part of the latent print was smeared or distorted and would have caused them to exert more effort in examining it.
3. If they had marked it as suitable and then couldn't identify it due to poorly recorded exemplars, they didn't feel good about going with inconclusive.
4. There is no in-office supervisor present here to oversee the actual work, so there was no one who could make them tow the line - after all, it's their opinion, right?

The problem this was causing us was in the technical review phase, where reviewing LPE could not concur with the conclusions because perfectly good latent prints were being left unmarked and uncompared. When you pressed them on it, they would stubbornly maintain that they just couldn't see it. That is why this policy was written.

But what Charles Parker quoted is not the entire policy. It goes on to outline a process of conflict resolution where the reviewing examiner in the case of a Murder, Armed Robbery, or Sexual Assault would take an unmarked print he felt was suitable back to the primary examiner to confer and try to come to an agreement as they both view an enlarged image of the print. If they couldn't agree, then the final decision would be made by the assistant lab manager (who has been an LPE in the past, but who is not currently practicing and whose office and responsibilities go far beyond just our little band of crime fighters).

As I read the policy I wrote, I admit it is poorly worded and I hope to make changes to it with the help of the other LPE in this office (Or maybe we should just scrap it). But it filled a need in a section where a supervisor is not closely monitoring, everything needs to be decided by consensus, and cooperation, reasonableness and the ability to be yielding is sometimes lacking. This policy was never meant to be considered as a point standard. In fact, I tried to word it so it would not be construed as such. The heading, you will notice, is entitled "suitability criteria" and is general enough to not pin down a specific number. I do dearly regret, however, using the phrase "suitable for identification" when I meant "suitable to be compared".

But I am one person. When it was written, a draft was passed around to everyone, we had a section meeting and discussed it. Probably not everyone liked it, but in the end, it was approved by the majority of our examiners and the assistant lab manager. If I was way off base with this, perhaps someone should have taken me aside and told me so.

Dennis,

Thanks so much for sharing the "origin story" of that policy. You did so with great honesty and humility!

The 4 suspected reasons leading to the policy hit home, b/c I have seen those reasons time and time again in various agencies. ANd #3 (discomfort with using "inconclusive") is probably the number one issue I see (and further complicated by as you described lack of leadership, enforcement, understanding and consensus).

Furthermore in agencies, there is a huge misunderstanding that comes from "inconclusive" and after some painful misunderstandings myself, it comes down to, as you noted, understanding "suitable for comparison" v. "suitable for identification". In some agencies, latents are ONLY retained if it is generally expected, that if presented with GOOD exemplars you WILL be able to exclude or identify. In other agencies, a more liberal approach is taken, that more "thin" latents are kept, compared, but one cannot reach a definitive opinion of exclusion or identification. Here "inconclusive" is also reported as an "i don't know" and it has nothing to do with known exemplars.

My post here is not to advocate either approach. One is not more correct or more scientific than the other. It is purely administrative. The solution is to understand the two approaches and know which one is in play. SWGFAST is working on trying to sort this out and falls along similar lines: choose which approach you are going for, spell it out, and then stick to it.

And Dennis, that seems to be a reason too for your policy, is different people in the SAME agency were applying this DIFFERENTLY. And that's a problem. I can respect different AGENCIES choosing different approaches, but you can't have both approaches in the same agency. Or you end up doing exactly what you did to hash it out!

Kudos though for trying to solve that conundrum and actually writing a policy on it.

g.