OSAC 5 Conclusion Scale/Research

[Boyd Baumgartner]

Isn't Support for Same Source saying the same thing as reporting an AFIS search that, while having some clear corresponding data yet not overwhelmingly so or at least not enough to ID, but still reporting a name and calling that an investigative lead (with the caveat that it's not an ID)?

Therein lies the problem. Are you saying Support for Same Source (SSS) associates a person to a case? Are you saying that I can't associate the person to the case but there is too much similarity to exclude it? And how is this threshold determined? Does it take into account similarities in pattern force areas? Again, was Mayfield a 'Support for Same Source' or an 'Investigative Lead'?

The paper indicates that there was a concern regarding the conclusions considered 'erroneous support for common source' and that:

If a detective or prosecutor interprets these qualified conclusions as something more definitive, this has the potential to lead to a miscarriage of justice.

It's not obvious from the OSAC document definition on what support for same source actually means, which was my point. Lacking guidance or criteria, the conclusions lack meaningful application.

Support for Same Source is the conclusion that the observations provide more support for the proposition that the impressions originated from the same source rather than different sources; however, there is insufficient support for a Source Identification. The degree of support may range from limited to strong or similar descriptors of the degree of support. Any use of this conclusion shall include a statement of the degree of support and the factor(s) limiting a stronger conclusion.

In the published paper that Tom linked, Figure 2 and Figure 4 seem to frame the problem correctly and that is: In the 5 scale model, how do you properly sub-divide Inconclusive in the 3 scale model? If it is as in Figure 2, then you need to have more clear criteria on what counts as a weak Identification vs a strong Identification and the QA that goes into those conclusions. There are certainly risks and trade offs with each model.

3to5.jpg

[4n6Dave]

We currently use a 4 point scale. There is room for the 5th point in there, but it just doesn't come up.

Our conclusions are: Identified, Cannot exclude (support for same source), Inconclusive, and Exclusion.

We also use incomplete to describe when we need better exemplars but I don't think that really fits into the scale as it is not a final conclusion.
Our conclusion scale and been published and peer reviewed and in use for a few years now and I don't think our latent print section has imploded.

Our analysts are comfortable describing when there is agreement but not enough to identify a person and also what the identification means.

The shift to a 5 point scale will have an impact on conclusions and the thresholds for those conclusions but I do not see it as the end of the world.

[4n6Dave]

Boyd,

I would ask what are your clear criteria for an Identification now?

Or what is meant by an inclusive now?

Or what is the standard for an exclusion?

Thanks,
David

[ER]

Boyd,

You raise a very good point that agencies should consider when moving to an expanded scale. However, I would suggest that since the 3-conclusion scale does not have clearly defined lines separating the categories, that translation from 3 to 5 looks more like the below image. One of the benefits of a 5-conclusion scale is to provide additional clarity when a comparison would fall into the blurry zone on the 3-conclusion scale. A comparison that's right on the boundary between ID and INC now fits comfortably in SSS. A downside to this approach is that there are now 4 blurry zones instead of 2. However, each agency should weigh the pros and cons for themselves. There is room for both approaches to fall under the same standard. (Look to Approach 1 and Approach 2 in the same SWGFAST standard as an example.)

I may be wrong, but it seems like you're suggesting that the lines between the categories of a 5 conclusion scale must be very clearly defined in order to be valid. However, the existing lines in a 3 conclusion scale are already very blurry between agencies, examiners, and even day-to-day with the same examiner.

I understand and share your concerns that there is potential for misunderstanding leading to miscarriages of justice. However, I don't find that argument convincing because many agencies are already reporting an expanded scale and because multiple other disciplines have been utilizing this approach for decades.

[ER]

David,

First, great questions about what the criteria or standards are now. Completely agree.

As an examiner at an agency that's using an expanded scale, I'm curious how you view the proposed standard.

• Is your agency considering adopting the terms and definitions?

• What do you see as the benefits or drawbacks of changing your "Could Not Exclude" to "Support for Same Source"?

• Are there any challenges that would prevent that change?

• Do you plan on expanding to 5 or staying with a 4-conclusion scale?

• Now that OSAC removed the "Draft" status of this document and changed it to "Proposed for SDO", does that change your agency's timeline for changes to your conclusions?

[4n6Dave]

Is your agency considering adopting the terms and definitions?
We are not currently looking at switching to the 5 point scale

What do you see as the benefits or drawbacks of changing your "Could Not Exclude" to "Support for Same Source"?
My main concern with "Support for same source" is that is sounds like "It is him" or "match" where I think that "cannot exclude" sounds more like "I can't say its not him." With the expanded scale our intent was to avoid the jury/investigators over valuing the weight of the evidence. We had originally thought about inconclusive with detail in agreement, Cannot identify, Detail in agreement - insufficient for identification, and a couple others. Cannot exclude seemed to communicate the meaning more directly while limiting the risk of overstating the evidence.

Are there any challenges that would prevent that change?
If you mean court challenges, I have not seen any. I think we could implement the 5 point scale. I dont think we would end up using the support for different sources however. Our suitability guideline for what we mark of value makes it so most of our latent prints should be able to be excluded. This is where our "incomplete" conclusion comes in.

Do you plan on expanding to 5 or staying with a 4-conclusion scale?
Current plan is to stay with 4 "terminal" conclusions. But we are adaptable if there is widespread support for the 5 point scale and it becomes common practice. I would like the OSAC to add Incomplete as an official results.

Now that OSAC removed the "Draft" status of this document and changed it to "Proposed for SDO", does that change your agency's timeline for changes to your conclusions?
No. Our current definitions of Identification (source identification), Exclusion (source exclusion), Cannot Exclude (support for same source), and inconclusive all fall in line with the document. I am not sure what they mean by a statement of the degree of support in the support for same source. Since we do not have a generally accepted way to measure agreement I am not sure how to qualify that.

[ER]

David,

Thank you very much for your responses. I believe there are many agencies in a very similar position to yours.

One last question:
I understand your points regarding "could not exclude", and that language closely resembles wording from the DNA discipline. If most agencies that report this type of limited association adopt SSS as the "standard" term, would that influence your agency to change that term? (You'd already answered that you are adaptable with widespread adoption of a 5 scale, so I was curious about use of individual terms.)

Thanks again,

[4n6Dave]

If "support for same source" became widely accepted and there were not issues with our customers or juries over weighting the evidence then I would switch.

When it comes to things like support for same source I worry that it will be like hair exam in the past where the analyst says one thing and the wording allows the prosecutor to say another. I know we cant eliminate the prosecutor from fluffing up our testimony in closing but I would like to limit the risk.

[Boyd Baumgartner]

I would ask what are your clear criteria for an Identification now?

Or what is meant by an inclusive now?

Or what is the standard for an exclusion?

Based on your answers, it sounds like our scales have some overlap. Here's ours.

Conclusions.jpg

Here's the criteria for how we rate the complexity of an ID. All the greens need to be checked for Basic. Any occurrence of an orange triggers an Advanced rating and any occurrence of a red triggers a Complex rating.

ID-Complexity-.jpg

Complex IDs get a technical review above and beyond verification and examiners are encouraged to utilize blind testing or consultation when dealing with a complex ID. There are times when QA measures result in disagreements and those go through a consensus judgement where 5 examiners chart what they see and create a consensus chart from which a conclusion is rendered. Obviously, all documentation is kept in the case for transparency. So the fuzziest boundary we have between scales is the Inconclusive: consistent but insufficient and the Complex ID rating.

I think our conclusion scales admit to the most transparency of the comparison process. 10 Print style IDs are not the same as a Zero Point ID or the Daoud ID. There's also a difference between something that falls just below an ID and a Complex ID especially when you know your testimony is going to be spun to the most favorable framing by the prosecution/defense. It's better to frame it yourself that you can't say it is that person and you can't say that it isn't that person, but here's what I saw. (we require charts on Inconclusive: consistent but insufficient conclusions)

[LPE123]

I have a case on my desk now in which I have numerous idents to two subjects. I have one latent that is well above the core almost all the way to the nail. I cannot find any target in the latent, no matter how far down toward the core I go, that I can find in any of the knowns, no matter how high above the core I go. I am not comfortable excluding at Level 2 without major case prints.

However, the ridge width, ridge thickness, and ridge texture of all the ridges in the latent are substantially different (thinner and closer) from the width, thickness, and texture of ridges in all of the known prints. We use a three conclusion scale and exclusions must be made at Level 2. Purely subjectively, I believe this is an exclusion using Level 3, but I cannot say that. I think that if our policy were five levels, I could reasonably justify "support for exclusion."

Thoughts? Comments? Does anybody have policies that allow this type of conclusion based on Level 3 as described above?

[anwilson]

LPE123,

what you're describing sounds like a tip impression. If I'm correct in that then the conclusion at our agency would be to report it out as Incomplete - need major case prints. I'd be hesitant to exclude on the data you're describing because ridge thickness and shape isn't as robust but I've definitely run into a similar circumstance. If you had major case prints and still couldn't find a clear target group, our agency conclusion would be Inconclusive - inability to locate due to ambiguity as it wouldn't fit our exclusion criteria of needing a clear target group as well as known area and orientation.

[LPE123]

what you're describing sounds like a tip impression.

Yes, of course. I didn't say "tip" because it represents about a 45° touch, not a straight down tip. Under my agency's policies, we report it out exactly the way you describe.

My question was to try and determine the feeling of a few other folks on whether you think we should report something other that "need major case prints" in that situation. To a more firm conclusion, what is your subjective feeling about excluding on something other than Level 2?

For example, if you have a latent that is so small in overall dimensions that it is clearly left by a small child, but you don't have that area of friction ridge detail in the inked prints of a grown male suspect, would you exclude the suspect based solely on physical dimension of the overall print?

Then, to take it to Level 3, if you had a latent of a tip left at about a 45° angle that does not go all the way down to the core, as described above, and you have inked prints that do not go far above the core, so that you don't have corresponding area of overlap between the latent and the inked, would you personally form a belief that the very fine ridges in the latent could not have been made by the same person who left thick, wide ridges in the inked prints from an area of friction skin just a fraction of an inch away from the area of the latent?

And the final question, if you form that personal belief, should we be able to report that as "supports different source origin" -- not a full exclusion, but something stronger than "inconclusive?"

[Boyd Baumgartner]

We use a three conclusion scale and exclusions must be made at Level 2. Purely subjectively, I believe this is an exclusion using Level 3, but I cannot say that. I think that if our policy were five levels, I could reasonably justify "support for exclusion."

Reasonable justification without standards for what counts as reasonable is meaningless. Did the prints included in the 7.5% erroneous exclusion rate in the black box study offer themselves up as unreasonable and unjustifiable when they were reached?

Then, to take it to Level 3, if you had a latent of a tip left at about a 45° angle that does not go all the way down to the core, as described above..

If you don't have an anchor to say how far above the core you are and you have unstable properties like impression shape and ridge width, the most prudent thing to do would be to ensure that you have the full tip area as contained by MCPs.

Don't forget about our good old friend CTS Test 10-516 whose total area and clarity far exceeded most tip impressions with only level 3 visible and how that went over. So, why push it without a more thorough examination?

[tombusey]

I apologize for going dark the last two months, but I had to prep for my online teaching. Now that I've survived the first week, I'd like to offer my thoughts on the really excellent discussion that you have been having in my absence.

First, if I can determine identities from usernames, it is really great to hear from some of my friends whom I've been missing. It was really hard not to have IAI this year. My year was complicated by brain surgery in February, and so I missed the Academy meeting as well. It will be great to chat and drink (and dance!) with folks again.

I really like some of the nuanced scales that folks have been putting together in this thread, and they make a lot of sense. I agree that there probably shouldn't be a one-size-fits-all conclusions scale (intelligence work and property crimes have different needs, for example), and I hope one take-away from our research is that there is a need within each agency to validate any change that is made to a conclusion scale. Our study simply demonstrated that if you change the number of items in the scale, people change how they use Identification. That seems a little scary. With respect to whether folks should overturn convictions that were done based on prior scale usage, that is a fascinating topic that while I think would be a procedural nightmare, should not simply be rejected. Lots of cases get overturned when science improves. In our case I don't think what was done was egregiously bad, at least not enough to warrant a new trial, but I wouldn't rule out at least looking at the facts of the case if some really extreme language was used. Let's assume that is above my pay grade and move on...

Note that this published study was the pilot data for a grant that was funded by NIJ to look at different conclusion scales in fingerprints, footwear and toolmarks. If you are an active practitioner in any of these disciplines, we need about 15 more fingerprint examiners, 30 toolmark examiners, and 30 footwear examiners. Just email busey@indiana.edu to get started. Feel free to recommend colleagues as well. There is a $30 Amazon gift card honorarium, and you can participate in more than one study if you are multi-disciplinary.

Thanks to everyone for your comments and I promise to check this thread more often.

I can't help but end this post with a question:

Do you think evidence accumulates along a single psychological dimension between 'evidence for same source' and 'evidence for different sources'? If not, what do you think the second psychological dimension is?

-Tom

[Bill Schade]

Tom

If I'm understanding the question, then I would suggest that the second dimension in all of this discussion is the "relevence" of the conclusion to the investigation. (relevence to both sides, prosecution and defense)

Agonizing over a conclusion for a blood print at a homicide is alot different from worrying about the fifth print comparison to a subject already identified on the hood of a stolen vehicle.

Bill Schade