Words mean things II: Exclusionary Boogaloo

[Boyd Baumgartner]

Over on the latest newsletter from the American Society of Trace Evidence Examiners there's an interesting article regarding a new term known as 'Exclusionary Differences' that came about via their subcommittee of the OSAC. (pg 11 in the newsletter)


The 30 second breakdown is that they used to define differences in terms of the of 'significant differences' and then switched to 'meaningful differences', while later combining the two into the term 'Exclusionary Differences' while burying and changing the qualifying language of 'substantial differences' in the definition and adding disclaimers that I would argue do the heavy lifting.


The 60 second breakdown is that there's such a concern about whether or not people will interpret words like 'significant' and 'meaningful' as being the result of statistical methods that the disclaimers associated with the definition are more meaningful than the definition itself.

The deep and ironic point is that more attention is being paid to the probabilistic aspect of language comprehension than models that describe similarity between trace samples themselves.

If the goal of expert witness testimony is to assist the trier of fact, this seems problematic. It seems we aren't the only discipline that is on a diet of word salad.

[josher89]

I'm not against their terms Distinguishable Sources and Indistinguishable sources

Distinguishable sources:
Sources are distinguishable when differences in detected peaks or relative peak intensities between
compared sets of spectra: 1) are outside the variability of spectra originating from the same source; and 2)
are not explained by considerations such as sample heterogeneity, contamination, different sample
conditions, or different sample histories.

Indistinguishable sources:
Sources are indistinguishable by spectral comparisons when differences in detected elements or relative
peak intensities between compared sets of spectra: 1) are within the variability of spectra originating from
the same source; or 2) can be explained by considerations such as sample heterogeneity, contamination,
different sample conditions, or different sample histories

[Dr. Borracho]

I'm not against their terms Distinguishable Sources and Indistinguishable sources

You're scaring me here, Josh old buddy old pal. Are you suggesting it's okay if latent prints go in that direction?

I can hear myself on the witness stand in a capital murder trial now testifying that

Latent A was distinguishable from the exemplars of John Doe,
but Latent B was indistinguishable.

Ai yi yi, yi yi, yi yi.

[Boyd Baumgartner]

Aren't their definitions really just technical speak for training and experience though?

1) are outside the variability of spectra originating from the same source; and 2)
are not explained by considerations such as sample heterogeneity, contamination, different sample
conditions, or different sample histories

What is the source of said variability? Is it not experience of what counts as ID/EXC?

Isn't 'Explanation by considerations...' just a combination of training and experience given the level of complexity? Think of it as rephrasing to 'what counts as sameness given the factors affecting deposition and recovery( aka complexity)?'

Couldn't these two ideas be encompassed by the phrase 'I have not seen nor would I expect to see this much disagreement in two items (prints, samples, items, spectra) that have come from the same source. And isn't the critique here just the same that examiner experience is privileged?


So of what benefit is there to retooling the definitions if the ability to overstate is just as present? The definitions may sound better, but do they perform better?

[Michele]

I remember years ago reading "A competent examiner is trained to recognize the difference between distortion within the impression and a true discrepancy..."

I wondered, is this true? How does a competent examiner recognize the difference? I'd love training in this!

Maybe examiners are a bit too confident in their abilities. The statement sounds good, but what are the rules to be able to tell the difference? Over the years I've found some clues that may exist, but visual clues are not always present. Even if there are signs of distortion, it doesn't mean that the distortion will cause false minutia to appear.

To me, relabeling things as 'exclusionary differences' sounds good, but where is the list of what is an 'exclusionary difference'? Without a list, the phrase is meaningless.

I wonder if this was brought up to the author? Even if the paper wasn't officially peer reviewed, it seems like most authors have someone review a paper before publishing. Was this not brought up?

I think this goes to show that just because something is reviewed and/or published doesn't mean that it's good information. Practitioners need to ask questions.

[Dr. Borracho]

Latent Print Examiners' eyes and brains work the same today as they did when the IAI was first formed in Oakland in 1915. An LPE today reaches the mental conclusion of "identification" (or "matching" or "individualization" or "indistinguishable") at the same instant as an LPE throughout the years of our discipline. Only the terminology and explanations have shifted and changed as we struggle to articulate the mental process we cannot even understand, much less explain accurately.

Aren't their definitions really just technical speak for training and experience though? . . . . . So of what benefit is there to retooling the definitions if the ability to overstate is just as present? The definitions may sound better, but do they perform better?

That's it exactly!

[John Vanderkolk]

The most fascinating aspect of OSAC that encouraged me to apply for membership, back-in-the-day, was the Scope, Bullet point "3.4, Harmonization: The OSAC standards efforts shall encourage harmonization to minimize redundant, overlapping or conflicting standards."

I agree, words mean things.

I anticipate there will be challenges facing the new OSAC Interdisciplinary Committee. That is why I applied for membership in that committee. Anybody else interested in applying? From their public website: https://www.nist.gov/form/osac-application-form

JohnV

[Michele]

John,

When the OSAC started, it was reported by Sue Ballou that all pattern evidence disciplines would have to be on the same page. Then when groups wouldn't compromise, that idea seemed to fizzle out. I heard they all claimed that they couldn't get on the same page because of the 'we are different' ideology. Are you saying that now they are going back to the original framework of the OSAC?

Michele

[josher89]

You're scaring me here, Josh old buddy old pal. Are you suggesting it's okay if latent prints go in that direction?

I was meaning in terms of trace evidence. They will sometimes get a homogenous submission of garbage and are expected to break down all of the components of what it contained - like DNA mixtures - so for those disciplines, it makes more sense.

I absolutely don't think we should be using the same conclusions as non-comparative disciplines because there is very limited correlation between them.

Significant / Meaningful = Toe-may-toe / Toe-mat-oh

[John Vanderkolk]

Michele, I look forward to seeing the complete job description for Interdisciplinary Committee. A small job description pops up on Interdisciplinary Committee within the application to join. I was enticed enough to apply.

As many in forensic comparative science know, I believe we are more alike than we are different, even though we are all unique. I believe in studying the source(s) (natural items, manufactured items, skin, shoes, tires, guns, tools, etc.) of our images first, then examining shapes within the images, no matter the discipline. We examine shapes. JohnV

[John Vanderkolk]

Josher89, you wrote: I absolutely don't think we should be using the same conclusions as non-comparative disciplines because there is very limited correlation between them.

Do drug chemists, DNA examiners, trace examiners, other examiners analyze the data generated from the unknown, analyze the data generated from the known standards, compare the data, then evaluate the significance of the analyses and comparisons to reach their conclusion? They often compare shapes within the charts and printouts that their technologies generate. There are correlations among us. We just do not talk about them very often.

[Dr. Borracho]

I believe in studying the source . . . first, then examining shapes within the images, no matter the discipline. We examine shapes.

Do drug chemists, DNA examiners, trace examiners, other examiners analyze the data generated from the unknown, analyze the data generated from the known standards, compare the data, then evaluate the significance of the analyses and comparisons to reach their conclusion? They often compare shapes within charts and printouts that their technologies generate. There are correlations among us. We just do not talk about them very often.

You are absolutely right, John. A DNA analyst examines a source sample with an instrument to generate a printout (image) which is then compared to other charts (images) to identify or exclude, although they use different terminology.

A chemist examines a source sample with an instrument to generate a printout (image) which is then compared to other charts (images) to identify or exclude, although they use still different terminology.

I wish you all the best in your efforts to unite more of the sciences on OSAC.

[John Vanderkolk]

Dr. Borracho, thank you! I do hope the OSAC Interdisciplinary Committee strives to encourage harmonization to minimize redundant, overlapping or conflicting standards. That hope is keeping me motivated to participate.

[josher89]

My limited understanding of how chemistry works is that they are looking for specific ions from a chromatogram or mass spectra - and their relative abundance or total ion count - before making a call of 'ID' or not. I was also under that same assumption of DNA analysts looking at electropherograms to determine if an allele is present or not. They make a call based on its presence or lack thereof across a set number of alleles.

What I've always tried to understand is how effective LPEs would be if our ridges were akin to brick walls and there were random mortar joints but everything was lined up. Then you could grid out a print, much like Galton did eons ago, and see if the break was there, or not.

In reality, since our ridges aren't straight, and there isn't a single 'light switch' choice of it's either there or not, we are faced with much harder statistics. "It" might be there, but "it" might be facing a different direction, or "it" might be there but "it" ends sooner or with a point instead of a rounded tip.

That was the basis of my comment and my naivety to how those other disciplines operate. Although I do remember an old organic professor that said back before computer assistance, their instrumentation would print out a chart of the ions or whatever and they would actually cut out the chart from the paper and weigh it and compare it's weight to a known standard and the weight of its graph on the same paper.

Believe me, I am all for the harmonization of terminology across disciplines, but too many people want to hold on to their terms and not give up comfort for responsibility so this Interdisciplinary committee is the first big step to making that right. I wish you and all others on that well and please give us the guidance we need.