CLPEX Chat Board

I think there's a bit of cross-debate here. I fully understand the concerns that some examiners have about reporting results in the vein of SSS, IwS, or Could Not Exclude. All being different phrasing of similar situations. The is some associative value that falls short of ID. Some have already decided to go down this road and report it. Others have not.

The new standard is a separate issue from the question of whether, how, and when to use this limited conclusion. As someone who testified to it, 4 points that all match would still be regular Inconclusive.

The new standard is simply trying to talk to examiners where they currently are. (Except for supporters of "Exclusion of All Others". Like the IAI and SWGFAST, it says to knock that off.) IF (and only IF) you already have wrestled with the complicated issue of limited support and have already decided to do it, then here is the common language to use. If you only use ID/EX/INC, then here is the common language for that.

The debate for SSS is over at some agencies, and they want a common term and definition. This document provides that. The debate over SSS continues elsewhere. This document makes no requirement on those agencies.

The debate over the merits of this document and the debate over problems/benefits of SSS are different and separate debates.

The debate over the merits of this document and the debate over "Exclusion of All Others" is very much the same debate. One that should have ended 10 years ago, but unfortunately continues.

So, obviously I can get a bit passionate about our field, and I hope y'all can figure me that flaw.

Instead of going around on the above points again, I'd like to pose a new question in the hopes of moving us all forward.

For those that currently only report ID/INC/EX, do you see any significant difference between the terms and definitions used in this new standard and the existing document from SWGFAST? What issues do you see with the terms and definitions for those three conclusions in this new document?

Bill,

I agree with you

but without the "flavors" I think we have a problem.
4 similarities in common is not the same as 12. And the type of similarities would have an impact... I think that without defining the flavors of support we are leaving out an important part of the equation.

I like that the document allows for examiners to adopt SUPPORT FOR SAME SOURCE as a singular category, if the agency chooses to. I think this makes it more digestible and an easier barrier to entry. But I think the real value of a verbal LR scale (limited, moderate, strong, extremely strong) is to describe (without numbers) the magnitude of the weight of evidence. As you point out, 4 average minutiae in correspondence do not have the same weight of evidence as 12. Calling them both SUPPORT FOR SAME SOURCE, in my opinion, creates the same thing we have right now: one giant category of Inconclusive.

I'd like to see more agencies adopt SUPPORT FOR SAME SOURCE (but also use the optional refinement). But I'd concede the option just to get more agencies using this consistent LR language. If we could move in this direction it helps to describe relative magnitudes of weight and gives common language that has been in use in UK, Europe, and academia for 3 or 4 decades.

g.

A friend emailed to ask my thoughts on the proposed/evolving five conclusion scheme.

A different friend on the Human Factors Working Group explained to me several years ago how a senior professional in virtually every field of endeavor has a relatively high “error rate” early in their career, but the ER goes down quickly as the person becomes an “expert” in the field. My friend explained how the ER stays low throughout a career until near the end, when it starts going back up again. My friend explained that it is not age or senility or carelessness or forgetfulness that ramps up the errors at the end of a career, but inflexibility. A person has been doing things the same, but policies and procedures have changed, and the senior person refuses to change with the times.

Thomas Kuhn noted that when talking about “paradigm shift.” Kuhn discussed two kinds of science, “normal science” and “revolutionary science.” Normal science is the day to day operation of a discipline using dogma on which everybody agrees. Revolutionary science is the replacement of the old dogma with new dogma. Newcomers devise what they believe is a better way to do the science, but the old guys in the discipline continue to insist on doing things the old way. When the new guys finally outnumber the old guys, the science changes.

Max Planck was somewhat more blunt. He said basically that the old guys never change, they need to just die before change can occur.

In another current/recent thread, Boyd opens a discussion of bias in a broader context than fingerprints or even forensics itself. Boyd initiated that tread on the “Bias of Bias Research.” Specifically, Boyd challenges Dr. Dror’s infamous conclusion that all fingerprint examiners are too biased to be trusted based on his sample group of five latent print examiners. I responded to Boyd by citing a concurrent trend in societal changes, which I posit is mirrored in the move to statistics in fingerprint identification, including by implication the five conclusion scheme. My earlier comments on that thread are completely apropos here:
viewtopic.php?f=2&t=2512

But to get back to the “five conclusion scheme” under discussion here, how do I feel about it? In a word, ambivalent. To give credit to the new kids (okay, maybe not so new anymore), I see where you’re coming from. In the context I discussed in the Bias of Bias thread, I believe the move to statistics in fingerprints is but one very tiny facet of broader societal changes. Yet much to my career detriment, I find myself living as one of the old timers in Thomas Kuhn’s description of a paradigm shift.

So I told my friend I am sitting here in resignation, just wishing I were on the North Fork of the Gunnison pulling in a few stocker trout for the evening campfire. Of course, I say “in resignation” meaning that I am resigned to the fact that in spite of what I really believe, change is coming. I have not actually resigned from the field yet. I figure I’ll keep on doing the job the way I have for almost fifty years (I’m aiming for that landmark goal), then I’ll see if I can’t find a comfortable boulder on the banks of the North Fork of the Gunnison where I can sit with a hip flask of scotch and catch a few of those little trout.

Unless I succumb to Max Planck’s curse first.

Just a few points from comments to my earlier post.

1. "Unable to locate" as *I'm* using it in my comment means that I have enough exemplars, I didn't find it, but I don't have a focal point. I would not expect to find it if I were given more exemplars. I do agree that if I feel I need more/better exemplars, that would NOT be support for different source and I would report out INCONCLUSIVE (need better exemplars). This is exactly why we need standard terms - because we're not even using "failed to locate" the same way. Both Michele and Glenn interpreted my use of that term as something else.

2. I also support documenting the complexity of the examination. However, expanding ID's into simple, advanced, complex seems to relay trustworthiness of the examiner's DECISION. IE - how likely are we to be right because of the complexity of the exam. It does NOT relay the weight of the EVIDENCE. (I'm going to throw in some numbers here for the sake of discussion, please don't take these numbers as dogma - they are simply to help draw a picture). I can have 45 L2D and come to an IDENTIFICATION but it could be an advanced or complex comparison if there is a lot of noise, maybe multiple touches, some superimposed ridges, etc. Alternatively, I could have between 6 and 12 very CLEAR L2D and go with SUPPORT FOR SAME SOURCE, not an ID, but it could be a very simple comparison because the print is high quality.

3. Dr. Borracho - sitting on a boulder with a hip flask of scotch sure sounds nice. Too bad I have far too many years left before I can even think about that

Looking through this thread, there are obviously a variety of approaches that different examiners and agencies have to conclusions. Many operate with with ID/INC/EX as options. Some have added a conclusion in between ID and EX. Others describe multiple steps from INC to ID. A brave few have even waded into a numerical scale between INC and ID.

Given this variety, I believe that the proposed standard does an excellent job of meeting the needs of all these examiners. There is standard wording very similar to SWGFAST language that fits perfectly for ID/INC/EX use. And the standard wording also fits for examiners that want to put something in between ID and INC.

Shouldn't this new standard document include the conclusions that many examiners are using?

Hi rsc55,

I helped write our agencies criteria on our complexity scale and review cases to see where inconsistencies may present themselves so I'd like to address your comment below.

2. I also support documenting the complexity of the examination. However, expanding ID's into simple, advanced, complex seems to relay trustworthiness of the examiner's DECISION. IE - how likely are we to be right because of the complexity of the exam. It does NOT relay the weight of the EVIDENCE. (I'm going to throw in some numbers here for the sake of discussion, please don't take these numbers as dogma - they are simply to help draw a picture). I can have 45 L2D and come to an IDENTIFICATION but it could be an advanced or complex comparison if there is a lot of noise, maybe multiple touches, some superimposed ridges, etc. Alternatively, I could have between 6 and 12 very CLEAR L2D and go with SUPPORT FOR SAME SOURCE, not an ID, but it could be a very simple comparison because the print is high quality.

Based on my review of cases for the past several years, examiners are pretty consistent in how they use the criteria we have so it largely takes the examiner out of their personal opinion and/or comfort level. It goes more to the weight of the evidence because it gives a more honest representation that not all IDs are the same and examiners should be articulating that. I'm not saying you can fully remove the personal feelings, but that's why as a supervisor I pushback on any reasoning outside of what can be demonstrated when I review cases. This is my expectation of the examiners in the office as well. As an example, if I call an ID basic with no ambiguous features that were relied upon to reach my conclusion but I marked ambiguous features on my documentation, the expectation is that the verifier would kick that back as being inconsistent. And I can say from my experience (as well as how many examiners will come and ask my feedback on how to approach these conversations with their coworkers) that this is routinely done. It's done on my cases as well so my staff have no issues pushing back on me as well if they feel I haven't demonstrated the weight I'm giving to a comparison. Another aspect of our criteria is that if an examiner is in doubt about the objectivity of a comparison, they rate up to drive additional QA.

For your first example, yes that ID could be either basic, advanced, or complex depending on the amount of ambiguity present. If there was some movement but the detail can easily be demonstrated to anyone (lay people as well as experts) it would fall into the "low ambiguity" criteria of being advanced. It would also be advanced if the orientation/area wasn't self-evident. The orientation being self-evident criteria isn't did I know the orientation, it's would anyone looking at the latent know where it came from. So there have been IDs I've called advanced because while I had a gut feeling about the orientation, there weren't any demonstrable clues that would lead anyone to orient the print the same way. If there was a lot of ambiguity to the point I had to heavily rely on the known to eek out features, then it would fall into a complex ID due to high ambiguity. It doesn't mean the ID is weak, it simply means more interpretation was involved and there may be disagreement on where I placed certain features. We have an additional QA measure for cases with complex IDs.

For your example of a comparison with between 6-12 L2D this could go several ways as well which is why you need clear criteria. If you had 6 L2D but also L3D that could be a complex ID with limited data (that just means I'd expect anyone doing that comparison to use all or mostly all of the same features). If the print only had L2D and nothing else it would most likely (I'd have to see the print) fall within our Inconclusive - consistent but insufficient to identify result. The reasoning would track pretty similarly with other latent's in the range you included.

That's not to say there aren't potential limitations, it is a human made criteria similar to the OSACs. I'm happy to share with anyone the criteria we use and would welcome feedback on ways to improve upon them further. I didn't include all of our criteria here because I'm already being really wordy

anwilson,

I just think that complexity scales and verbal likelihood scales are not the same thing. I am all for including complexity if that is deemed valuable by your agency. But you can have both a complex and non-complex ID, and a complex and non-complex SSS. One does not replace the other. One explains the reliability of information present, and the other relays how much that information supports the proposition. There can definitely be crossover, one can certainly influence the other. But there is a slight difference in what they are saying.

Baby steps. These are needed. The rest will come later. Compromise is integral to our success and these conclusions make sense. Even if you don't currently use them all. Having these will make us, in this field, stronger, and it will help so many of us just trying to get through testifying.

rsc55 has a good point that should be recognized.

The weight of the evidence (the LR) is NOT the same thing as the complexity of the case/comparison/examination.

I thought of an analogy: A DNA analyst says that they would expect a profile to appear by random chance in approximately 1 in a million people. This statistic says nothing about how complex that examination was. Did it include multiple profiles? Did it have artifacts? Did it have dominant profiles? Allele dropout? and so forth.

The risk of error and the level of interpretation and expertise required to generate that 1 in a million is not reflected in that number. The complexity (and risk of error) needs to be addressed separately.

rsc55 astutely separates this for fingerprints. The LR (strong support for same source) does not tell us if it was a complex examination, or how much interpretation was required to get to that value.

A complexity scale is fine. I'm all for it. IF 1) it is separate from the LR scale (weight of the evidence in support of the propositions and 2) anwilson and Michele, specifically to you: IF IT IS REPORTED. I am asking either of you that seem to use this: is it REPORTED in your reports or is it in your notes or is it not all recorded, but rather something verbally communicated between examiners.

As rsc55 points out, these are not mutually exclusive. Michele proposed a complexity scale INSTEAD of the OSAC conclusions. I don't see why that is necessary. Both are valuable and can give the end user(s) more information.

g.

PS Dr. Borracho, you make fine points, except I'd to add one thing about Kuhn's observations. The most common result of a Scientific Revolution is a blending or merging of the two approaches. There is usually merit to both approaches, which is why they are vehemently argued, but in the end, the most successful approach tends to be taking the best of both worlds. For example, it's one reason I support using statistics to enhance and support our conclusions (not entirely replace them; sorry Henry Swofford, we disagreed on this point, but we both see value in the use of models).

I was reading from Les Prophéties last night as I nipped a wee shot of comprehension from my hip flask, when I stumbled across an interesting passage in that much maligned book. Of course, it took some serious thought and a few more wee nips from the flask to interpret, Nostradamus being cryptic as was his style, but here is how I understood his prediction.

On All Hallows’ Eve, ano domini 2019, a child was conceived somewhere in the middle of the North American Continent (that would mean that today, as you read this, her friction ridges are in the stage of very active growth and formation around the volar pads). She will grow and mature into a preeminent scientist, becoming a world-renowned expert in forensic identification.

In her 50th year, she will stumble across an archaic text on the use of friction ridge skin impressions. Dusting off the crumbling cover, she will make out the book’s title, Ridgeology. As she reads, she will be amazed that so much knowledge had existed at the dawn of the 21st Century, which was shunned and lost to the world for decades by an invasion of statisticians, lawyers, and academicians. She will shine new light on this ancient text and it will once again assume its rightful place as the true science of friction ridge skin impression identification.

And as the discipline of Fingerprint Identification goes through yet another paradigm shift, there will be much wailing and gnashing of teeth by those who fought so hard to infuse statistics and qualified associations into the field. Notable rebels from the former generation who led the paradigm shift of the early 21st Century will now find themselves ignored in their protests, just as those who led the shift away from the statistical ideas of Galton and Locard in the mid-20th Century were ignored by the current generation of new scientists.

And so it goes, g. Hang in there, my friend, but don’t be surprised if you find yourself on the losing side of the next scientific revolution. I’ll be watching from a boulder somewhere on the North Fork of the Gunnison, hip flask in hand and a big smile on my face.

Dr.,

shunned and lost to the world for decades by an invasion of statisticians, lawyers, and academicians

I think that's a bit of an oversell. Sure, there are those out there that are calling for an end to "identification". But identification remains in this proposed standard. What has the current "paradigm shift" eliminated from Ridgeology? The same term has exists with an improved and clarified definition. The proposed OSAC definition for ID is a pretty minor change from the last SWGFAST one.

I've reviewed some of the reports that g. has written recently. I think it's helpful to take a look at how he describes an Identification and how that's different from what's existed over the past ten years.

1. Identification - this word and this conclusion is clearly stated in the report and the report includes the current definition. SWGFAST a couple years ago and OSAC now. No change here.

2. Complexity - the report clearly states whether, in the examiner's opinion, the examination was complex or not. Even operating under a pure Ridgeology framework, what would your response be when a judge asks, "Was this a complex or non-complex examination?" We should be able to give a straight-forward answer, especially since this has been defined by SWGFAST for 8+ years. No change here, except maybe being a little more transparent.

3. Extremely strong support - this simply restates the definition of ID. A definition very similar to what's existed for about 8 years now.

Does this method of reporting an ID shun the knowledge in Ridgeology? What's lost from that book? I see all of that knowledge preserved, carried forward, and built upon. The proposed conclusion standard isn't destroying the past or abandoning what's come before. It's preserving and building upon a strong base. Does an Identification, described and report as above, sound like a replacement of the past? Or does it sounds like new improvements incorporated into the strong base that's come before?

I have a blind job applicant who is a computer savant. I hire her to run AFIS. She can scan latent lifts and launch searches. Then she can take the candidate list and run each pair through an LR app. If we're using a multi-binned scheme of support for identification, then why can't she go to court and testify to the LRs in the top bin?

After all, isn't that pretty much what chemists do? They use color tests and instrumental analysis, then report and testify to the results. Why is that not where we're headed?

Why is that not where we're headed?

Quite a few reasons actually.

First, in an Identification report, describing the complexity of the comparison or restating the definition with the term "extremely strong support" has absolutely nothing to do with a multi-bin system. It can be used in that system, but it can also be used in an old-school ID/NoID binary system. Extremely strong support is part of IDs definition and complexity is part of the notes taken during the examination.

Second, a multi-bin system (let's say 5 conclusions) does not tend towards blind examiners or sole reliance on instrumental analysis. If it did, then it would have happened for footwear and QD examinations decades ago. All the fears and worries that have arisen over an expanded conclusion scale are proven unfounded by these disciplines.

Your scenario links a multi-bin scheme with a purely auto-coded LR. Might as well ask, "If my car has four doors and a trunk, why can't a blind person land a 747?" Cars and planes are both related to transportation and 747s have lots of autopilot features, but...

It is possible to have a 5 (or even 9) conclusion scale with or without an LR model. It's also possible to have a 3 conclusion scale with or without an LR model.

why can't she go to court and testify to the LRs in the top bin?

I guess she COULD if the judge allowed it, but it's more likely that she can't. Currently, she can't because there isn't a way to blindly auto-select the latent, auto-code and auto-pair the minutiae, and auto-generate an LR (or at least not an LR that's conditioned on that level of automation). In the future she likely can't because of the direction that we're heading with statistics. In my opinion the most likely deployment of a valid statistical model will be in support of the examiner's decision. Meaning that the examiner has to do ACE-V before the model gives a number.

Why do I think that? Look around. In Glenn's company, he does ACE-V, reaches an ID, sticks it in the model, and reports ID with 10^x LR. Even DNA analysts look at and interpret the peaks before crunching the numbers in the computer.

TL/DR - Expanded conclusion options ≠ LR. LR's will add to and not replace ACE-V.

I am not at all a fan of Support for Same or Different Source. There. I said it. I would prefer to be able to use a valid mathematical model to support and articulate those types of conclusions , but that is my very personal opinion and a different conversation. Having said that, we use a classic 3-conclusion scale and the OSAC standard is not at all in conflict with anything we do or how we do it. I would have no problems articulating on the stand why we don't use the additional conclusions in the standard based on our own quality system policies and procedures.

I've been doing this work for 15 years now and I am frustrated that the field is still bogged down over the semantics of the word "identification". Seriously????? I would like to think that competent examiners everywhere understand and support the need for standards. I may not be the brightest crayon in the box, but I understand that there has to be a baseline beginning to everything. It can always be improved upon later. If it were my choice, I would adopt all the existing SWGFAST documents and build from there. Alas, it is not my choice and I understand why it's not being done that way. What I do see, from the perspective of a grunt bench examiner, is a standards development process that is broken and failing us. A process that is intended to be informed, guided and deliberate just comes to a dead stop, for no apparent reason other than certain people just not getting their way. I've read a lot of arguments for and against here, most are valid. I've also read a lot of complaints that don't come with the requisite suggestions for correcting the perceived issues. I work under the principle that if you see a problem, you should bring a possible solution. Otherwise you're just part of the problem.

I don't know what ASB's criteria for acceptance or rejections are, but I'm sure they exist. What I see as lacking is a time limit. Evaluate these documents, adjudicate or address public comments and either accept or reject them in a more timely fashion. Again, as an examiner, I want/NEED to have these standards published sooner rather than later. I would also rather have a very basic, baseline standard that works for all rather than one that requires I have to go back to school just to understand the minor nuances of it. IMHO, if a document is valid in the face of the current body of knowledge in the field then it should be acceptable. Personal preferences or agendas should not be a consideration. Standards can always be refined and improved upon. The arguments and discussions to drive the state of the art forward will go on forever long after we are all dead and gone, or just enjoying a sip from the hip flask looking in from the sidelines. /endrant