Verifying negative processing results

[Snyder22]

I apologize if this topic is already on here, but I couldn't find anything with the searching I did (if it is, please just point me there!)

When an examiner at your agency processes evidence, and determines that no latent prints of value were developed (no photos or lifts are taken), does a second examiner look at the evidence to verify that decision?

Just a question that has come up as we revamp for ISO 17025:2017 and our ANAB inspection.

[Boyd Baumgartner]

It has been discussed before in this thread.

[ekuadam]

There are two of us in my lab that process and currently we don't. We look at each others images we take and if we happen to see some possible ridge detail that should have been captured (like if there is some cut off on the side of the image) we will discuss that with each other.

Side note though. Our section is split into two sides, processing and comparing. There is discussion on the comparison side about whether or not they will start verifying suitability. Our tech lead has asked us on the processing side to think about how we would/could implement it on our side as well. I don't think we will start it anytime soon, but it is something being considered.

[Snyder22]

Thanks for the link and comments. Looking at everything from 2011, I guess I'm wondering if other agencies are starting to consider it again like ekuadam says? My experience has always been that it wasn't done, but that doesn't mean the field/reasoning isn't changing.

We obviously want to stay in line with current practices, while also balancing the fact that there are limits to our resources of examiner time...

[josher89]

Since it was I that originally posed that question, I will chime in.

We are a small enough lab with a small enough backlog that we verify every decision and conclusion (that means suitability for analysis and ALL examination conclusions).

Even so, we will not go back and re-vist (reexamine evidence) that the other has processed just to 'check' to see if all ridge detail was captured. We are certified, proficiency tested, and accredited and while those things are only pieces of paper, they do show a measure of professionalism. I guess we are treating each other like they're innocent until proven guilty. I trust that they are capturing all ridge detail via photography that is potentially suitable for analysis (since this is our policy). We review those images for suitability. If we determine otherwise, we make a note of it in the case file and proceed with the examination.

If a quality review, internal audit, or other trigger leads one to believe the other is just being lazy or missing potential evidentiary ridge detail, we have procedures in place to check. My 'tell' would be if, over the last three weeks, my partner didn't develop any latent prints on the evidence they processed, I would begin to declare shenanigans and advise our QAM to look into it.

I'm not saying to not do it; I'm saying your system should be robust enough that you won't need to do it but also have something in place to verify if it became an issue.

I remember an episode of CSI: Las Vegas where Gil was testifying on the stand about a book of matches and when asked to review the evidence, he saw a nin print that had developed that wasn't previously there at the time of processing. That's an extreme (and entertaining!) case but that could happen in real life so should we reexamine porous evidence months later to confirm no additional latent prints developed? That'd have to be your policy, not mine.

[NRivera]

Just out of curiosity josher, what is your criteria to define what is "potentially suitable for analysis"? Just wondering because we've had similar discussions here.

[ekuadam]

Since it was I that originally posed that question, I will chime in.

We are a small enough lab with a small enough backlog that we verify every decision and conclusion (that means suitability for analysis and ALL examination conclusions).

Even so, we will not go back and re-vist (reexamine evidence) that the other has processed just to 'check' to see if all ridge detail was captured. We are certified, proficiency tested, and accredited and while those things are only pieces of paper, they do show a measure of professionalism. I guess we are treating each other like they're innocent until proven guilty. I trust that they are capturing all ridge detail via photography that is potentially suitable for analysis (since this is our policy). We review those images for suitability. If we determine otherwise, we make a note of it in the case file and proceed with the examination.

If a quality review, internal audit, or other trigger leads one to believe the other is just being lazy or missing potential evidentiary ridge detail, we have procedures in place to check. My 'tell' would be if, over the last three weeks, my partner didn't develop any latent prints on the evidence they processed, I would begin to declare shenanigans and advise our QAM to look into it.

I'm not saying to not do it; I'm saying your system should be robust enough that you won't need to do it but also have something in place to verify if it became an issue.

I remember an episode of CSI: Las Vegas where Gil was testifying on the stand about a book of matches and when asked to review the evidence, he saw a nin print that had developed that wasn't previously there at the time of processing. That's an extreme (and entertaining!) case but that could happen in real life so should we reexamine porous evidence months later to confirm no additional latent prints developed? That'd have to be your policy, not mine.

I guess for us it is different, as there are only two of us that process. I was an examiner for 9 years (recently just decided to go back and become a full time processor)and know for the most part, what is probably suitable and what isn't (even though everyone's suitability is different, and is a discussion for another day), but our other processor has only been doing it a few years and never has been an examiner. Same with the couple of people before him. So when they were trained, they were trained on some analysis and were told that if they see 5 minutae to take a picture, the images then go to the examiners and it is up to them to see if they are suitable. We (and our management) trust each other in that we are not missing anything, and there are times where we will show each other our evidence and ask if we should take a picture of something. If it comes to that, we usually take a picture of it, just in case someone down the line might think it is suitable. Our current processing backlog I think is only around 100. It hasn't been decided if we will go the route of going over evidence again, as since there are only two of us, how much will that slow us down?

[josher89]

In order for a latent print to be considered suitable for an ‘of value’ determination
(sufficient for comparison), it should contain a target group of minutiae such as a core or delta,
orientation and/or anatomical source (i.e. fingertip, phalange, palm or foot). Special
circumstances may dictate that a latent print be considered ‘of value’ that do not conform to
these criteria; if so, proceed using the described methodology.

We are a Type 2 lab (I think that's the one but I haven't used the two types in a while!) in that we keep any latent that can be used for exclusion purposes as well as identification.

So, if there is sufficient information to render a conclusion, we will keep it. I am thinking of giving my CSIs the training and giving them a numerical metric (like if they see four minutiae) to photo. If there is a doubt, I would tell them to photograph it.

[ER]

My previous agency started a procedure to do a proficiency test for processing. However, since a traditional PT would be difficult to grade, it was an observation test of a actual case.

That could reveal someone who was missing latents of value and lead to further reviews of past work, reexamination of previously processed evidence, and retraining.

Another possibility is if during tech review there is a trend to take very few photos on types of evidence that typically have more photos. Checking the evidence back out of property could confirm if they were missing latents or were just unlucky in what was processed.

Finally, a review of each photo can sometimes show a latent of value partially included on the edge of one picture that is not fully captured in its own photo. This can also be a clue to go back and view the evidence for what was missed.

To fulfill ISO/ANAB requirements I'd say to set up a observational PT. Then have general procedures in place so that someone with a concern can voice that to QA and that QA can work with the examiners and the unit in reviewing things.

Above all, it is imperative that none of this comes off as or is interpreted as punitive. Everyone needs to treat this as steps to reach the best and most accurate answer.

[josher89]

What ER said.

We implemented an annual observation (we don't call it a PT per se) and that has actually given us additional opportunities to observe casework outside of the admin and tech reviews. Obviously, when you are being examined, you are typically on your best behavior but it does allow for the observer to determine if things are being overlooked due to complacency, ignorance, or lack of professionalism. And, there are polices in place to deal with those if they crop up.

[josher89]

To respond to ekuadam, I would ask if reviewing the evidence prior to checking it back in would be worth it. Much like blind verification. So far, I haven't seen research that suggests it finds more errors. Or reviewing an AFIS candidate list that contains 30 or more; usually, if they person is in the database, research shows it's in the top three like almost 85% of the time.

A Note for FBI NGI Users...
One of the most valuable pieces of fingerprint-related information shared at the 2018 IAI Conference involves FBI Lab and CJIS research about about NGI latent print search result candidate scores. At the IAI Conference in San Antonio, FBI Latent Print Examiner Kyle Tom explained that preliminary research showed that when the matching scores of the #1 and #2 candidates have a difference of 1,250 or more, 83.5% of the time it will be an identification.

This is important because it means all agencies should consider implementing a policy that any such 1,250 or more difference in NGI candidate responses should require review by more than one examiner - either because the first examiner made an identification, or because there is an increased chance of an erroneous exclusion. Hopefully, the FBI will publish research on this topic in the future.

Some agencies are of the opinion that allowing examiners to see the matching scores in candidate lists biases them and should be precluded. The current chair of the OSAC Friction Ridge Subcommittee AFIS Best Practices Task Group (Mike French) and I are both of the opinion that AFIS matching scores are an important objective measurement which can lend valuable information to the examination process and aid quality assurance.

- Ed German
Certified Latent Print Examiner, IAI
Certified Biometrics Professional, IEEE
14 Oct 2018

Do a trial for a month and review each other's evidence before sealing and keep track of the time it takes. In doing so, does the backlog grow and/or take sufficiently more time to complete a case that ultimately winds up that you aren't finding additional 'of value' impressions? I don't like seeing labs implement changes without there being a solid foundation for doing so. I think a month is a fair amount of time to see. If it's catching enough to be significant, then you better do it. If not, it was worth the look.

[ER]

Absolutely!

Even more so, every agency should have an idea what the score curve looks like for ALL of their AFIS systems. Last year I gave a presentation to users of the AZ AFIS system on how to do this. Sometimes hits are missed, and we don't all have the staff to review each candidate. If just the candidates with scores above an established threshold are reviewed, then that will undoubtedly catch most of the misses.

(Now, I'm not sure if any agency actually started looking for these misses. I just made sure it was possible when the system upgraded, and then showed each agency how. There are obviously some agencies that aren't in a position to deal with misses in a positive manner, and some examiners that are extremely fearful of their agency looking for their mistakes. But this gets into a bigger discussion of how our field treats errors.)

[Snyder22]

Thanks all for your replies. We're accredited, do a processing proficiency test (switched to having it be observational this year), and have recently started reviewing no value decisions when 1) an entire item of lift card(s) submitted by an agency is NV and/or 2) an entire photo or lift taken during processing is NV. Our policy states we preserve latent prints which may potentially be of value during processing. So, if we're agreeing about the latents that are photo'd and then determined to be NV, anything else that was developed but not photo'd would be below that level of information.

As Josher said, we would notice if all of a sudden everything was negative (there are only three of us, so we each end up seeing some part of almost every case).

I like the idea of testing it for a month and seeing what happens, and keeping track of the extra time it takes, if the question won't go away.

[NRivera]

It's good that you mention your agency's approach because that has a huge bearing on the decision. I guess it becomes a matter of where on the graph you want to dwell based on your operational approach. We are approach 1 (sufficiency for ID) so a 5 minutiae threshold for preservation of a questioned print would make sense. My concern is that just putting a number on it is still somewhat arbitrary. Operationally I could live with that given our volume, but it's not likely to be a one-size-fits-all solution for most agencies out there. There are additional factors that should be considered to choose what's best for you and yours.

[Steve Everist]

I remember an episode of CSI: Las Vegas where Gil was testifying on the stand about a book of matches and when asked to review the evidence, he saw a nin print that had developed that wasn't previously there at the time of processing. That's an extreme (and entertaining!) case but that could happen in real life so should we reexamine porous evidence months later to confirm no additional latent prints developed? That'd have to be your policy, not mine.

This reminds me of a trial I was testifying in. It was a case with a lot of processed paper. I don't remember if my testimony included ID's or just exclusions. However, there was a break prior to my testimony. It was either the prosecutor or the paralegal in the case who mentioned to me that the evidence items (paper) were put into sheet protectors. I didn't think about it until she handed me the pages and I saw a beautiful latent print that had no markings next to it that would accompany a print that I would have photographed. This seemed strange to me as there were prints in its vicinity that had been marked, although they were very faint by this time. I couldn't understand how I would miss since a beautiful print until I remember being told about the prints being put into sheet protectors. Then I realized that, because we package the evidence as we receive it (but with warning stickers regarding the chemical treatment and to wear gloves when handling), that someone was walking around with a bright purple/magenta finger as a result of viewing the evidence unprotected.