Outside Verification Policy

[IdahoDJ]

Looking for written policy or ideas for creating one based on the following introduction; I am a clpe working for a state lab (accredited). On a regular basis we do "outside verification" for small departments that may only have one examiner who need us to look at work product (Idents) or for conflict resolution when a third set of eyes and a third opinion are warranted.

At a staff meeting recently we discussed several hot topics that were presented at the recent IAI Conference in Spokane, namely John Black's lecture on 100% Verification (or How often do you want to be wrong?) and Examiner Disagreement: Roundtable Discussion with Kathryn Suchma, Glenn Langenburg, Charles Parker.

From the staff meeting we discussed how helpful and even necessary it is to have a verification policy that covers not only in-house verification (we verifiy all conclusions) but also that covers verification of outside conclusions. In the past this has been done informally by a phone call from the outside examiner whom we know asking for a meeting to have us look over and "verify" their "identifcations." We will spend as much time as we need reviewing the work and end with placing our initials/date on the latent and known fingerprint card and making notation on their worksheets -- but none of this has been documented IN OUR SYSTEM and we have no policy in place regarding outside verification.

Can you please tell me if your lab has a WRITTEN policy regarding outside verification and what it entails? We discussed treating it like regular evidence and submitting it into our lab system as a new case, however with backlogs and workloads it would take some time to get to these cases in the order received, and there is no-way in helena montana a case like this will get priority status-- so we all agreed this was not practical, however in today's ASCLD-ISO environment it may be going in that direction. I sure hope my QA/QC Manager is not reading this.

I appreciate any comments, concerns, suggestions or questions this post may produce.
THANKS!

[Charles Parker]

IdahoD

I must say you have a nice lab. Our policy is that we cannot verify another agencies work unless that other LPE has been competency tested by our agency. I know of several others the same way. They will verify only if the outside examiner has been competecy test by the inside agency.

It was my understanding that was an ASCLD-LAB thing or it was something that someone in QA made up thinking it was an ASCLD-LAB thing.

2nd point -- Is verification in your agency an analysis. If you consider that it is, then you will need to document it just like any other analysis. I am pretty sure that is an ASCLD-LAB thing.

Like you I hope your QA is not reading this.

I think it is great that your agency (a state lab) will verify smaller agencies work. I wish more would do that, but I am afraid most will not.

In our situation you have to send it in and we treated it just like a new case, and don't care what you did or did not do. Kind of wasteful of time management.

PS John Black gave a good presentation didn't he. That son-of-a gun has me thinking now. Pretty soon him and g. will change my whole outlook on life. Oh well times are a-changin

[IdahoDJ]

Thanks Charles, that is what I was thinking is the case.

Just to add to this idea a bit, there is an examiner down at the Tuscon, AZ Crime Lab who I hear routinely makes trips up to the Phoenix to have his stuff verified. I hope he could jump in and tell how they do things down there.

DJ

[L.J.Steele]

I'm not an examiner, just an attorney, but might I suggest having the outside policy include some close non-matches in the mix so that the outside examiner can't assume that he or she is verifying a match?

[Charles Parker]

L.J. Steele Wrote

I suggest having the outside policy include some close non-matches in the mix so that the outside examiner can't assume that he or she is verifying a match

That is an interesting suggestion. It would certainly make an interesting work flow in obtaining close non-matches to bolster the verification from outside agencies. Not practical I would image based upon the time required to find close non-matches.

[L.J.Steele]

Ms. Steele, great idea but could you please tell me your definition or description of what exactly is a 'Close Non-Match'? How does one come up with them on a case coming in off the street?

Again, I'm not an examiner, just a humble attorney. Dror's work seems to talk about ambiguity and low quality samples as being an imporant part of bias situations -- it is hard under test conditions to over-ride clear, unambiguous matches or non-matches with even strong baising info.

What I'm thinking of is the stuff under discussion in other threads. The place where an examiner looks at a latent and says "gee, this is close. I can't call it a match because of its quality, or this unexplained dissimilarity over here, etc., but I can see how another examiner might disagree with me." Or the place where you'd like to say "consistent with", but not good enought to say "yup, that's the guy". This seems like the kind of situation where getting that second opinion from an independent examiner that the quality isn't good enough or the dissimilarity is inexplicable, would be useful.

[Neville]

Hi Lisa

Charles is right it hardly ever happens, on AFIS you get close matches in that their score is so close it could be a hit, as in the first 5 on the list may only have a score of 20 seperating them, but the ident may have a score of 1000 below those, and when you look at the first 5 they are nothing like what you are looking for, the hit maybe 10th on the list. This is one of the reasons I have my doubts about using LR models. Though as pointed out I have not been to the brain wa.... sorry I mean the training workshop.

But to get back on track to do it manually would take possibly a year of work to find one more and may not even then. What you and person with little training/experience may see as a close match, most of us wouldn't give a second glance.

Unfortunately many experts in the field to day have little experience of manual searching so that it is even less likely to happen mainly because AFIS has been around for so long (close to 20yrs in NZ) and those without AFIS would not have the large collections to manually search anyway.

[Charles Parker]

IdahoD Sorry about that -- I did not mean to steal your post.

Of the agencies in Texas the majority I am aware of that service a large population do not perform verifications for other agencies.

1. Because they did not train or test the outside LPE and 2. If it is analysis then they have to start from scratch and do it just like any other case.

I suppose there are some mid-size agencies that would do it for other smaller agencies in their region. I will ask around.

You are the only state lab I know that has been doing it and own up to it.

We currently do not perform any outside agency work -- at all.

[IdahoDJ]

Ahem, excuse me... this has been an entertaining discussion about close non-matches, thanks. It is now confirmed what I have suspected from the beginning, everyone nutty enough to work in forensics has a little ADD in'em.

Now could we get back to the question about outside verification? I am wondering after reading Charles Parker's comments just how often outside verification actually occurs.

Here in Idaho we don't have alot of qualified examiner's bringing work product to us to verify, but it does happen. We are all quite familiar with the quote, "if it isn't written down, it didn't happen" or something like that so, what kind of documentation does your lab or agency require when reviewing outside work?

Once again, much thanks!
DJ

[Pat A. Wertheim]

. . . there is an examiner down at the Tuscon, AZ Crime Lab who I hear routinely makes trips up to the Phoenix to have his stuff verified.

There are two CLPEs in Tucson whose identifications have to be verified in Phoenix. First, myself. I am currently the only qualified LPE working in the Arizona Department of Public Safety (DPS) satellite crime lab in Tucson. Our policy requires verification of all conclusions. Therefor, all of my identifications, exclusions, and inconclusives must go to one of our other labs for verification. We have satellite labs in Flagstaff and Lake Havasu City, as well as the main lab in Phoenix. The Flagstaff and Lake Havasu City labs have two LPEs in each lab, so they can do their verifications without sending them to Phoenix. Because Phoenix is closer to Tucson than the other labs, I send my verifications to Phoenix. I usually get my work back verified in a week.

The other LPE who sends verifications to Phoenix works at the Pima County Sheriff's Office (PCSO) in Tucson. Twenty years ago, when I verified an ident for PCSO I merely did my exam and if I agreed with the ident, I simply initialed the latent print lift and inked print. I did not take separate notes or issue a formal report. In most cases, the whole process took five minutes or less. Starting sometime in the mid to late 1990's, we began issuing reports on outside verifications. That meant formal submission of the evidence to our property section, then the taking of separate notes, and the issuance of a report stating that the PCSO identification was verified. That process took a couple of days. But about two years ago, the DPS policy changed to require both an initial identification and verification in house. Because I need to send all of my stuff to the main lab anyway, it became more efficient for the PCSO examiner to send her verifications to the main DPS lab in Phoenix rather than send it to me, have me identify the print, and send it to Phoenix for verification. I have been advised by the PCSO examiner than it now typically takes five to six months to get back her latents and the report when she sends a verification to the main DPS lab in Phoenix.

There are two trainees in the DPS Tucson lab. It is anticipated they will complete their training in six to twelve months and can start doing verifications in Tucson without the need to send anything to Phoenix. Likewise, PCSO has one trainee working with the CLPE. When the PCSO trainee is cleared to do her own casework, they will no longer send anything out for verification. That is expected to happen any day now.

[L.J.Steele]

Charles is right it hardly ever happens, on AFIS you get close matches in that their score is so close it could be a hit.

(Didn't Charles have another post on this responding to me -- it isn't showing up in the thread today.)

I may be unclear -- I'm thinking not thinking of going back looking for instances of close non-matches just to create work for the outside examiner. I'm thinking of stating in the policy that the organization will submit to the outside examiner both "matches" fitting certain criteria, and close "non-matches" that fit certain criteria, and will not specify what conclusion the originating examiner reached. Thus receiving examiner doesn't know if the original result is a match or a non-match.

I infer from what you and Charles are saying, there will not be very many close non-matches. (Situations were the latent is so ambiguous that the examiner is troubled by the non-match conclusion and/or thinks reasonable examiners might differe might be a place to start.) One would have to talk to Dror and others in the psychology field to know if the potential for rare close-non matches being submitted would be enough to reduce the potential for bias.

[Kathy Saviers]

Charles - a question for you -

Why does it matter if your agency trained/tested this person or not? A match/identification/individualization is a match/identification/individualization or it isn't. The review should be of the evidence and not a judgement of the person. It seems a little snobbish to belittle someone who doesn't work for an accredited lab just because yours is. Maybe that other agency cannot afford to pay for the title but they do most of the same procedures as you do. The result of the examination is the question, not the training credentials of the person.

Sure, if it isn't a match/ident . . . (you know what I mean), it causes some issues with that person and outside agency, but how can there be any liability on your part if the person who brought you the evidence is right?

Another question - Would your agency verify for another CLPE even if he/she wasn't tested by your agency? Or doesn't IAI certification count?


Kathy

[RL Tavernaro]

IdahoDJ,
My experience mirrors Pat W's from a few years ago, when I retired from the Arizona DPS Lab. I will add one thing though; when I was still there, we had the capability of transmitting digital images of the latent print & known prints over a secure computer network to the main lab. Under special circumstances, we would use that capability to expedite the verification (it could typically be done the same day). If memory serves correct, a signed worksheet or report was then transmitted back to the submitting satellite lab, to be included with the case file.

If I may digress a bit, Kathy S's post reminded me of something you may want to cover in your policy: unexpected comparison requests in court.

The short story is that while in court, I was asked by a judge to complete a comparison that resulted in an identification (individualization). I suggested to the court the identification could be 'verified' (since another examiner from our lab happened to be at the courthouse that day). The defense strongly objected, & asked the judge to prevent any further examination, which was granted, and the court ordered me to testify regarding my examination. The defense later proceeded to argue to the jury that a verification should have been done, & requested the court throw out the fingerprint ID.

The judge allowed my evidence, the defendant was found guilty, and the defense later appealed. The court's decision was upheld, specifically citing my testimony that although verification was an accepted practice and good policy when practical, the absence of verification does not automatically invalidate an identification. Like many others, I routinely made identifications for many years without a formal verification policy (other than periodic random review of some cases).
Regards, RLT

[Charles Parker]

Cathy, let me try to answer your question(s) and keep it short. First though:

“The following are the thoughts of the author and should not be considered representative of any organization – past or present”

1. Like you I have no problem with other LPE coming into the agencies I worked at and asking for something to be verified. Did that for 25 years, but it has not been that way the past 7 years.

2. I do not think it matters, but I do not make the rules.

3. Person ‘A’ says it is snobbish for accredited labs to be that way while Person ‘B’ says the rule is needed to limit liability and to keep the untrained from exploiting the lab. (Besides you do not see Drug or DNA Exams being sent to other Labs for Verification) Both could argue that point and it would depend on who you are and what corner of the block are you standing on. I do not subscribe to either one but opt more that it is dealing with misinformation and some decisions made by non-subject matter experts.

4. You have a good point about it being the same --- But I do not make the rules.

5. The liability comes when the Lab rubber stamps one that gets by.

6. The liability comes when the other LPE states “well my examination has thoroughly been verified by the State (or whatever agency is doing the verification).

7. Lastly if that Certified LPE was not competency tested by that agency and does not take the yearly proficiency by that agency, then that agency is not going to verify their work. Certified or not.

8. This is my understanding of the rules. My understanding could be flawed as Idaho has been doing verification for outside LPE. Good for them and I hope they continue. I hope you continue doing it.

9. To me verification was someone did the work up, brought it in and the LPE reviewed it, signed off on it and the other LPE left the office (generally). The moment you take a latent print in, do a full examination, do a report to be released is not in my book verification.

10. Some of the examples given in previous responses are from verifications performed by satellite labs under the same umbrella. These are all same people that have been (I think) competency and proficiency tested by the same general agency. Example a state lab with 4-12 satellite labs.

I am for outside verification. You are right it is the evidence that counts and not the individual. I think it is interesting that an agency can prohibit verification for other agencies, but must run AFIS searches for other agencies because they have a State Terminal. No one has every claimed that there is logic and sound reasoning in all aspects of Forensic Science. 8)

[Kathy Saviers]

Hi Charles,

As you so noted, my opinions are those of my own and not necessarily those of my or any other agency.


It is unfortunate that those who make the rules have the power but not the logic to see the big picture.

Your example 7: So, if g., Pat Wertheim, or Ron Smith were working a case in your area and needed a review and verification, your rules would prohibit you from doing so because your agency has not personally tested them. To go one step farther, if any of these illustrious, well-known, and distinguished gentlemen mentioned were conducting a class in your agency and asked for a piece of case work as an example and made the suspect in class, again, your rules would not allow you to accept their analysis because your agency didn’t train and test them. Of course, in this case, THEY would be training YOU, but still their analysis would not be accepted.

Let me open another can of worms. Please read below the follow line items from the Certification Program Operations Manual. Are we violating either the Code of Ethics or Standards of Professional Conduct by refusing to assist our peers or others in the profession?

Kathy

Code of Ethics – Section IV

To cooperate with others of the profession, promote improvement through research, and disseminate such advancement in my effort to make more effective the analysis of the expert.


Standards of Profession Conduct – Section V

Every applicant/certified person shall cooperate with peers in the advancement of the science and profession of his/her respective discipline(s).