CLPEX Chat Board

So I just got a chance to read this week's Detail a few minutes ago and read about the ruling on the Daubert Hearing in New Hampshire. Is it disturbing to anyone else that a full day ago, an article describing an honest-to-goodness court criticism of our methods of documentation and verification resulting in the (let's hope temporary!) exclusion of a latent print was posted and there is not a single reference to it yet on this board? Yet we have now 8 pages of mudslinging and nonsense rehashing the Shirley McKie case and speculating on what Iain McKie's reasons for wanting to drop it could be (who cares and who could blame him for being sick of this nonsense?!?).

I think it's high time we re-examine our priorities and get to looking at issues that are of actual and current relevance to our profession.

The two issues the judge had with the New Hampshire ident were (1) documentation of the application of ACE-V during its application and (2) safeguarding against verification bias through blind verification procedures.

Regarding (1), Stephen Meagher points out (quite correctly) that this would be best addressed by "the establishment of more detailed guidelines for case documentation of ACE-V" by SWGFAST. I wholeheartedly agree and call enthusiastically for them to get to work on providing such guidelines for us all to adopt into our SOPs.

The second issue is a more tricky, but no less important, one. How do we avoid verification bias? Mr. Meagher is right -- the topic has come up before. But everytime it does, the response seems to be along the lines of, "well, it's a nice idea, but impractical for x,y, or z reason of time, operations, etc." Obviously this excuse is not cutting it. A judge has set a precedent of saying that this IS in fact important, and necessary to address. We can't drag our feet or ignore the problem any longer.

The judge states in her opinion that "The court can envision possible methods of creating a plausible blind verification process, such as assigning verification cases without identifying them as such." I personally don't see this as an operationally workable solution. Perhaps I'm not looking at it right. But I certainly recognize the need to come up with some kind of feasible, operationaly practicable method of blind verification that will satisfy both the courts and our own senses of propriety.

So I throw it out to this community: Leave off arguing over the past and let's focus on the future of our science. What are your thoughts and suggestions? How can we feasibly introduce blind verification into our SOPs without hopelessly bogging down the process? Has anyone got a blind verification procedure in place that has been working? If so, how does it work?

Respectfully, etc...

Heidi,

I think that people may be taking a wait and see approach. We may not want to jump the gun too quickly to implement new procedures until it is determined how far we need to go.

Blind verification will most likely be a little more work than normal, but I guess that I dont see it as being that much more work. Its hard to think of adding even a little more work though, especially when you think of the amount of time spend doing administrative stuff as opposed to 5 10 or 15 years ago.

All of my verifications are done in a blind manner. I am a one man agency (for now) and have a co-operative relationship with another local examiner. I developed an Adobe PDF document into which I can import the digital images of my two prints. On this document there are 3 choices that boil down to it is, it isn't, or there's not enough info to tell. I provide no biasing influence on the doc at all, its isn't even named a 'verification' form its a 'comparison review form'. The second examiner makes their choice, signs it, keeps a copy and sends it back to me.

Of course, my situation is different from most agencies and definately wouldn't be applicable in all situations. One example is that my workflow is entirely digital, which isn't the norm. At my old agency (state lab with 10+ examiners where manual comparisons with cards and magnifier is most prevalent) I would suggest some sort of review form. The first examiner takes a comparison to a second examiner without any indication of the first examiners conclusion. The second examiner makes their determination on the form and thats included within the case file.

Is it a perfect scenario? No, but no scenario will be perfect for every different agency. But at least it would go a long way to thwarting any claims of confirmation bias.

As far as the documentation of the ACE-V process? To me that is a slippery slope. Even the judge recognizes that its not possible to completely articulate in writing your thought process during comparison. And where do we stop with the ACE-V documentation? Just ID's, ID's and exclusions, each comparison? Lots of questions here that could greatly impact our effeciency.

Just my .02

Andrew

Andrew,

I agree completely. I think that we need to see what the final decision is and how it will impact us, etc. However, seeing as how these are topics that keep recurring, I didn't think it would do us any harm to do a little brainstorming now.

Blind verification to me is a tricky one. We all recognize the benefits, but as you say, the same scenario may not be appropriate for every agency. I understand managers' concerns about the time that basically double-comparing every print in every case would take. It quickly becomes a totally unfeasible numbers game. That's why I'm soliciting ideas from people who are making it work in their agencies. I'd really like ideas on how this can be successfully implemented. It sounds like you've got a great procedure in place -- but you also seem to have a quite unique situation in staffing, etc, that may not apply to the "average" agency. I really appreciate your input.

I also agree with your comments on the documentation. Deciding what is necessary for a given situation, and even what is prudent, could be a very daunting task. That is why I am happy to leave it in the hands of the good people of SWGFAST who are experienced in coming up with these types of standards. All the same, I will eagerly await their recommendations.

Reading over my last post, I see that it sounded pretty critical and I don't want to give the impression that I feel the community is off-course here. I really don't. I'm just frustrated with the amount of childish behavior that's been going on lately on some of the threads and thought we could use redirection into a more positive venue -- coming up with solutions from which we all can benefit.

Thanks for your reply,

Andrew,

Most agencies only verify latents that have been identified. Is this true of your agency? Because if it is true then the person receiving the images already knows you’ve ID’d it. I’m just thinking that some people may not consider that blind, maybe they’d consider it an independent evaluation.

Another thought….Is your report written prior to the other examiner looking at it? Is his or her conclusion sent back to you or someone else? If it’s sent back to you then this could be viewed as a consultation with another examiner and may not be considered blind verification either.

Just wondering.

The court's initial decision here was twofold, as I understand it. First, documentation as to the methodology and its application under Rule 702; and Second, blind verification in single identificaitons.

Neither of these are new concepts in some Federal agencies. In Federal Court under Rule 702 you have to provide a description of your bases for conclusions, which I believe is the documentation refered to by the judge. Federal agencies and a lot of local agencies in some states use a "boiler plate" description of the comparison methodology, in other words, the same wording copied and pasted on every report. And blind verification for single identifications is already practiced at the FBI, which is probably where the judge got that idea. Multiple identifications to the same suspect can be verified normally, i.e., when the verifier knows the conclusions.

With all due respect to my esteemed mentors Dave Grieve and Ernie Hamm, I understand their point about being strong enough to resist confirmation bias (another thread in the past week). But just as we all thought that the Daubert issues were settled decades ago, we still had to go through hearings to prove "error rate, testing and validation, general acceptance, and peer review" dozens of times before Daubert challenges started to die off. Well, now that defense attorneys are tapering off on Daubert, you can bet they will come up with an imaginative new attack and it appears to me that the new attack is "confirmation bias" and it will arrive in earnest this year. So we had better be ready for it with research and explanation in court. Blind verification is all about dealing with confirmation bias.

But remember Judge Pollack (Llera-Plaza case). His first decision was not good for fingerprints, but after a full hearing he reversed himself. I would not go screaming in despair over this latest decision just yet, either. Let's sit back and watch. I'll bet things end up much less damaging than our imaginations would lead us to fear.

Michele,

Is this true of your agency? Because if it is true then the person receiving the images already knows you’ve ID’d it. I’m just thinking that some people may not consider that blind, maybe they’d consider it an independent evaluation.

True, and currently I have only sent over ID's for review. We are planning on implementing a system where we send over non ID's sometimes to preserve the true "blindness" of the verification. As of yet we haven't implemented that, but its in the works. Trying to figure out the line of how many to be sent over and coordinating with with my boss and his so that he doesn't think that I've bummed a print yet at the same time doesn't know that I am intentionally sending him a non-ID.

I've only been working in this capacity for about 8 months now and we are still trying to get all the wrinkles ironed out, but I agree with you.

Is your report written prior to the other examiner looking at it? Is his or her conclusion sent back to you or someone else? If it’s sent back to you then this could be viewed as a consultation with another examiner and may not be considered blind verification either.

He never sees my report, all he gets is the review form.

Some Interesting Past Cases:

1931 Graves v. State Cite: (119 Tex. Crim. 68, 43 S.W.. 2d 953)

Latent prints in a burglary were found in an office where the defendant had worked shortly before the crime had occurred. The court ruled that fingerprints do not warrant conviction if found under such circumstances that they could have been left innocently. No great leap there.

1933 Davis v. State Cite: (125 Tex. Crim. 6, 66 S.W. 2d 343)

Latent print in a burglary of a store and after testimony and reviewing the Graves case above the judge ruled that “Fingerprints while admissible are not conclusive as to identity of the individual”. Ouch---fingerprints not conclusive to individual identity.

1938 Conners v. State Cite: (134 Tex.Crim. 278, 115 S.W. 2d 681)

Latent print in a burglary just as Davis and based upon the testimony of the witness and reviewing the Davis case the judge ruled the same. Fingerprints are admissible but not conclusive to the identity of the individual”. Ouch a second time.

1941 Grice v. State Cite: (142 Tex.Crim. 4 151 S.W. 2d 211)

Latent prints presented and the judge corrected the previous cases. Judge Beauchamp ruled that “fingerprints alone are sufficient to convict”. He also ruled that fingerprints are not duplicated and is the responsibility of the defense to prove otherwise.

For eight years Texas had to deal with the situation that “Fingerprints are not conclusive to a person’s identity”.

In the 1960’s a judge in Florida threw out palm print testimony because there was no state law that stated that police could take palm prints when a subject was arrested.

Let us wait and see. Things have a tendency to straighten out or maybe even get better.

Mr. Parker has hit the nail on the head. In 1999, I recall a number of examiners taking the doomsday outlook on the Daubert challenges. Yet, in hindsight, our science has emerged stronger than ever. If our critics chose to attack us over "confirmation bias" and "blind verification" now, while we may know those "issues" are, in truth, non-issues, in the end we will do the research and once again emerge stronger than ever. As for our critics, the extreme judges, the aggressive opposing attorneys, I say, "Bring 'em on!" They don't threaten us, they only make our field stronger as a science.

I think technology could eventually solve both problems (documentation and blind verification). As you mentioned, currently the problems seem so insurmountable because of the labor involved, so insurmountable that we run from the problems and deny they exist. But I think we are just selling ourselves short if we don’t brain storm and debate this issue. Thanks Heidi!

Kasey mentioned in another thread about research around computer algorithms designed to grade the quality of latent prints. I didn’t read much about it at the time, and yes I know AFIS currently performs this function in a rudimentary manner, but think of this: A far more sophisticated electronic workflow than anything we have now. In that workflow a computer would flag poor images (among other factors) for blind review. Kasey, if this is the point you were trying to make please let me know.

The algorithm could even be programmed to sprinkle in a few non idents into the workflow to throw verifiers off, so that when they see a certain type of verification come their way they won’t know if it’s the real deal or if they are being tested. I think the trick is a good, well thought out, electronic workflow. One that costs a LOT of money. Like the type of money that is budgeted into DNA analysis and research. I think all the judges should be very angry and demand we have such a thing! 8)

As far as documenting the ACE-V process. We could come up with a gazillion ways to do that with, AGAIN, the right software. It could be very streamlined and effective. The only major concern is that the written conclusions would not be shown to the verifier prior to their decision.

I agree with all of you that this is going to be the next big issue in court.
I have been challenged in cross-examination by defense counsel who imply that since verification of my findings was not blind it was somehow suspect. ("Oh, so you decide that a fingerprint was made by a certain person, and you take it to one of your little friends in the office and they sign off on it......"). My only counter to this implication is to state that all of the technicians are trained to act independently and to recognize and not be influenced by bias of any kind. (I almost told one lawyer that I get a lolipop for each mistake that I find, but I stopped myself in time). It's a tough sell, though.

I have always thought that there should be a way of programming AFIS so that when an identification was made and the operator clicked on "ident", if it was the first time that the latent was identified, the packet disappeared and re-appeared at another terminal as a run-of-the-mill comparison (i.e. no indication that the comparison had been looked at by anybody else). Once the second operator looked at it and marked it as an "ident", the blind verification was achieved and the process of notifying the investigator could be undertaken.

If the second technician did not verify the blind ident, then you have a training issue. The computer program would flag the file so that it wouldn't just languish in the database without being acted upon.

This wouldn't help in non-AFIS suspect comparison cases, however. Like the rest of you, we just don't have the time to sift through the same list of suspects that the first examiner did. Maybe the person who makes the ident. could include five (or ten) other ten-print cards (either suspects or random cards pulled from the file) along with the suspect's fingerprint card. The verifyer would "only" have to sift through a limited number of ten prints before finding the suspect.

This wouldn't help with major cases (like homicides) where you could have dozens if not hundreds of fingerprint lifts. Having only ten suspects still means one hundred "known" fingerprints to compare.

Job security is one thing, but really.....

Strict Scrutiny:

I think the trick is a good, well thought out, electronic workflow. One that costs a LOT of money. Like the type of money that is budgeted into DNA analysis and research. I think all the judges should be very angry and demand we have such a thing!

Lol! Yes, wouldn't that be lovely? If only we could all jump on the DNA funding bandwagon...

Cindy Rennie:

Maybe the person who makes the ident. could include five (or ten) other ten-print cards (either suspects or random cards pulled from the file) along with the suspect's fingerprint card. The verifyer would "only" have to sift through a limited number of ten prints before finding the suspect.

I like this idea, Cindy, only it still keeps the problem where the verifier knows that an ident has been made. It's sort of like a training exercise where you know there are no "No ID"s in the packet. Yes, it's a good tool to learn to compare, but you have the advantage of knowing the "right" answer is there -- you only have to find it. So, we've still sort of lost the blindness of the verification. It's still better than giving the verifier a note saying "I've identified latent number 1 to the third finger of Suspect B. Do you agree?" but I would ideally like to find a workable solution where the verifier doesn't know an ident has happened.

I like the AFIS suggestion you made as well, Cindy, of having hits automatically appear on someone else's workload so they can independently call it a hit without knowing it was already done once. I'm not sure how well this would work with the way I currently use AFIS. I understand that different systems operate differently and I certainly don't know the capabilities of all of them, but on my particular system, we sign in under our own name, enter our prints for searching, and then they are saved onto our jobqueue for checking later. In other words, I know which prints are mine because I entered them, I am holding the casefile, and I am checking them under my own sign-in name. I'm not sure how a new print could appear in my jobqueue without my immediately knowing that I hadn't put it there. I like the idea... I'm just still looking for ways to keep the verifier unwitting and thus, truly blind.

I'm wondering if anyone has a protocol by which sometimes an ID is passed on to verify (i.e. you give your verifier a latent that you have IDed along with 2 or 3 "suspect" cards and you wait to see if they come to the same conclusion -- without you writing your conclusion on the notes they see) and other times, a non-ID is passed on to verify (i.e. you give your verifier a latent that you have NOT IDed along with 2 or 3 "suspect" cards and you wait to see if they come to the same conclusion -- again without you writing anywhere what that conclusion was). It seems to me that this is the only way to maintain blindness without resorting to double-comparing every single print in every single case.

So, it would be more like Quality Assurance than full blind verification. You wouldn't go into court and say, "yes, every print in this case and every other case I've ever done has been subjected to blind verification" because I just don't think that is feasible. Instead, you would say something like, "Yes, this individualization was blindly verified." And the defense would say something like, "How is it blind? Doesn't the verifier know by virtue of the fact that they are verifying that an ID was already made?" And you would be able to answer, "No, they don't, because as part of our Quality Assurance program, occasional non-ID cases are also given out for verification randomly so that the verifier never knows upon receiving a case for verification whether it is an ID or whether it is a QA check and thus they must take each case for verification on its own individual merits."

Does anyone think that something like this might work? Does anyone already have something like this in place?

First, I have spoken to Langill's trial counsel. The State is likely to ask for a reconsideration, so the decision is good law now, but could be changed in a month or two. If I do get the Langill motions and transcript, I'll let Kasey know.

Second, I'm pleased that the Court is giving due consideration to these issues instead of following the "admissible since 1920s" line of reasoning. I think the Mayfield error, the Stacey report, and Dror's studies have all butressed the argument that there is a potential for error here.

Ms. Rennie, I'm sorry to hear you are getting the "implication of intentional impropriety" approach. IMHO, that's the wrong tack for defense counsel to take unless there is reason to suspect malfeasance. I've suggested on the NACDL lists that counsel consider the "honest-but-mistaken" approach we use on eyewitnesses.

See
Doyle's Champion article on the "forced choice" and "contaminated evidence" approaches to dealing with the honestly mistaken ID witness.
http://www.nacdl.org/public.nsf/698c98d ... ht=0,Doyle

See also Doyle's Champion article on accuracy and confidence.
http://www.nacdl.org/public.nsf/698c98d ... ht=0,Doyle

(If you have trouble with the links, go to www.nacdl.org, follow the links to the Champion, and type Doyle into the search engine -- you should get his articles (and one of mine) on eyewitness ID issues.)

I know there are limits to the analogy between an examiner and an eyewitness -- the biggest is that so long as the original latent and original exemplar exist and are unchanged, someone else can look at the match, albeit not from the naive state of not knowing a match was made and likely to whom.

Dr. Busey did a very nice presentation on the problem of information on how one views an image. The underlying study, Dolan, et als, How the brain learns to see objects and faces in an impoverished context, 389 Nature 596 (Oct. 1997) is worth reading. If you can get to a conference where Busey is doing a demo, I think you'll find it illuminating.

One also may wish to look at Berns, et als., Neurobiological Correllates of Social Conformity and Independence during Mental Rotation, 58 Bio. Psych. 245 (2005)
Downloadable here:
http://www.ccnl.emory.edu/greg/
which deals with the effects of peer pressure on what one perceives.

The dicta in Patterson about suggestion and about blind verfication, and the Langill decision are two datapoints, which do not yet a trend line make. But I think they should not be ignored. Judges are listening to well presented arguments about the potential effects of suggestion and confirmation bias on well-trained, honest, hardworking examiners.

So what to do? Take these thoughts with a big grain of salt, as I'm not an examiner and don't pretent to be one.

Contemporaneous documentation will help. This does NOT mean cut-n-paste boilerplate text. This may work for a while, but experienced defense attorneys will start to recognize boilerplate for what it is and use it in cross to show its weakness. This means even a few lines about what you see in the latent before seeing the exemplar and a line or two about what you know about the crime, other evidence, and about any possible suspects.

After seeing the exemplar, again, a few lines written at that moment about what you see and why you think it is a match. This will also help you later in preparing for trial.

[I'm curious how the Langill court expects the examiner's agency to come up with a contemporaneous report that isn't tainted by knowing a match was made -- this may be interesting.]

What about blind verification? I understand the practical problems. May I suggest some red flags when blind verification would be a particularly good idea:

The print is a difficult or close match (yup, hard to define that, but Dror et als say suggestion won't overide an obvious non-match or match, but can be a factor in ambiguous situations).

The print is the sole or primary evidence linking the suspect to the crime. (Ideally, the examiner won't know this, but a supervisor or the lead detective will and can flag the problem.)

The case is high profile or the examiner feels pressured to solve the case. This might be a good thing to do in all likely capital/death penalty cases for example.

The examiner knows the suspect, or picked out the exemplar having recalled something familiar about the print.

You might be able to solve the "only matches submitted" problem by also submitting close exclusions. Heck, if a second examiner can find something you didn't see, then you might solve a case that way. (Of course, then you'd need yet another blind examiner to verify.)

The suggestion issues aren't going away and courts are listening, so I suggest taking reasonable precautions to avoid having your evidence excluded.

What about this:

For every case where there are comparisons performed (even if there are no ID's), the examiner must take the known card(s) and at least one latent print to another examiner. The "reviewer" will do the comparisons and indicate their results on a form. This form would be kept in the case file as a record of either the verification of ID(s) or a record that non ID's are being used as 'controls'

I sort of like that, Andrew.

It gives a nice sort of "spot check" to every case, assures blindness (depending on how it is implemented), and sees that every case is given some kind of verification attention, without bogging down workflow unnecessarily. The only specifics I would add are:

(1) Clearly, all IDs would have to be included in the verification packet.
(2) I'm thinking that protocols would state that in every X number of cases with multiple latents, at least 2, or 3, or some number must be given for verificiation. Just a little safeguard to make certain that verifiers aren't only getting multiple prints when they are IDs. I know you said something to this effect in your post, but I'd want a sort of set rule so that people have a specific guideline to make certain that it's happening with enough frequency.

Blind verification is not required in ANY OTHER FORENSIC SCIENCE...why latent prints? I work in a lab that does firearms, chemistry, DNA, and polygraph, as well as crime scene processing. NONE of them employ "blind verification," although they all do "Technical Review" on their cases, which requires checking the conclusions of the analyst, as well a number of other things. We in the Latent Print Section do verifications on specific identifications, although not "blindly." And we do Technical Review on homicide and persons crime cases.

As an ASCLD lab, we already document the living crap out of everything we do. Personally, I am never going to sit down and do the kind of detailed narrative on each and every little smudge that the defense thinks is necessary. We all know that's merely an obstruction of justice move, anyway. In a lab with any kind of volume of cases, this is impractical and obstructive.

We don't know what kind of notes this examiner took, whether it's an ASCLD lab, or what. This decision obviously upholds the science of latent print analysis and the validity of conclusions. What it is stuck on, is the documentation process. I think someone should ask the court what IT thinks would be adequate documentation and see if you get an answer. It's very easy for people who don't do latent print work to have opinions on what we should do, without the slightest knowledge of what the procedure actually entails.

If you work in a lab that gets 10 cases a month, then, sure, sit down and describe in loving detail every single powder particle you find on the lift card. If you work in a lab that gets 150 cases a month (like I do), then that's crap.