Page 1 of 1
Disagreements between experts
Posted: Fri Aug 15, 2008 2:43 am
by supersleuth
I'm curious to how other fingerprint departments deal with the following situation.
An examiner identifies a latent print. The case is given to their line manage for 2nd or 3rd verification and because the line manager 'doesn't like it' they conclude the latent print is insufficient. I know of instances where the 2nd checker has agreed with the identification but the 3rd checker has 'insuffed' the latent print and therefore vetoed the identification.
Is the first examiner incorrect? If the 1st and 2nd checker agree is it right pass around the identification until you find a 3rd checker that will agree?
I also know of a case where the identifcation was checked by 10 experts all agreeing but the line manager had already contacted the Investigating officer and told them it was 'not identical'!!! Unhappy with this action they sought advice from senior management and were told to 'leave it' because the final decision had been made.
How do your bureaux deal with these instances. I'm sure they occur all over. Do you have a panel of experts that can analyse these cases??? In small bureau is that even possible?
Re: Disagreements between experts
Posted: Fri Aug 15, 2008 7:26 am
by Michele
I’m guessing this isn’t an accredited lab or they’d have a conflict resolution policy. I’m also guessing this lab doesn’t follow the IEEGFI because they have a similar recommendation. And SWGFAST guidelines too or they’d have to state which recommendation for ‘Standards for Conclusions’ an ID goes against. I think we’d have to see the policies of this lab to understand the reasons behind the situations you’ve given.
While it used to be common for agencies to only record the examiner and the verifier, I know of many agencies that have raised the bar and now also document anyone who doesn’t agree with a conclusion. When agencies undertake this additional QA measure, 1st) I think it shows that they’re trying to be as transparent as possible, 2nd) this policy diminishes the possibility of verification shopping and 3rd) when an agency is being this transparent and someone disagrees then the agency must go beyond just having 1 verifier that agrees with the conclusion. When this happens some agencies require additional verification (3 or 4 people, a panel of people, or an outside agency), some agencies require blind verification, some agencies require some sort of documentation (a chart), and some agencies require written justification behind all conclusions (the basis for it being an ID and the basis from the person who thinks it’s not).
Re: Disagreements between experts
Posted: Fri Aug 15, 2008 7:55 am
by Gerald Clough
My situation in the past is not typical of most examiners who participate here, but it's not all that unusual overall. I was the only examiner who could really be called current in my agency, perhaps the only one in the county. There were others with the basic training, but they didn't pretend to competency in latent comparison. My verifications were done elsewhere, through the good offices of associates at other and larger agencies. I had determined that if one of us believed the latent to be insufficient, I would not sponsor it as an identification. That would apply if it was reported to me that another examiner has also worked on it and had decided it was insufficient. Although I would have to rate any of the potential verifiers as more experienced than I was, if I could not settle in my own mind that it was sufficient, I would not sponsor it. The difference in my situation and that of most here was that I either had ultimate control of the entire investigative case package, or I had a lot of influence on how it would be presented.
It is my position that in non-trivial cases, there is no absolute threshold that I could apply to decide the matter. In these non-trivial cases where sufficiency is close to some speculative limit, the close call goes to the side of inconclusion. This, of course, involves an attempt to discriminate between degrees of tolerance and degrees of experience. Even there, I find it difficult to argue convincingly that, somehow, a more experienced examiner can "see more." They can, however know more. Note that in the first paragraph I did not reject the case where a more experienced examiner can show me, in a way that I can understand and rationally accept, why the identification can be made. We have all done that - extended our knowledge and abilities through having difficult cases explained by mentors.
Of course, the posted question is really more about organizational matters than a global issue in examination. It would be absurd to claim that there will not be conflicting opinions or that every question can be settled by discussion and argument. I think that, one day, when defense expert consultation becomes the norm (as it seems to be becoming the norm in some areas, at least according to a telephone conversation with another examiner), prosecutors can reasonably ask to be informed of conclusions that are likely to face reasonable attack. Internal difference of opinion certainly indicates that it could be expected. What then? While I was writing, Michele's post appeared and presents exactly what I would want, if I were the prosecutor taking the evidence into court, information that the conclusion may have some vulnerability and development of the demonstrative evidence that will be used to argue.
Re: Disagreements between experts
Posted: Fri Aug 15, 2008 9:07 am
by Charles Parker
This kind of topic comes up on ocassion throughout the years and before I get into the negative side I want to say that probably 98% of the Latent Print Examiners I KNOW would agree with Michele and Gerald. However there are a few that might respond with.
1. Sweep it under the rug and forget about it.
2. Don't be waking sleeping tigers.
3. We do not want to air our dirty laundry in public.
4. If you do anything it will just look bad for the discipline.
5. If you do anything then it will be harder for us in court.
And probably several more I cannot remember.
As Gerald and Pat have said in a previous thread and Michele has repeated in this one. We need to be transparent. Yes it is going to hurt but it will heal fast. Besides what does not kill me makes me stronger (cliche).
Supersleuth---sounds like you have a management problem. I wish you luck in fixing it, because if they cannot see it themselves then no one is going to be able to show them.
Re: Disagreements between experts
Posted: Fri Aug 15, 2008 8:50 pm
by Les Bush
Hi Charles,
In our office we have a policy similar to SWGFAST. If I was on the end of an 'insufficient' outcome there are two things that would need to occur to satisfy my interest. Firstly and coincidently it is an Olympic year so how do we gauge selection of athletes to represent our countries; trials and tests. Since fingerprint examinations rely on the personal ability of humans they become available for examination as to who is the best in the office. The second part is the tool or instrument needed to display the result, a measured outcome such as fastest time in swimming or longest distance in the javelin. Fingerprint science in this generation badly needs an appropriate tool that will capture the decisions of experts. To date the best criticism Cole has offered is that we currently operate in the realm of the 'black box'; the mind of the expert. A tool is required to record the decision process and allow all the information to be collectively discussed and assessed, a true forensic forum. Since we now operate in the digital environment it is easy to put together a computer program specific for fingerprint examinations. Thankfully our office has done that, we called it the digital comparator. Ive recently completed a research paper called "Metric tool for fingerprint examinations" in which I recommend the McKie print Y7 should have all the experts who are presenting evidence undertake their examinations using the metric tool. Our future relies on being relevant, current and innovative, gone should be the days of 'illustration charts' that dont reflect the actual fingerprint examination.
Re: Disagreements between experts
Posted: Sun Aug 17, 2008 7:09 am
by Charles Parker
Hi Les, I hope things are going good in the land of “Oz”. Our office is accredited so we have policies to comply with that. We also have a smattering of SWGFAST since we do not do any processing, 10-prints, or digital photography those policies are taken up by other sections.
… end of an 'insufficient' outcome there are two things that would need to occur to satisfy my interest. Firstly … Since fingerprint examinations rely on the personal ability of humans they become available for examination as to who is the best in the office. The second part is the tool or instrument needed to display the result, a measured outcome….
For the first part our verifications and reviews are on rotation (no shopping trips for the best or someone likely to agree). However we compensate by having an open office of consultation and presentation on those latent prints that are deemed borderline, or complex. Your second part has got me hooked.
To date the best criticism Cole has offered is that we currently operate in the realm of the 'black box'; the mind of the expert.
Personally I think his best is “Error Rate”, however I would also think that Stoney’s “how low can we go” is also right up there instead of the documentation issue, but like I said you have me hooked so proceed.
A tool is required to record the decision process and allow all the information to be collectively discussed and assessed, a true forensic forum…… we called it the digital comparator. Ive recently completed a research paper called "Metric tool for fingerprint examinations"
Hopefully you are going for publication soon as I would be interested in reading it. Is it something you do on all latent prints (or examinations) or is it just for those “Mother Bears” that are borderline or have been determined to be complex with areas not reliable for exact feature determination? I fully agree that more documentation of the decision process on those that do not show a robust representation of the 3 levels of detail would be called for. If the policy for such documentation would be ALL examinations then I would want to ponder that carefully and determine how user friendly it is as well as other areas of operability. It has been my limited experience that persons who develop systems often overstate the ease of use since they have been with the project from its conception. They are sometimes bias on the operability since they have been with it for so long and gave birth to it from their own concepts and ideas.
Perhaps you could give us more information such as does the documentation also contain other information such as substrate, matrix, distortion, levels of detail (clarity) and placement of focal areas (if any).
Code: Select all
Our future relies on being relevant, current and innovative…..,
I cannot argue with the relevant, current and innovative approach, but the key word being relevant. Being a minor historian the floors of this discipline are littered with the bodies of concepts and ideas that might have been considered innovative but were just not relevant and applicable to real time problems. I can remember Brattley, and then Miracode or the Osborn or Osterburg grids, or the polygonal model and the list can go on. Then there are those that were touted as the best practice but over the years have been relegated to very specific or narrow tasks. But then for every five ideas or concepts that are brought forward one or perhaps two rise to the occasion while others float to the bottom to become the sediment of our history. I do hope your idea, concept and hard work is one that will rise to the occasion of the needs of the discipline and look forward to seeing it in more detail.
.....gone should be the days of 'illustration charts' that don’t reflect the actual fingerprint examination.
Now this statement caught my eye. A number of years ago the agency I worked for at the time the Firearms Examiners (FAE) would photograph the striations on some of the L&G of bullets or the impressed markings on casings for the purpose of documentation and on occasion to present in court to the jury. Several other agencies found out and were upset that we were doing that. Their reasoning was the same in that the photographs do not reflect the actual firearms examination and they should not be presented in court because we have the training, experience and knowledge and you cannot impart that to a jury by a photograph---we are the experts and they need to learn to take our word for it. Their reasoning still escapes me to this day. I agree with you that they do not reflect the “actual fingerprint examination” anymore than “ACE-V” reflects the “actual fingerprint examination”, but the ‘illustration charts’ do represent a part of the process and I cannot find anywhere where it is supposed to reflect the “actual examination”. I personally think they represent the part that is important to those persons who have no background or little experience. I personally like my Power Point presentations in the court room and they do not represent “the actual examination” but hopefully they do present the basic concepts and help some understand the process a little better. You are going to have to have one heck of a whiz bang program to make me drop my illustration charts and power points to the lay person.
Now back to the problem as originally posted on this thread. One solution I would offer is that SWGFAST write a more detailed standard on “Conflict Resolution” AND provide some models that comply with the standard. The current one is vague and in my POV really does not help at all. One model is not going to work for everyone so 2 or 3 acceptable models would go a long way in pointing the herd in the right direction.
I look forward to your publication and keep up the good work down in Oz.
Re: Disagreements between experts
Posted: Sun Aug 17, 2008 9:56 am
by RL Tavernaro
Supersleuth,
In situations similar to your scenario, I can recall a couple of instances where an examiner from a local agency had the courage (or was foolhardy enough) to take a comparison to an outside agency further up the chain (state or federal lab). The examiners really believed in an identification and it was verified by a peer, but was being overruled by a supervisor. The identification was confirmed by the outside agency, forwarded to an additional outside agency for a second confirmation, and then the original examiner presented their case to their agency. After the fireworks subsided, and amid bruised egos and the further embarassment of dealing with the fallout of an ill-advised formal complaint by an agency manager to a state agency in one case, and a federal agency in another (for correctly reporting an identification & thereby "interfering with personnel issues"), policies were changed for conflict resolution.
Perhaps more importantly, I recall another case in which a supervisor directed an examiner to report an identification that the examiner did not believe in. The examiner was uncomfortable enough to take the comparison to an outside state lab, where the "identification" was determined to be erroneous. Again, there were fireworks & bruised egos, however the responsibilities of the supervisor involved were changed, as were policies at that agency.
It should perhaps be noted that the examiners I am referring to had built up professional relationships (including on a personal level) with the outside agencies they eventually sought assistance from, and it was within the jurisdiction of those agencies.
I am not advocating subversion of agency authority, as I am a firm believer in resolving problems at the lowest possible level. I am merely reporting what others have felt compelled to do when confronted with situations they deemed serious enough, and they had the personal confidence to proceed. Individual circumstances differ, as do the potential risks and rewards. There may also be legal implications to consider.
Regards, RLT
Re: Disagreements between experts
Posted: Sun Aug 17, 2008 5:26 pm
by Les Bush
Hello Charles,
Always good down here in oz. The olympics are in full swing and our little nation seems to be holding its own against the power houses. What about Phelps, a true olympian, absolute class above.
Forgive me but I had a bit of a chuckle when I read your post and found we were in disagreement about the subject we were posting on. That will happen, consensus among specialists is probably up there with being able to win at the high jump. The greatest agreement reached so far is the policy of no set number and even though we have been on board with that since the 1980's it is still in transition across the globe. The wheels of our science turn ever so slowly and maybe that reflects the careful and cautious way we defend our patch and approach our work.
Error rates as a Cole criticism are connected to the black box by the same factor of the human element. What we observe, record and are able to prove will produce the outcomes. How we do that is either traditional methods or emerging technologies.
The Metric tool paper is with the R&D manager of my organisation. There has been interest raised following this thread so I will take the step and seek approval to publish. Alternatively there are at least three copies in your country with friends in the science. The Metric tool has the purpose of use for (Mother Bear?) examinations that require specific result presentations. The digital comparator is used as the centre of our business operations for all latent examinations and within our movement to a paper-less process.
Your issue on the 'relevance' of any technology is appropriate. The digital comparitor really only came about due to the availability of flat screen monitors that prevented curvature distortion. The consensus of choice as to what comparator program is best fit for national/international use is very similar to how the DNA science sorted out which process was best fit for them. Whichever way it goes the science needs ideas brought to the table for discussion and assessment.
The charting issue for court presentations of actual results has come about by reading various daubert challenges and other comments. The clincher has been the diversity of presentation styles used in the Justice One committee for Y7. My hope is that before the Public Inquiry begins there will be a move to have all experts use a relevant standard and one that is technologically sound. To present the courts with an actual examination result requires that the process of ACEV is followed and that an appropriate hypothesis is validated. Would be nice to see how the former SCRO experts handle that task.
Thanks for the thread Charles, rest assured I will keep contributing in the future. The next paper is nearly finished and it discusses theories about how the primary ridges develop. This underpins the purpose of the paper in presenting the observation of the primary ridge surface groove. Cheers from oz. les