Quality Assurance Measures

[Michele]

In Charles Parkers post on “Who or What is the Latent Print Community?” he mentions a great topic in the below statement, referring to SWGFAST:

“In looking at “Trained to Competency (ver.2.1) I see several key words such as “Understanding”, “Knowledge”, “Ability”, and “Application”. These are all fine words, but there is nothing in the document to measure when a person reaches understanding, or knowledge. There is nothing there to test ability or to check on application.”

In this weeks Detail, Dave Charlton states:

“Findings in earlier studies have highlighted that fingerprint experts are vulnerable to contextual influences. It will be important to try to ascertain how these influences manifest themselves, how such influences might be minimized or eradicated, and what steps could be taken to enhance training, accreditation and methodology to improve the resistance of experts to such influences.”

The need for better quality assurance measures seem to be continuously noted but it seems like little is being done to implement such measures.

First, as Charles points out, there is nothing to measure an examiners understanding. Many industries have some sort of testing process to determine competency. I’ve looked into a few of these tests (Professional Engineers, CPA’s, Amer. Medical Association tests, and the Bar exam) and it looks like the most credible professions have tests that are justification based, not just conclusion based. In the US we’ve had the IAI Certification Test available to us and the CTS proficiency test (both require conclusions only, no explanation of how an examiner arrived at the conclusion). Recently other companies are coming out with proficiency tests (two companies that I’m aware of), and these tests also only require a conclusion. I asked one company that if their goal is to have a better test, why don’t they make it a justification based test? Their reply was that the industry doesn’t promote that at this time, neither ASCLD or SWGFAST. I’ve also heard that the IAI is looking into an additional certification test (for those that do this job but can’t take the present test because they don’t qualify yet). It seems to me that if we want to be considered competent we need to show that we understand what we are doing. External testing seems to be more objective than internal tests, but if the external tests available aren’t sufficient then I think agencies should boycott these tests and use internal testing until adequate external tests are available. Or maybe continue the external tests and add quality internal tests that assure examiners came to their conclusions reasonably. I think justification could also be implemented in actual casework. If the examiner's in the Mayfield case (and Dror’s study) were required to document their reasoning then it would have been easier to determine why the error happened. Without this information, it seems like people are just speculating as to the cause.

In addition, Dave mentioned enhancing training, accreditation and methodology. When Roy Huber first articulated ACE + Verification he said the verification process was “the most reliable form of proof”. The entire method (referring to scientific methodology) is “constantly critical of itself, entertains no dogma, maintains no absolutes or infallibilities. It is both cautious and skeptical”… “In childlike fashion it constantly probes by asking why? And constantly provokes by challenging “so what?” (1962)

I know I’m reading a lot into this but this but it seems like Huber advocated verification as complete peer review and over the years our industry has accepted a limited view of verification as merely reproducing the conclusion. I think in 99% of conclusions reproducibility is fine but to ignore that complete peer review may be needed in some instances is dangerous to our conclusions, our industry, and our credibility as experts. I think that in some cases the verifier should be saying to themselves, “yes, I can reproduce this conclusion but (as Huber states) “so what”, does that mean the original examiner saw this or that? As the verifier (or perhaps the peer reviewer), I think additional documentation is needed or I think the examiner needs to explain a certain dissimilarity”.

In summary, I think expanding our methodology to include complete peer review (the verifier asking, “What were you seeing and basing your conclusion on?” - even if used in a limited capacity) and acknowledging the benefits of justification based conclusions, not just conclusions, are all things that would help examiners. I believe that solutions to some problems are right in front of us but we may just be ignoring their value. These might not be perfect solutions but could these be solutions that are headed in the right direction?

[Whisler]

To be “competent” and “able” can be taken into a few different arenas. A USDA stamp would help to say who’s qualified and who’s not.

We have to keep in mind that regardless of the standards the community sets in place, the courts have their standards that supersede ours. Certainly one set of standards can influence the other but none the less what my credentials say are subject to the courts interpretation and scrutiny. They will determine if I’m qualified or not, and unfortunately that can change from day to day even if it’s in the same courtroom.

As far as using the IAI Certification as a standard, I personally don’t feel that is necessarily a fair standard. I know a few people who I believe are qualified for the test but it’s not as easy living in rural America without Departmental support. When you look at testing fees, personal expenses, then you have to add in travel, lodging, and then there’s always that potential for failure. When you’re raising a family you have to think, “Could my family have used that money for something more important?” I’m curious of what demographic data is collected on the CLPE licenses and what conclusions could be drawn from that information.

It’s just hypothetical, but imagine if there were ever an IAI certification course on-line or a course at the local community college, you’d see some pretty serious numbers then.

I strongly agree with you Michelle that justification based examination would benefit Accreditation and Certifications processes, especially in this forum where we’re primarily dealing with fingerprints and opinion based testimony. It seems that it causes examinations to remain fluid, however many governing bodies don't seem to like the concept of having their tests change. I’ve spoke with agency Administrators about this concept and they’ve expressed their fear of liability and allegations of being unfair since they “changed the test for this person and for that person but not for them.” From an administrative position I suppose it’s just simpler to avoid it altogether.

I am deeply concerned that we are beginning to cater to a lesser standard. Especially with the technology boom in our labs. I remember when we used to have to dust for prints outside in 3 foot of snow, on a hill, barefoot. Now days, it’s all lasers and stuff. It’s just getting too easy and I’m afraid that some are loosing their grips with where it all began, and why, and how we got here, and what it’s all about. Training is the key, but you can’t force it on them, and you can’t make them accept it, and you can’t make them pick it up and run with it when you done with them. From a trainers point of view, we have a unique burden and obligation to face these challenges and come up with new ways to try to overcome them.

Now, I don’t want to sound contradictory hear, but Michelle said something new about the IAI possibly offering a certification for those who don’t qualify for a regular one? Does this mean that my daughter can get a license to drive because she’s not too bad of a 13 year driver due to her lack of experience? I hope to get the rest of the story of this potential IAI ½ certification issue. I don’t believe there should be a lowering of a standard because not enough people are qualified, or to boost membership numbers, or to get membership fee$. I believe Oklahoma has a State law stating that latent print testimony can only be performed by personnel IAI certified of employed by ASCLD Certified facility. In that case I can understand it having varying degrees of IAI certifications, however I’d prefer to see the IAI allow each state chapter oversee the individual State Certification process of this “lesser standard” in compliance with the IAI’s guidelines. There may be other States in a similar situation, which is why I believe this should be a State issue, governed by the IAI.

[Steve Everist]

To be “competent” and “able” can be taken into a few different arenas. A USDA stamp would help to say who’s qualified and who’s not.

We have to keep in mind that regardless of the standards the community sets in place, the courts have their standards that supersede ours. Certainly one set of standards can influence the other but none the less what my credentials say are subject to the courts interpretation and scrutiny. They will determine if I’m qualified or not, and unfortunately that can change from day to day even if it’s in the same courtroom.

This is a good point, but in lieu of the courts having enough knowledge to determine what is and what isn't, it would be nice if the largest association regarding latent print identification could give them something to hang their hat on.

I strongly agree with you Michelle that justification based examination would benefit Accreditation and Certifications processes, especially in this forum where we’re primarily dealing with fingerprints and opinion based testimony. It seems that it causes examinations to remain fluid, however many governing bodies don't seem to like the concept of having their tests change. I’ve spoke with agency Administrators about this concept and they’ve expressed their fear of liability and allegations of being unfair since they “changed the test for this person and for that person but not for them.” From an administrative position I suppose it’s just simpler to avoid it altogether.

I think another issue is that it's a lot easier to objectively grade a result as compared to a process.

I am deeply concerned that we are beginning to cater to a lesser standard. Especially with the technology boom in our labs. I remember when we used to have to dust for prints outside in 3 foot of snow, on a hill, barefoot. Now days, it’s all lasers and stuff. It’s just getting too easy and I’m afraid that some are loosing their grips with where it all began, and why, and how we got here, and what it’s all about. Training is the key, but you can’t force it on them, and you can’t make them accept it, and you can’t make them pick it up and run with it when you done with them. From a trainers point of view, we have a unique burden and obligation to face these challenges and come up with new ways to try to overcome them.

I think what's happening is that there never has been a standard, we are actually looking back to how we got here, and we're deciding that we need to do more than what we've always been doing. The courts aren't necessarily raising the bar as much as they're finally giving us one. We can't just come in and say that we've always jumped over it even though it was never there. But we do need to show that we can now.

Now, I don’t want to sound contradictory hear, but Michelle said something new about the IAI possibly offering a certification for those who don’t qualify for a regular one? Does this mean that my daughter can get a license to drive because she’s not too bad of a 13 year driver due to her lack of experience? I hope to get the rest of the story of this potential IAI ½ certification issue. I don’t believe there should be a lowering of a standard because not enough people are qualified, or to boost membership numbers, or to get membership fee$. I believe Oklahoma has a State law stating that latent print testimony can only be performed by personnel IAI certified of employed by ASCLD Certified facility. In that case I can understand it having varying degrees of IAI certifications, however I’d prefer to see the IAI allow each state chapter oversee the individual State Certification process of this “lesser standard” in compliance with the IAI’s guidelines. There may be other States in a similar situation, which is why I believe this should be a State issue, governed by the IAI.

I would say that it’s more like your neighbor’s 20 year old son who’s old enough to drive, went through driver’s education training, lives in a town where there are no laws requiring him to be licensed for driving a car, and decides that he may want to travel in some towns that do require the license. He could get a Commercial Driver's License but he doesn't have any immediate plans to drive a commercial vehicle where the license would be required. Or, he wants to get a job delivering pizzas and they do require a driver's licens in his town to perform those duties. He may also look for the same work elsewhere in towns that require a driver's license in general. He never needed it before, but realizes that it would be a benefit for him to have it, even if he decides not to deliver pizzas.

It’s also about his 45 year old father who has been driving in that same town for about 30 years and never left. He never needed a license either, so he didn’t get one. He’s been driving a commercial truck for years too, but never outside of the town. He’s got the option of getting a Commercial Truck Driver’s license, but the expense to do so doesn’t make sense to him as his work won’t reimburse him and he’d likely never need to as he doesn’t make deliveries that take him where it would be required. He knows that if he were to take a job out of town that he may need it, but that isn’t in his future and the costs outweigh the benefits for him.

But seeing how his son is considering a move that would mean that he’d possibly travel into an area that requires the driver’s license, he may want to consider it. He could possibly get by without as he’s never had any problems with his driving and has been fully trained to drive and many consider him an excellent driver of both automobiles and commercial vehicles. But if the need ever came up, it would be a lot easier to show that he’s got his basic driver’s license showing that he’s reached a level of state-mandated competency, even though he wouldn’t bother getting his CDL…

You mention the idea of “lowering the standard,” but at this point there is no standard. The IAI’s current certification level hasn’t been determined to be a standard for competency, and it is more often referred to as a level of excellence. Whether that’s true or not could be a discussion in and of itself. I think the idea behind this new level of certification would be to set that standard for competency, while allowing those that are in a position that they would have to testify, but do not meet the requirements for the current certification level, to be endorsed as competent by the IAI.

It also allows those that have all the issues that you mention regarding the certain certification test, along with those that are completely competent in practice but don’t want or may not be able to pass that level of test, to at least show in court that they do meet a minimum competency as put forth by the IAI.

I’m a little confused by your mentioning of certification being “a state issue, governed by the IAI.” The current certification test is proctored by a member of the LP Certification Committee from your local division. A quick look at the Texas division site shows three members of the LP Certification Committee. So isn’t that how it’s currently being done with the single certification level, or am I misunderstanding what you mean?

I would also hope that if there were a competency level of certification, that the current level would be tweaked so as to truly represent a level of excellence in comparison to that which is considered the standard of competency.

[Graham F]

I don’t believe there should be a lowering of a standard because not enough people are qualified, or to boost membership numbers, or to get membership fee$

Have not the IAI already embraced the concept of acceptable lower standards? The certification examination has now moved from 6.5 hours duration to 8 hours. Could this be construed as a lowering of standards?

We witness employment announcements stating that a Certified LPE is required who has successfully completed the test. A “grandfathered” CLPE is not acceptable. Are we soon to see the appearance of vacancies demanding Certified LPE’s required who have completed the 6.5 hour exam, in preference to the new 8 hour exam?

I am aware that examination applicants have purposefully failed their comparison portion of the exam, but successfully complete the Pattern Interpretation and True/False exam. Subsequently when re-sitting the exam, are allowed to utilize all the allotted exam time for the comparison section (Section 3. F.5 Certification Requirements). Being cognizant of and allowing people to circumvent the system, thus lower standards have already become acceptable, or not?

[Whisler]

Steve, I re-read what I wrote and it confused me a little too. I know what I meant, but it didn't come out right. It goes into a whole other issue of beliefs I have about the parent IAI and State chapters roles and certifications. Perhaps that is something for another post.