I FAILED THE IAI RE-CERTIFICATION TEST

[Carl Speckels]

My name is Carl Speckels and I work as a Forensic Scientist IV, supervisor, at the Phoenix Police Department Crime Lab. I have overseen our Latent Print Comparative training program for approximately 14 years, serving as both coordinator of that program and trainer. During that time, I've successfully trained approximately 30 Forensic Scientists, almost all having obtained LP certification through the IAI.

Now, I'm not proud to have failed this RE-certification test, in fact, it's embarrassing and deflating. But the reason I capitalized the subject of this posting is because I'm hoping it captures the attention of those that have either failed (and may feel isolated) or those that have knowledge of, and an opinion on, the "new" IAI RE-certification test. There are many like me that have failed this test, the stats tell me so, and there are many more to come unless the LPCB decides to change the current test.

A few more things about me...

1. I've been comparing fingerprint impressions since 1997 as a Latent Print Examiner
2. I've been an IAI certified examiner since 2003 (passed on my first attempt)
3. I had successfully passed two previous IAI RE-certification tests (yes, the easy ones)
4. I've been required to take and pass an annual proficiency test since 2002 and have never failed (have taken both CTS and FA)
5. I've never had a quality inquiry into my casework, of any sort

A few facts about the IAI RE-certification test...
(there are IAI rules limiting what I'm allowed to disclose about the test)

1. The LPCB reconstructed the test and it became active in April, 2016
2. The new test consisted of 10 latent samples, it was previously 5
3. The new test contains Exclusions, the previous test only contained ID's
4. The test requires a binary choice of either ID or Exclusion with 100% accuracy to pass (unlike other testing vendors that do allow for Inconclusive - specifically, Forensic Assurance)
5. An Inconclusive decision is not a viable option; it results in an automatic failure of the test

I am currently in the process of appealing my test. I am attaching the appeal that I submitted to the LPCB to this posting and I would encourage you to read it in its entirety. A few key points that I make in the appeal:

1. The test has a 9% failure rate; this equates to approximately 80 certified examiners losing their certifications over a 5-year cycle (mostly due to Erroneous EXCLUSIONS); this error rate is higher than the published Black Box Exclusion error rate
2. The test has not been properly validated
3. Unlike other testing vendors such as Collaborative Testing Services and Forensic Assurance, the LPCB does not allow access to the testing materials or answers. Consequently, affected individuals and their respective agencies are not able to assess root cause for any claimed “errors”.
4. The emphasis that the LPCB places on erroneous exclusions is excessive
5. As a RE-certification test, it does not test to the same level or test under the same rules as the certification test

Obviously, I have already failed the test and disclosing this fact to you will do nothing to change that. My goal here is to bring awareness to the LP community that this test has inherent flaws. For those that will fail this test after me, consider that it may be due to the construction of the test rather than the skills of the test-taker. Some taking this test may have promotion, rank, and/or pay, even employment, tied to the success of this test. As such, the test should be properly constructed and validated.

[josher89]

After reading through this post (and I will read the appeal in a minute; I promise!), it is starting to seem like maybe I should just let my certification expire, then retake the knowledge, pattern, and comparison tests over again. I'd be starting from scratch but would be less likely to make an error?!?!

[Carl Speckels]

Several of my trainees (new grads) have taken the initial certification test in the last couple years and said the comparative portion of the test took them approx. 3-4 hours. They passed just fine. As you know, the cert test allows the option of choosing 12 of the 15 to compare. That is not an option for the RE-cert. You must compare all 10 and get all 10 correct. So, your "expire" strategy is not all that crazy sounding. I considered the same.

[BuckOFive]

I recently took the new re-certification test. It was awful. I have over 25 years of experience and I found this test ridiculous. I passed the test but the stress was crazy. The old re-certification test was on the other end of the spectrum. Ridiculously easy. Was there no middle ground? Why wouldn't the new test at least be like the certification test in which you have to get 80% correct (no erroneous) conclusions? My understanding is that the new chair of the cert board doesn't think a test is a good test unless a certain percentage of people fail. What? Seriously? The test should be as close to case work as possible and this test is the exactly opposite of that and everything we in this profession preach about not over extending yourself etc. There is no transparency and so many other things I don't have time to go into now. It was cavalier and reckless for this new test to be put together this way. It has put many examiner's career on the line. I am not advocating for the test to go back to the easy re-cert test. But lets be practical and professional.

Thank you for putting this out on this board. My fear is that many examiners have been stressed out by one of these tests but passed and won't rock the boat or have failed and be ashamed of talking about it. This is exactly how some bad decisions never get corrected.

[Bill Schade]

I am also a re-certification failure!

I'm not proud of that (It's kind of embarrassing) however I don't think that a "Miss" (erroneous exclusion) is an error in the same sense that an erroreous Identification is. So I have no problem admitting it in a public forum. I too have heard that I am not alone in failing the test on a "miss". I've posted before on my view of exclusions so I won't repeat it here. I seem to have lost that argument so I'll accept the need to test an examiners ability to exclude.

I don't think the answer is to take the complete test though. Although the comparison portion may be more forgiving, the written test has become a challenge for examiners. I say this because as a former test proctor I saw how many candidates for certification failed the written test after correctly completing the comparisons. This is a complete reversal from the days when everyone passed the written portion of the test but many could not complete the comparison protion.

I agree with Carl that it might be time to stop reacting to issues one by one and rethink the entire concept of testing for certification. Changing the time allowed for the test, re-doing the comparison examples, changing the focus of the written portion, changing the rules on re-testing after a failure, and offering the test electronically instead of hard copy are all band aids to issues as they arise.

Historically the test has been an evaluation of general fingerprint knowledge and "can I find it" comparison testing. In todays environment perhaps it should be a test of an examiners ability to apply procedures and methods correctly. More of a white box test than a black box test.

I would like to see certified practitioners rise up and ask for changes before they are negatively affected. It's kind of hard to protest after you fail the test.

[Michele]

I took my re-certification test a few months ago and I was disappointed in the test... I felt like the test was only testing one thing, my ability to know where to look. The reason this was disappointing to me was because I don't feel this is a skill that is needed by experts, AFIS does this part of the job for us now. To me, the more important elements are a) is the analyst over-interpreting information b) are they overstating conclusions c) do they understand their role and responsibilities? None of these things are tested.

Before receiving my results, I started feeling that it was wrong to use certification as a measure of my abilities; it does not measure my knowledge or ability, so I removed CLPE after my name at the top of my CV. Since that time, I've known quite a few people who have failed the test and I've told them all the same thing. It's a shame that this test does not test the important aspects of expertise, and therefore it is not a proper measure of anyone's knowledge or abilities. I don't think anyone should feel bad for not passing this test, to me, finding a needle in a haystack is a game of chance, not a skill or an ability. I know that for me, passing the cert test was just luck. I'm hoping that this test, as well as others, will someday state what skills are being tested, instead of just testing to see if practitioners can guess at ground truth conclusions.

Carl, I'm so sorry this happened to you but I really think that this is the only thing that makes people look at problems within the discipline, and make improvements. Someday testing will be better, and I really believe it will be due to those of you who are willing to speak out. I know it's not easy but I applaud your willingness to share your story.

Michele Triplett

[Steve Everist]

Thanks for sharing, Carl.

Hopefully this will spark some good discussion about what a failed recertification means.

In our office, we do not have separate tiers of LPE. However, there is a $50/month premium that is added for CLPE status. This is collectively bargained through our guild. So the financial cost of failing a test exists, but is not as significant as it could be for those people in a position that requires obtaining and retaining CLPE status for the job position.

I'll give an example, based on currently available jobs on the IAI's website:

Both are from Santa Clara County, which may be helpful since Tim is active on CLPEX. Or it may not and I may be looking for another presenter for Trending Topics at the conference in San Antonio... :

Latent Fingerprint Examiner II

Salary
$39.35 - $47.61 Hourly
$3,148.00 - $3,808.88 Biweekly
$6,820.67 - $8,252.57 Monthly
$81,848.00 - $99,030.88 Annually

Position:
A successful candidate would be expected to obtain and retain certification as a Certified Latent Print Examiner. Possession of a Bachelor's Degree is highly desirable.

Latent Fingerprint Examiner III

Salary
$44.28 - $53.57 Hourly
$3,542.16 - $4,285.68 Biweekly
$7,674.68 - $9,285.64 Monthly
$92,096.16 - $111,427.68 Annually

Employment Standards:
Possession of and the ability to maintain current Latent Print Examiner Certification from the International Association for Identification (IAI) upon appointment.

AND

Four (4) years of experience developing latent prints and analyzing fingerprints for identification in a public jurisdiction or law enforcement agency, two (2) of which include experience as a Latent Fingerprint Examiner II with Santa Clara County or equivalent.

Looking at these, and reading (and to some degree being familiar with) Carl's background, I would expect he would qualify for LFE III (or above). The difference in salary range between the II and III is about $1000/mo. It looks like a III would go down to a II and take a corresponding salary loss, for failing a practical test that is referred to in the IAI documents as one that shows proficiency? In the case of an erroneous exclusion, the waiting period is 6 months and one year for erroneous ID (or multiple misses). I would assume there's a contingency that would be worked out if this were to happen, but what if there isn't? So, this is also a heads-up to agencies that have these provisions. You may want to have this figured out.

The stakes can be pretty high, seeing how some jobs are now requiring certification (as well as states).

And as an aside, I wonder if one of the tests has had a higher failure rate than the other knowing that there are a couple of different tests that are used. And if so, is there one print that's been the cause of more failures than another. With the RS&A results that have previously been discussed, we can get a feeling for what happened based on the data that's released.

EDIT: clarification based on requirements of the positions at Santa Clara PD, now that I've read it better.

[Carl Speckels]

Hey Steve, yeah, the implications can be pretty steep for some, as you pointed out. We have an implied cert requirement here at PHX also. Must be certified to qualify for FSIII and above. Seemed fine enough since nobody failed the RE-cert test given its ease. But now, our administration may consider modifying that requirement in light of the information on the new RE-cert test.

BTW, I asked LPCB Anne Steinmetz, for the associated error rates of both tests, assuming that there were two, and she only provided me an overall fail rate of 9%. One of the latents in the test that I took has an erroneous exclusion rate of 7.6% so I’m going to assume that since that makes up the bulk of the 9%, and it’s only included in half the tests (based on the numbers Anne provided), that the “other” test must be easier, or at least not failed as many times.

[Steve Everist]

Hey Steve, yeah, the implications can be pretty steep for some, as you pointed out. We have an implied cert requirement here at PHX also. Must be certified to qualify for FSIII and above. Seemed fine enough since nobody failed the RE-cert test given its ease. But now, our administration may consider modifying that requirement in light of the information on the new RE-cert test.

And this could affect the IAI too. If more agencies are having to readjust requirements for job qualification, because they don't feel comfortable with the current test, it could result in a reduction in people applying for the test. The IAI has recently raised prices for their (re)certifications, and they may realize a drop in revenue as a result of less interest in taking the test. Certification losing agency support could negatively affect the income generated from the testing process. This could also be a good thing in the long term, because it may force a complete revamping of the system. But it's also tough when they're trying to get the certification testing processes accredited.

The first bullet points of what the FCMB Director's duties are relative to accreditation under 17024 (emphasis mine):

Section 5.01 Forensic Certification Management Board (FCMB)
Section 5.1.1 FCMB Director
Scope:
• The Director of the FCMB is charged with the responsibility of the credentialing boards and testing programs sponsored by the IAI and the maintenance of accreditation standards. This position is a four year appointment.
Qualifications:
• Must be a member in good standing and be familiar with ISO 17024 standards.
• Prior service on the IAI Board of Directors, elected office, or certification board chair, and ISO 17024 Auditor training are recommended.
Duties:
• Standardize the administration of all IAI certification boards.
• Assess and prepare for meeting ISO 17024 requirements for the IAI certification programs or other such accreditation program the BOD shall direct.

[Pat A. Wertheim]

Following is some boring historical perspective and personal commentary from an old guy who applied for CLPE in 1980, was tested & certified in 1981, and has maintained certification continuously since then.

When latent print certification was first implemented, the test had fifteen latents with fourteen idents and one exclusion, or “non-ident” as they were called then. To pass the test, an applicant needed to mark twelve latents correctly. An erroneous identification was automatic failure, but an erroneous exclusion did not carry a penalty. It didn’t take long to realize that a lot of people were passing the test by making eleven idents, then marking the remaining four latents as non-ident. After a few years, the test was changed to fifteen latents that were all ident. After all, our job was to say who DID touch a surface, not speculate on who did not touch it.

Back then, a single erroneous identification in casework was a career killer. You were through. Finished. Fired, or transferred to some other unit in the police department. Look for a new career in fast food, because nobody will ever touch you in a latent print unit ever again.

But while an erroneous identification was a cardinal sin, an erroneous exclusion was pretty much a nothing-burger. Sure, if somebody consistently missed the Helen Kellers, they were reassigned to crime scenes or ten prints. But it was universally recognized that every latent print examiner occasionally misses a partial latent with no delta or core. No big deal.

The idea was that we can never prove who did NOT touch a surface. Even excluding a person as the source of a particular latent does not prove that person did not touch the surface. Our job was to identify who DID touch a surface.

We also weren’t too concerned about missing a difficult identification. The philosophy was that erring on the side of conservatism was okay, whereas stretching too far to make an identification was totally wrong and dangerous, even if you were correct.

In the early days when there was no recertification test, some of the old grandfathered examiners could skate into retirement and remain certified for decades after they had last looked at a fingerprint. Those of us pushing for a recertification test started applying for new certification every three years (the certification period then) to make our point. That was never encouraged by the certification board and was finally stopped a few years ago. Today, you MUST take the recertification test, not reapply for new certification.

The shifting concept of “forensic science” since the NAS Report has come to view erroneous identifications and erroneous exclusions as equally serious. Any conscientious latent print examiner with a fair grasp of our role in the criminal justice system recognizes that the two are NOT equal. Nonetheless, now it seems that “science” demands we consider an erroneous identification and an erroneous exclusion equally bad.

We are told that the certification test and proficiency tests are supposed to be handled like routine case work. If we miss an ident or make an erroneous exclusion in real case work and it is caught through consultation or verification and is not reported out, there is little or no penalty in most agencies. But you cannot seek consultation or verification on either the certification test, the recert test, or most proficiency tests. Combine that with the fact that the penalties for a technical error on a test are far greater than the penalty for a similar technical error in real casework and you have a terribly lopsided situation.

Make an erroneous exclusion in case work, it gets caught during verification and technical review, and, at most agencies, nothing happens except maybe you get the stink-eye from the boss. If you make an erroneous exclusion on a proficiency, you may get hit by a CAR. I’ve been hit by those CARs twice now. The first was on the 2010 spring CTS test in which about 20% of examiners made the erroneous exclusion on the same latent. Even though CTS retroactively through the latent out, those of us who made that erroneous exclusion still carried the stigma.

The second erroneous exclusion I made was more recently when I fell victim to the expectation Bert Rivera referred to recently in another thread on this chat board when he questioned the ethicality of CTS having two exclusions for a number of years and creating an expectation there would always be two exclusions, then throwing in a test with a different number. While I agree that there should be a varying number of exclusions from test to test, I recognize Bert’s point and confess to stupidly and unwittingly falling victim to it.

My fault entirely on both erroneous exclusions on those two CTS tests. But in my defense, I would argue that I have never made an erroneous identification on any test and I have never been found to have made an erroneous identification in case work.

In our business, a tragic injustice occurs when an examiner makes an erroneous identification in case work and those errors should be treated with the utmost gravity. However, rarely does any serious injustice occur when an erroneous exclusion is made. The penalty for a single erroneous exclusion should not be great and the penalty for frequent erroneous exclusions should be left to the supervisor or the department where an examiner works.

Especially when there is a forced mandate to either identify or exclude and inconclusive is not an option, decertification and potential loss of job is far too great a penalty for an erroneous exclusion on a test.

[Boyd Baumgartner]

I went to the IAI's current certification roster page, downloaded the list and then went to the Wayback Machine's cached version from Mar 5, 2016 and downloaded the same document. I then imported them into a database and queried for what was on the 2016 list but NOT on the current list, meaning the certification had expired.

As you would expect, Carl and Bill were on the list as they have stated and so were 175 other people. This represents a net loss of $53,100 to the organization. Now, I realize that the list will contain people who just let their certification expire for whatever reason and there's an embarrassment factor potentially associated with the list, so I won't post it. However, error or no error I find it odd that one of the people on the certification board is on the list as no longer being currently certified. I would think that would be a condition of being and remaining on the board. Who knows, maybe it's just a timing issue based on how frequent the list is updated.

Also, I'm curious why people are taking the test when it's not only not 17024 accredited, but also not 17043 accredited considering the IAI describes the certification as both a proficiency test and a measure of competence. However the re-cert specifically only includes comparisons. If the history/oral board(testimony) portion is so essential to the certification, why isn't it re-tested during re-certification? The change in testimony has been a hot topic over the last five years.

Additionally, I found several things odd that on the IAI's latent certification introduction page :

Throughout history, man’s environment has dictated the necessity for every workable system to include a means of checks and balances. This need has long been recognized in various professions to ensure the moral character, integrity, and competence of the individual practitioners.

Until recent years, however, the latent print identification discipline has been divided over the means and types of checks and balances which should be utilized. Many believed that control should be maintained on the local level, thus allowing each individual laboratory to set its own standards of conduct and professionalism. Others have contended that one is an expert only after having been accepted by the court as an expert. Neither view is acceptable, primarily because the examiner is not being judged by his peers but rather by individuals who in most cases have limited knowledge of the latent print discipline..

So, if checks and balances are essential, what is the check on the IAI's Latent Print Certification Board?

There's also this blurb:

The need has been recognized to objectively and unequivocally identify latent print examiners qualified to provide essential professional services for the criminal justice system.

It seems the re-cert is nothing more than a garden variety proficiency test, but it's not accredited and it doesn't even conform to the conclusions on the DOJ's Approved Uniform Language For Testimony And Reports For The Forensic Latent Print Discipline. It also fails to ensure proper testimony, given all the consternation surrounding Identified versus Individualized, exclusion to all others, etc. How many people submitted their initial certification with transcripts containing mentions of individualization? That is no longer fashionable, so their own standard of 'unequivocal' cannot be measured.

Lastly there's this:

Given the successful operation of this certification program for more than three decades, the first question asked of a latent print examiner prior to their offering expert testimony should be whether the individual is, in fact, certified. If the answer is “no,” it is the right and obligation of the criminal justice system to demand a satisfactory explanation as to the reason the witness has not been certified.

So, as someone who is not certified my satisfactory explanation is this: You cannot ensure to me that people who are on the board can pass your test. The test fails to conform to the best practices of accreditation as outlined in the NAS report and it does not ensure accurate conclusions on the stand. Not only does the test not conform to those conclusions as outlined in the DOJ's Uniform Testimony language it does not test for them in the comparison portion because it fails to provide standards by which to arrive at those conclusions. Lastly, considering that the language of testimony has changed, a re-certification cannot ensure that the testimony of the certified examiner is acceptable by today's standards.

[josher89]

Everything Boyd said

(mic drop)

[Carl Speckels]

However, error or no error I find it odd that one of the people on the certification board is on the list as no longer being currently certified. I would think that would be a condition of being and remaining on the board.

So, if checks and balances are essential, what is the check on the IAI's Latent Print Certification Board?

Given the history, authority, and implications, that this test carries, I expected that it would have been properly constructed and validated. I was disappointed to find, per LPCB member(s), that the test had not been properly validated. Only those on the board, those that selected the test samples, those that constructed the test, and those with intimate familiarity of the test, were the ones, the only ones, to "validate" the test. In fact, it was communicated to me (again by a LPCB board member) that the board reviews the test annually and based on the "high pass rate" their assessment was to make no changes. Their philosophy is that since 91% of the test-takers are passing that it must be okay. Note: there appears to be two tests, only one of those tests (half of the test distribution) gets the latent with the 7.6% associated error rate so it appears that just one of the tests (the one with the 7.6% latent in it) is responsible for the bulk of the failures (this is purely conjecture on my part, I don't know it to be true). Anyway, the board fails to realize that 91% pass also equates to 9% fail. That is very high for a test that is being given to seasoned, certified examiners. Imagine if the medical field was losing 10% of its doctors.

A previous post by BuckOFive referenced the LPCB's Chair maintaining that the test must have an inherent failure rate for it to qualify as a valid test. I have heard the same thing. This is skewed. By this logic, if the top 100 examiners in the world (whoever they may be) were assembled to take the RE-cert test, 9 of them would have to lose their certification in order for the test to be valid. Does that make any sense? People that once held certification, have demonstrated and proven their skills and abilities on a daily basis, have passed annual proficiency tests (ones that are actually validated), are losing their certification. Not because they aren't worthy of certification but because the LPCB has failed to produce a properly constructed and validated test.

[NRivera]

There are obvious issues with the current re-certification test. Thankfully I passed mine. We can spend eternity debating what those issues are. I agree with many of the issues already raised here and strongly disagree with others. The truth is, none of what we debate here will matter unless it is communicated to the cert board and put on a meeting agenda. I would strongly suggest that those fortunate enough to be able to attend the conference this year (I can't go as it's "not my turn".) make time to attend the certification board meetings. Talk to the members of the board of directors. Make your voices heard and participate. If you feel up to the challenge, throw your name in the hat for nomination to the certification board. It seems to me that there needs to be some philosophical discussion as to what this test is supposed to be and evaluate vs. what it currently is. That discussion happens in the board meetings and as IAI members we vote for those representatives who sit in the meetings. I would throw my name in the hat if I could attend, but maybe next year.
FWIW, our discipline recognizes 3 possible conclusions that can result from the evaluation process. Any test of competence should use the same criteria that we apply to casework. My perspective is that a competent examiner should be able to correctly assess the data that is there. That doesn't always equate to arriving at the same conclusion and it shouldn't equate to failing some test. The failure rate should be irrelevant to the validity of the test.

[ekuadam]

I also failed my re-cert test this past December. I guarantee I knew which one I missed because I searched for it up until the final day I had to mail it back. It cost me on my paycheck as where I work we get paid a little extra if we are certified. I am waiting to be sent the recert test to take again and hopefully pass it. I have no knowledge of how the old re-cert test was ( I have heard it was extremely easy) so I can't comment on it. People I talked to who haven't taken it yet were surprised to hear that people actually don't pass it as they were used to taking the test that was 5 latents where they were all ID's. I do believe you should be allowed to go Inconclusive on the test as that is a conclusion that most labs have.

Having moved back over to the processing side as the Lead Processor I don't need my certification as I don't compare but I want it back as it's a matter of pride for me as I worked my #$$@ off to get certified in the first place.