I "FIXED" THE IAI RE-CERTIFICATION TEST

[josher89]

@LPE100--thanks for the clarification and it sounds like they were being very approachable about this (and other) issues. I'm glad that they were open to listening to people, understanding the concerns, and making appropriate adjustments as necessary and also taking suggestions under advisement to keep the testing process fair and accurate.

[Carl Speckels]

There was no admission or implication that there were issues with the two latent prints that were shown at the meeting. The Board informed the group that they had decided to share the two impressions that had been brought up specifically in this and/or the other thread. By showing the two impressions to the meeting attendees, the latents would no longer be included in the test.

After having seen the latent prints, in absence of the knowns, it is difficult to say with certainty why an examiner may have erroneously identified or erroneously excluded either of these two latent prints (the Board says that both erroneous conclusions have been reported for at least the first latent). I think based on the latent prints, provided the knowns were of sufficient quality, an inconclusive conclusion, even with documentation, wouldn't coincide with a consensus answer.

I doubt the board would make any such admission. What's interesting is that there was only one latent that was really talked about on this chat board to any degree that would have possibly (emphasis on possibly) compromised it as a test latent, and yet, they removed both. Why both?

As for "having seen the latent prints"...you haven't seen what the test-takers were given. You've only seen what the LPCB wants you to see. The LPCB has published latent #12 on Onin as a very clear latent with little distortion; in fact, the entire latent is visible, edge to edge, unlike the test latent. So, I would agree that it would be difficult to understand how/why a certified examiner could have erroneously excluded it. And yet, many have, including myself. Makes no sense...unless, that's not the actual quality that the test-takers had to work with. And, it's not. I almost laughed when I saw what they represented as latent #12 on their PPT. Doesn't really pass the smell test, does it? The latent that test-takers were given was far worse, areas of the latent were barely visible if at all, and the amount of available information was far less and much more distorted and ambiguous. They are misleading you and the public, in their defense, because they can. Who can say and prove different after all? I would challenge the LPCB to release MY test latent #12. Why not? It's retired now, what could it hurt?

[CarrieB]

[quote=Please forgive my disinclination to read the report, but can anyone advise me whether, having removed that latent and tacitly admitting its inappropriateness, those who failed the test on account of that latent will be allowed a second chance at salvaging their reputations and careers?

That question was asked of the board directly and they stated that those who failed are still subject to the policies in place at that time. Essentially, they are the unlucky test takers who had their recert date fall within this specific two year time frame, where these latents (now removed for all future test takers) were included, and which may (will?) affect them forever.

[Dr. Borracho]

Thank you, CarrieB. I'm disappointed, but not surprised. Good people who are good, career latent print examiners fell victim to an ill-considered test. In my eyes, they are not diminished. The certification program is.

[ER]

As for "having seen the latent prints"...you haven't seen what the test-takers were given. You've only seen what the LPCB wants you to see. The LPCB has published latent #12 on Onin as a very clear latent with little distortion; in fact, the entire latent is visible, edge to edge, unlike the test latent. So, I would agree that it would be difficult to understand how/why a certified examiner could have erroneously excluded it. And yet, many have, including myself. Makes no sense...unless, that's not the actual quality that the test-takers had to work with. And, it's not. I almost laughed when I saw what they represented as latent #12 on their PPT. Doesn't really pass the smell test, does it? The latent that test-takers were given was far worse, areas of the latent were barely visible if at all, and the amount of available information was far less and much more distorted and ambiguous. They are misleading you and the public, in their defense, because they can. Who can say and prove different after all? I would challenge the LPCB to release MY test latent #12. Why not? It's retired now, what could it hurt?

Carl,
Respectfully, I think that this post contains serious accusations and implications and is extremely detrimental to this discussion. We should be focused on improvements to the test, not on implications of misconduct and calls for an end to certification. While your original post was helpful in bringing this topic up for the community to discuss, from my discussions with other examiners, some of your later comments have either turned people away from your side or distracted from the real issue at hand. I've had discussions where I wanted to talk about how a single erroneous exclusion shouldn't result in a failed test, but the conversation keeps returning to how many hours you spend on comparisons vs. supervisory duties. In my opinion this thread has spent way too much time on unnecessary topics that were easily dismissed by most in the community. That dismissal then effectively ends the much-needed conversation into how the re-cert test needs to be improved.

Despite all this, I will briefly respond to your post about example #12 and then expound on what I believe are the true issues that should be addressed.

The LPCB displayed sample #12 on screen along with 22 marked minutiae. The audience questioned the validity of some of the markings, and the board responded that some of the points may be slightly off due to a PowerPoint issue. The LPCB also handed out the actual samples from the test to those that asked. I have one of these "test latents" on my desk right now. If anything, the PowerPoint slide is as good or slightly worse than the actual test latent. Please do not continue to suggest that the board provided you with poor-quality materials. It's not supportable, and it distracts from the real issues at hand.

In my own analysis of sample #12, I found 5 green, 7 yellow, and 4 red points. I find that I typically mark more minutiae than most other examiners, so if I could only find 16, then the 22 displayed in the PowerPoint seems to be reaching. But again, don't let this distract from the actual main issues.

Which are....

Why is there a higher erroneous exclusion rate associated with this latent print?
AND
Should a single erroneous exclusion result in a test failure?

This latent print lacks a clear core or delta. You can see the suggestion of the core at the bottom right, but its exact position is unknown. Orientation is fairly certain but can vary by about 45 degrees. The clearest (green) points are around 12-18 ridges up and left of the core. This leads to a fairly large search area, especially when you have to look at 10 fingers of multiple candidates. Finally, the most likely target groups for examiners to choose consist of opposing minutiae. There is an opposing ridge ending / bifurcation pair towards the bottom left. There is a long over-under in the center right. Just under that is a pair of minutiae facing opposite directions separated by a ridge. This leads to a similar situation as the 2010 CTS test. Slight distortion in the exemplars can obscure both points in opposing features. Now we have the holy trifecta that leads to erroneous exclusions: no core/delta, large search area, target groups with opposing features.

Was the latent that test-takers saw barely visible and worse than the PowerPoint image? Absolutely not.
Is this latent more susceptible to erroneous exclusions? Absolutely yes.

Erroneous exclusions need more study and examiners need more training on this topic. Our erroneous exclusion rate can be reduced. However, it will always be a part of normal casework. There will always be a low background level of these errors in our discipline due to how distortion can cause impressions from the same finger to appear different. The certification (and re-cert) tests should account for this reality in the test design. Two possible solutions are 1) do not require answers for all exercises (current design for the certification test), or 2) treat a single erroneous exclusion as a foul ball that isn't a fail but also isn't a pass. The test is just retaken (which totally should have been the name for Taken 2 with Liam Neeson).

So, to Carl in particular, but any others that are upset with the current state of the re-cert test... Please focus on these two main questions without getting bogged down in the irrelevant issues that keep coming up. You are risk of being tuned out because of it. Instead, focus on positive ideas that can lead to improvements.

[ekuadam]

As I stated previously, I also failed the most recent re-cert test (It was my first time taking it). I remember this print, I don't know if my erroneous exclusion was this print or not. Was I upset? Absolutely. After a coupled days of being angry, I moved on and told myself that I should have found the identifications, but also, that they would have hopefully been found in verification. I currently am awaiting another re-cert test to be mailed to me so I can try again. I never have taken previous re-certs so I can't compare the quality, but from what I have heard, the older tests were extremely easy. I found this one to be very fair, and on par with a CTS test. If people feel the test needs to be improved, I am all for it, but have ideas on how to improve it. Currently I don't have any so I just voice my opinion that even though I had an erroneous exclusion I found the test to be fair. I had 30 days to take it and I didn't pass it. Doesn't make me any less of an analyst.

[LLT]

To ER,

I am another unfortunate victim of the recert test, LP #12 - "Erroneous Exclusion". I have 28 years of experience and correctly identified 14 of the 15 latent prints on the certification test (I contemplated looking for the 15th but was too tired by that point) and 100% every year on CTS tests. So, needless to say, I'm pretty upset by this.
I applaud Carl for bringing this issue to light and for challenging the LPCB board on its validity. While the board has decided to remove #12 from the test (and I understand a second latent has been removed as well) this may not have been necessary had they provided the test takers with the same quality latent print they showed the examiners in attendance at the IAI. So, I take some offense in your critique of Carl and how he should stop suggesting that the print was of poor quality because I'm here to support that.........the latent print photo we were provided was EXTREMELY POOR quality and a far cry from what the LPCB provided for the audience to view. The photo we received was blurred and completely over-exposed! It insults my experience as a seasoned examiner to have colleagues believe that I could not identify this print when they are not privy to the latent print that we had been given. I would like to see the copy they provided those that chose to take it because I can guarantee it's of a different quality then what is the real truth.

[Carl Speckels]

Eric (ER),

From whom did you get your "actual samples"?

If you're looking at anything resembling the quality in the Onin print then you are definitely NOT looking at what I was looking at on my test.

Again, you haven't seen MY test latent. And you haven't seen what others have seen on their tests. You've only seen what the LPCB has shown you.

I'm willing to surrender my privacy for the recert test that I took. The LPCB is welcomed to post MY test latent, latent #12, provided they have the integrity to post my actual test latent.

[ER]

The board handed out the actual test materials at the conference (since they wouldn't be using it in the test anymore). The image has the normal blurriness that comes from printing 1:1 images on photo paper. The print has heavy deposition pressure which would tend to make finding features more difficult. While the overall image of the item is over-exposed, the latent itself is properly exposed.

Let's set that aside for the moment and ask ourselves, "What's my goal here?"

For me, my goal is to continue the discussion in our field about erroneous exclusions. Why do they occur? How often do they happen? How can we reduce the rate of occurrence? What training or retraining is appropriate? What quality assurance methods are appropriate? At what point does retraining need to occur after errors? How should an erroneous exclusion be handled when discovered in casework? In a proficiency test? On a certification test? Should comparisons that are more prone to errors be included on tests? I think these are important, difficult, complicated, and multifaceted questions with complex and nuanced answers.

I don't know whether my copy of the latent is better than the version you got or the same quality or even the exact same sheet of paper that you worked on. I don't know whether you (and others) got a bad batch of printouts. I believe that arguing about the quality of the printout is a separate issue that is distracting from more important questions and a more important discussion.

After the 1995 CTS test, our community blamed the test. We said that anybody can take it and everyone that failed was an untrained, unqualified foreigner. That wasn't the case. Many well-known and qualified experts made that error, but never admitted to it or discussed it (except secretly, late at night, and at the bar at conferences). But we didn't ask why. We just blamed the test.

After the 2010 CTS test, many examiners again sought to blame the test. The latent wasn't properly labeled. It had to have come from a round object not a flat one. "Everyone who identified it was guilty of under-analysis." Again, this wasn't the case. The latent lacked a core or delta. The most likely target groups were opposing features and looked different in the known. Again, it's not our fault. It was the test's fault.

Similarly after the Mayfield error, there was an initial narrative that the poor digital copies without scales was the reason for the error. In the end though, the OIG did not buy it. That report essentially said that the examiners just made a mistake. And then most importantly, asked how things could be made better. And in so doing, the latent print field was revolutionized with new research from the Noblis group.

Now we're again at a point of blaming the test or the testmakers. What is that going to accomplish? Is the IAI going to reinstate everyone that failed? No. Are they going to admit that some people got bad printouts while the ones handed out at the conference were from a good print batch? No.

Our field needs those that make mistakes to take a step back, admit that an error happened, and then move forward with positive ideas to improve the discipline? Think of what can be gained by that!
Or are we only going to discuss how the materials were sub-standard and the testmakers dishonest? What is there even to gain by that?

[LLT]

To Eric (ER),

I understand and agree with some of your thoughts and arguments surrounding this heated issue. I can admit to mistakes and failures when warranted but I find this to be a bitter pill because I believe the quality control with this test was lacking and such a poor quality test print should never have been distributed, especially when people's financial lively-hood and careers depend on their certification. I'm not asking for an easy test by any means but make it fair and legit.
You talk about the big problem of examiners making erroneous exclusions and I agree, that is a problem. But, how many examiners would have made an erroneous exclusion had this been actual casework? I would not have made an erroneous exclusion but an inconclusive opinion based on the print's ambiguity and lack of solid anchors (in the test print WE were given); we were not given that option unless we wanted an automatic fail. When we're paying good money to continue a certification we shouldn't be set-up to fail. And, in light of your understandable concern with erroneous exclusions, I'm not sure this is the forum to address it, necessarily, because I don't believe the percentage would be nearly as high if this were actual casework.

[ER]

how many examiners would have made an erroneous exclusion had this been actual casework? I would not have made an erroneous exclusion but an inconclusive opinion

LLT,

That is an excellent point, and I totally agree.

In actual casework I would expect most examiners to identify this latent. However, due to the issues that I've described above I would also expect some examiners to report inconclusive. I would then also expect that most verifiers would find the similarities and the final conclusion to be reported as ID.

Getting back to the cert/recert tests. Since we know that erroneous exclusions happen 1) with some frequency, 2) to everyone, and 3) more on latents without a core/delta, then how can the test be improved going forward?

While it may take some time for the board to consider allowing inconclusive (without it being an instant fail) or designing a test that purposefully includes inconclusive, in the meantime, the test could be immediately improved with a relatively simple change. A single erroneous exclusion is not a pass or a fail. The examiner must retake another test and remains certified during the interim.

At this point I feel that the IAI, the LPCB, CLPEs, and IAI members are best served by 1) understanding why this latent in particular led to multiple errors, and 2) discussing how best to reform the test.

And remember, I'm agreeing with you, and now asking that we move the discussion forward in this direction.

[Carl Speckels]

I wanted to chime back in and address a few things…

1) As indicated by my original post…I failed the IAI recertification test, I’ve made no excuses for that. Some may see it as an excuse when I point out that the quality of LPCB’s recently published test latent #12 didn’t match the quality of MY (I can only speak for mine) test latent #12. But it’s not. You’ll notice that this quality disparity never came up before the publishing of their #12. I recognize that my test latent has the potential to be an anomaly, maybe an innocent printing error, I don’t know. But, I don’t think the IAI maliciously issued Carl Speckels a test latent that is of inferior quality, more so than what they would normally issue, only that it happened.

2) I don’t want this thread turned into a personal thing where people feel the need to take sides. My original post was with the intent of identifying a few issues relating to the re-cert test that could be improved upon, and I believe some progress has been made. Hopefully, as a result of the discussions on this very chat board, and consequently through the legitimate concerns that have been brought forward to the LPCB by the informed community, the board will consider these items as they continue to pursue and design the best test possible.

3) As for me, I personally no longer have much of a stake in the outcome of any of this. Unless required, I will no longer seek re-certification and I haven’t a need for IAI membership, I’ve not been a member for years. I do however have an interest, maybe even a vested one at this point, to help those who have taken and will take the re-certification test. Those who have failed and will fail this test deserve(d) a well and properly designed test; the test should be worthy of their failure, these are the examiners that are supposed to be the best in the world after all. And some things deserve the attention of the LPCB, like proper validation of the test; a critical review of the test’s associated failure rates; a reconsideration of what is considered an acceptable failure rate; consequences that are commensurate with the type/nature of the “error”, i.e. erroneous exclusions; updating the testing convention to digital or online so that all quality variables are insured to be consistent from test-taker to test-taker, etc. There are several areas that could be improved upon.

So, with that, I hope that if this discussion is to continue, that it continue with the goal of addressing and solving problems with the (re)certification testing processes; holding the LPCB accountable to its paying test-takers; and insuring that the best interests of the IAI’s membership are being met.