It seems as though some of the trending topics we've discussed this year are making their way to the comments of the tests and that's good. Specifically some standards around Exclusion and the need for an inconclusive. A related topic, also mentioned is the issue of performing to the test, akin to something NRivera said in the Spring test thread. Things that may not meet your casework standards, you'll be lax on just because you know the test's standards. That was echoed in the comments (See emphasis in quote below)
This comment sums these issues up nicely:
It would seem like a good first step towards moving proficiency tests towards a blind state (incorporated into casework unbeknown to the Examiner), would be to start offering tests for labs with varying conclusions. There should be a version with 'not identified' as an answer because some labs don't exclude. You could have one with an exclusion standard. You could have one where the prints are less than stellar and include some sort of 'incomplete-need better exemplars'. You could also had one with consistency but not enough to ID and include an 'inconclusive' result. Then agencies could purchase the test that best matched the SOP's and conclusions they follow.Three of our certified latent print examiners took this test and felt that two of the palm standards were of extremely low quality in the interdigital region, which was needed for comparison with L1. All three examiners indicated on the test that in active casework they may have concluded an inconclusive result for L1, as better standards were needed. The low quality of the standards caused concerns as to the value of the test, as pushing examiners to form a conclusion is not necessarily a test of their proficiency. Inconclusive may be stated as a valid conclusion, however, analysts know that ID or EXC are supported in the testing environment. (emphasis mine)
Since exclusions are where we make the most errors, many agencies are moving in the direction of more detailed procedures for exclusion determinations; including the necessity of comparison of two target groups, sufficient level two discrepancies be present, standards be of sufficient quality, known orientation and anatomical source, etc. Our concern is that L1 did not appear to support those trends in the community.
Nothing real salacious in the results of this test. Although these two statements on the very last page seemed odd.
I'd say the first observation negates the second observation as all of the inconsistent answers were confined to one agency. That, by all definitions, constitutes a pattern.1. A total of seventy (70) participants representing seventeen (17) agencies/entities submitted responses to test 18102. Of those 17 agencies, a total of only one (1) agency submitted responses inconsistent with our assigned values.
2. The total number of inconsistent responses was six (6) and these inconsistencies did not show a pattern and were spread across multiple demographics.
Also, Appendix 11 seems odd since all the inconsistent responses came from one agency and that agency apparently has two ways of doing things.
What else did you notice?
