Thanks for posting this Josh, I tend to think of garden variety Examiners like myself as the ultimate consumer of science in the discipline. As such I think it's healthy to view any publication through the lens of Richard Feynman's famous quote:
Science is the belief in the ignorance of experts
By ignorance, I believe he means fallibility. This is especially healthy in our line of work where the dogma in the literature and culture was that fingerprint identification was infallible and the expert was unassailable. A look at many of the critiques still reference this historical claim of
infallibility and it's offspring, overstating conclusions.
That being said, I think you need to characterize categories of published works. We have five main categories in my estimation. Physical Science publications, Psychological Science publications, Technological Publications , Policy Publications and Academic Publications.
Physical Science publications are the most straightforward and would include chemical validation studies like the effectiveness of an
HFE7100 formulation of Indanedione. These studies might present themselves in the discipline specific publications like the JFI or other similar peer reviewed publications.
Psychological Science publications are a bit more thorny but can also present themselves as
performance studies (black box, white box, etc) and/or
expertise and will also be found in peer reviewed publications like
PLOS One and
Elseiver (these will contain many types of publications including physical science and technological publications as well)
Technological publications include using technology to advance the discipline in some capacity, either through
image processing technology, or
AFIS style detecting, feature extraction and matching technology. These types of papers are typically found in
trade specific journals under the banner of discipline specific associations like the
IEEE
Policy publications include the bigger aggregate analyses that want to make sense of trends and gaps within the discipline usually to inform policy makers and influence policy decisions. This includes the
NAS Report, the
Human Factors Report and the
PCAST Report. These publications make use of peer reviewed publications listed above and will include recommendations based upon how these studies are interpreted.
Academic Publications include your text books like those from
CRC Press or academic collaborations with government funding like
The Fingerprint Sourcebook. These will be a mix of evangelism, academics and scientific translation of physical science, psychological science, technology and policy papers.
That being said, I think you need to keep in mind that there are some prevailing pernicious problems that permeate the perspectives (hello alliteration). Chief among them is the
publish or perish mentality of the publication world which biases research towards the novel and towards affirmative findings over replication and null findings. This has lead directly to what is known as the
'Replication Crisis' and is going to be the most problematic in areas like bias studies, sufficiency studies, performance studies and other psychometric studies which form the basis for error rates according to the PCAST reports. I would say Dror's original bias paper is the poster child for such a phenomenon. Is it methodologically sound? Has it been replicated? Is it biased itself towards finding an effect?
It would seem too that we have somewhat of a problem with pseudo-fame as well, in which seemingly good actors want to rescue the discipline from itself. Just look at who are the prominent speakers year after year in the conferences and their 'one size fits all' solutions. As the
article which helped launch the replication crisis movement states:
Scientific claims should not gain credence because of the status or authority of their originator but by the replicability of their supporting evidence
Anyway, what you posted is definitely of value and that's how I go about supplementing those techniques with the broader aim of putting findings in context to the discipline.