Daubert Debate in MD

[Pat A. Wertheim]

But wait, Patrick, let's take Lisa's example of a complex chemical experiment involving multiple reactions. Now, there are some chemists more talented at what they do (measuring, pouring, guaging results, etc.) than others. And there are some chemists that always get a drop too much or too little of a reagent when trying to measure accurately. So even in Lisa's example, you might have some chemists who never make a mistake, some who on rare occasion make a mistake, and some who frequently make mistakes. But if you average them all together, then the good chemist looks worse than he is and the bad chemist looks better than he is. It seems to me it would be of more value to the court to consider the error rate of the individual than the average error rate of everybody.

[L.J.Steele]

Don't we all as human beings compare things...anything...basically the same way. We look at (gather data from) one object and see if there is corresponding detail (data) in the other object and vice versa.

Oddly, the answer may be no -- experts, at least fingerprint experts, see prints differently than laymen.

http://newsinfo.iu.edu/news/page/normal/1966.html

Your ideal test for IAI certified examiners....I read that and I see a competency test for examiners. You are testing whether the examiner is making an error, its not testing the act of comparison.

No. The only reason to use IAI certified as a pool is to avoid the complaints about tests that lump experienced examiners in with trainees and new hires. I assume that IAI certified would set a qualification floor. Unless you can train a computer to do identifications, I don't know how you can test the method without testing examiners -- the idea behind a statistically significant sample size and number of questions is to smooth out individual outliers.

I'm not a statistician, but I'd expect that the reason they use a series of questions is to try to eliminate Pat's issue with the guy or gal who always get things wrong. They become the statistical outliers. We're looking for the middle of the bell curve to see what happens there.