Are we scientists?

[David Fairhurst]

... Newton's "standing on the shoulders of giants,"...

The full context and wording from Newton was that he was responding to Robert Hooke's allegations of plagiarism. Newton retorted, "If I have seen further, it is only through standing on the shoulders of giants."
This is generally seen as being a dig at Hooke, who was "not a tall man". The implication being that if Newton had "borrowed" material from other scientists, he certainly didn't trust anything that Hooke might have produced.

Do we treat each other as professionals or do we resort to name calling?

Would you like to withdraw this question now Michelle?

<Second edit, now adequately researched >
And while I'm in a historical mode, and as he's been mentioned already; Einstein did not accept the uncertainty and probabilistic postulations of quantum mechanics as proposed by Bohr and Heisenberg. Being a scientist doesn't mean you cant be wrong occasionally.

[David L. Grieve]

Yogi Berra had two principles that would assure success: (1) Hit the ball where they pitch it, and (2), pitch the ball where they can't hit it. Someone was bound to fail. I chuckle when the cause of an error is failure to follow scientific principles, ACE-V or instructions from a fairy godmother.

I once interviewed a man applying to be a latent print examiner. The position required experience, and the applicant stated he had more than 16 years experience in fingerprints with the US Postal Service. Just a few questions revealed he was totally unqualified. When I asked about that experience, he informed me he had been the one in the post office to tack up the wanted posters, and when he did, he carefully studied the included fingerprints. I have no doubt the man believed he was qualified and during the interview he made several astute observations.

Years later, I attended the second trial in which the defendant was accused of killing a judge by sending his honor a pipe bomb through the mail. The US Postal Service (not the same guy) had identified a latent print from the wrapping paper. The defense presented its own expert witness, a forensic science professor at the local university who testified his ten year career with the San Francisco PD lab had included latent print cases. A check with SFPD revealed the man signed in latent print cases when a latent print exiner was not available. Whether he considered himself qualified or not I'm not sure, and he was allowed to testify about the identification after the jury heard his claim to qualifications.

A judge in Chicago will not allow a witness from the lab to call himself/herself a forensic scientist, even though that is the job title. A judge down in these parts would correct any latent print examiner who said there was no scientific basis for a minimum standard by yelling, "Yes, there is. I just don't known what it is."

As noted by other posts, what we call ourselves doesn't mean all that much but how we do what we do does. All judicial rulings are not perfect, all gatekeepers don't see things the same way and the system has flaws. But the system offers the best chance of getting it right than another other proposal if everyone tries to contribute to the wall of justice envisioned by Hume.

Whether Simon Cole believes his own PR or not is unimportant. Simon Cole is a historian who has put his own spin on past events that support his predetermined conclusions and ignores those events that do not. Under oath, he admitted his understanding of the identification process was gained by "luring on the Internet." He may be a nice guy, he may even provide wisdom now and then. But he is no more qualified than someone putting posters on the wall or signing in cases.

[Michele]

Pat,

I wasn't speaking of protocols as the step by step procedures, I was speaking of the bigger picture. One part of science is to have an in-depth knowledge level about something. In your former job, you were a technician following procedures but didn’t have the knowledge required to qualify under science.

I agree that a technician can testify to what they do, as long as they aren’t claiming to be a scientist.

[Pat A. Wertheim]

Dave, old friend and mentor,

All judicial rulings are not perfect, all gatekeepers don't see things the same way and the system has flaws. But the system offers the best chance of getting it right than another other proposal if everyone tries to contribute to the wall of justice envisioned by Hume.

I firmly believe that the criminal justice system in the US is the best in the world. It's fun to sit around in the hopitality rooms at conferences and complain about everything that is wrong with the courts and Daubert hearings, but in the final analysis, or system is still the best. Old friend, you and I have both gained personal insight into systems less good than ours.

Whether Simon Cole believes his own PR or not is unimportant. Simon Cole is a historian who has put his own spin on past events that support his predetermined conclusions and ignores those events that do not. Under oath, he admitted his understanding of the identification process was gained by "luring on the Internet." He may be a nice guy, he may even provide wisdom now and then. But he is no more qualified than someone putting posters on the wall or signing in cases.

There is a basic difference between the poster hanger or the case intake clerk testifying to an identification or an exclusion, and a critic who raises questions. Simon Cole has never tried to testify to an identification or to an exclusion. He raises questions, some valid and some not, for us to face in court, which is not always a comfortable situation. I would submit that in the past decade he has come to understand science a wee bit better than many of those who do testify to identifications and exclusions. He asks the hard questions, as is his right, and if we can't answer them, shame on us. I would submit that, to date, we have been answering his questions pretty well, indeed, because to the best of my knowledge, Simon has never kept our evidence out of a trial. What Simon has done is stir the pot and bring about the climate in which younger, brighter scientists like g. can conduct research that was never envisioned a century ago when you and I listened to John Ferrier and sat in the gallery in the Chicago courtroom and heard the Jennings case.

[16_Points]

I think it could be characterized as beginning with all that can be reliably known at the time, making careful observations while being mindful of factors that can influence perception, and forming a rational opinion of the nature of some reality. When you divert from that, engaging in the exercise without Newton's "standing on the shoulders of giants," without disciplined observation, or leaping to a conclusion that is not rationally justified, you're outside the scientific mode of inquiry.

I'm going to nit pick here, Gerald, and say carefully recorded observations preserved for posterity. That is how, as Newton says, we stand on the shoulders of giants. Fingerprint examiners do not have a 100 year cumulative record of experience. Rather we have hundreds of individual bubbles of experience which burst whenever a fingerprint officer retires/resigns (or dies). Only a tiny fraction of this collective experience is ever recorded. Again, I would encourage people to consider what science would be like if it only existed as an oral tradition. Instead of passing on journals and notes, a researcher works one on one with the scientist replacing him imparting the knowledge and wisdom accumulated over the course of his career. Is this really what we would want?

I would suggest that this is a critical difference between a scientist and a technician. A technician must master a skill, like driving a forklift. With some practice (experience) and expert guidance (OJT) you can become a proficient fork lift operator. A scientist, on the other hand, must be able to assimilate both the extant body of knowledge (as already suggested) and new or novel circumstances etc. It's an over simplification, but I suppose you get the point.

It's impossible to say when some human effort passes from technical development to "science" or even if there's a difference at all. Rarely is there a clear line in any real field of endeavor between empiricism and science. Arguably, if you're using all the available knowledge, making careful observations, and reaching rational conclusions, you're working scientifically, because you're not relying purely on observation but also applying solid knowledge of physical reality. It's a mode of thinking. But it can get tricky when you try to label something "not scientific." People may differ in what they consider valid knowledge upon which to base an inquiry. Research scientists argue such things all the time. People may differ in their confidence that some observations were accurately interpreted. And they can, of course, differ in what they conclude. But they may all be working scientifically.

To say that something is impossible is to say that the attempt should not even be made. You're saying, "Don't even try!" Really? Is that how science progresses? I would suggest that science tackles impossible tasks on a daily basis. Not by solving all the mysteries of the universe in a single blow but by inching along, each step bringing us closer to an answer. While there may not be a proverbial "line in the sand" where technical development crosses over into science, we can define science based on certain characteristics we expect to see in a scientific endeavor. Things like carefully recorded observations preserved for posterity which add to a body of knowledge available for peer review and a sincere effort to quantify the inferential limits of the data set (error rate).

Michelle's comment about errors is correct. They failed one or more of those three qualifications. But they did not all fail on account of working unscientifically. In general, it is definitely possible to proceed scientifically to an error, because we cannot know the limits of reliable knowledge and can often not observe with absolute accuracy. And some examination errors failed in the rigor of observation. The test of an opinion process is that, when it is applied in a scientific manner, it is highly reliable, which means that someone is justified in joining the opinion of the expert. Latent print examination stands very high in that regard. I think that if because the threshold for reaching a belief is treated so conservatively. It takes quite a significant failure in one of the three parts of scientific process to produce a demonstrable error.

(emphasis added)


Gerald, I think we've kicked this horse before. I reject the notion that the limits of reliable knowledge are "unknowable". The relevant question, before scientists and courts, is do we know as much as is currently possible or practical? It is indeed possbile to give a better answer than, "The error rate cannot be determined." Or my favorite, "The methodological error rate of friction ridge individualization is zero." I suspect that some or all of g's research (like performance testing) is leading in the direction of an empirically based, quantitative decription of error rate (my apologies, g, if I am mischaracterizing your work). So it is indeed possible to make meaningful, data driven statements about practioner error. I suspect, in part, that the work has not been done heretofore because someone convinced the fingerprint community that such an endeavor is impossible. Go figure.

[David L. Grieve]

Pat, perhaps you are right. I like to think good people like Glenn, Alice, Kasey and a host of others I do not know as well would be addressing these issues on the own. Maybe pestering from Cole urged them on. No doubt we became way too compacent before Daubert and a wee bit lazy. There is plenty of abuse and abusers to go around.

[Pat A. Wertheim]

Dave,

I believe we have had bright young scientists in the business since before you and I saw our first fingerprint. The problem has not been the lack of desire to do research. The problem has been that we work exclusively for police agencies that wanted only casework, not time "wasted" on research. It has been through the prodding of the critics like Simon Cole, through the Daubert hearings, and through the NAS report that some police administrators have admitted (grudgingly, no doubt) the value of further research. That, in turn, has enabled the bright young scientists among us today to initiate their research.

[Boyd Baumgartner]

I have to chime in again to critique the use of the word technician in this discussion. Technician here is being used as a synonymn for 'Operator'. Technician in the classical understanding is someone who has a practical understanding of theory, and a high level of understanding of technique. Another way of looking at a technician is a skilled worker. An Operator is someone who pushes buttons, levers, pulleys, etc to produce a result. In the fingerprint realm you can find AFIS Operators, or Live Scan Operators who do not perform analysis, only enter data into those systems through an interface.

Some examples of Technician Defined:
http://opm.gov/fedclass/gs0404.pdf
http://www.calmis.ca.gov/file/occguide/crimnlst.pdf
http://en.wikipedia.org/wiki/Technician (yeah, yeah, I know; Wikipedia)

Deconstructing it a bit further, the problems we get into as a discipline is that some are more skilled than others and the technique 'ACE-V' has been ill defined. Glenn's own Performance study from JFI 59 (2), 2009 fails to explicitly outline the ACE-V method/technique . The study then goes on to show precision variance, which is exactly what you’d expect for a skilled endeavor reliant on a technique that is not defined.

The difference in the attitudes reflected here, I would venture to say is that some people have motivation to ground their conclusions in accepted scientific principles since ACE-V can be so nebulous. The dilemma here is that Technicians by definition only have a practical understanding of some of those scientific theories. For instance, as complexity of a print grows, how many people are putting a more heavy reliance on rationalism (deduction) versus empiricism (sensory observation)? What is the QA measure for not becoming entwined in a narrative fallacy for a false justification based on too much reliance on deduction; Crtical Rationalism (aka Falsifiability)?

The spectrum Glenn alludes to, in my opinion is a spectrum of Quality, which mirrors a spectrum of scientific philosophy of Empricism to Rationalism. If we have "A" quality prints, our skill level need not be as high, our justification needn't be as robust, as "Any other examiner should be able to see it". Empricism = Sensory Observation.

On the other hand, if we have a very complex print, we're deducing a lot more, like 3 ridges go in to this area of distortion and only two come out, therefore there must be a ridge event in the midst of the distortion. If you have a ratio of deduction to observation that’s too high, you’re running a risk of making up a narrative to justify your conclusion and you might be consuming too much data to allow yourself to attempt to falsify your conclusion.

So, you can see that technique in this scenario could lead to a few different results, which are largely attributable to how the technique is being employed, which is why I call examiners technicians.

[16_Points]

OK, Boyd. If we want to define something, let's look at an authoritative source on the English language, the OED.

technician, n.

1. A person knowledgeable or skilled in the technicalities of a particular field; esp. an expert in the formal or practical aspect of an art, sometimes with implications of a corresponding lack of creativity.

2. A person qualified in the practical aspects of one of the sciences or mechanical arts; (in later use) esp. a person whose job is to carry out practical work in a laboratory or to give assistance with technical equipment.

[Gerald Clough]

Gerald,

What are the three qualifications of science that you are talking about, I think I missed it.

Just that what distinguishes a scientific mode is:

- Beginning with all that is known. Meaning it's pretty silly to pretend to working scientifically to any serious end without knowing what's already been discovered. All sorts of reasons for that, but one is that unless you make a study of what's supposedly known, you can't rationally begin at an appropriate point.

- Rigorous observation, which includes being mindful of factors that can influence perception.

- Rationally drawing an appropriate conclusion.

You could write long treatises on each of those points, but I think they can be used to distinguish the kind of thinking we're interested in. Now if there's no existing prior knowledge at all, I don't think that excludes scientific thinking, just that the lack of knowledge becomes itself a kind of knowledge that means any progress requires developing some body of knowledge. Some would call that the empiricism stage, a special case and one that is traditionally a difficult one. Using the humeral hypothesis of medicine again, the inquiries became scientific when people recognized that the "humors" were merely imagined, and they had to start all over from the beginning, largely with anatomical work to establish some basic knowledge. (The first real advances in that area happened in a general period where a lot of fields were making advances in exactly the same way by recognizing the incorrect "knowledge" of earlier times and when they were learning to think in this new way.) In our case, we're not without a body of existing knowledge in general, and we're privy to a mass of experience and prior observations that we apply both in examinations and in research.

It would be truly silly to conclude without rigorous observation. I guess I would generalize that to mean careful accumulation of data. And what constitutes a rational conclusion is simply a conclusion that can be articulated from the combination of knowledge and observation. I think that's the minimum set of requirements. The intent to produce a conclusion only succeeds if each of those is productive. If my attempts to observe fail because there's no data I can have faith in, I don't get to a conclusion, other than reporting that I can't. If I have insufficient knowledge to allow me to put meaning to the observations, then I'm just empirically poking around for ideas of what they might mean. If I conclude using other than the knowledge and observations, the conclusion is inappropriate for the kind of inquiry I intend.

I consider those three point to apply when the expert pretends to concluding scientifically. But I recognize that not all expert fields will be seen to clearly proceed with all three. Or perhaps they do. For instance, psychology. I think it's fair to say it's almost entirely empirical, entirely clinical. The prior knowledge that comes into play in psychology conclusions in law is largely the knowledge that one or another condition has been defined in law, and the expert is mostly applying testing and observation to see if the features specified by law are present and genuine and meet whatever other conditions are required in law. The conclusion is often whether or not expert examination of the subject shows the subject to meet the test the law has laid out.

Some expertise is intended to be used in deciding if some action was reasonable or not. For instance, a physician may testify as an expert in a matter trying whether another physician was reasonable in a diagnosis or selecting a therapy. It's not to establish precisely the nature of the ailment but whether the physician performed as physicians should. For that kind of inquiry, the knowledge is knowledge of what is reasonable practice. The observations are reviews of what the physician knew or should have known and how he acted in response. The conclusion follows from those two things and is rational.


16-points:

I'll give you an example of what I was thinking of about it being impossible to say when technical development becomes "science." A Neanderthal is sitting there chipping away at a chunk of chert with the notion of making a tool for a specific task. He knows nothing of geology. Nothing of the analysis of stresses and fractures. Nothing of moments and angles. Nothing of the mass/velocity components of force. He chips and chunks out an object that he thinks might be worth trying in the task. When he's done, he fetches, say, a deer skull and taps the tool on the skull, "teaching" it to strike deer in the head. He decides he's pretty much finished making the tool, and he will go out and try it on a live deer. Perhaps he's satisfied with how it works. Perhaps he's not, and the next time maybe he makes it a little different or goes through a different sympathetic magical ritual to see if the next one works better. Is he working scientifically? Hard to say in a way that everyone would agree with. Is he capable of working scientifically? (Maybe not. One of the striking things about Neanderthal culture is it's persistence in using the same tools throughout nearly their entire existence, when there was no environmental reason preventing improvement.)

By it being impossible to know the limits of reliable knowledge, I mean we can't know what may one day be known, that it's impossible to know what isn't yet available to know. I guess that's obvious, but the point was that we might form an erroneous belief in fact, or we might, presented with the same case in the future, present a somewhat differently qualified conclusion. But we have to work in every moment with what we have at that moment. Maybe the ultimate test of fully functioning scientific mind is how someone deals with new knowledge when it appears. History is filled with cautionary examples of respected scientific figures of the day refusing out of hand to acknowledge new knowledge. We can be pretty sure they didn't just decide to be contrary and pigheaded as they appear in retrospect.

But I do agree about being able to make statements about error. That seems logical to me, since when I look at errors (excluding fraud, mislabeling, etc.), I find that for every error reliably known to be an error, we can describe the specific intermediate errors that led to the erroneous conclusion. And that suggests that those particular mistakes can also be identified in any examination where they appear or can be offered for examination to see how often they are repeated.

[16_Points]

16-points:

I'll give you an example of what I was thinking of about it being impossible to say when technical development becomes "science." A Neanderthal is sitting there chipping away at a chunk of chert with the notion of making a tool for a specific task. He knows nothing of geology. Nothing of the analysis of stresses and fractures. Nothing of moments and angles. Nothing of the mass/velocity components of force. He chips and chunks out an object that he thinks might be worth trying in the task. When he's done, he fetches, say, a deer skull and taps the tool on the skull, "teaching" it to strike deer in the head. He decides he's pretty much finished making the tool, and he will go out and try it on a live deer. Perhaps he's satisfied with how it works. Perhaps he's not, and the next time maybe he makes it a little different or goes through a different sympathetic magical ritual to see if the next one works better. Is he working scientifically? Hard to say in a way that everyone would agree with. Is he capable of working scientifically? (Maybe not. One of the striking things about Neanderthal culture is it's persistence in using the same tools throughout nearly their entire existence, when there was no environmental reason preventing improvement.)

We can make statements about this transition if we define what science is, if we establish the criteria necessary to be science. If you look at science as an entity that exists outside of human agency and human definition, then, yes, it becomes a murky question. Again, as long as we are using a consensus opinion of what constitutes science (by that I mean some identifiable criteria), then we can make some statements about what is scientific as opposed to just technical.

By it being impossible to know the limits of reliable knowledge, I mean we can't know what may one day be known, that it's impossible to know what isn't yet available to know. I guess that's obvious, but the point was that we might form an erroneous belief in fact, or we might, presented with the same case in the future, present a somewhat differently qualified conclusion. But we have to work in every moment with what we have at that moment. Maybe the ultimate test of fully functioning scientific mind is how someone deals with new knowledge when it appears. History is filled with cautionary examples of respected scientific figures of the day refusing out of hand to acknowledge new knowledge. We can be pretty sure they didn't just decide to be contrary and pigheaded as they appear in retrospect.

As stated here, I agree with you. Thank you for the clarification.

[L.J.Steele]

Are judges scientists?

Heck no, at least not the vast majority. Ditto for the vast majority of attorneys. Moreover, judges and attorneys are often generalists -- I may not know much about the structure of a particular gang or about narcotics distribution or embezzlement or the legislative history of a particular statute, but I have an ethical obligation to learn enough about it to competently represent a client (which often means consulting with experts) if I take on a case.

The judge has limitations on what he or she can look at that's outside the evidence -- so if the attorneys make a hash of presenting or defending a challenge, a judge may be limited to making a ruling on the facts before him or her, even if there are important other facts that neither side presented. (One of the reasons I have rarely seriously considered applying for the black robes is that limit on research.)

[L.J.Steele]

[Qualification as an expert is usually not an issue. Few litigants will try to put up an unqualified expert.

I'd qualify this to most litigants won't try to put up an expert they do not believe is qualified -- and there are some ethics rules about candor to the court that ought to preclude putting up a known fraud. However, one of the problems here is that there are some frauds and charlitans out there (Zain, anyone?) who talk a good game and who apparently no one looked into deeply enough to catch them at an early stage.

I'm quite concerned about experts who aren't keeping up with their credentials or education in their field, or who otherwise shouldn't be in the courtroom, or who worse tell attorneys that something isn't challengable, or is correct, leading the attorney not to catch bad evidence.

[printlady]

While my job by definition may qualify me to use the title ‘scientist’, I am concerned that the term might be misleading to a jury. In the colloquial sense, I think the average person views a scientist as someone with a string of letters after their name and extensive experience in scientific research. If you look at it from that point of view, even an MD may not qualify as a scientist. Using the term to try to make myself sound more important strikes me as similar to the hubris I have accused Valverde of. When I testify to my findings in a case, representing myself as a technician doesn’t make me any less of an expert to the court.

[Gerald Clough]

While my job by definition may qualify me to use the title ‘scientist’, I am concerned that the term might be misleading to a jury. In the colloquial sense, I think the average person views a scientist as someone with a string of letters after their name and extensive experience in scientific research...

Aw, come one. Make an effort. White lab coats aren't that expensive. (And they last a LONG time, if you only wear it to court.) And I figure that with some careful thought, you can come up with some letters to go behind your name. What's the acceptable ratio to equal a Ph.D.? Four to one? Five to one?

Gerald Clough, BSEd, MA, CSCSA, FFPS, EMT-LP, LARO, MPO, LIH, CSI

Opps. Left off LD (Licensed Driver), NT (Native Texan), and BOI (Born on Island [Galveston])


But, no. I'd not lay claim to Scientist unless forced to a yes or not answer, and then I'd say I wasn't a research scientist but understand and apply scientific methods. For all the reasons you gave and because I think folks are just plain suspicious of scientists as not very practical people and would have more faith in their mechanic than in an automotive engineer when it came to diagnosing their car trouble.