Forensic board suspends former FBI analysts for error in '01

[charlton97]

Just seen this. I assume this is an erroneous exclusion, but the article is a little confusing to me at least. In which case, do I understand this correctly that the IAI will suspend for commiting erroneous exclusions? This is interesting because that could add added pressure to call matches leading to a type 1 error?? As always, my sympathies with those involved in this incident, there but for the grace of....and all that.


Forensic board suspends former FBI analysts for error in '01 Monroe murder case
June 29, 2009
Because they made a technical error in misidentifying a fingerprint on a gun in a Monroe County murder case, two retired FBI analysts have been given one-year suspensions on their forensic expert certifications by the disciplinary body which gave them those certifications.

Ivan Futrell and George Wynn are appealing the International Association for Identification's decision to suspend them for a year after they provided fingerprint analysis results requested by defense attorneys in the Helen Biank murder case. The suspensions prohibit Futrell and Wynn from testifying in any further court proceedings anywhere as experts for that one year.

On Halloween 2001, Biank, 37, was fatally shot in her Barrett Township home, where a fire then was set.

The investigation led police in 2005 to arrest Biank’s ex-boyfriend, Mark Miller, 42, in Florida, charge him with Biank’s murder and bring him back to Monroe County to face prosecution. Evidence against Miller included the murder weapon with a fingerprint believed to be his on it.

Florida defense attorneys William Grant and Milan Samargya contacted Futrell and Wynn, who analyzed the fingerprint on the gun and concluded the print was not Miller’s.

The defense sought to publicize Futrell’s and Wynn’s finding through the local news media, but assistant district attorneys Michael Mancuso and Michael Rakaczewski became aware of the defense trying to do so and requested a gag order. Monroe County Court Judge Margherita Worthington granted that gag order, preventing attorneys on both sides from commenting to the news media about the case until after a trial verdict or guilty plea.

Miller later pleaded guilty to third-degree murder, saying he was playing with the gun when it accidentally went off and shot Biank, who had rejected his marriage proposal. He was sentenced in November 2007 to 15 to 40 years in state prison.

After Miller’s guilty plea, Mancuso contacted the International Association for Identification, which had certified Futrell and Wynn as forensic experts.

The IAI conducted a lengthy investigation into the matter and in February had its attorney, Phyllis Karasov, send Mancuso a letter stating, “Mr. Futrell and Mr. Wynn were found to have made a technical error in their misidentification of the latent fingerprint and, as a result, disciplinary action will be taken.”

Trooper Phillip Barletto of the state police barracks in Fern Ridge, who investigated the murder, released news of the disciplinary action to local news media on Monday, saying authorities are free to discuss this now that the IAI has ruled and the appeal process is underway.

"If you're a forensic expert suspended by the IAI, that's a blow to your credibility," Barletto said.

Futrell declined to comment to the Pocono Record when contacted at his Stafford, Va., home Monday evening. The Record has not found a way to reach Wynn.

A phone call left Monday afternoon at Grant's and Samargya's Florida law firm has not been returned.

[Michele]

Dave,

Why would people feel pressure to make a match? Couldn't they say inconclusive?

Michele

[charlton97]

Michelle,

indeed they could call inconclusive. However, it is my contention that fear of error has an impact on decision making. A fear of a type 1 will make examiners more conservative in their decision making and a fear of false exclusions will have a similar impact the other way I predict. It is the balance of reward over sanction that is the key here. Now, if the biggest reward, or, the least possibility of sanction is to err toward inconclusive, then that is what you will get, far more inconclusive decisions. I guess if that was to happen then we may need to look at a probabilistic tool to help the courts out? I am just stirring the debate Michelle, asking questions is what I love to do;.))

Cheers

Dave

[RedFive]

There are some important unanswered questions in the article. Of course I would like to clarify if it was an erroneous exclusion or an inconclusive - I can't imagine getting suspended for an inconclusive. The article doesn't mention if any other examiner did a later comparison - who is to say that the latent on the gun is that of the boyfriend? More CSI effect here - since the boyfriend admitted to shooting the victim, it has to be his fingerprint on the gun and nobody elses.

[Pat A. Wertheim]

I checked the IAI website and it says a certification can be suspended for "technical error." Strictly interpreted, that would mean erroneous identification, erroneous exclusion, and erroneous inconclusive.

I remember not too many years ago there was an exclusion among the fifteen latent prints on the certification test. To pass the test, the applicant had to have 12 latent prints marked correctly with no erroneous identifications. A lot of certified examiners passed the test by making 11 identifications and then marking all four of the remaining latents as "non ident," which meant exclusion in the FBI terminology. The way I see it, every certified examiner who passed the test by doing that was guilty of three erroneous exclusions, even if they did not make any erroneous identifications. If an erroneous exclusion is now considered a technical error deserving of suspension, should the CLPEs who passed the test by marking four latents as "non ident" have their certifications suspended retroactively?

Or if erroneous inconclusive is grounds for suspension, then shouldn't only those who correctly identify all fifteen latents on the test be certified?

There are some important unanswered questions in the article.

Indeed there are!

[sharon cook]

[Pat A. Wertheim]

Thank you, Sharon. My point exactly. If the rules 10 years ago encouraged erroneous exclusions on the certification test and if the prevailing philosophy for the first couple of decades of the certification program was that erroneous exclusions were no big deal, then it would be wrong to punish an erroneous exclusion today with suspension. I am trying to avoid speculation and just ask "If" questions, but RedFive is still correct:

There are some important unanswered questions in the article.

[Ernie Hamm]

This is not a simple exclusion error, but (according to the article) diverse positions in opinion regarding an association. One side reported the latent print was made by an individual and the other side reporting it was not made by the individual. These individuals were not retained for “inconclusive” findings, they were in a “yes” or “no” situation. They decided, “No”.

There have been two OCONUS publicized cases of this type of differing opinions in recent years. One was amicably resolved by the ‘opposing’ agencies and the other, well…that is a different story.

A review of the “Code of Ethics for Latent Print Examiners” in the Certified Latent Print Examiners Pamphlet, 1987, states, “Any Latent Print Examiner whose professional or personal conduct becomes adverse to the best interests and purposes of the profession of Latent Print Identification shall be liable to censure, suspension or withdrawal of Certification as a Latent Print Examiner”. The referenced examiners are listed in this pamphlet.

The article states that the sanction imposed by the IAI Board of Directors, based on results of a review by the IAI Latent Print Certification board, is being appealed. If some individuals have a concern with the actions of the IAI, there is precedence for having the latent and record in question posted on an internet site for review, critique and comment (open or anonymous) by the general populace. The actions and decisions of the reviewing individuals can then be scrutinized for competency and correctness.

[Gerald Clough]

It's really hard to say much without being in possession of everything in the file. Exclusion is just as absolute as identification. But to invoke the ultimate sanction, one might arguably look for a clear error that falls solidly outside any reasonable standard. By that, I mean that you might not look so harshly at something like a latent that would easily be taken to be a finger pad, perhaps looking more like it on account of distortion, and missing the source area in an unlikely place. Or a peculiar circumstance that gave no hint that the latent was from a foot. Absolute exclusion can require exhaustive work to say with certainty that no part of an individual's body could have made that mark.

Without reference to this particular case of individual examiners, I would think a board in any discipline would view an error with more alarm if it appeared the erroneous conclusion required a plain and unlikely error, and the conclusion suited the client, leading to a suspicious of something more than just making a mistake while being entirely neutral. I have no reason to think that happened in this case, but I have more than once seen it done, including an expert rendering two different and contradictory opinions on the same evidence at different times.

I can't imagine this sanction from a mere threshold difference of opinion, which is the sort of thing probability studies address. And I don't think it will necessarily have any bad effect on conclusions. It may, in fact, make people more mindful that erroneous exclusions can be just as damaging as identification errors. They are indeed different sorts of evidence in most cases, identification speaking specifically to the source's contact. But a mistake in excluding a suspect as the source of a latent on a murder weapon is a major problem that places the gun in the hand of a "Mister X," and that's the sort of thing that can set a murderer free.

And, am I remembering wrong, or does it often just depend on whether or not someone is willing to make a formal complaint?

[Michele]

Can you have an erroneous inconclusive?

Typically we are taught (or at least I was) that we can arrive at 3 possible conclusions; ID, exclude, or inconclusive.

The problem with this is that by definition inconclusive means that you didn’t arrive at a conclusion. Maybe it would be better to say that we can have 3 responses; 2 of these are conclusions and 1 is the lack of a conclusion.

Isn’t ‘no value’ a conclusion? If so then we have 4 responses, 3 of these are conclusions and 1 is a lack of a conclusion.

If one examiner says something is inconclusive but another examiner makes an ID, the 1st examiner may not have arrived at the correct conclusion but they didn’t arrive at a wrong conclusion either (because by definition they didn’t arrive at a conclusion). It’s a failure to arrive at a conclusion, not a failure in the conclusion. These may both be errors but one could be considered an oversight while the other is clearly wrong.

Another question is the phrase ‘not identified’. Is it a conclusion or is it a lack of a conclusion? I think it’s a lack of a conclusion while Pat’s stated that other people refer to it as an exclusion. Does the IAI state the meaning? It seems like they should if they’re going to hold people accountable for it. If we’re all in the same discipline and we don’t know what each other are referring to then how are the courts suppose to interpret what we’re saying? I think the NAS report was correct when it said we need to standardize terminology.

I don’t believe we’ll ever be able to resolve these types of issues until we start defining things the same way.

[David Fairhurst]

Can you have an erroneous inconclusive?

As we're breaking it down, as I see it there are 2 types of inconclusive result.

1. The latent print contains low quantity/quality of detail and, though the source area is present in the exemplars, the examiner is unable to individualise.

2. The latent print has sufficient detail to be individualised but the source area is not present in the exemplars and the examiner is unable to exclude.

To call an error in the first one would have to show that, at a required minimum ability level, an examiner should be expected to individualise the latent. Failure to do so shows a lack of ability to individualise. (see Wertheim P. "The Ability Equation" and Vanderkolk's Q-Q curves)

To demonstrate that the examiner is in error in the second conclusion we could show that the source is indeed present in the exemplars and the examiner failed to find it. The examiner's search skills are in question here but not their ability to individualise.
Alternatively a lack of analysis skills could lead an examiner to be less than fully confident of the source area before they start searching. I'm going to give an extreme example for this one and trust that you can all make the reality adjustment.
An examiner is presented with a latent print. It's not a difficult latent. From analysis of the shape and size of the latent the examiner correctly concludes that it is a thumb. The pattern is a symmetrical arch. The examiner is unable to determine which thumb he has because he lacks knowledge of "fault ridges", which are present and would lead him to correctly conclude that it is a right thumb. So our friend continues into the search phase not knowing whether he has a right or left thumb. One of the exemplar sets is incomplete, it does not have a left thumb. So what does the examiner do? He cannot exclude this person but if he could correctly analyse the latent he would be able to.

So I have shown errors in each of the three parts of ACE.

Analysis error leads to inability to exclude when exclusion should be possible.
Comparison error leads to inability to exclude when individualistaion is possible.
Evaluation error leads to inability to individualise when individualisation should be possible.

The seriousness of each of these I leave for you all to discuss and evaluate.

Have fun,

[Steve Everist]

Can you have an erroneous inconclusive?

Typically we are taught (or at least I was) that we can arrive at 3 possible conclusions; ID, exclude, or inconclusive.

I don't remember if we were taught that specifically over 10 years ago during our training, but those are currently the three acceptable conclusions based on SWGFAST's "Standards for Conclusions" (9/11/03 Ver 1.0).

If one examiner says something is inconclusive but another examiner makes an ID, the 1st examiner may not have arrived at the correct conclusion but they didn’t arrive at a wrong conclusion either (because by definition they didn’t arrive at a conclusion). It’s a failure to arrive at a conclusion, not a failure in the conclusion. These may both be errors but one could be considered an oversight while the other is clearly wrong.

Although this may be true based on a somewhat black and white definition of what is the correct conclusion, the inconclusive may be the correct conclusion for one examiner while another examiner's ID could also be correct when evaluating the same latent and known exemplar. Otherwise there is a presupposition that all examiners will agree to what is ultimately the correct conclusion based on two prints that likely will not have ground truth knowledge. This subjectivity is inherent in all comparisons.

Another question is the phrase ‘not identified’. Is it a conclusion or is it a lack of a conclusion? I think it’s a lack of a conclusion while Pat’s stated that other people refer to it as an exclusion.

Maybe we should ask, "what would the average jury assume it to mean?"

Does the IAI state the meaning? It seems like they should if they’re going to hold people accountable for it. If we’re all in the same discipline and we don’t know what each other are referring to then how are the courts suppose to interpret what we’re saying? I think the NAS report was correct when it said we need to standardize terminology.

I agree that the terminology must be standardized, but should we expect that every examiner comparing latent print evidence come to the same conclusion given the same prints to compare? To expect this would remove the subjectivity in the examination process - otherwise referred to as the training, experience, etc... of the individual examiner doing the comparison. As much as subjectivity seems to be presented as negative and unscientific, it isn't and it's a part of the latent print comparison process. Since we're given the three conclusions, I'm referring to any two conclusions where one of them includes inconclusive.

I don’t believe we’ll ever be able to resolve these types of issues until we start defining things the same way.

I don't think we can even start to consider these issues until we agree on the definitions.

[Michele]

I remember having this conversation years ago on this forum (that some people use exclusion to mean that the person didn’t leave the latent and others are using it to mean that it’s wasn’t present in the exemplars they had). We’ve also discussed that if someone’s tolerance level is higher then they might label something as having ‘no value’ while other’s would not label it this way. Some people might have a lower tolerance level thinking it is of value and others may say it has value for exclusion.

We seem to discuss a topic and in the end we don’t resolve it, we just stop talking about it because people don’t agree. We are silent on the topic until later when a problem is seen.

The issue seems to be that we all have different standards. Both Dave and Steve alluded to this by saying that it’s depended on someone’s ability or skills or training and experience. In the majority of cases we all agree but when the quality and quantity are low then problems arise. Is it a problem or is this just something we have to recognize? Statistical values can help with differing tolerance levels, when the quantity is low, but what do we do when the quality is poor (some people see items that others don’t)?

I know examiners have been asked in court, “How do you know when you’ve arrived at an identification?” I’ve heard a variety of answers and most of them include something about having a sufficient amount of information. Sufficiency is so subjective that I don’t necessarily think that’s a standard. Maybe that could be used as a personal standard but is it a scientific standard?

I’ve often debated with people that our conclusions were scientific but maybe they aren’t? I’m not quite there yet but I’m starting to see the value of their point of view.

[Gerald Clough]

I agree that the terminology must be standardized, but should we expect that every examiner comparing latent print evidence come to the same conclusion given the same prints to compare? To expect this would remove the subjectivity in the examination process - otherwise referred to as the training, experience, etc... of the individual examiner doing the comparison. As much as subjectivity seems to be presented as negative and unscientific, it isn't and it's a part of the latent print comparison process. Since we're given the three conclusions, I'm referring to any two conclusions where one of them includes inconclusive.

I have to differ, and I'll tell you why.

That's something I hate to see, the misuse of "subjective." There is properly no place for subjectivity in examinations. By this, I mean that the applicable plain definitions of these opposite terms, objective and subjective, are in whether the product of thought is the result of the character of the subject (the person observing) or the object (the thing being observed). It is the difference between the object being characterized according to its own traits or being characterized according to the peculiarities of the observer. When examiners collect data, that most always are objective. It must always be the accumulation of actual characteristics of the impression. It must not be driven by external factors.

This does not mean all examiners are equal. One may be more experienced or possess more information about skin and impressions and distortion or whatever. But they are each strictly observing the evidence and working from strictly what they see. In medical practice, you can think of objective signs as those anyone else can see, rather than, for instance, imagining sings that might suggest the presence of a condition within the physician's special area of interest. Objectivity does not require all observers to be identical. Mechanical analytic devices are not all identical. But they are all objective.

Nor do different results imply subjectivity. And a poorly-defined process or vague threshold does not imply subjectivity. Not every process invoked when humans analyze observed conditions is well-defined. But, so long as the product of analysis is driven strictly by the object being observed and the analytical process neutrally applied, it is objective. In latent print examination, the process is objective when all examiners with identical training, experience, and threshold conclude similarly, because that means that the identical analytical machines are operating without biasing input. (But objective examination does NOT require identical analysts.) It would be subjective when a factor that did not contribute to that strict technical analysis corrupted the analysis. Many things that fall under the heading of bias induce subjectivity. A drive to identify a particular individual does, if it influences decisions, injects subjectivity. Knowledge of a wealth of other evidence associating an individual with the crime scene does, if it causes the otherwise objective threshold to be lowered, introduce subjectivity.

Many people try to use "subjective" to suggest uncertainty. On the contrary, people who are making a subjective determination are commonly very certain and may be quite correct, but they have reached their conclusion by making decisions, at least in part, on account of factors that alter the process of strictly letting the object contribute the data. Their interpretations have become plus-or-minus the effect of something other than their standard interpretation.

So, no. We may not expect every latent print examiner to reach the same conclusion about every case. If we were counting ducks, and the number of ducks was such that we could each count them one by one, we would expect, barring plain mistakes, to all produce the same count. But if we were asked to say how many ducks there were in a huge flock sitting on a lake, we might each apply a different process. One might count those in a limited area and multiply that number by how many such areas there were. Another might count the number in an area to get a density and calculate the total area from the dimensions of the lake. Different counts, but both are strictly objective, since they are the product of observing only the ducks with no other motive than to get a count. Yet another, vastly experienced in duck counting (as various officials are at estimating crowd sizes), might carefully observe the ducks on the lake and conclude a number without making any actual count. All three are objective, because all three arrive at their counts strictly by applying a neutral process to what they observe. But if another observer who may wish the count to be high because they wish for a hunting season to be extended alters the process to increase the resulting number, that's subjective. The desire for there to be a great many ducks has nothing to do with the ducks themselves or their actual number and improperly became a factor in the result. We see this subjectivity all the time when proponents and opponents produce different counts of the number of people in various demonstrations and marches.

When you look at your child at night and are deciding if it's time for them to go to bed, you are objective when you base your decision on how THEY appear and behave and your belief about what is best for their physical health and performance. You are subjective when your decision depends to some degree on how badly YOU want them to be gone.

Arguably, no human always carries out any process without some subjectivity. But the process they invoke and that defines the way objects are analyzed is strictly objective. Even astrology, for all that it assumes an invalid relationship between astral bodies and human character, is objective, so long as it's not influenced by a desire to provide the client with a predetermined result. Objectivity and subjectivity have nothing to do with whether or not the process is valid or accurate. Point-counting was not subjective on account of no proof that a particular number was reliable. It was not subjective on account of examiners blindly accepting having been told by their trainers that it was the right number. Individual professional thresholds are not subjective on account of being undefined. Both point-counting and undefined thresholds are object-neutral. They do not impose any meaning on the evidence as a result of anything but unbiased interpretation.

Specifically on the subject of Inconclusive, that determination is also strictly objective when the analytical process is properly carried out. The conclusion is strictly the product of all the observations being made and interpreted according to the best abilities of the examiner and the examiner's strictly neutral and professionally derived threshold. If I am asked, "Isn't your conclusion really subjective?" my answer is, "No. It is entirely objective. I look only at the evidence and apply my analysis without regard to any wish for any particular result and without any other influence." My being objectivity doesn't mean I'm correct. But it means that I'm not incorrect on account of improper influence.

Sorry. I'm kind of a bug on subjective/objective, because "subjective" is so often used as an attack, and "objective" is so often used to imply accuracy, both being commonly used incorrectly, according to my objective judgment.

[Phil Barletto]

Just to set the record straight on the case details. I see this board has gone in several directions, so here it is.
The homicide occurred in Oct. 2001. One of my co-workers developed the latent on the right side of the Remington XP-100 rifle, that was used to kill the victim. In April of 2002 the latent print was identified to the #8 finger of Mark Miller. There were several interviews with Miller by investigators, during all of the interviews Miller denied seeing or touching the weapon. Ivan Futrell was hired by defense attorneys to review the ID. In March of 2007 he generated a report indicating "the latent fingerprint images of one fingerprint from a remington XP-100 weapon is not the left middle finger of mark Miller". In turn Ivan Futrell had a verification conducted by George Wynn (according to the defense attorneys letter to the prosecution). The defendant pled guilty the day of jury selection. During his plea, he then stated to the judge, that he killed the victim with the Remington XP-100.
The district attorney then sent all availible items, reports, Power Point presentations, photographs to the IAI for review. The professional review board of the IAI agreed that Mr Futrell and Mr Wynn had made a technical error in their misidentification of the latent print. Thank you for your intrest in the case, Trooper First Class Phil Barletto, PA State police Forensic Services Unit.