Statistics and Misidentifications - The weeks Detail

[Outsider]

1/4

PROSECUTOR’S FALLACY

The victim of a street robbery could not describe the robber but could describe a distinctive piece of jewellery that he was wearing. Police inquiries revealed that the jewellery was made locally, only 5 of the type were made and they were all sold recently in the town. Let’s consider 2 situations.

Case A
The police ask the public for help to solve the crime but they do not mention the jewellery. Someone overheard a man shortly after the time of the robbery bragging about getting some money “the easy way”. The police are called and find that the suspect is wearing the exact type of jewellery described by the victim.

Case B
The police describe the article of jewellery when asking the public for help. Someone thinks they have seen a man wearing it. The police are called and confirm that the suspect is wearing the type of jewellery described by the victim.

In both cases the suspect was arrested and accused of committing the robbery, which he denied. During the court case the prosecutor told the jury that the town has a population of 250,000. 5 examples of the type of jewellery were made so the odds of someone having one are 1 in 50,000. Chance, therefore, can reasonably be ruled out as an explanation for the jewellery being found with the accused. The only reasonable explanation is that the item of jewellery is the one that the robber was wearing, so the accused must be guilty.

In case B the defence counsel pointed out that 5 of the type were sold so there are another 4 people in the town who could equally have carried out the crime. The odds that his client is innocent are not 1 in 50,000 but 4 in 5.

How can it be that in case A the rarity of the jewellery provides reasonable certainty that the accused is guilty (odds that he possessed the jewellery just by chance, and therefore he is innocent, are only 1 in 50,000), but in case B any perceived certainty is an illusion?

The answer is that the odds of 1 in 50,000 only apply to people who are chosen AT RANDOM. In case A the suspect was chosen BEFORE anything was known about possession of the article. With respect to the jewellery this is random selection. In case B the suspect was chosen AS A RESULT of possessing the article. This is non-random selection.

The prosecutor was right in case A but in case B he fell foul of the Prosecutor’s Fallacy. The suspect changes from being almost certainly guilty to probably innocent just because of the selection method (of course the suspects would be different people).

The Prosecutor’s Fallacy in a nutshell is:You cannot use the rarity or low likelihood of something as proof of guilt if the suspect was selected because of that thing out of a very large pool. If you do, you have committed the fallacy.

[Outsider]

2/4

Now consider two outwardly similar fingerprint cases.

Case A
A police officer is spotted going in to a crime scene and failing to make a note in the attendance log kept at the entrance. The Chief Inspector questions everyone who has been given permission to enter the crime scene. All of them can prove that they were elsewhere at the time of the un-recorded entry. The Chief Inspector is angry to think that an unauthorised officer entered the crime scene and he wants to know who it was. A few days later the fingerprint department announce that a police officer was identified from a fingerprint in the crime scene. This officer did not have permission to enter and vehemently denied it when told of the identification.

Case B
A fingerprint department announce that a police officer was identified from a fingerprint in a crime scene. This officer did not have permission to enter and vehemently denied it when told of the identification. Despite a thorough investigation no evidence is found to suggest that anyone had entered the crime scene who did not have the necessary permission.

In both cases the Chief Inspector knew that fingerprint misidentifications are extremely rare. Given that there is no reason at this point to question the competence of the fingerprint department, the odds of a misidentification must be 1 in many tens or hundreds of thousands, maybe even more. A misidentification, he concluded, can reasonably be ruled out. The only reasonable explanation was that the officer deposited the fingerprint, so he will accuse the officer of lying and initiate disciplinary proceedings.

In case B, a statistically-aware Quality Assurance (QA) officer in the fingerprint department heard about the case and it made him feel uneasy. He remembered from his statistical awareness class that unless we can be sure the error rate is zero, if there was an unusually low likelihood that a fingerprint was deposited in the manor described by the prosecution, there was a heightened chance that the case was based on a misidentification (something to do with “prior probability” he recalled). He called the Chief Inspector and urged caution, pointing out that before the identification nobody had any reason to imagine that any unauthorised person had entered the crime scene. He also pointed out that there is no general perception that crime scenes are frequently visited by police officers who are not authorised to do so, who are also dishonest and reckless enough to lie about it when confronted with a verified fingerprint identification. In fact, in the history of fingerprinting he knows of no such case but knows of a number of cases of misidentification. So although a misidentification is a very rare event, he would consider a disobedient, dishonest police officer entering a crime scene to be equally rare. Since there is no independent evidence that this happened, the QA officer thinks that misidentification is the more likely explanation, but he cannot be sure.

How can it be that in case A the rarity of misidentification provides reasonable certainty that the officer is lying, but in case B any perceived certainty is an illusion?

Answer follows:

[Outsider]

3/4

How can it be that in case A the rarity of misidentification provides reasonable certainty that the officer is lying, but in case B any perceived certainty is an illusion?

The answer is that the very low likelihood of misidentification only applies to identifications chosen AT RANDOM (or selected out of a small group which was chosen at random). In case A, we KNOW that a disobedient officer visited the crime scene. This gives a fair chance before the fingerprint analysis (prior probability), that a fingerprint will have been deposited by the officer. The latent prints in the crime scene became the limited group of potential sources from which the disobedient officer might be found, and this was BEFORE any identifications were announced. This is random selection (and this is the way it works in normal crime cases). In case B the identification was chosen AS A RESULT OF THE DENIAL. With regard to accusing someone of lying this is non-random selection.

The Prosecutor’s Fallacy in a nutshell is: You cannot use the rarity or low likelihood of something as proof of guilt if the suspect was selected because of that thing out of a very large pool.

The identification with the denial along with the rarity of known misidentifications is the only “proof” that the police officer was lying. In case A it was selected out of the small group of identifications which had a connection with the act of wrongdoing, which we know happened. In case B it was selected out of an unlimited pool of police elimination identifications at all crime scenes, and the act of wrongdoing is hypothetical, having to be inferred from the identification. If misidentifications happen but are very rare, the police officer in case A is almost certainly guilty but the police officer in case B is probably innocent.

All that is necessary to create a case B is for a fingerprint department somewhere to fail to achieve full individualisation during the police eliminations (or for a police officer to actually enter a crime scene unobserved and deny it). The vast number of police eliminations throughout the world every day means that extraordinarily close matches will have to be handled from time to time, so misidentification, if we don’t specify the location or time period, is not unlikely. It is impossible to estimate the balance of probabilities between this and a disobedient police officer entering a crime scene and lying about it.

Case A on the other hand originates from a physical act. The limited number of latent prints connected with the act means that a failure to fully individualise would be unlikely to alter the outcome. A misidentification in case A is extremely unlikely so it can reasonably be ruled out.

The only other situation that I can think of where full individualisation is required is database searching. But unlike case B, a random match from a database search would be unlikely to hit on someone who has a feasible connection with the crime. I am sure that this must help to trap most misidentifications. Once again, the pre-existing act of wrongdoing is the vital safety factor.

Of course, if case B was generated by a grossly incompetent or corrupt fingerprint department then the police officer would be even more likely to be innocent.

[Outsider]

4/4

Fingerprint Case B continues:

Although he didn’t quite understand the bit about random selection the Chief Inspector agreed with the QA officer that an independent opinion on the identification should be obtained before proceeding. At this point a Texan was spotted outside the crime scene. “Are you a sharpshooter?” asked the Chief Inspector. “Nope, fingerprint expert.” came the reply, so he was invited in to look at the prints. The Texan told the inspector that he excludes the police officer as being the donor of the fingerprint in question. The Chief Inspector and QA officer decided that other identifications should be checked starting with the murder case that was under investigation when all this started - and a lot of trouble was avoided.

http://www.stevehornsc.pwp.blueyonder.co.uk/pf.htm

[Daktari]

Steve, your imagination is staggering. I can honestly say that it really exceeds your statistical knowledge and understanding.
Perhaps you should consider writing a book on the McKie case. Ooops someone with an even greater imagination, but equally distanced from reality, has beaten you to it. Don't be too despondent though, it didn't sell very well
Even the Big Red Bus did not help it's sales. Let's hope that the next time the name McKie appears on a vehicle it's chalked up on a Wee White Van that has Reliance Prison Services on the side.

[Pat A. Wertheim]

The book I would love to read, but doubt I ever will, is the one that will be published decades from now once all the dust has settled and all of the truth is known, written by some excellent researcher not currently involved in the case. That may be a long time coming. There are things about the Harry Oakes case from 1943 that are still hidden in the files of various national police agencies on both sides of the pond. I suspect the same may be said of this case fifty years from now.

You may not like Steve's speculation, Daktari, but I really enjoy it. I think that his theories are closer to the mark than yours. And I look forward to a quiet visit in a little Scottish pub when you and I can banter back and forth in person. What fun we shall have!

[Outsider]

Steve, your imagination is staggering. I can honestly say that it really exceeds your statistical knowledge and understanding.

Give details of where you think my statistical knowledge or understanding is inadequate and let's discuss it.

[Daktari]

Did you not say that probability changes over time?
Did you not say that if everyone in the UK had there fingerprints taken that would produce a database of 50m prints? OK, maybe it's the ten times table you struggle with.
Did you not confuse the odds of winning the lottery with the chances of meeting a lottery winner?
Did you not completely fail to realise that the Texas Sharpshooter refers to clusters of data not a single point?
I still have your early postings from the McKie website. If I have more time I'll go through them and really embarrass you.
In the meantime can I draw your attention to a certain Allan Bayle, another self styled expert, who jumped aboard into the McKie bandwagon and paid the price? Although some think he got off lightly, at least he won't be spending many more night in the McKie household!

[Outsider]

Did you not say that probability changes over time?

Probability changes after non random selection. Probability does not change after random selection. Think about picking a ball from a bin, what is the probability of selecting the one red ball out of all the white balls if you pick blindfold or use you eyes to select the ball you want.

Did you not say that if everyone in the UK had there fingerprints taken that would produce a database of 50m prints? OK, maybe it's the ten times table you struggle with.

Eh?

Did you not confuse the odds of winning the lottery with the chances of meeting a lottery winner?

No.

Did you not completely fail to realise that the Texas Sharpshooter refers to clusters of data not a single point?

Some people use cluster examples to explain the fallacy because they work in fields where they work with this type of data. Academics are hopeless at explaining concepts like this, they see life through formulas and talk a different language from the rest of us. Do a Google search for Prosecutor’s Fallacy and you will be frightened to death by the formulas. The real world concept is quite simple. I see the Texas Sharpshooter and Prosecutor’s Fallacies as sharing the same logical root and they are about selection.

Even if the error rate of identifications is one in tens of millions, if you have a non-random mechanism that efficiently selects that one mis-ID out of all the rest, you cannot say that misidentification will be unlikely in that case. I cannot think of a more efficient mechanism for selecting a mis-ID than assuming that some previously unimagined wrongdoing has occurred just because someone denies that they deposited a fingerprint, with no other evidence of anything, not even evidence that the assumed wrongdoing has happened.

[Thomas Taylor]

Dear Steve Horn, Pat Wertheim, David Grieve, Les Bush, Ian McKie, and others who insist on using their real names to attempt to win debates with these anonymous losers, both on this topic and others recently:

I posted a comment last year warning against wrestling in the mud with pigs. The pigs enjoy it and you only get smeared with their mud. Gentlemen, all of you have the winning arguments but the pigs will never admit it. They will continue to do nothing but deny your truth with their anonymous lies. If you lower yourselves into their mud pit, you cannot emerge clean.

Please desist from playing their games. Stay out of the mud and above the fray. Let the anonymous tellers of truth wrestle with the swine. Through their anonymity, like the pigs, they will remain free of the mud.

[Daktari]

Look stupid, you claim probabilty changes over time then say
Probability changes after non random se ... you want./
You are talking about two different EVENTS, not two different
PROBABILITIES. That's why you do not understand what you are talking about. Get real Steve, go to your local library and get Statistics for Dummies. You certainly need it!

[Outsider]

Daktari is right that events are different things from probabilities or proportions but there is nothing complicated about it. A human decides to do something, create an event like starting a prosecution, because of something that he or she finds, for example a fingerprint identification. But we don’t prosecute all fingerprint IDs, we only prosecute some so there is a selection process involved. We must understand this selection process because it is a vital part of the whole system.

Before the fingerprint work starts in a location there are only two types of latent print, those deposited by a wrongdoer and those deposited by honest people. Fingerprint work has only two outcomes, good identifications and misidentifications. When two binaries interact, four outcomes become possible. So after fingerprint analysis there are four possibilities:

A. Honest people’s fingerprints which have been correctly identified
B. Wrongdoers’ fingerprints which have been correctly identified
C. Honest people’s fingerprints which have been misidentified
D. Wrongdoers’ fingerprints which have been misidentified (sounds weird but I suppose it is possible)

The police or prosecutor decides to fire the “event” of starting a prosecution because the identified person cannot give an innocent explanation for being in the location and denies that they have deposited the fingerprint. This is where selection happens. B and C will both trigger the event. The safety or reliability of fingerprinting at the point of accusing someone of lying is the ratio of B to C. Let’s do some “back of envelope” calculations (scientists and engineers do this all the time, the purpose is to understand processes, not to reach mathematical conclusions).

Let’s think about a crime scene where there are 400 latent prints and 2 of these were deposited by the wrongdoer. Lets take the proportion of good identifications as 999,999 out of every one million identifications (misIDs are 1 in 1 million).

B = 2 times (999,999 / 1,000,000) = 1.999998

C = 398 times (1 / 1,000,000) = 0.000389

So the ratio of B to C is 1.999998 to 0.000389 or 5141 to 1

If we want to express this as a proportion or probability, the calculation is B / (B + C). So the proportion of prosecutions based on good IDs with these figures is 1.999998 / (1.999998 + 0.000389) = .9998 or 99.98% (someone please check my calculations).

We have just done all that Bayes Theorem stuff but without the symbols like p(X|~A) * p(~A). It is important to understand that to estimate the reliability of fingerprinting at the point of an accusation of lying, we need to be able to estimate both the error rate of identifications AND the proportion of wrongoers’ fingerprints in the crime scene (the so called “prior probability”).

If we play about with the figures and let the prior probability drop right down to a similar figure to the error rate of identifications, then the reliability of fingerprinting (at the point of an accusation of lying) drops to an unacceptably low figure, even if the reliability of identifications is still very high. Lets try it with a prior probability of the wrongdoer depositing a print of 1 in one million (999,999 out of 1 million are deposited by honest people), the error rate of identifications is the same as before:

B = 0.000001 times (999,999 / 1,000,000) = 0.0000009

C = 0.999999 times (1 / 1,000,000) = 0.0000009

So the ratio of B to C is 0.0000009 to 0.0000009 or 1 to 1, a 50% chance of error. B and C are both very small numbers so they are both very rare events. The problem is that we cannot distinguish between B and C so to be safe we must make sure the B is always much more likely than C.

The way to get a miniscule prior probability of a wrongdoer depositing a fingerprint is to have no evidence that a crime or act of wrongdoing has taken place.

Of course the calculations are a simplification, for one thing the error rate will not be a constant. But I think that an understanding of the mechanisms at play can only be a good thing, even if mathematical accuracy is impossible.

Daktari, I am a computer programmer, you won’t beat me on logic.

http://www.stevehornsc.pwp.blueyonder.co.uk/bayes.htm

[Daktari]

A human decides to do something, create an event like starting a prosecution, because of something that he or she finds, for example a fingerprint identification

The reason Shirley McKie was prosecuted was not because her print was found at 43 Irvine Road. That was not a criminal offence. As everybody knows McKie was banned from entering the house because of her previous history. The real reason was because it was believed that she lied at David Asbury’s trial.
McKie’s then Advocate (Barrister) Herbert Kerrigan QC visited Peter at his Wakefied office (some 200 miles from Glasgow) and reported back to the McKies. He was dismissed from the case. I wonder why?

The safety or reliability of fingerprinting at the point of accusing someone of lying is the ratio of B to C.

Fingerprint officers do not accuse people of lying. They simply offer an opinion on whether a match has been found or not. You will of course be familiar with the ident found on the stolen car in Kilmarnock. The so-called fifth misidentification. It is not mutually exclusive for a ident to be correct, but the person who deposited it to be completely un-associated with the actual crime. But that’s not logic, that’s common sense!

[Outsider]

As everybody knows McKie was banned from entering the house because of her previous history.

Daktari, you are straying close to a reasoned argument here, but you can’t see it. So let me present the argument for you:

Look stupid, what would you say the “prior probability” was of leaving a fingerprint in the wrong place for someone who has a history of leaving prints in the wrong place and lying about it? Have you not heard of the “baby in the bag” case? Get real Steve.

I would then reply:

On the face of it a police officer with a personality that leaves fingerprints in the wrong place and lies about it COULD be a factor affecting the probability of doing so again, although not as big a factor as real proof before the fingerprint analyis that a police officer was in a forbiden location (as in my fingerprint Case A above). But this would only be a factor if we can be reasonably sure it happened.

The baby in the bag case revolves round whether McKie was wearing gloves when her fingerprints were found on a court exhibit. I did some quick tests with materials I had easily available. I have proved that with a thin smooth-surfaced glove it is extremely easy to deposit a fingerprint while wearing it. Here is the proof. The picture on the right is with a smooth-surfaced glove, the one on the left is rough textured.

http://www.pbase.com/stevehorn/image/80449870.jpg

Fingerprint departments now advise double gloving to prevent transfer of fingerprints while handling exhibits. This is a researched, documented and accepted problem. So let’s put this one to rest. There is no reason to believe that Shirley McKie was lying about the baby in the bag case. OK. Let’s not hear that one again.

It does seem like a big coincidence that McKie was involved in both cases. I always urge caution when inferring something from coincidence because unless you specify what you are looking for before the events happen, co-incidence is not nearly as rare as you might think. Have you ever met someone from your neighbourhood while on holiday thousands of miles away?. “What an incredible coincidence” you think, but it is not unless you specified some of the details before the meeting (yes, it’s the Texas Sharpshooter again).

But if there is a thread to link the baby in the bag case to the McKie/Y7 case we should look to research to find it. I am thinking about Dr Dror’s research on bias. Here are two situations

Case 1
A police elimination match identifies a well-liked and well-trusted police officer. The officer says that he was not in the location. Nobody believes that this officer would lie. “It was only an elimination, we didn’t use the 16 point standard, just forget it.”

Case 2
A police elimination identifies an officer who is not particularly liked - not a team player. The officer says that she was not in the location. People remember the previous case, they never really believed her then. “It’s that bl***y woman again”. “I don’t like police officers who lie.” “How can you press fingerprints through gloves, the idea is ridiculous”. “Make sure the 16 point standard is applied, she isn’t getting away with it a second time”.

[Pat A. Wertheim]

Steve, allow me to take Case 2 a step further. One of the people who has reacted against the female officer says, "Let's give that bl***y woman something else to worry about. Newspaper reporters have been hounding us for more informations, so let's give it to them." So some of the people who are upset with the female constable call a few of the reporters and say, "The real reason her fingerprint was on that door frame is that this woman likes kinky sex. She has a married lover and they snuck into the scene to 'do it' on the exact spot the murder occurred. It was while she was gripping the door frame during the act that her fingerprint was deposited."

That, Steve, is exactly what happened. I was asked by one of those very reporters in the spring of 2000 what my thoughts on that story were. He told me an officer had just given him this information "off the record" and wanted some of the tabloids to publish it "from reliable inside sources." He said other reporters had been told the same thing and there were several detectives and officers spreading the rumor. None of the reporters believed it and so none printed it, even in the most raunchy of the tabloids. The story was just too obviously a lie. The reporters saw it as a desperate effort to discredit the female constable with a story designed to distract her from the real fight, i.e., the fight against the perjury charge the male officers had lost in court, but were still insisting was correct to the press.

When some of the anonymous liars on this site insist Iain McKie invented the story about sex, they are lying yet once again. I know because I heard it from that reporter months before Iain mentioned it to me, and may well have heard it even before Iain did.

Your analysis of this case is remarkably accurate. Thank you for helping expose the lies. The truth, as you imply in Case 2, was that Shirley McKie was a strong woman fighting to survive in a man's world. She was the youngest female ever promoted to her rank in Strathclyde Police and the chauvanistic men could not accept an uppity female on the fast track. They had to destroy her. When the fingerprint (actually, a palm print, I believe) on the garbage bag with the dead baby failed to slow down her progress, they had to resort to something else.

Okay, Daktari and caledonian, go ahead and smear me with some more of your mud, the way Taylor says you will. I have no doubt he is correct. You can allow no truth to go unbesmirched.