Statistics and Misidentifications - The weeks Detail

[Daktari]

And can you explain how you can simply ignore the other “so many experts” who disagree with you and your colleagues?

You make the distinction yourself when you say, selective people "looked at the original material".
And, may I ask you again, please stop misquoting me.
The way you distort what people say I am beginning to believe you may be an SNP supporter, schurley(sic) not!

[Taggart]

Daktari,

Once again you have lost me! However I do apologise for making the question far too complicated and difficult for you to answer.
So in fairness let me try again to get your clarification.

You stated, and let me quote you word for word

“The probability of so many experts making the same two 'mistakes' in the same case, given their experience and previous reliability, is mind boggling.”

I did a copy and paste so I don’t misquote you!

What I was wondering was this. You mention “so many experts”. You agree with that? And are you happy you haven’t been misquoted? Good, let’s continue.

We know six experts at SCRO saw the original material. We know Peter Swann did, we know Malcolm Graham did. We also know by his admission that Martin Leadbetter did NOT use original material, and we also know by his admission that John Berry did NOT use original material.

Let me go back and quote (copied and pasted, in fairness to you) your colleague Miss McBride.

“The fact that so many experts came to different conclusions about the print, undermined the science of the process itself. Ms McBride says the answer lies in the fact the SCRO experts were the only ones given access to the original print - a claim bound to be refuted by others.”

So your colleagues states in a National Newspaper that the answer lies in “the original print”. So she takes Berry and Leadbetter out of your equation. This leaves you with what? Let’s recap. Six Experts at SCRO?

Six Nil to one side.

Malcolm Graham. You remember Malcolm Graham? The same expert who apologised for his error? The same Expert who was paid to examine a metal tin and he concluded it was covered in Fingerprint Powder when in fact it had been treated with superglue and dyed, and there wasn’t a single particle of powder on the tin! An easy mistake to make?

Seven Nil to one side.

Peter Swann, who admitted in his evidence that his conclusion was initially based using the Court Chart produced by SCRO. The chart Swann also admitted was of poor quality. I’m sure you can recall the blurred cropped chart.

Eight nil to one side.

Well I guess it would be correct to call eight experts “so many experts”. And you stated it would be “mind boggling” for them to have made mistakes “given their experience and previous reliability”.

Let me add a few others to this equation daktari, although I realise you know where I am going with this. Let me add the names of Pat Wertheim, Arie Zeelenberg, Torger Rudrud, Geoff Shepherd, Mike Thompson, Kristian Rokkjaer, and Frank Rasmussen.

International Experts with no connection with each other. Experts with no vested interest in the case. Experts who are completely independent. Not six experts who all worked within the same organisation. Not forgetting Robert Mackenzie’s long association with Peter Swann.

Seven International Expert who have all examined “original material”. A fact which exposes Miss McBride as having lied by stating to The Herald newspaper that SCRO were the only ones to have the original material!

“Ms McBride says the answer lies in the fact the SCRO experts were the only ones given access to the original print - a claim bound to be refuted by others.”

Well daktari, I do refute it!

So the score is now Eight Seven.

But I suggest the seven is stronger as it is clear their independence is far greater. So let me try again so you can give me some clarification.

“The probability of so many experts making the same two 'mistakes' in the same case, given their experience and previous reliability, is mind boggling.”

We have Six Experts all working in one office, Swann and Berry and on the other side we also have Seven International Experts, with no vested interest, and with total impartiality.

So please clarify your statement and tell me which group of “so many” experts" you would find it mind boggling to be incorrect in their conclusions? It is obvious to us all you believe it to be the first group.

So maybe you can explain why your statement does not apply to the Seven International Experts? Are the probabilities significantly less for seven independent experts to be incorrect, over six people who work in the same office? I would have thought the chances of seven totally independent experts making errors would be considerably less than six experts working in the same office, arguably with a vested interest? After all both sides cannot be correct can they????

So please do explain and prove to me your mind is not already boggled!

[Outsider]

In the paper “How the Probability of a False Positive Affects the Value of DNA Evidence” (Prof. William C. Thompson, Chair, Department of Criminology, Law & Society, University of California, Irvine and others) it is noted that courts require the random match probability (RMP) of DNA profiles when DNA evidence is used, but the frequency of false positives (lab errors) is neither known nor required. This is of concern because when the RMP is very low, false positives would be the significant source of errors. The paper also shows that the importance of the error rate is not the same across all cases.

Particularly in cases in which there is little other evidence against the subject, ignorance of the true probability of error creates a disturbing element of uncertainty about the value of DNA evidence

This is similar to the point I have been making about the McKie case. I thought it would be interesting to see how varying the error rate altered the likelihood of guilt in different situations. The paper of course gives a Bayesian proof but I am more comfortable with a mechanistic/frequency view so I did an imaginary exercise using fingerprints. Here it is:

-=-=-=-=

Consider 3 fingerprint cases

Case 1 - A house is robbed and someone is identified from a fingerprint who denies having been in the house. The police visit this person at home and find the stolen goods there. The suspect refuses to explain how the goods got into his possession.

Case 2 - A house is robbed and someone is identified from a fingerprint who denies having been in the house. This person could have carried out the robbery but no other evidence is found to link him to the crime.

Case 3 – A house is robbed and someone is identified from a fingerprint who denies having been in the house. This person can prove that at the time of the robbery he was thousands of miles away attending a conference of forensic statisticians. He says he cannot explain why his fingerprint was found.

We need to make some simplifications and constants to make the principle clear. All crime scenes contain 100 identifiable latent prints. When someone enters a house without the knowledge of the householder they will deposit one identifiable latent print and we will have that person’s prints on file. This person will deny having been in the location, as will someone who has been misidentified.

Let’s try the exercise with 3 after-verification error rates, (a) zero, (b) two errors per million identifiable latent prints and (c) two errors per 10,000 identifiable latent prints. Half the people misidentified will be charged with the crime under investigation but for the other half this is not feasible. The question is - what is the likelihood that the identified person is lying about visiting the house? For cases 1 and 2 this would be in connection with the robbery, for case 3 this would be for some unspecified reason? Case 3 cannot be answered without more information.

Additional Information
A survey has recently been published. It found that in one house in every 10,000, a neighbour creeps in unseen for no obvious reason, leaves no trace except a fingerprint and will lie about it (this was a very difficult research project).

[Outsider]

For case 1, I think that no matter what the error rate is, it is certain or nearly certain that the identified person has visited the house. The fingerprint ID caused the police to visit his home but after the goods were found we have additional evidence that he is implicated in the crime. Even if the error rate was very high, finding the goods suggests that this identification is not one of the errors. A statistician could give a Bayesian explanation but a normal person could figure this out for themselves.

In case 2 with an error rate of zero he is lying, infallible is infallible. For the other error rates, the crime scene contains 100 latents so the chances of an error occurring that could incriminate in the robbery are (b) 1 in 1,000,000 times 100 = 1 in 10,000 and (c) 1 in 10,000 times 100 = 1 in 100. This will have to be compared with the chances of identifying the perpetrator of the crime. We have simplified this to certainty by saying that he will leave a print and we have his prints on file. The odds that an identified person is telling the truth because of misidentification in these circumstances therefore are (b) 1 in 10,000 and (c) 1 in 100 (if it was not certain that we would identify the perpetrator the error rate at this point would rise because the number of misidentifications would stay the same).

The starting point for case 3 is the knowledge that one house out of every 10,000 has had an uninvited visitor for reasons unknown. Scenes of unrelated crimes will be subject to the same statistic. 10,000 crime scenes contain 1,000,000 latent prints and one of these will have been deposited by the visitor. The same 1,000,000 latent prints will contain (b) 1 or (c) 100 misidentifications that cannot be connected with the crimes. So the odds that an identified person is telling the truth when they deny depositing a print in these circumstances are (b) 1 in 2 (evens) and (c) 100 in 101.

So for the three cases and three error rates the chances of guilt (lying about entering the house) are:

Case 1: (a) guilty, (b) guilty, (c) guilty
Case 2: (a) guilty, (b) 1 in 10,000 of innocence, (c) 1 in 100 of innocence
Case 3: (a) guilty, (b) evens, (c) 100 in 101 of innocence

The trend can be seen but the absolute values have no meaning. I just chose the input values to make the calculations easy. I see a scale which starts at case 1, goes through case 2 and ends with case 3 (and Shirley McKie). At one end knowing the error rate is not important but at the other end it is vital, as is knowing the frequency of the act of wrongdoing (we usually know this, it is 1 per crime scene). That scale is the strength of all the other evidence in the case (the probability of guilt prior to considering the effect of the fingerprint evidence).

I am not sure where database searching fits into this picture. It would seem that there would be an increased risk of error because the database will contain a number of non-matches that are very similar to the real match. But if the closest match from 50 million records hits on someone with a very close connection to the crime, that must have a significance.

If someone denies depositing a fingerprint and they cannot be linked to the crime and there is no other evidence to suggest that another act of wrongdoing has occurred, this should be treated as a very special case. It is essential that both the error rate and the frequency of the proposed act of wrongdoing are known, or at least the approximate ratio of the two. To assume dishonesty without knowing this would be highly dangerous and irresponsible (a discrete but thorough investigation might be appropriate). It might seem that an independent out-of-department verification would settle the matter but the Cowans, Mayfield and McKie cases all show that independent verification can not be trusted to catch every, or indeed any, misidentification.

The point about the survey of secret house-creepers being a difficult research project is not a flippant remark. For case 3 we need to know how often people enter places and leave no trace or any reason to believe that someone has been there - and lie about it. Did the research team employ a clairvoyant? This paradox might apply to many or perhaps all attempts to use a forensic result as the only substantial evidence that an act of wrongdoing has occurred that someone is lying about (you have to know how often something occurs which leaves no reason to believe it has happened).

"How the Probability of a False Positive Affects the Value of DNA Evidence":
http://www.bioforensics.com/conference/ ... %20Pos.pdf

[Daktari]

A survey has recently been published. It found that in one house in every 10,000, a neighbour creeps in unseen for no obvious reason, leaves no trace except a fingerprint and will lie about it (this was a very difficult research project).

Come off it, this is pure Mickey Mouse stuff!

[Pat A. Wertheim]

No, Daktari. Yours is the delusional fantasy that crosses the line from any semblance of reality. Steve's musings have value to them. You, on the other hand, are flat wrong in everything except for one -- your stated commitment to a full and open judicial enquiry is commendable. Perhaps the debates here, if read by some of the MSPs who care, might sway enough that such an enquiry will come about. But frankly, I am beginning to believe that any dreams you and I shared of an enquiry are pure fantasy, too.

[Daktari]

Trouble is there are a number of things going on just now that may take precedent over an Inquiry given the sub judice rules that exist (but not always obeyed) over here. Asbury's compensation claim and a certain Employment Tribunal are examples.

I fail to see how Steve Horn's postings offer anything but a right good laugh. His understanding of police procedures, judicial procedures, how a fingerprint bureau works, the theory of probability, simple arithmetic and much else leave a lot to be desired.

I think that we will see Judicial Inquiry as there must be a limit to the number of promises that the SNP will break and they used a few up yesterday with their Budget.

Clearly there will be a lot of pressure to drop the Inquiry, cost, time consuming, nothing to be gained, etc, etc, we have already heard similar from Iain McKie and SPSA boss David Mulhern. I

On the other hand there are people who know what went on at 43 Irvine Road and are not prepared to let the matter drop.

[Daktari]

Oh Taggart, I almost forgot about you.

Since you have resorted to playing the numbers game again
here's a few more to add to your list.

Allan Bayle why does no one mention him anymore?
Gary Dempster wonder what happened to him?
John MacLeod who only found one matching characteristic

Hardly three wise men I know. But on your side none the less.

[Taggart]

The numbers game dear boy? The only numbers I am interested in is as follows:

Explanations asked for by daktari in my previous posting – ONE
Explanations offered by daktari to evidence his claims – ZERO

If nothing else that statistic has been pretty consistent now for quite a while!

Out of curiosity, have you any idea why your name was recently been made known to me???? I am truly amazed that someone has revealed such information to me, but looking back over your postings it indeed makes perfect sense. In fairness it would be wrong of me to post it so publicly on this forum; however it will be extremely fascinating to see if your evidence under oath at any Public Enquiry is consistent with your comments on here? And there was me thinking you were all going to stick together, but appears not!

If you read back over my previous postings did I not mention one of your colleagues has already betrayed you all by their actions? Don’t worry that evidence is safe and will be kept until the Enquiry and then produced. If of course there is no Enquiry for whatever reason, then that evidence will be submitted to the Crown Office in due course. So rather than worry about anybody else, and what they may or not be doing daktari, I suggest you have a long hard look at those closer to you. Of course will any of them admit betraying you a second time? I assume you already know who betrayed you the first time and what awaits you all at the Enquiry? Of course if you have worked it out, I can only assume you have all by now got your stories sorted, as clearly they will have had to have changed considerably from the previous versions told, which in turn will expose lies told to the Parliamentary Enquiry?

[Daktari]

Oh dear, you've found me out at last.
How will I ever sleep at night?
You may as well 'publish and be dammed'.
It looks like it's all over for me!

[Outsider]

A few thoughs about database searching.

In terms of risk of error, it could be argued that the McKie case has similarities to searching large fingerprint databases. In both, a single verified error in a huge number of fingerprint comparisons will lead to a false accusation. In a database search one latent print is compared with millions of records in the database and an error will cause the wrong person to be accused of the crime. If there is a verified error in the very large number of elimination comparisons performed throughout the world every day, a McKie-type case will be initiated if it is subsequently handled in the same way.

A while back g. offered the idea that the prior probability of guilt (before the effect of the fingerprint evidence is considered) could be a Normally distributed random variable with the assumption that those with closest proximity to the crime have more likelihood of committing it. If a database search hits on someone who lives far away from the crime and who appears to have no connection with it, then it would be very unlikely that this person has deposited a print in the crime scene, if the fingerprint match had not occurred. According to Bayes theorem, a very low prior probability will lead to lower than normal posterior probability after we apply the fingerprint evidence (if the error rate is not zero). It could be argued that the Mayfield misidentification is an example of this.

http://www.usdoj.gov/oig/special/s0601/PDF_list.htm

By this analysis and my own, the two most discussed and notorious misidentifications of all time, Mayfield and McKie, could have been predicted as being especially risky because they both have abnormally low prior probability, Mayfield because of the large database search and weak connection with the crime and McKie because there is no proof that the proposed act of wrongdoing happened - by anybody.

One major difference is that database searching creates a very large number of comparisons, but the number of identifications in any inquiry is fixed, defined by the number of crime scene latent prints. A McKie-type case can arise out of an unlimited population of elimination identifications. The fingerprints of people on elimination lists are expected to be found in crime scenes, so bias might be a factor leading to an error which prompts a denial.

If a database search (and the verification) is done fully blind and it hits on someone who has a strong connection with the crime then it would be very unlikely that this could have happened by chance due to error. So perhaps a database search that finds someone who has a connection with the crime is very safe, despite the large number of records being searched and the fact that many of them will be very similar to latent mark.

[Outsider]

Found this fun site about logic and probabilities:

http://www.dcs.qmul.ac.uk/~norman/paper ... ility.html

The fourth link down is about evidence in court.

[Outsider]

Some people find analogies helpful. The following could help give a mental image of how selection affects certainty and when we need to know an error rate precisely.

THE PAINTED PING PONG BALL PROBLEM
Imagine several cartons each containing thousands of ping pong balls are emptied on to the floor. The balls should all be white but sometimes a coloured painted ball is included by mistake. The manufacturer takes every care to avoid this and they are very rare but with such a large quantity of balls on the floor it may not be unlikely that one or two coloured ones are included.

As long as the manufacturer’s error rate is low, if we close our eyes and pick a few balls from the floor (random selection) it is extremely unlikely that a coloured ball would be among those picked. But with our eyes open we could pick a coloured ball because it has drawn our attention. Statisticians call this self-selection, the item is selected because it possess the property we are interested in.

Someone enters the room without being seen and maliciously paints some of the ping pong balls. We must find these balls and we must avoid the factory-painted ones. Unfortunately they are indistinguishable from one another but when a ball is painted a little paint is nearly always left on the floor nearby.

How can we tell if a coloured ball was painted in the room or was an error? The answer is to look at the context. If there is paint on the floor nearby, a coloured ball is extremely unlikely to be an error. We don’t need to know the manufacture’s error rate exactly to have this confidence, we only need to know that it is low. Balls selected for their proximity to paint on the floor is random selection with respect to the colour of the ball, like picking with our eyes closed. We can then go on to select a ball by colour from within this small group with very little risk of it being an error.

What if we notice a coloured ball and there is no paint on the floor nearby?. This ball has self-selected because of its colour. The odds that it was painted in the room will be the ratio of the number of balls painted in the room where no paint was dropped (we have said that this is very unusual) to the number of painted balls supplied by the manufacturer (errors are also very unusual). Unless we know these two numbers precisely and one of them is much bigger than the other, uncertainty is the only rational conclusion. It would be wrong to say that because the manufacturer’s error rate is low it must have been painted in the room and it would be wrong to say that because there is no paint nearby it must be an error. Both of these are errors of logic.

I have got a longer version of this at my web site where the connections with fingerprinting and the McKie case are discussed:
http://www.stevehornsc.pwp.blueyonder.c ... ngpong.htm

[Outsider]

Some people think that mass medical screening is not a good use of resources. It leads to large numbers of people who don’t have the disease being told that they test positive.

"False Positive Screening For Cancer Found To Be Frequent And Costly"
http://www.sciencedaily.com/releases/20 ... 002224.htm

"Over 10 years, one third of the women screened had abnormal test results requiring additional evaluation, even though no breast cancer was present. … Physicians should educate women about the risk of a false positive result of a screening test for breast cancer.”
http://content.nejm.org/cgi/content/abs ... 38/16/1089
Note that is one third of all the women tested who get false positives. I am sure that false positives will outnumber good positives by a very large margin.

Here is why this happens. A medical test used for mass screening might be 99.9% accurate, in the sense that out of 1,000 tests on people who do not have the disease, 1 of them gets a positive result and 999 get a negative (a false positive rate of 0.1%). Everyone who has got the disease also tests positive. With this test, when someone is told that they test positive, what are the chances that they have got the disease?

The answer is not 99.9%. That is the answer to this question, “what are the chances that someone will test negative given that they are healthy?” The question we want answered is, “what are the chances that someone has got the disease given that they test positive?”

To get the answer we need to know how likely it is that a person who takes the test has got the disease. Say, in the general population we know that 1 in 10,000 has got the disease. Of 10,000 people tested during the screening, the 9,999 who are healthy will all be subject to the 0.1% false positive rate, so 10 of them will test positive. The one person who does have the disease will also test positive. So 10 out of every 11 people who test positive do not have the disease.

You might be shocked that a test that is 99.9% accurate can have 90% of its results wrong, but that is not the right way to think about the test. The test modifies prior odds of 1 in 10,000 to posterior odds of about 1 in 10. The test, on its own does not prove the person has the disease but it might be very useful to know that someone is in a group where about 1 in 10 has got the disease.

If the test was only performed on people who we have reason to suspect might have the disease - maybe they show symptoms or a previous independent tests gives a positive result – then things would be very different. If the prior chance of having the disease is now 1 in 10, then out of 10,000 people tested, 1000 of them will have the disease (rather than just 1 with mass screening). This will result in 1000 good positives and 9 false positives. The same test changes from 1 positive out of 11 having the disease to 1000 positives out of 1009 having the disease.

Using one of the vast number of elimination fingerprint identifications to indicate that some hidden act of wrongdoing has occurred has similarities to mass screening to uncover hidden diseases.

Women who are repeatedly tested for breast cancer have a cumulative risk of getting a false positive. Police officers who are repeatedly in elimination lists have a cumulative risk of misidentification over their careers so I wonder why there have not been more Shirley McKie-type cases. Either fingerprinting is, indeed, very safe or something similar has happened in the past but when it did, it was handled differently.

[Outsider]

I’d be interested to know if it happens occasionally that someone who has been identified from a fingerprint denies depositing it, but they are not charged with the crime under investigation.

I think I read somewhere that if the subject was being checked for elimination, the identifications are normally done with lower quality assurance procedures than for a suspect. If they deny depositing the print, what happens if they do not become a suspect for the crime under investigation. Is there anything like a standard procedure for handling such a thing?

Lord Johnson when he summed up at Shirley McKie’s perjury trial said:

“The second hurdle is why would Miss McKie do this and stick to her position from Day 1, moment 1, apart from the very initial: reaction where she didn’t think very much about it because obviously, as we know, rogue, to put it loosely, fingerprints can turn up.”

Any ideas what he meant by “obviously, as we know, rogue, to put it loosely, fingerprints can turn up”. If a rogue fingerprint means an identification that is denied yet there is no evidence to suggest why it might be denied then an interesting line of enquiry for Lord Justice Campbell might by why was McKie’s case handled differently from other "rogue fingerprints".