CLPEX Chat Board

I would like to know why the use of likelihood ratios are objective if there is a subjective probability?

I would also like to know also how Bayes theorem is going to take into account the fingerprints of those in the future and the past? The reason why I ask this is because we are always told that we cannot be certain of identifications because the worlds population have not been fingerprinted and that people in the past and the future have not been fingerprinted either.

The short answer will do if possible.

I don't think Bayesian inference gets us very much with fingerprint identification. We don't have much of a mathematically useful specific prior probability, although that doesn't exclude Bayesian analysis. But if we do take the Bayesian fork in that road, we need better data to refine the probability than we have today. We don't have good objective definition of how we decide if we are observing a "match" or not. Using a "personal probability," a belief, as the prior probability is accounted for in Bayesian inference. But the next parameter is presumed to be additional objective knowledge with a known likelihood. While this is rarely if ever absolutely certain knowledge, to be useful, it is "scientifically certain," as is, for instance, the known incidence of a disease in the population, in the classic calculation of likelihood of a false positive diagnostic test.

One is tempted to retreat to a pure frequentist view, but I think that becomes rather limited, since we lack well-defined experimental parameters. For instance, we don't have much way to test if an identification truly selected the correct source. Even if we limit our experimental observations to absolutely known situations where we are observing comparisons of impressions of known sources, we are short on knowledge of how much data triggers the match decision, and we also have to deal with the fact that that quantum of data is a minimum required for a sufficient belief to form, there is obviously some range of confidence levels, albeit all very high confidences. That's a very different situation from something like establishing a prior probability of guilt as one person in a population of, say, 50,000 and applying Bayesian analysis when we add DNA data, a situation where Bayesian analysis provides a useful likelihood number for a legal finder of fact.

Bayesian inference can't take the future into account, except that it provides an expected likelihood. And it is the millions of unexamined fingerprints that is exactly the reason we would like to draw inference with those we have seen. The arguments in Bayesian probabilities are always subject to change. It's intended to be dynamic, refining likelihood as knowledge accumulates. But we simply lack enough knowledge to get off the ground with either approach. A substantial amount of research is aimed at developing better knowledge of the sort that could be used for likelihood calculations. There is a strong desire to produce numerical results, because the "science" part of us dislikes relying on opinion. But given the current state of knowledge, attempts to be objective using the little that is known today are uninteresting, except as exercises to help design studies intended to obtain better data.

The greatest difficulty in applying probability to latent print examination is the not unreasonable assumption that we're working with very small likelihoods. (If that wasn't the assumption, we wouldn't be here.) We can't confidently say how specific quantities of detail move the likelihood of unobservable difference closer to or farther from zero, because we deliberately are very conservative with how we defined identity and have avoided working with so little that duplications would have been observed. We haven't the foggiest notion of how much data would be required for the numbers to converge. We haven't much studied that question. I don't know that it's even studiable, given the practical requirement of making judgments of many sorts of observations as we work through an examination. Probability calculation, Bayesian or otherwise, belongs in the realm that explores latent print examination as a brand new concept, because, so far as what is required to make use of those calculations, that's really where we are. Our forensic application belongs in the realm of opinion based on experience in practical problems, opinions that can be subjected to argument as being well or poorly justified.

Actual numbers for likelihood ratios aside, the role of Bayes' Theorem in the Subjectivist account of evidence is probably the most in tune with the ACE process I can think of outside of Hypo-deductive reasoning.

http://plato.stanford.edu/entries/bayes-theorem/#3

Subjectivists maintain that beliefs come in varying gradations of strength, and that an ideally rational person's graded beliefs can be represented by a subjective probability function P. For each hypothesis H about which the person has a firm opinion, P(H) measures her level of confidence (or "degree of belief") in H's truth.[6] Conditional beliefs are represented by conditional probabilities, so that PE(H) measures the person's confidence in H on the supposition that E is a fact.[7]

One of the most influential features of the subjectivist program is its account of evidential support. The guiding ideas of this Bayesian confirmation theory are these:

•Confirmational Relativity. Evidential relationships must be relativized to individuals and their degrees of belief.
•Evidence Proportionism.[8] A rational believer will proportion their confidence in a hypothesis H to their total evidence for H, so that their subjective probability for H reflects the overall balance of her reasons for or against its truth.
•Incremental Confirmation.[9] A body of data provides incremental evidence for H to the extent that conditioning on the data raises H's probability.

The way I read this and apply it to Friction Ridge Identification is that we keep a mental balance in our brains and weigh the cumulative agreement and disagreement which is applied incrementally as we incorporate new holistic elements. Prior to reaching a threshold of sufficiency, we are dealing with degrees of belief towards that hypothesis of individualization (or exclusion if the cumulative disagreement is observed.
This is why I tend to shy away from the ‘One Discrepancy Rule’, as one is not very cumulative and the domain on which the ‘one’ is applied is vague. But that’s an all together different topic now, isn’t it.

Gerald and Boyd,

I enjoyed reading your comments. Much appreciated thanks.

Boyd Baumgartner wrote:The way I read this and apply it to Friction Ridge Identification is that we keep a mental balance in our brains and weigh the cumulative agreement and disagreement which is applied incrementally as we incorporate new holistic elements. Prior to reaching a threshold of sufficiency, we are dealing with degrees of belief towards that hypothesis of individualization (or exclusion if the cumulative disagreement is observed.
This is why I tend to shy away from the ‘One Discrepancy Rule’, as one is not very cumulative and the domain on which the ‘one’ is applied is vague. But that’s an all together different topic now, isn’t it.

A lot of people don't like to think of it that way, but it's just the way the mind works, has to work, when there's no "bright line" threshold. I think of it in terms of tensions. There's a very natural tension that tends to increase as observations trend toward the match. And I don't think the one discrepancy rule is really outside the topic. We presume that the rule comes into play (at least in the way we're talking about it here) when the examiner has otherwise reached the threshold for forming the belief opinion. It essentially purports to add a new term to the likelihood calculation that has the effect of a zero result. Prior to that, the likelihood was taken to be high. Now it's zero. Therefore, the appearance of the new term say in effect that the high likelihood was not to be trusted. But if the new term was never added, perhaps because the discrepancy was not observable, the high likelihood was accepted. It is this combination, that the otherwise presumed valid threshold can be invalidated by the appearance of an additional datum that forces likelihood to zero AND that no matter how badly we would like to see more data, it's not available, and some discrepancy that we acknowledge would force it to zero might be available if only it were impresses, that I think takes the process out of the realm in which such calculations are reliable and places it in the realm of "expert guessing."

And that possibility of an unobservable discrepancy is really what's at work in the creation of degrees of confidence. It becomes a matter of how confident I am that no such unimpressed discrepancy exists. We've already admitted the possibility of discrepancy, as evidenced by the existence of the rule. When I have a great deal of observable data, I'm obviously more confident that seeing more data won't throw my confidence to zero. Another examiner might, for some quantity of data, opine that his confidence is not sufficiently high to form the belief. Because we have an absolute limit of available data in every case, we are constantly creating a belief that is much like the Bayesian prior probability. Such a prior probability need not be indifference, the uninformative prior. I'm perverting the Bayesian idea there, but the point is that we can't move beyond that, on account of no data.

It is indeed largely subjectivist at this state of the art. It meets the subjectivist condition that absolute truth is unknown and generally unknowable (in latent identification). And that accords well with the purpose of legal "fact" finding, which seeks the best rational decision on what the truth would be if it could be known and makes that decision to a stated degree of belief. Bayesian principles are nevertheless worth some attention, because improper prior probabilities are capable of corrupting the decision, and evidence should be assigned correct weight.

And I don't think the one discrepancy rule is really outside the topic. We presume that the rule comes into play (at least in the way we're talking about it here) when the examiner has otherwise reached the threshold for forming the belief opinion. It essentially purports to add a new term to the likelihood calculation that has the effect of a zero result. Prior to that, the likelihood was taken to be high. Now it's zero. Therefore, the appearance of the new term say in effect that the high likelihood was not to be trusted. But if the new term was never added, perhaps because the discrepancy was not observable, the high likelihood was accepted. It is this combination, that the otherwise presumed valid threshold can be invalidated by the appearance of an additional datum that forces likelihood to zero AND that no matter how badly we would like to see more data, it's not available, and some discrepancy that we acknowledge would force it to zero might be available if only it were impresses, that I think takes the process out of the realm in which such calculations are reliable and places it in the realm of "expert guessing."

I don't think I'd necessarily view it in this manner. Belief opinion starts at time = 0.1, it just carries a lower degree of persuasion. I would describe encountering counter confirmatory data as a change in the degree of belief over time. As time (t) moves forward and your sample size of friction ridge comparative elements (n) increases, your degree of belief (b) will increase in either the direction of individualization or exclusion based upon cumulative agreement or disagreement. If t, n and b are small, encountering some data that is contradictory to the tentative direction of your conclusion will represent a challenge to the degree of belief that will have a different weight than if you encountered it later in the process, especially if the orientation/location is taken in context. It is irrational to believe that such an occurrence where t and b were high would lead to the resetting of the probabilistic calculus to zero unless your n was extremely small or the intersubjectivity of your data was faulty from the onset.

The short answer will do if possible.

OK, maybe it's something to do with the difference between confidence (in a statistical sense) and certainty (in an absolute sense).
It's one thing to say I am 100% certain that this is an ident and another to say I am 100% confident in this identification.

Boyd Baumgartner wrote:... encountering some data that is contradictory to the tentative direction of your conclusion will represent a challenge to the degree of belief that will have a different weight than if you encountered it later in the process, especially if the orientation/location is taken in context. It is irrational to believe that such an occurrence where t and b were high would lead to the resetting of the probabilistic calculus to zero unless your n was extremely small or the intersubjectivity of your data was faulty from the onset.

I agree that there's a rationality problem, and I think I see the reason for your feeling about the single discrepancy rule. Is it really a rule about belief or a rule about officially sponsoring a belief? Is viewing it as a rule having to do with official expression the only way to make it work in a system in which the threshold of sufficiency is undefined. This "single discrepancy" (by which we mean a clear and otherwise unexplainable feature) cannot be said to have different weight that depends on when it appears during observation. That depends on where you start and where you are when you see it. If it can take on different weights, then that has to depend on the total amount of consistent data that accompanies it, because the data doesn't accumulate in any particular order.

How can a quantity of consistent data be justified as creating a belief in identity, if a single inconsistency can invalidate it? If I imagine these easily imaginable components:

1 - A typical latent impression, limited, as always, to a portion of the total contiguous friction ridge skin and therefore having some border beyond which no clear data can be observed.

2 - A clear record impression that presents clear features well beyond the limits of the latent clarity border.

3 - A quantity of observable consistent data within the clarity borders, just sufficient to form a confident belief in identity.

Not an unusual situation at all. We have a perfectly acceptable opinion of identity. (Another examiner might disagree about the sufficiency, of course, but would not declare that the first examiner was no expert on account of the more liberal threshold. So we need not consider this second opinion here.)

Now, extend the area of clarity in the latent just a bit, so that it now presents a clear feature that is simply not in the record, a feature for which no good specific explanation can be made. What of the sufficiency that was presumed, but for that small change in the area of clarity?

Now, for the sake of exploring this, I significantly increase the amount of consistent observable data in the clearly impressed area. For instance, previously it might have been 8 good points, and now it's 20. I don't think it matters whether to do this I make the data more dense in the same clear area or increase the clear area. The clear inconsistency is still there, though. Is it now viewed differently? What does this say about how the sufficiency threshold is treated in individual examinations?

In following this thread I wonder if any experts worldwide might like to share some information with me.

In particular with regard to the Official reports being produced by the various agencies. If it will help I will make reference to the SCRO Statements which became Official Crown Production in the case against Shirley Mckie.

Before doing so I make reference to the official correspondence sent by the then Director of SCRO, Harry Bell, in the immediate wake of Shirley’s acquittal:

http://www.thefingerprintinquiryscotlan ... O_1022.pdf

‘The debate on the actual fingerprint centred on the upper two-thirds of the fingerprint detail. This was an area which the SCRO experts dismissed as being too distorted to evaluate.’

Now let us look at the actual statement prepared by these very same experts, looking in particular to how they disclosed or reported on the two-thirds of the print which they could not evaluate. Bear in mind at Justice 1 and elsewhere the experts openly admitted to finding points in disagreements in this area. Points in disagreement that to this day they cannot give an adequate explanation for.

http://www.thefingerprintinquiryscotlan ... B_0010.pdf

’The result of our examination is as follows:-
It was found that the aforementioned photographed impressions were both identical in the sequence of ridge characteristics with the left thumb print on the finger and palm print form in the name of Shirley Jane McKie or Cardwell.’

It is the use of the word ‘identical’ that interests me. How could two prints be ‘identical’ when in fact the experts have openly admitted to finding point in disagreement? To me ‘identical’ means that there are no discrepancies, yet we find later that two-thirds of the print could not be evaluated. This is of course a lie, as evidence is available to show that the experts found points in disagreement. ‘Too distorted to evaluate’ infers that there was no examination of this area, when in fact we know they did ‘evaluate’ this area and saw points in disagreement.

There is no doubt if the SCRO Experts had disclosed their true findings this would have set alarm bells ringing at the Crown Office when it was apparent that only one-third of the print had been ‘identified’. This in turn I believe would have led to the Crown Office having the evidence independently assessed and the mistakes uncovered at an early stage. In real terms this report is not only misleading it could be argued, given expert evidence is withheld, fabrication of material evidence.

Now to my questions:

Do agencies use a standard blank template to report their findings?

If so, what wording is used in relation to identified prints?

Do agencies disclose in any report if there are any part of a print that is not identified, and if so does this report offer any explanation for any discrepancies?

If not, why not?

What training do trainees or indeed experts receive in prints which are ‘distorted’ or are subjected to ‘movement’, or indeed ‘double touch’?

Do bureaux have a file of such prints which are used in training?

Do experts ever carry out any scientific analysis on any such areas to show what happens to the print?

On this thread my bestest friend Daktari offered the following:

OK, maybe it's something to do with the difference between confidence (in a statistical sense) and certainty (in an absolute sense).
It's one thing to say I am 100% certain that this is an ident and another to say I am 100% confident in this identification

.

Does the SCRO report reveal whether they were 100% certain or if they were 100% confident?

’We have no doubt that the aforementioned photographed impressions contained in the book were made by the person whose fingerprints are shown on the finger and palm print form in the name of Shirley Jane Mckie or Cardwell’

That to me shows 100% certainty. And yet despite ‘certainty’ in their findings they were wrong.

First, I will say that I do not believe that in an LPE conclusion there is any difference merely on account of language between "confidence" and "certainty." It is always, and can only be, a statement of expert belief in what the true source would be, if that true source could be known to a factual certainty. There is, of course, the situation in which an examiner comes to a private belief that the examiner does not belief can be supported within the presently accepted standard for official publication. But the threshold for identification among examiners is generally so conservative that that sort of private belief or suspicion doesn't often appear as an opposing opinion by another examiner.

We find it easy to talk about discrepancy rules in the abstract and in the context of simple illustrative situations, even if we don't find an agreeable consensus of what they mean. But that discussion isn't much helpful in a real case in which the physical relationships among portions of an impression aren't so easy to determine. Without any specific reference to SCRO/McKie, it may be useful to consider a hypothetical case latent with multiple or broken impressions that was reported out as identified but was later declared erroneous, either factually (physical impossibility of that source) or by broad consensus. At that point it is virtually impossible for any subsequent examiner who knows of the error to judge whether all portions of the impression were properly evaluated. Reexamination of such cases is instructive, but I think any examiner who does so and is prepared to state with certainty how they would have evaluated every portion must properly admit that their certainty is to at least some degree suspect, on account of what they would have done having been judged under the unavoidable influence of the strongest imaginable bias, the certain foreknowledge of the correct conclusion. No one should get too much of a nice warm feeling from looking at a known error case and deciding that they could never have made that mistake.

Gerald thanks for taking the time to reply. The real point I was trying to make was as follows. In 1997, prior to any mention of any misidentification, evidence shows that four experts prepared a report, in the full knowledge, that there was not one unexplained discrepancy, but at least 4 or 5 discrepancies that they could not explain.

Instead of disclosing this they reported the two prints as being ‘identical’.

I am trying to discover what experts report these days. If there are known discrepancies that cannot be explained, are they disclosed? If so how? If not, why not? Is the defence made aware of such discrepancies?

I am talking about known discrepancies that cannot be explained. You might suggest that such a scenario might not exist, and experts would be able to explain discrepancies. The reality however is in 1997 such a scenario did exist.

You're assuming that they saw the discrepency before anyone pointed it out to them.

Taking a step back to look at the situation, I find it plausible that they all saw the same 4 or 5 points that looked about right and called it an ID because it was 'just an elimination'. It wasn't until outside experts were called in that they took a "close" look at the print and started explaining things by calling them discrepencies.

To specifically answer your questions about how things work now... Well, that depends on the agency. I would almost guarantee that a 'discrepency' is not mentioned in the report. However, some agencies are starting to write detailed notes on distortion present in the latent print. These notes would be available to the defense on request.

Taggart wrote:I am trying to discover what experts report these days. If there are known discrepancies that cannot be explained, are they disclosed? If so how? If not, why not? Is the defence made aware of such discrepancies?

I suspect the issue is not often directly addressed in policy. When I try to think about how it would be worded, it occurs to me that it would be very difficult to say just what should be formally reported without forcing an agonizingly detailed formal analysis on a great many, perhaps most, examinations. Every latent has "discrepancies," features that don't appear in the record impression, both positive and negative differences. I don't think it's reasonable to say that a discrepancy is "unexplainable" simply because the examiner can't say just what caused it. In other words, I don't think you'd find many rational examiners who treat any difference between impressions as contrary to identification on account of not being able to say exactly how they were formed. I suspect the more common thought process is that it falls into a broad category of artifact or extraneous impression and that it is treated as not a valid skin feature or not part of the contiguous portion being used for identification. I would say, though, that I've never seen and never expect to see an identical latent-record pair. I understand what's meant by "identical," but it's very poor language. And I have never seen and never expert to see an examination report that addresses every difference between the latent and record. No matter how elaborate the report, I can always find some unaddressed difference. Whether that unaddressed difference matters becomes strictly a matter of opinion. It cannot be otherwise.

I have always taken "unexplainable" to mean that I accept the object as a genuine impression of a skin feature that can be related to the other, consistent features as part of the continuous portion being addressed. There are two possible realities. (1) The impressions do not share the same source portion of skin, an exclusion, or (2) they share the same source skin portion and that the feature did not impress in the record or did not impress in the latent. If it was a feature apparently missing from the record, I could probably resolve that with a better record impression. If the feature is missing from the latent, I cannot resolve it that way. At the same time, all discrepancies are not equal. Some frankly destroy the constellation of consistent features, an example being a clear additional ridge that destroys the relationships among the otherwise consistent features. We wouldn't even be talking about that one - it would just become an interesting example of two collections of features being different by one ridge. A single ridge unit appearing within the consistent constellation would call for more judgment and might very likely be arguable as a difference of opinion. An outlying discrepant feature that is taken to be contiguous with the consistent constellation only by treating a rather vaguely impressed area of ridges as true contiguous impression is also arguable, and there might well be different opinions as to whether or not the consistent constellation was sufficient to help make that determination.

I think the bottom line here is that there is no documentation standard capable of settling the issue for all parties and for all purposes. If we imagined that some standard was sufficient for that purpose, it could still not accurately express all the factors that went into the various judgments made during the examination that contributed to the conclusion. I think we seem to be yearning for something that would make any conclusion so reliable that there would be no question of or need for any alternate examination. That's a chimera. That is not the nature of opinion evidence. In fact, that's the nature of all criminal evidence as it relates to guilt. What is sufficient revelation to defense? I don't think that can be answered, except that every tangible thing and recorded circumstance that's available for analysis is revealed. And nothing should be taken to assure adversarial counsel that everything that might be subjected to opposing expertise has been explained. It's always disclosed. It's there for them to see in the evidence.

Hi Ray, I think that how you interperate the output of such things could be considered subjective, though I must say I do not see that because some thing is subjective that is a negative.
Is that short enough.