CLPEX Chat Board

Here is the definition from Biology-Online.org:

a method of testing in which an independent observer records the results of any test, drug, placebo, or procedure without knowing the identity of the samples or what result might be expected.

http://www.biology-online.org/dictionary/Blind_test

Michele T: I will double-check on the defense expert issue. As I recall Cole's article (and my own research in the area), none of the 22 mis-IDs were first caught by defense experts. I'm told, for example, that two defense experts looked at the Cowans materials prior to his trial, but I can't seem to get their names from anyone. Of course, in some of the cases, such as the Vegas mis-ID on the body, there was no defense expert, so the universe of potential cases is smaller.

Charles P. -- As I understand it, double-blind ID methods are used by the Northampton Mass PD (one of their senior detectives was on the TWGEYE group that created the FBI Guide on Eyewitness ID in 1999). I think it is in common use in Mass., N.J., a couple of places in Minn., and at least a couple of places in Cal., but I'd have to double check. Chicago did a recent study (IMHO deeply flawed) and is resisting double-blind and sequential arrays.

To some extent, you don't need double-blind if the test is wholly objective -- if the question is does the reagent turn a color or not or what does the meter read, there's not as much room for bias errors. (Tho it can happen, there's an early study on differences between the folks who had to count blood cells on a slide and the first machine counting devices.)

There's something about this statement that makes me uneasy.

Me too.

When you get to the heart of the matter (look at the two definitions on Charles' post) blind verification boils down to one thing. Confirmation of a conclusion without the influence of bias.

That is the end game, that is the goal. BV doesn't increase the accuracy of the initial conclusion, all it does is support that conclusion (hopefully) independently.

How does that help? Well, anytime you have 2 trained and qualified experts reach the same conclusion independantly your odds of them being correct are dramatically increased. Imagine a scenario where two examiners have an error rate of 1 in every 100,000 comparisons (completely hypothetical numbers). The odds of those two each reaching the wrong conclusion independently is 10,000,000,000. Without blind verification (when you have the possibility of confirmation bias) you can't accurately estimate the chance of error (and I would actually argue that it could cause the error rate of the second examiner to be greater).

Does BV completely eliminate error? Nope, but let me know when you find a model that will. BV doesn't directly increase accuracy, it increases the probability of accuracy.

Now back to the heart of the matter: How do we go about developing a model that will promote confirmation of conclusions without the influence of Bias? And I still say that its easier than we are making it out to be.

Does BV completely eliminate error? Nope, but let me know when you find a model that will. BV doesn't directly increase accuracy, it increases the probability of accuracy.

Well said. Also, I don't believe blinding is a concept foreign to the hard sciences. It may not be used in wholesale fashion, but it is used where needed. And whether it is used to increase accuracy or just the probability of that makes no real difference. Science is about attaining truth and knowledge with the highest degree of accuracy possible. Bias takes away from scientific accuracy. Whether we are studying N rays, canalli on Mars, or an Oregon lawyer's fingerprint on a bag half way around the world, increasing the "probability" of accuracy is a very good thing.

For The Record

TWGEYE is a NIJ funded group that developed guidlines and procedures for the implementation and documentation of "Eyewitness Identification" in it's many different forms. They have published a book (55 pages) that specify the guidelines for showups, lineups and photo arrarys.

In the 55 page book there is no guidelines for "Blind Verification". The only place that even talks about "Blind Verification" in the complete NIJ booklet on Eyewitness Identification is the following quote.

Similarly, investigators’ unintentional cues (e.g., body language, tone
of voice) may negatively impact the reliability of eyewitness evidence.
Psychology researchers have noted that such influences could be avoided
if “blind” identification procedures were employed (i.e., procedures
conducted by investigators who do not know the identity of the actual
suspect). However, blind procedures, which are used in science to
prevent inadvertent contamination of research results, may be impractical
for some jurisdictions to implement. Blind procedures are not included in
the Guide but are identified as a direction for future exploration and field
testing. In the interim, an enhanced awareness on the part of investigators
of the subtle impact they may have on witnesses will result in more
professional identification procedures.

In the guidelines they do go into the importance of not "leading" the witness.

TWGEYE was comprised of 30+ experts in their respective fields.

TWGEYE is a NIJ funded group that developed guidlines and procedures for the implementation and documentation of "Eyewitness Identification" in it's many different forms. They have published a book (55 pages) that specify the guidelines for showups, lineups and photo arrarys.

Correct. The DoJ Guide, published in 1999, aludes to double-blind testing. At the time, there was considerable resistance among parts of TWGEYE to two of the psychologists' recommendations -- sequential testing and double-blind testing. (Check out Doyle's book True Witness for an insider's view of some of these debates; also check out Judges, Two Cheers for the Department of Justice's Eyewitness Evidence: A Guide for Law Enforcement, 53 Ark. L. Rev. 231, 256 (2000); Wells, et al. From the Lab to the Police Station: A Successful Application of Eyewitness Research, 55 Am. Psychol. 581 (2000))

Since the Guide, several departments have tested and implemented both sequential testing and double-blind testing. As I mentioned upthread, there's some controversy in the area based on a study in Chicago last year which seemed to find that (1) non-blind tests were better at identifying the officers' suspect [no surprise] and (2) in one or two of the studied departments no witness picked a distractor/foil when given a non-blind test. Unfortuantely, there was no follow up to see if there was evidence supporting a link between the police's suspect and the offender (DNA, prints, confession, etc.). NACDL is in the middle of a FOIA-like suit to get the raw data from Chicago to see what happened.

Again, there are departments implementing sequential, double-blind ID procedures on a routine basis and the trend seems to be growing, but we'll have to see what the fall-out is from the Chicago study.

Lisa,

I don't know if I'm missing the obvious here but can you please explain what sequential testing means.

Thanks,

For The Record

The TWGEYE Guidlines do incorporate Sequential Lineups as stated in Section V, Subsection C pages 34 and 36.

Section V: Procedures for Eyewitness Identification of Suspects ..... 29
A. Composing Lineups ................................................................. 29
Photo Lineup .......................................................................... 29
Live Lineup ............................................................................ 30
B. Instructing the Witness Prior to Viewing a Lineup................... 31
Photo Lineup .......................................................................... 31
Live Lineup ............................................................................ 32
C. Conducting the Identification Procedure .................................. 33
Simultaneous Photo Lineup ................................................... 33
Sequential Photo Lineup ........................................................ 34
Simultaneous Live Lineup ...................................................... 35
Sequential Live Lineup .......................................................... 36
D. Recording Identification Results .............................................. 38

Michele Triplett wrote:I don't know if I'm missing the obvious here but can you please explain what sequential testing means.

Commonly, when suspects are shown to an eyewitness in a line-up or photo array, they are shown all at once (simultaneously). The witness gets to see the suspect and 5 to 7 other people (foils/distractors). Normally, the administrator of the procedure knows which person is the suspect (non-blind).

The psychologists had been studying this procedure and found a problem called "relative judgment". People looking at simultaneous choices pick the person who looks "most like" their memory of the culprit. If the culprit is present, then he looks most like himself and will be picked by a large majority of the witnesses.

When the culprit is absent however, witnesses are far less likely to pick "none of the above", then to pick the person who looks most like the culprit. (Generally the police suspect, who wouldn't be in the procedure if he didn't bear some resemblance to the description of the culprit.) When the witness sees folks in the line-up or photo array one at a time (simultaneous) and has to make a "yes/no" judgment on each, the studies show they are more likely to not pick the wrong person. (There are also slightly less likely to pick anyone, which was one of the concerns for the TWGEYEE folks.)

Double blind presentation means the person administring the array or line-up has to not know who the suspect is. The problem here is not deliberate hints, but body language, tones, manner of questioning, etc. The TWGEYEE folks had some concerns about staffing issues, particularly for small departments. One of the members (the detective from Northampton, MA) came up with a clever idea for doing arrays where the pictures are placed in a series of folders which are shuffled. The witness looks at each one at a time and indicates yes/no. The administrator can't see which image the witness is looking at and can't react subconsciously. There's also some new methods using computers to show the images where again the administrator can't see which image is being shown.

[quote = "Charles Parker]The TWGEYE Guidlines do incorporate Sequential Lineups as stated in Section V, Subsection C pages 34 and 36. [/quote]

You should be able to get a .pdf of the Guide here...
http://www.ncjrs.org/pdffiles1/nij/178240.pdf

There's also a Training Manual here...
http://www.ncjrs.org/nij/eyewitness/188678.pdf

I'm looking at my hardcopy of the DoJ Guide -- It does describe how to conduct sequential arrays and line-ups. As I recall, it deliberately does not express a preference for simultaneous vs. sequential and doesn't address double-blind, noting only that the administrator should "avoid saying anything to the witness that may influence the witness' selection".

If you want to see more of the debate:
The Wisconsin AG's response to the Chicago study and discussion of double-blind, sequential arrays....

http://www.doj.state.wi.us/dles/tns/ILRptResponse.pdf

For more on this area, try Dr. Wells' home page:
http://www.psychology.iastate.edu/facul ... mepage.htm

Ms Steele, I have those documents or have read through them. I have also spent about a half of a day on Professor Wells site.

My interepretion is different than yours.

Charles Parker wrote:Ms Steele, I have those documents or have read through them. I have also spent about a half of a day on Professor Wells site.

My interepretion is different than yours.

If you'd like, email me off-list and we'll see if we can figure out where we disagree. It has been a while since I read the DoJ Guide itself and may well be conflating it with the commentary and research that has come out since the Guide was published.

My recollection is that double-blind and sequential v. simultaneous presentation were the two contentious issues. I agree that the sequential method is described, but I don't recall the Guide taking a position on whether it is preferable to simultaneous, tho many of the researchers do think it is better. As I recall, the Guide mentions the risks of non-blind presentation on page 9, but doesn't recommend double-blind presentation, tho again, most of the reserachers strongly supported it. (As I recall, there is one major critic of the sequential, double-blind procedure, but most of his criticism is in materials on his website, not in any published, peer-reviewed venue.)

If you get a chance to read it, Doyle's _True Witness_ on the history of eyewitness ID research in the Courts is a lively read and has some discussion of the politicing among TWGEYEE. A librarian should be able to track it down via interlibrary loan.

Wells' view are here:
http://www.psychology.iastate.edu/FACUL ... tation.pdf
Pages 585-86 (5 and 6 of the pdf) talk about the sequential/simultaneous research.
Pages 594-95 (14-15 of the pdf) talk about the shortcomings of the Guide in Wells' view -- the double-blind and sequential presentation issues.

The text is what it is, the question is how to interpret it. If you credit Wells and Doyle, who admittedly are proponents of the research, one concludes that the problem isn't the research, but resistance by prosecutors and some police for other reasons.

For further reading, let me recommend Saul Kassin's page on eyewitness ID, which has many of his papers in .pdf. His 2001 piece on general acceptance among researchers is often cited in this area.

http://www.williams.edu/Psychology/Facu ... itness.htm

His page on false confessons
http://www.williams.edu/Psychology/Facu ... ssions.htm

Is also useful for discussions of how one's memory can be manipulated by suggestion and the risks of accepting a confession at face value.

Malpass' page is also useful...
http://eyewitness.utep.edu/

and has a whole section on sequential v. simultaneous presentation and the Illinois/Chicago study.

Ms Steele

[L.J. Steele Says]If you credit Wells and Doyle, who admittedly are proponents of the research, one concludes that the problem isn't the research, but resistance by prosecutors and some police for other reasons.[/quote]

1. I do not think the researchers are going to find fault with the research. That would be like several leading latent print examiners all of a sudden decide that friction ridge skin is not permanent or unique. It is not going to happen.

2. What do you think the reasons for the resistance by prosecutors and some police were?

Is it because they are playing politics, cannot see the forrest for the trees, or simply just have a different perspectives.

I spent a lot of time on Professor Wells site. Probably too much time. It is an interesting site. From what I can gather it is his personal site that may be partially funded by others with his view. If one wanted more information on a specific point of the much larger eyewitness identification function of criminal investigation then his site would be the one to go to. If one wanted to explore all aspects of eyewitness identification from Showups, Live Lineups, Photo Lineups, Mug Books, and Composite Art, then his site would certainly leave something to be desired.

It is a single site with a single purpose. But it is his site, and he is paying for it, so he can do what he wants with it.

"What is written is not always Accurate and what is Accurate is not always written". ANON.

Take Care Ms Steele, I will be gone for quite a while and will not have the opportunity to contact you or follow through on any more posts.

Continue arguing (logic wise) for what you belive in and I will at some time in the future continue with mine.

1. I do not think the researchers are going to find fault with the research. That would be like several leading latent print examiners all of a sudden decide that friction ridge skin is not permanent or unique. It is not going to happen.

If you look at Kassin's article about general acceptance, you will see that the researchers agree on some of the topics and disagree on others. As far as I know, there's only one major scientific critic of double-blind, sequential ID procedures, Ebbe B. Ebbesen.

http://psy.ucsd.edu/pages/people/faculty/eebbesen.html

Unfortuantely, his key papers in this area are unpublished and available only on his website. In this area, I tend to discount material that hasn't been published in a peer-reviewed journal.

I've heard Lt. (now Det., I think) Ken Patenaude & Sgt. Paul Carroll speak about their experience with TWGEYEEE.
See
http://www.paulbcarroll.com/eyewitnessevidence.htm

My recollection is their interpretation is consistent with Doyle and Wells -- that the problem was more about politics and the potential administrative and fiscal issues involved in changing things than problems with the science. (It's been a long while since I heard them talk, so again, I could be conflating them with having heard alot of talks over the past few years from Wells, Steblay, Penrod, Kassin, etc.) Also, note that the Guide was published in 1999 -- there's been a lot more research in this area in the past 7 years and several field tests in various departments. With the exception of the Chicago study (which has its critics), it seems to back the double-blind, sequential supporters as being practical and seeming to reduce mis-IDs without a significant decrease in valid IDs.

In my own experience with appeals, I've had one appellate prosecutor who took the time to read the studies, think about the issues, and write a thoughtful brief discussing the policy implications of my arguments about the flaws in a particular procedure. I've had three other prosecutors effectively try to argue that the field is all pseudoscience and that the court should ignore the research on a variety of procedural grounds. (That approach hasn't worked for them yet, the Courts seem interested in the topic, even if they don't often rule in my favor.) Thus, I'm inclined to believe that some prosecutors might dig in their heels just because it is the defense bar who's been pressing for changes and because they fear that agreeing to changes might be taken as a concession that the older cases were flawed -- with all sorts of unpleasant implications (they believe) for cases in progress, on appeal, and in habeas, that were processed under the old procedures.

I'm proud to say that in Mass, after a series of high-profile DNA exonerations involving flawed IDs, the prosecutors themselves insisted that the police implement double-blind, sequential IDs, so I think once folks have gotten burned a couple of times, they are more willing to listen to folks proposing solutions.

I don't work much with police directly. I'd like to think that they are interested in getting the right bad guy and willing to listen to well-supported arguments on how they can do it better. I'd agree that in eyewitness ID, as with the double-blind and documentation discussions we've had here, that there practical issues that need to be resolved, but I think New Jersey and other jurisdictions have shown that the new procedures are feasible.

When I'm doing research in this area, I tend to start with Wells' site for the latest stuff, then check out (in no particular order):

Steblay:
http://www.augsburg.edu/psychology/steblay.html

Penrod:
http://web.jjay.cuny.edu/~spenrod/penrod/

and the NACDL's internal eyewitness ID site. Since I've worked on a couple of eyewitness ID appeals, I tend to keep abrest of the major developments in the research, legislation, and litigation.

I will be gone for quite a while and will not have the opportunity to contact you or follow through on any more posts.

I hope you stay well -- I'd be glad to discuss it further here or by email when you've got a chance.

L.J.Steele wrote: I don't work much with police directly. I'd like to think that they are interested in getting the right bad guy and willing to listen to well-supported arguments on how they can do it better.

Take it from someone who has done a substantial amount of work with the police. They do want to get it right. The caveat is in their minds they are always right. Even when they are wrong.

A classic example of this was when I was at a meeting with several homicide detectives, a prosecutor, and two crime lab managers. I was trying to make a point that forensic science is often about opinion and how important it is to allocate adequate resources so people are more apt to come to the "right opinions". I gave the example of the Houston lab blundering some DNA results and sending the wrong guy to prison (he may have even been executed).

Later a private lab exonerated the alleged suspect. With that example I attempted to convince the room of how important the intellectual capacity of the decision makers are to getting the right guy, not just conformity and groupthink. The response from the detectives was that they had trouble accepting the multitude of DNA exonerations. The crime lab managers called the Houston incident isolated.

The police culture is one of conformity and "trusting your gut". That culture is slow to change because even if they get the wrong guy, well there was something about him that was fishy. He had to be guilty of something else even if we made a mistake here (and there's no real proof we got the wrong guy), and on to the next case. There is also intense pressure to clear cases and as some of the experts have stated on this site "hey I just give my opinions, its the jury's who convict".

I wish I had some sort of dividend for every time I've heard "my gut tells me…" Every time I have heard that saying I have wanted to ask if the person thinks they have more nerve endings in their brain or their innards? I doubt it would do much good, because without overwhelming evidence the gut all too often wins.

Thank God for the adversarial system of justice.