Article "Military crime lab evidence tossed..."

[RedFive]

http://www.mcclatchydc.com/2011/05/25/1 ... ossed.html

She testified that no one had detected any problems with her work," Gittins said. "But the judge told the (jury) to disregard her testimony and said it was unreliable.

It sounds like the judge's decision to suppress the examiner's testimony was based more on discrepancies with the examiner's performance on the proficiency test than with the discovery issue.

[Michele]

I agree that a missed identification doesn’t mean that someone isn’t proficient. If an agency agrees that this is true then is it possible for an agency to state that 1 missed identification is acceptable on a proficiency test (and is still considered passing)? An agency could also state that they would remove someone from casework and review prior work if a proficiency test wasn’t passed (where the 1 missed ID would be considered passing). Is this a crazy thought?

[Tazman]

I agree that a missed identification doesn’t mean that someone isn’t proficient.

Is it true that Collaborative Testing Service deemed one of the latent prints on a proficiency test last year to be inappropriate and threw it out post test, based on lack of concensus among those taking the test? I've seen some dozzies on proficiency tests, both internal and external. And what about that one a few years ago that had a 20% erroneous identification rate. If we are going to berate somebody for missing an ident, what should we do when there is a 20% erroneous identification rate? Fire them all?

Just wondering . . .

[Michele]

Was it my triple negative that was confusing? I wasn't berating anyone. I meant that missing one doesn't deem a person incompetent, does it? If it doesn't, why can't that be stated by an agency as the standard for passing (one missed ID is acceptable)?

[Steve Everist]

Is it true that Collaborative Testing Service deemed one of the latent prints on a proficiency test last year to be inappropriate and threw it out post test, based on lack of consensus among those taking the test?

Yes, you can see their report on this test (10-516) at: http://www.ctsforensics.com/assets/news/3016_Web.pdf

Although 71% did answer correctly, 29% did not put an answer (Not Identified) and this was determined not to be a consensus result. I haven't seen it defined anywhere what is and what isn't consensus however.

They do have the comments listed in the report. One of them was "additional standards required". This was definitely not the case regarding this print as there was plenty of corresponding ridge detail between the known and latent that a consensus individualization should have been met. However there were some things about this print that would lead someone to possibly exclude on what may be assumed to be the pattern type. There are also some criticisms about this print and that it may not have been a naturally left print. Specifically see the comments from CN89Y9.

So should someone who erroneously "NI'd" 5D be not considered proficient, even though CTS threw it out? Is 71% not consensus? I would guess all of those who were otherwise proficient in the test, but didn't ID 5D, would have if simply told which finger it came from. It was more than likely a searching error.

[Pat]

So should someone who erroneously "NI'd" 5D be not considered proficient, even though CTS threw it out? Is 71% not consensus? I would guess all of those who were otherwise proficient in the test, but didn't ID 5D, would have if simply told which finger it came from. It was more than likely a searching error.

I missed 5D. I was among the 29% who answered it as "inconclusive."

Although I am a wee bit biased in analyzing my "miss" of this latent, I will adamantly defend my "miss" as NOT a searching error. If anything, it was "over analysis," although I would argue alternatively that those who correctly identified it were guilty of "under-analysis." The latent print 5D was represented in the test instructions as having come from a flat surface. The "A" of ACE-V is Analysis. Under Analysis, we consider a number of factors, including matrix, substrate, development medium, anatomical features, etc.

If the "Substrate" of 5D was a flat surface, then the latent most probably would have been the outside edge of a left thumb. It could have been the left side edge of another finger. My search was based on a combination of substrate analysis for the sides of fingers that might possibly have touched a flat surface.

In reality, 5D was the inside of a right thumb. It was probably lifted from a lab beaker or other such curved surface. It was NOT lifted from a flat surface.

I presented a two year research project into fabricated latent print evidence at the IAI in 1994. One of the most common signs of fabricated evidence is a substrate evident in the lift that is different from the substrate listed on the lift card or in the report of the person developing the print. The shocking thing was that during that talk, I asked how many people in the audience had actually seen fabricated fingerprint evidence -- not merely heard of it, but how many had actually held a fabricated lift submitted to them in an attempt to have the print identified and testified to by them. Out of about 125 people in the room, 25 raised their hands.

That should scare the pants off of any honest latent print examiner. Granted, that was not a scientific survey. But can it be true that 1 out of 5 latent print examiners will receive fabricated evidence in their careers? That question at the conference, supported by personal conversations with hundreds of other examiners during classes I taught, convince me that the likelihood of seeing fabricated evidence is great. If you are in this business for your career, you must be cognizant of the clues of fabrication or you run a very real risk of being caught up unawares in a fabrication scheme.

So my concern was over those examiners who correctly identified 5D without even noticing that it could NOT have come from the surface from which it was purportedly lifted. I have talked to a number of other well respected, senior latent print examiners who also missed 5D for the same reason I did. Substrate analysis is an important part of their analysis, which I believe was the primary reason so many people missed that latent, NOT a "searching error."

And I believe there must have been a big "opps" at CTS when the comments came rolling in (no, comment CN89Y9 was not mine, but I wish I had commented on that latent at the time of the test; I did not.) I believe both the high "miss" rate, combined with CTS's error in labeling the latent print, played a part in the decision to throw out the results on that latent.

As I commented once before on this chat board, I was part of the validation study of the CAS proficiency test a few years ago and, in my opinion, it was a much better designed proficiency test. The latent print images on that test contained several images per "lift." They were all actual size latent prints with background in the lifts, not merely the isolated latents with all background cropped out that CTS puts on their tests. One lift might contain exclusions, prints of no value, inconclusives, and latent prints correctly identified to more than one person, all on the same lift. The CAS test I reviewed did NOT contain any tricks, manipulated latent prints, or mislabeled latent lifts.

But back to the main topic and implication of this thread. The defense in the case that was the subject of the news article claimed that the examiner was incompetent and her results could not be trusted because she had missed an ident on a proficiency test. The judge agreed and threw the fingerprint testimony out. I firmly believe that all of us miss identifications once in a while, whether from over analysis or searching errors or just because it was a bad day. But a missed identification can not reasonably be interpreted to mean that an examiner's identifications are in doubt. Talk about comparing apples and oranges!

[Michele]

Pat,

Let me plan devil's advocate here.

On one hand we are saying that missed ID's aren't a problem.

On the other hand, most agencies have some sort of root cause analysis that is implemented if a missed ID occurs on a proficiency test (removal from casework, review of past work).

Verbally we are saying missed ID's are not a problem but our actions indicate that it could be a problem (otherwise, why would we look into it).

Out of all the personal emails I've recieved on this topic, not one has indicated that they think someone needs to get 100% on a proficiency test in order to be considered proficient. Maybe that shows that the root cause analysis that agencies are using is really a bit excessive.

[kevin]

I think part of the problem with using CTS to gauge proficiency is that CTS is not a good indicator of proficiency. I'm not saying that proficiency tests are a bad thing - just that CTS is doing it wrong. How often have we seen them go overboard with the soft focus, Gaussian blur, or whatever it is they use to make such poor quality photos? It is just passing off substandard quality for difficulty.

How many CTS prints even resemble a casework latent? Or even a livescan or AFIS image? Or a nin print? It is just the same cropped images that look like they went through CS4 a few too many times...

I sometimes look at a CTS print and wonder if I am looking at a ridge ending or some artifact that was pasted in there to appear as minutiae?

[Pat]

Maybe that shows that the root cause analysis that agencies are using is really a bit excessive.

I agree wholeheartedly. The IAI accepts a 20% erroneous inconclusive error rate on the examination for Certified Latent Print Examiner, doesn't it? How many CLPEs actually identify all 15 latent prints on the test correctly? And yet, we treat an erroneous inconclusive in casework as a serious error, and unless you score 100% on a proficiency test, most departments pull you off case work, do a root cause analysis, and issue a corrective action report. And then, in this case, a judge throws out fingerprint identifications based on an inconclusive on a proficiency test.

SWGFAST allows less than 100% correct answers on a proficiency test, provided a department's policies and procedures provide for the lower score:

5 Test Evaluation
5.1 Criteria for successful completion shall be defined in the program plan. Failure to effect all
possible individualizations may be acceptable depending on an agency’s policy, test design, and
test purpose.
5.2 Corrective actions
5.2.1 Corrective action must be taken when an erroneous individualization or exclusion occurs.
5.2.2 Other departures from the expected results must be reviewed for corrective action.
5.2.3 Administrative and clerical errors must be reviewed for corrective action.

I believe the problem with "erroneous exclusions" and "erroneous inconclusives" is that the profession is breaking relatively new ground in dealing with erroneous inconclusives and erroneous exclusions. In our rush to address all "technical errors," many agencies have written policies that treat all errors with the same degree of severity. I believe we would all agree that an erroneous identification should be treated seriously, whether in casework, on a proficiency test, or on the CLPE exam. But I think most of us would agree that an erroneous exclusion is less serious, and an erroneous inconclusive is less serious still. In time, the profession will come to grips with this issue. The sooner, the better.

[RL Tavernaro]

Pat wrote:
In reality, 5D was the inside of a right thumb. It was probably lifted from a lab beaker or other such curved surface. It was NOT lifted from a flat surface.

I have not seen the print in question, however am curious if further info was available about the item/surface the '5D' was lifted from. For instance, could the item have been a relatively small cube, such as a 'kleenex' box, or a plastic photo cube (approx 5 inches)? Technically, each side of the cube is a flat surface, & handling the item would allow for the right thumb to 'wrap-a-round' an edge to make contact to a flat surface with the right edge of the thumb. Or was other info available that would negate that possibility?
Regards, RLT

[Pat]

Hi Bob

As I recall, the scenario on that CTS test was that a desk had been broken into and cash had been taken from a cash box inside one of the drawers. All of the lifts had been taken from the desk and the cash box.

No lift was individually labeled, which would have pretty well negated their use in court anyway had it been a "real" case. But, of course, we are supposed to take a proficiency as if it is a "real" case. If it had been a "real" case, I would have missed the ident, perhaps a tech review would have caught the miss, it would have been sent back with the reviewer's note about the ident he/she had found, I would have reexamined it and claimed the ident, but with mention in my notes and in the report that this lift had most likely come from a curved surface and not a flat one like the desk or cash box (assuming it was the kind of rectangular cash box about the size of a shoe box). But while proficiency tests are supposed to be handled like "real" cases, the truth is that at that agency where you and I used to work, they were not. I got the proficiency back with a suggestion I look at it again, but without benefit of actually being told what the reviewer had found, which would have happened in a real case.

So a double or triple standard is applied. In a real case, a missed identification would be returned with notes on the reviewer's findings for reexamination by the original examiner. The quality assurance policies and procedures would have ensured that only accurate results were reached and reported. In a proficiency test, open communication is not allowed and the quality assurance policies and procedures only partially apply ("root cause analysis" and "corrective actions" are taken, but without the benefit of open communication), even though the entire proficiency is supposed to be handled like "real" casework.

What I think Michele was getting at in some of her posts was that an agency has the option of writing their policies and procedures to allow for such an occurance on a proficiency test. But the problem is that most agencies do not write their policies that way. Then they are stuck having to treat an erroneous inconclusive or an erroneous exclusion with the same severity as they would have to treat an erroneous identification and, in truth, the proficiency is not handled in totality as a real case would be handled.

This is an area of lab procedure that has not fully matured. Until it does, both individual examiners and entire agencies will likely suffer embarrassment over these less serious errors that occasionally happen to all latent print examiners. If proficiency tests are not well thought out or do not truly mirror actual case work, then those errors and embarrassments will be compounded.

[Michele]

From what I’ve seen, a lot of SOP’s seem to be borrowed from other agencies (since so many of them are alike) and the agencies implementing them have no idea why they are implemented. Managers don’t seem to consider the repercussions of the procedures they are implementing. They end up creating procedures that look good but are unrealistic and potentially damaging to practitioners. In the end, it’s the examiners testifying in court and the ones hung out to dry. I don’t think missing an identification on a proficiency test as a problem. In this situation, I think the problem is management implementing harsh procedures and then continuing to support the procedures they implemented. Hopefully agencies can learn from this and consider changing how they handle missed identifications on proficiency tests.

In addition, if missed identifications are considered a potential problem in casework (as in the first case that was in the media) then maybe more agencies should consider starting to verifying exclusions. Letting the practitioner take the fall when the real problem is due to agencies procedures, is not good management.

At our agency, SOP’s are not written by management for employees to follow. They are written by employees for employees. Management signs off on them showing that the procedures are something that everyone can live with.

Regarding SOP’s, does anyone know why so many agencies SOP’s state they won’t review another agencies work? If the defense wanted my work reviewed, I’d rather have it reviewed by someone that I knew was competent. One person told me that it was written so that it didn’t open the flood gates of having to review so much work since nobody has the staff for that. I don’t buy that answer. SOP’s are written for employees, not to tell attorney’s what we will and won’t do for them. I’ve also heard that agencies don’t want to get in the middle of problem cases. I can see their point, but if there is an error in a case, wouldn’t you rather figure it out sooner than later? Isn’t it our jobs as experts to give our conclusions? I’d think it would be even more important in these problem cases. Anyone know where this procedure started?

[Graham F]

Regarding SOP’s, does anyone know why so many agencies SOP’s state they won’t review another agencies work? If the defense wasted my work reviewed, I’d rather have it reviewed by someone that I knew was competent. One person told me that it was written so that it didn’t open the flood gates of having to review so much work since nobody has the staff for that.

From the thread - What is the threshold for inevitability?
Show me an examiner who is out of training and doing independent casework who says they've never missed one and I will show you a person who is either a fool or a liar

Do I detect a subtle undercurrent of bureau/PD latent examiners questioning the abilities of examiners undertaking independent casework?

Is there an inference that bureau/PD latent examiners are better trained, have more experience, are better skilled and are more competent than examiners undertaking independent casework?

[Tazman]

Do I detect a subtle undercurrent of bureau/PD latent examiners questioning the abilities of examiners undertaking independent casework?

If you inferred a bias against independent examiners, rest assured that none was implied. Let me explain. In the police agency where I work, and I assume in most others, a trainee goes through basic training, then begins a period of doing casework under the watchful eye of a senior examiner. That is called a period of "supervised casework." After the new person demonstrates competence, usually six months to a year, the new examiner is allowed to work cases without constant oversight by the senior examiner and is said to be doing "independent casework." The identifications and exclusions are still verified, and another examiner will do "technical review," but the new examiner is free to make decisions, do comparisons, and claim identifications without consulting the senior examiner who trained him/her. It is in that sense that the new examiner is said to have become "independent."

My point was that anybody who does this work overlooks, or "misses," the occasional identification that he/she should have made. It has nothing to do with whether you work for a police department or you are a private consultant.

[Michele]

Update about the Army Crime Lab Issues:

http://www.mcclatchydc.com/2011/06/26/v ... crime.html

Posted on Sun, Jun. 26, 2011

Beyond missteps, military crime lab roils with discontent
Marisa Taylor | McClatchy Newspapers
last updated: June 24, 2011 06:48:46 PM

WASHINGTON — The military's premier crime lab should be a place of sober scientific research, but lately it seems more like the set of a soap opera consumed with scandal and intrigue.


In less than four years, at least six internal investigations have been launched and six complaints filed against managers. The accusations and counter-accusations include racism, sexual harassment, assault and fraud.


The disputes have embroiled top managers and pitted them against one another. The lab's former lawyer says she was retaliated against for blowing the whistle. The military counters that she made off with official records.


Amid the upheaval and finger-pointing, a lab analyst was convicted of embezzling almost $70,000 from a professional association to pay for his gambling addiction.


"The place is a rat's nest," said Mike Jellison, a former firearms examiner who worked at the lab for 14 years. "It's not conducive to science."


Interviews and thousands of pages of court and military documents that McClatchy obtained reveal a litany of concerns about the lab where analysts handle evidence from all the military branches. Each year, about 3,000 criminal cases are processed at the facility called the U.S. Army Criminal Investigation Laboratory.

Acrimony and backbiting permeate the lab, military officials have found. Employees accuse the lab of protecting bad managers and ignoring serious complaints such as conflict of interest and waste. Prompted by the swirling allegations, the military ordered sensitivity training for lab officials and conducted an employee survey to assess conditions.

"There are perceptions that managers are biased for a variety of reasons," Army Col. Eric Belcher concluded after one inquiry in 2009 that described a brewing problem with "extremely bad relationships between managers."

Military officials, however, continue to back the lab's top officials, including its longtime director.

The Army's Criminal Investigation Command, which oversees the lab, described it as "an exceptional organization, staffed by talented and committed scientists and managers."

In a statement, the CID said it couldn't legally discuss the employee claims: "In short, individuals may bring their side of the story to the media but CID cannot respond."


However, the statement said it takes "all credible allegations of wrongdoing and misconduct very seriously and continue(s) to take appropriate actions when and where warranted."

Last week, command officials sent an investigator to determine whether missing records that detailed the misconduct and forensic mistakes at the lab had been destroyed or stolen. The Army realized that the documents had disappeared from the lab when it began responding to McClatchy's questions about two discredited analysts.

Separately, the Defense Department's inspector general confirmed last week that he was conducting an inquiry into the lab's handling of one of the analysts' misconduct at the request of Sens. Charles Grassley, R- Iowa, and Patrick Leahy, D-Vt.

Started in 1943 by two soldiers at the Allied front in North Africa, the lab now has 180 employees in Fort Gillem, Ga.


Evidence in the military's highest-profile investigations has been analyzed there — from the friendly fire death in Afghanistan of former NFL player Pat Tillman to the mass shooting at Fort Hood, Texas.


Five years ago, however, misconduct by two analysts tarnished the lab's reputation.


A McClatchy investigation revealed in March that one of the analysts, Phillip Mills, was found to have falsified a report, prompting a three-year, $1.4 million retesting of his cases. The lab concluded that he'd made dozens of mistakes, often when testing evidence in rape cases.


Making matters worse, the crime lab didn't always inform defense attorneys about mistakes, including evidence testing that wrongfully convicted defendants.


Since then, the accusations of impropriety have only multiplied, court and military documents show.


At one point, misconduct by an employee prompted an FBI search of one of the lab's offices. The investigation resulted in the arrest of Allen Southmayd, a 63-year-old handwriting expert.


Southmayd had a serious gambling problem, court records indicate. After he joined the lab in 2000, he began to write checks to himself from a professional organization where he served as treasurer. He spent the money from the American Board of Forensic Document Examiners at casinos. In 2007, the group caught on and Southmayd resigned from the lab. He pleaded guilty to embezzling in federal court and was sentenced to probation.




To some, the morale problems began when the Army tapped chemist Larry Chelko in 1993 to be the lab's first civilian director and began replacing most of its military examiners with civilians. Although many of the lab's first civilian hires were former soldiers and Army investigators, many of the younger hires have no military background.


"In the military, the mindset is different," said Jellison, a former Army warrant officer. "It's mission first. The military does what the military tells it to do and things run smoothly."


"Now it's a military lab, but it's run by civilians," said Jellison, who recently decided against returning to the lab as a civilian. "I loved working there at one point but the people in management don't want any ex-military people there, period."


John Cayton, a retired Army investigator who ended up taking the job Jellison turned down, said he noticed hostility not only toward retired soldiers but also older workers. He resigned within months.


"I didn't feel welcomed," he said.

Lab officials have denied trying to weed out employees with military backgrounds.

"The focus is to hire qualified folks, and because they're in the military doesn't mean they're qualified," the lab's chief of the forensic analysis division, Richard Tontarski Jr., testified in an inquiry.



Other divisions have emerged.


After a black temporary employee, A.D. Bell, was passed over for a permanent position, the lab's lawyer, Lisa Kreeger, testified in May 2010 that she'd overheard a manager make a racist remark about him.


Donald Mikko, the chief of the firearms branch, backed Kreeger, alleging that his boss resisted hiring Bell because he's black.


Lab officials have denied the allegations, saying Kreeger misheard the remark. They point out that when the director confronted Kreeger about her claim, she conceded that she might have misheard it.


But a Defense Department investigator found merit in the discrimination claim, concluding, "Management has not articulated a legitimate nondiscriminatory reason for the action contested."

Kreeger and Mikko have filed their own complaints, alleging that officials retaliated against them.


Fueling the racism charges, other African-American employees raised concerns about several comments by whites, from an examiner telling a black IT worker in an email that he had "banana breath" to insinuations that the black employees were less educated, the lab's director testified in Bell's discrimination lawsuit.

Belcher, the investigator who raised concerns about lab management, pointed out that only 9 percent of the "technical side" of the lab was black.

"A concerted effort should be made by the lab to increase diversity," he wrote.


Some former and current employees say officials have made matters worse by obsessing over damage control and punishing employees who raise concerns. After a recent series of articles by McClatchy chronicled the mistakes and misconduct by two analysts, the lab held a meeting in June to "set the record straight" and, according to several people familiar with what was said, implied that discussing the lab's problems with the news media could violate military conduct codes. The sources asked to remain anonymous for that reason.


Recent hires have only worsened morale, the employees said.

W. Mark Dale, a former director of the New York State Police laboratory system, was hired to oversee training despite a recommendation by New York's inspector general in 2007 that he be criminally investigated for a scandal there. He was accused of keeping secret misconduct at the lab and mistakes by a discredited analyst. He was never prosecuted.

The managers, who've been accused of stoking the resentment at the Army's lab, say they're being unfairly attacked themselves by disgruntled employees who are unable to keep up with modern science and high standards


Col. Martin Rowe, the chief of the lab's expeditionary forensics division, who's observed the disputes since he joined in 2009, testified that he'd noticed "half-truths circulating" and a "general lack of communications." He dismissed the tensions as due to "growing pains."


But the conflicts don't appear to be resolved.

Managers and employees argue for hours in closed-doors meetings. Adversaries exchange countless accusatory emails. At times, confrontations spill into the lab's hallways.

One female lab technician became upset when a male examiner put a lab brush in her lunch pail. An employee she'd accused of sexual harassment years before had used the brush. Ordered to apologize, the examiner appeared to grab her. Investigators concluded that it constituted assault. The examiner resigned.


Some employees have claimed that the tense environment has made them ill. Mikko testified in one ongoing lawsuit that another midlevel manager and war veteran appeared to have developed post-traumatic stress disorder "severe enough a 59-year-old man who has served God and country and his organization for years (was) literally in tears and shaking."