NYT Opinion: Sessions Is Wrong to Take Science Out of Forensic Science

[Boyd Baumgartner]

https://www.nytimes.com/2017/04/11/opin ... .html?_r=0

Prosecutors applauded the April 10 announcement by Attorney General Jeff Sessions that the Department of Justice was disbanding the nonpartisan National Commission on Forensic Science and returning forensic science to law enforcement control. In the same statement, Mr. Sessions suspended the department’s review of closed cases for inaccurate or unsupported statements by forensic analysts, which regularly occur in fields as diverse as firearm and handwriting identification, and hair, fiber, shoe, bite mark and tire tread matching, and even fingerprinting analysis.

If all you knew about forensic science was what you saw on television, you might shrug off this news, believing that only the most sophisticated and well-researched scientific evidence is used to solve and prove crimes. But reality is different.

D.N.A.-exoneration cases have exposed deep flaws in the criminal justice system’s use of forensic science. Reforms have not come easy, but slow and plodding progress has been made. In 2005, the F.B.I. said that it would no longer conduct bullet-lead examinations after a review panel found matches essentially meaningless. A blue-ribbon panel of the National Academy of Sciences raised the same concern in a 2009 report that found nearly every familiar staple of forensic science scientifically unsound.

Prompted in part by that report, the Justice Department initiated a review of thousands of cases involving microscopic matching of hair samples. In 2015, the F.B.I. announced its shocking initial findings: In 96 percent of cases, analysts gave erroneous testimony. At a meeting last spring of the commission that Mr. Sessions just disbanded, the department said it would expand the view to include a wider array of forensic disciplines.

With the announcement by Mr. Sessions, this momentum comes to a screeching halt. Although forensic science would seem a low priority for an incoming attorney general, it is not altogether surprising that it was in Mr. Sessions’s sights. As a senator (and former prosecutor), Mr. Sessions made forensic science a priority. He sponsored and shepherded to passage the Paul Coverdell National Forensic Science Improvement Act of 2000, which remains the signature federal funding mechanism for state all-purpose forensic labs. That might suggest that Mr. Sessions would care about the integrity of forensic science, but his enthusiasm has been for more — not better — forensic evidence. When the National Academy of Sciences’ scathing report was released, Senator Sessions simply waved it away, remarking, “I don’t think we should suggest that those proven scientific principles that we’ve been using for decades are somehow uncertain” — ignoring the panel of experts who had concluded just that.
Continue reading the main story

Every independent critique of our forensic science system comes back to the same basic conclusion about both the root of the problem and how to fix it: Forensic science rests under the exclusive control of police and prosecutors, and its legitimacy and integrity have suffered as a result. Even Obama-era law enforcement officials had a tenuous relationship to reform. Just last year, the President’s Council of Advisors on Science and Technology issued a report that reaffirmed and extended the 2009 findings. The F.B.I.’s response was to vehemently disagree, the Obama Department of Justice basically replied, “Thanks, no thanks,” and the professional association for the nation’s district attorneys criticized the report for its insufficient attention to “the ancient debate over precisely what constitutes ‘science’ ” while asserting that the final arbiter of good science should be lawyers and courtrooms, not scientists and laboratories.

The 2009 report concluded that the only way to ensure effective oversight of forensic evidence was to protect its independence from law enforcement. But its recommendation of a national, independent oversight agency was met with intense resistance from federal and state law enforcement. Instead, the national commission was formed as a compromise solution that brought in the National Institute on Standards and Technology as co-chairs and stewards of scientific values. Some forensic scientists — at times grudging partners in the process of reform — even came to embrace greater professionalization, and one major professional organization recently declined to support the Justice Department’s proposal to move forensics in-house.

The loss of an even partially independent national commission is no trivial matter.
In its brief two years of existence, it drafted 43 standards that actually changed forensic science, on the ground, for the better. The commission’s guidance covered issues like certification requirements for forensic examiners (who, unlike your local manicurist or food server, typically must not pass any basic competency exams still), discovery rules (providing criminal defendants with at least some of what parties receive in civil cases) and reporting standards (discouraging the use of the popular phrase “reasonable degree of scientific certainty” as it has “no scientific meaning and may mislead fact-finders”). The National Commission on Forensic Science was even poised to issue a raft of best practices for the wild west of digital forensics, which has exploded without supervision over the years. It seemed that a promising new era of accuracy, transparency and accountability in forensic science had dawned.

There are jurisdictions and even prosecutors committed to meaningful forensic reform, but none with the resources, expertise and authority of the commission. And so long as forensic science is under Department of Justice control, reformers will be cut off from the primary purse for scientific research and setting best practices.

We know what happens when prosecutors and police officers control forensic science, instead of scientists. We have already lived through an embarrassing parade of wrongful conviction, tragic incompetence, laboratory scandal and absurdly unsupported forensic findings. We have commissioned the studies, read the reports. They brought us to the place we are now, at the cusp of something better. Sadly, with the flick of one prosecutor’s self-interested pen, that vision is now gone.

Erin E. Murphy, a professor at New York University School of Law, is the author of “Inside the Cell: The Dark Side of Forensic D.N.A.”

[Dr. Borracho]

(lengthy editorial critical of all forensic science), by:
Erin E. Murphy, a professor at New York University School of Law, is the author of “Inside the Cell: The Dark Side of Forensic D.N.A.”

An anti-forensic science editorial printed in a liberal newspaper, written by a liberal university law professor whose previous editorial criticized the "Dark Side of Forensic D.N.A." -- Now there's a good unbiased opinion on the reliability of forensic science in general!

[Bill Schade]

Instead of questioning the writer of the article, we should be asking ourselves if the article contains any valid points. It is up to the reader to determine whether an editorial or article is fair and unbiased.

Otherwise any thing written and published by a practitioner could be dismised as biased.

Science doesn't base value on who said something, it measures the strength of the argument.

Shouldn't that be the goal of Forensic Science?

[Ernie Hamm]

The following is from an Editorial by Dr. Roger J. Davis in the Journal of the Forensic Science Society; 1991; 31(4):407, titled “Forensic science on the quality track”. Dr. Davis’ closing statement makes a very valid argument on what is important in the goals of forensic science, as follows:

“Scientists need to be concerned with quality, no more so than in that peculiar hotch-potch of analytical sciences that has come to be known as forensic science. Few would dispute that our citizens have the right to expect uniformly high quality from a public service. Fine, but an individual will generate true quality from within (Pirsig, Robert M., Zen and the art of motorcycle maintenance, London: Corgi Books, 1976), not as result of external pressure to conform to “quality” standards. A conscious fostering of professional pride, commitment, experience and enquiring minds must go hand-in-hand with formal accreditation. Otherwise, quality in forensic science will be like a veneer, having more to do with keeping up appearances than the pursuit of excellence.”

Bold added for emphasis.

Back to the tar pit.

[Alan C]

(lengthy editorial critical of all forensic science), by:
Erin E. Murphy, a professor at New York University School of Law, is the author of “Inside the Cell: The Dark Side of Forensic D.N.A.”

An anti-forensic science editorial printed in a liberal newspaper, written by a liberal university law professor whose previous editorial criticized the "Dark Side of Forensic D.N.A." -- Now there's a good unbiased opinion on the reliability of forensic science in general!

Rather than just throwing around the word "liberal" as if that allows us to dismiss the author's arguments out of hand, how about actually addressing what she says? And I acknowledge that at the very least she makes some overstatements (e.g., that the NAS report "found nearly every familiar staple of forensic science scientifically unsound"). And by the way, this is clearly labeled an opinion piece so the author is under no obligation to be "unbiased."

[josher89]

I think what's interesting here is that there are all of these lettered organizations, and each one answers to a different segment of the federal government. The NAS/NRC does its thing for Congress, the now-defunct NCFS does its thing for the Attorney General, PCAST does its thing for the POTUS. Like typical government, it seems there's a lot of waste going on. OSAC is doing its thing under NIST, which is at least not directly tied into the executive or legislative side of things.

Everyone is entitled to their opinion so here's mine.

I think what has happened post-NAS report has been good. It's generated a ton of research (some better than others) but most importantly, discussion. Just because someone is a critic of us doesn't mean we just dismiss them. We should be at least interested in what they have to say regardless of whether or not it's relevant. These critics may wind up being judges or jurors one day so we need to be aware of their concerns and be prepared to educate them (sorta what Bill said). If we can't educate them (outside of their obstinance), then perhaps we need to rethink what we are doing and saying. For a long time, people swore the earth was flat or that spontaneous generation was a thing; once they were educated, they realized that wasn't true. If we can't prove what we are doing is valid, then we probably shouldn't be doing it--or at least not calling it science. At a minimum, we should be prepared to discuss the shortcomings of what we do and embrace a better way to do it.

There are some critics that just have a hand out; waiting for their payday to say something ridiculous. There are others that are true scientists that can add value to forensics in a way that isn't self-serving. Those are the people that we should be working with.

[Boyd Baumgartner]

I think what's interesting here is that there are all of these lettered organizations, and each one answers to a different segment of the federal government.

What's the old adage? A camel is a horse built by committee. I think we see that here. We've got several different committees wrestling with the same question. The lingering problem is and will be: acceptance of these suggestions by various interested parties as they are not necessarily best practice, even outlined by their own members, they lack enforcement, they lack financial resources to further them, and they're potentially over the head of the very people who need to present them. These are some major hurdles in my estimation.

I think what has happened post-NAS report has been good. It's generated a ton of research (some better than others) but most importantly, discussion.

I would tend to agree, the shortcomings however are that good science doesn't always equate to good policy. James Gates, the gentlemen referenced in that link spoke at the NCFS meeting where he was critical of the forensic science community as being inflexible. These are contentious topics for some people as was evidenced by the argument that broke out between Jules Epstein and William J Fitzpatrick.

While it's not posted yet, in the latest NFSC Webcast there were a few highlights. One was Elham Tabassi's lunchtime presentation on day one where she outlines devising, testing and validating likelihood ratios for Iris Scans. It was a great talk that outlines some of the fights that will have to be had such as which philosophical school of statistics will win out (objective vs subjective) and other pretty nerdy topics which we are knee deep in.

Overall, I tend to agree that the forensic science community tends to be a tad inbred when it comes to research, policy and recommendation and benefits greatly from outside influence from more established science. As a discipline we have tended to produce untenable statements, and positions and attitudes that have been at the center of critique and outside the norm of mainstream science. While we're coming out of it to some degree, obviously there's more work to be done.

[Dr. Borracho]

James Gates, the gentlemen referenced in that link spoke at the NCFS meeting where he was critical of the forensic science community as being inflexible.

I was flattered recently when Jim Gates was touring my lab and gave me the opportunity of visiting with him for a half hour or so. Dr. Gates is one of the less inflexible non-forensic scientists on the PCAST. Jim and I talked about the philosophies of Karl Popper, Carl Hempel, and Thomas Kuhn. He was not unreceptive to my position that applied sciences, including the various disciplines of forensic science, should not be viewed through the same paradigm as the theoretical science for which he is famous. He did not object to me applying the metaphor of hammering a square peg into a round hole when I talked about the efforts by lawyers and academicians to enforce mathematical standards in fields that are not well suited to such applications. At the end of our visit, Dr. Gates complimented me in front of the laboratory director for my grasp of science.

Boyd references Dr. Gates' comment on the inflexibility of some in our field. I would suggest that there is just as much inflexibility on the side of many of our critics.

My comment earlier in this thread was not meant to impugn liberals in general (I weigh in at the "bleeding heart" end of the liberal/conservative spectrum myself). Those such as Erin Murphy (whose editorial was quoted by Boyd to open this discussion) would seemingly "throw the baby out with the bathwater" and banish forensics, as currently practiced, from the courtroom altogether.

Errors do occur in forensics. Efforts should be made to diminish the rate of such errors. But overall, our "error rate" (if by that one means identifying the wrong person) is exceedingly low. I would speculate that fewer such errors are made in forensics than are made on the blackboards of theoretical physicists in their ivy-covered towers where they practice the statistical applications they criticize us for not using. I would further suggest that forensic scientists make far fewer errors than lawyers.

Lawyers and law professors who hold forth as experts on science and who demand that forensic science be free of errors are not trying to improve our practice, regardless of the motives they profess.

Assuming that there is only one definition of "science" for all fields is fallacious. Applied sciences and descriptive sciences are different from theoretical physics and exact sciences. Lawyers who present themselves as knowing science better than those of us who practice it on a daily basis are not helpful to either science or to justice. They are a hindrance to both.

[NRivera]

IMO this article makes a mountain out of a molehill. The Comission's own charter states very clearly that "The objectives and scope of activities of the Commission are to provide recommendations and advice to the Department of Justice (DOJ) concerning national methods and strategies for:..." blah blah blah.

https://www.nytimes.com/2017/04/11/opin ... .html?_r=0

The loss of an even partially independent national commission is no trivial matter.
In its brief two years of existence, it drafted 43 standards that actually changed forensic science, on the ground, for the better. The commission’s guidance covered issues like certification requirements for forensic examiners (who, unlike your local manicurist or food server, typically must not pass any basic competency exams still), discovery rules (providing criminal defendants with at least some of what parties receive in civil cases) and reporting standards (discouraging the use of the popular phrase “reasonable degree of scientific certainty” as it has “no scientific meaning and may mislead fact-finders”). The National Commission on Forensic Science was even poised to issue a raft of best practices for the wild west of digital forensics, which has exploded without supervision over the years. It seemed that a promising new era of accuracy, transparency and accountability in forensic science had dawned.

The NCFS had zero authority to issue "standards". It was never a Standards Development Organization or anything remotely close. What it could do was make policy recommendations based on research and best practices. The AG or a designee still had to come up with meaningful and enforceable policy which, aside from the mandate for AUSA's to use only accredited forensic service providers, never really happened to my knowledge. Furthermore, the DOJ can only dictate policy to federal courts. State and local courts usually follow but that is a voluntary undertaking, oftentimes tied to federal funding. What I understand this to mean is that the new AG simply doesn't want a commission to "advise and recommend" but would rather have some internal person take on that role. Can that individual, whomever it may be, do it better than the 40+ members of the NCFS? We'll soon find out.