CLPEX Chat Board

Good morning!
We are trying to approve Photoshop for use in a (county) latent print lab, and we are running into issues with our QA manager stating that we must complete a validation and competency on Photoshop before we can use it on casework. I’ve also worked for another (state) agency in the past and used Photoshop for years while there was no validation or evne performance checks completed, so I’m confused on why it’s necessary to do it no. In addition, if we do have to do it, how we’d even go about doing that. To make things even more interesting, our crime scene unit uses photoshop to enhance latent images prior to submitting them to us, however they do not do validations as supposedly they were “grandfathered” in without needing that…

Despite ISO 17025:2005 5.4.7.2 Note mentioning that commercial software used for the purpose it was designed may be considered to be "sufficiently validated", I think the disconnect is because someone is considering photoshop as a “procedure” and/or “method,” which I disagree with. I’m just trying to understand where our QM is coming from and quite honestly, to determine if it’s really necessary to validate photoshop since no other lab I’ve spoken with has done anything like this before.

- Do you use photoshop for latents? Have you been using it for a while, or is it something newer that you've started?
- Was it validated prior to use?
- Any other disciplines use it in your lab (crime scene, etc)?
- Suggestions on how to convince others that photoshop is not a procedure or method?

Thanks!

Adobe itself touches on this very issue in this document which I think was written by George Reis who posts here from time to time. You can find his contact information by going to his site: http://www.imagingforensics.com/contact/

The document says in part:

When a technology is challenged in court, a Kelly-Frye hearing or a Daubert hearing may be called to determine if the technology is valid. Digital imaging technology has gone through three such hearings since 1991. In his paper 'About Forensic Digital Imaging' Erik Berg describes a significant case that occurred in 1995 (State of Washington v. Eric Hayden). Berg states, 'State of Washington vs. Eric Hayden serves as an affirmation of the conclusion reached in the Commonwealth of Virginia vs. Robert Douglas Knight case. It also imposes the same requirements for digital images as those placed upon other types of evidence. ...Any enhancement techniques must be reproducible, so that notes about the enhancement process, as well as who did the work should be maintained

So, to your point specifically, in addition to meeting the ISO definition of validation, Photoshop has been legally validated as well.

Commercial, off-the-shelf software, shall be considered fully validated. I'm sure Ski could chime in here but I think your QAM is reading into the standard a bit more than necessary.

I think someone should definitely have training in the use of PS before being turned loose on using it for casework. My argument would be if we're using Word to type a report, we should probably have that validated as well. Let's make sure that what I type is actually being rendered on screen and printed on paper. That was obviously facetious but that's my point.

I would urge them to look at the new 17025: 2017 for additional guidance on the requirements for software. 7.11.2 says,

Whenever there are any changes, including laboratory software configuration or modifications to commercial off-the-shelf software, they shall be authorized, documented and validated before implementation.

Further (same section):

NOTE 2 Commercial off-the-shelf software in general use within its designed application range can be considered to be sufficiently validated.

I don't know that there needs to be much more to it than that. If you have proper training in image enhancement, and as long as you are doing what you should be doing, you're good. How do you prove that to anyone (court, peer, QAM)? Look at the History Log and/or metadata and they can recreate the enhancement.

Here's my recommendation:

First, ask your QM which ANAB requirement is being referenced.
Second, ask your QM if any latent print unit anywhere is being asked to do this by ANAB.
Third, ask your QM to contact ANAB for clarification. It makes sense that things be checked before lots of unnecessary work be done. And a high-up employee of ANAB should be contacted. Not just some volunteer that goes on an assessment once a year and maybe in some other discipline.

If you listen to the podcast interview with Anja Einseln, she's pretty clear that ANAB isn't telling agencies HOW to do things, but just making sure that the basics are met in some way (agency's choice).

Thanks for sharing!

Josher89, I agree that QAM is looking deeper into things than what I was expecting, and I agree that the Note mentioning commercial off-the-shelf software seems pretty self explanatory to me ... however, the issue is that they seem to think using photoshop for latents is a METHOD/PROCEDURE. Based on that tidbit of info, it's causing issues within our own quality manual ISO 17025:2005 (we're still moving toward 2017) as well as a state regulation. I do have Ski's contact info and will be reaching out to him early next week.

Thanks for the legal perspective, Boyd! I'll look into that a bit more.

Although this is long and clause-heavy, it explains a bit more of where i'm coming from from an 17025 perspective ( even though I realize you guys are not familiar with my state regulations)



"A validation study prior to bringing Photoshop online for use by the LPU in casework IS required by this Laboratory as stated in our quality manual:
[Our laboratory's] Quality Manual states: “The Laboratory validates all new PROCEDURES prior to their use on casework.”

Also of note, both ISO/IEC 17025:2005 and ISO/IEC 17025:2017 require labs to confirm that they can properly operate testing METHODS they employ before using the method on casework – as per the following:
--- ISO/IEC 17025:2005, Clause 5.4.2: “…The laboratory shall confirm that it can properly operate standard METHODS before introducing the test.”
--- ISO/IEC 17025:2005, Clause 5.4.5.1: “Validation is the confirmation by examination and the provision of objective evidence that the particular requirements for a specific intended use are fulfilled.”
--- ISO/IEC 17025:2005, Clause 5.4.5.2: “…The validation shall be as extensive as is necessary to meet the needs of the given application or field of application. The laboratory shall record the results obtained, the procedure used for the validation, and a statement as to whether the METHOD is fit for the intended use.”
--- ISO/IEC 17025:2005, Clause 5.4.5.3, NOTE 3: “Validation is always a balance between costs, risks and technical possibilities.”
--- ISO/IEC 17025:2017, Clause 7.2.1.5: “The laboratory shall verify that it can properly perform METHODS before introducing them by ensuring it can achieve the required performance. Records of the verification shall be retained. If the METHOD is revised by the issuing body, verification shall be repeated to the extent necessary.”

Further, [State regulations] describes internal validation as a review of accumulated test data within the laboratory to demonstrate that an established method or procedure performs as expected in the laboratory and to specify laboratory interpretation guidelines. [State Reg] goes on to state that “The licensee shall separately perform a validation or conduct a performance check on standard methods before use.” This concept is again stated in “All new procedures and modifications of current procedures are validated and documented before being used in case work.” There do not appear to be any software associated exceptions in [State Reg].

I have confirmed in a discussion with [lab director]that our Laboratory considers validation of NEW METHODS prior to their use on casework a significant tool in our kit for use in ensuring the quality of results and protecting the interests of the Department, the Laboratory, and our customers. We currently do not see anything in our read of the current accreditation requirements, nor in the collective 2017 requirements (both ISO and ANAB), that would cause us to modify the Laboratory’s approach, though this topic will be evaluated as we continue our transition to the 2017 requirements."

Wow. If you QAM spent THAT much time writing a response to why a validation is required and seems THAT convinced that Photoshop is a method and a procedure, then you're just done. Unless you can get another QAM who yours respects to explain how Photoshop isn't a METHOD, I'd say to wave the white flag, follow the path of least resistance, and ask your QAM to help you design the validation study.

When someone puts that many references into their decision, they're not in a state to listen to or consider the other side.

Here's your next line to your QAM, "Would you be able to help me design the validation study? I don't want to miss anything."

In our lab we have to validate our photoshop yearly. We put a cd in and open up, what basically is a color chart, and have to adjust brightness and contrast. Change to CMYK to make sure it changes in the program and do a couple other basic things.

We also validate our scanners and our camera lenses as well.

ekuadam wrote: ↑Sun May 20, 2018 12:21 pm In our lab we have to validate our photoshop yearly. We put a cd in and open up, what basically is a color chart, and have to adjust brightness and contrast. Change to CMYK to make sure it changes in the program and do a couple other basic things.

We also validate our scanners and our camera lenses as well.

Wow - Congratulations! You're actually the very first person that I've talked to that has had to validate photoshop :-p
Would you mind sharing exactly what you do to validate in your lab? I can message you my email address if you are able to share attachments or worksheets, etc.

When I was at my previous agency, we DID do a validate/performance check (i forget what we actually called it) on our scanners and cameras annually as instructed in the SWGIT guidelines.

Please message me your address and I will get with my manager to make sure I can send you the worksheet that has directions on what we do to validate our photoshop.

ER wrote: ↑Fri May 18, 2018 1:40 pm When someone puts that many references into their decision, they're not in a state to listen to or consider the other side.

Here's your next line to your QAM, "Would you be able to help me design the validation study? I don't want to miss anything."

Right?! yes, we thought it was a bit excessive and were under that same impression regarding not being willing to consider another side. I just still dont understand how it's considered a procedure/method though. I'm not sure that our QAM understands to what degree we're using PS anyway ... in the same email quoted above, it was concluded with:

"We look forward to receiving your validation plan for review once it is available.
Thank you for all of your hard work in connection with this validation project!"

The application of enhancing isn't the same for every image, and there's no way we can validate every single thing we MIGHT use in PS to prepare for that - it's just doesn't logical (OR necessary based on the ISO clause saying commercially purchased software can be considered validated). Part of the issue is that we're unable to find written definitions of Method and Procedure in our documents, so I'm assuming if we cannot disprove that it's NOT a method or procedure then we definitely have zero argument.

ekuadam wrote: ↑Mon May 21, 2018 9:34 am Please message me your address and I will get with my manager to make sure I can send you the worksheet that has directions on what we do to validate our photoshop.

sent PM. Thanks!

Heather wrote: ↑Tue May 29, 2018 5:53 am

ekuadam wrote: ↑Mon May 21, 2018 9:34 am Please message me your address and I will get with my manager to make sure I can send you the worksheet that has directions on what we do to validate our photoshop.

sent PM. Thanks!

Sent them to you a few days ago. Hope you got them. If not, let me know.

We have been accredited since 2010 and we have not had to validate our DCS systems. It's never been an issue in court nor in assessments or surveillance visits.

While I have already provided my response to Heather, I decided to respond to the question here as well.

As stated in the introduction of ISO/IEC 25051:2005, this standard defines quality requirements for COTS software products. Software applications such as Microsoft Windows and Adobe Photoshop, et al, have been developed and tested in accordance with these guidelines.

Quality requirements for COTS software product include “… the product description, the user documentation, and the software contained on a computer sensible media. The quality requirements, functionality, reliability, usability, efficiency, maintainability, portability, and quality in use are consistent with the definitions of SQuaRE.” (source: https://www.iso.org/standard/37457.html)

ISO/IEC 17025:2005(E) section 5.4.7.2 reads:

When computers or automated equipment are used for the acquisition, processing, recording, reporting, storage or retrieval of test or calibration data, the laboratory shall ensure that:

a) Computer software developed by the user [must be] documented in sufficient detail and [must be] suitably validated as being adequate for use; (emphasis added)

ISO/IEC 12025:2005 also states that commercial-off-the-shelf (COTS) software (e.g., word processing, database, image processing and statistical programs) in general use within their designed application range may be considered to be sufficiently validated.

Moreover, I have taught the use of Adobe Photoshop in the digital processing of evidentiary photography to law enforcement agencies throughout the United States, Canada, Europe, Australia and New Zealand for the past 20 plus years. (ISO/IEC 17025:2005 (E), section 5.4.5.2 (Note 4) states that “[v]alidation studies can be conducted by the scientific community (as in the case of standard or published methods…”) During these training programs, I have continually demonstrated the three primary keystones of the ISO/IEC 17025:2005:

• Accuracy – the agreement between the accepted and the obtained value (e.g., analysis and comparison of Level 1, Level 2 and Level 3 fingerprint characteristics).

• Repeatability – demonstrating that one examiner using a particular instrument (e.g., a computer) using a series of methods (e.g., Ninhydrin, Black Powder, R6G, Amido Black, Basic Yellow, Indanedione, Zinc Chloride, RAM, and more) process the same sets of samples (digital images) on different days and obtain the same (or similar) results that were demonstrated to be acceptable.

• Reproducibility (i.e., the ability of a result to be replicated by someone else independently) – demonstrating that the image processing technique can be reproduced reliably and satisfactorily by a competently trained and experienced practitioner in another laboratory with the same equipment and resources.

In addition, the use of Adobe Photoshop has been commonly accepted by the courts for 20 plus years. There is sufficient case law that clearly states that Adobe Photoshop fulfills the evidentiary requirements for the admission of digitally processed photographs. (I have a list of cases that have been used as precedent in the courts)

The bottom line is that there is absolutely no reason (or requirement) to validate the use of Adobe Photoshop as it is considered “sufficiently validated” as being adequate for its intended use.

This was a topic that came up at the annual Quality Assurance Conference in Scottsdale, AZ this year. To expand on Eric's earlier point- have your QAM get in contact with other Quality Assurance personnel. They have a private list serve too. Often QA feels like examiners resist their interpretations just because they don't want to have to put in the work. If you could (politely and tactfully) point out that their interpretation is in the minority of their own group they might drop the issue. No one likes to be in the minority amongst their peers. Your success may impact the rest of us, I'd hate if this interpretation gained more acceptance and we all had to comply because of a small minority gaining traction.