Introduction
The accreditation process for digital forensics laboratories is complicated and time-consuming. Accreditation processes have been in place for decades, and in recent years accreditation has been applied not only to traditional labs that conduct fingerprint, blood, drug and other analyses; but also to the discipline of digital forensics.
Accreditation, as it applies to laboratories, should not be confused with certifications that apply to individuals, or to certifications that apply to items of equipment and software. The National Voluntary Laboratory Accreditation Program describes the difference as follows:
"The terms "accreditation" and "certification" are sometimes used interchangeably, however, they are not synonymous. Certification is used for verifying that personnel have adequate credentials to practice certain disciplines, as well as for verifying that products meet certain requirements."
See: https://www.nist.gov/nvlap/accreditation-vs-certification
This work explores a few of the considerations that organizational leaders should think about when contemplating whether or not to seek laboratory accreditation.
Regulations, Policies, and Standards
The International Organization for Standardization
The International Organization for Standardization researches, develops, and publishes (for a fee) requirements and guidance for various industries with the motto, "Great things happen when the world agrees". At least two of the regulations, ISO/IEC 17025:2017 and ISO/IEC 17020:2012, provide guidance to many digital forensics laboratories. The regulations are available for purchase at the ISO web site: https://www.iso.org/standard/66912.html
ISO/IEC 17025:2017 and ISO/IEC 17020:2012 include requirements for testing, calibration, and inspection of laboratories, digital and otherwise. 17025 is generally considered to be more applicable to the daily work of a digital forensics lab. Compliance with the requirements is recognized by ANSI and A2LA towards the accreditation of digital forensics laboratories. Compliance with the ISO's requires specific management activities, record-keeping, policy-writing, and document control.
Selected Topics from 17025 and 17020
Below are some of the topics from the contents section of ISO/IEC 17025:2017 for testing and calibration of laboratories. The topics give you an idea about the number of requirements; each of which must be supported by an organizational policy, detailed record-keeping and periodic audits.
Below are some of the topics from the contents section of ISO/IEC 17020:2012 - requirements for inspecting laboratories.
The Business of Providing Accreditation Services
Two of the organizations that assist agencies (for a fee) towards completing the accreditation process are ANSI and A2LA. The ISO/IEC 17025:2017 standard is typically used to guide accreditation. The organizations are briefly described below:
The American National Standards Institute (ANSI, also ANAB)
The National Accreditation Board is a non-governmental organization that provides accreditation services to public and private-sector organizations for a fee. ANAB is jointly owned by the American National Standards Institute (ANSI) and the American Society for Quality (ASQ). Their services are funded by the accreditation fees charged to member organizations.
See: http://webstore.ansi.org/RecordDetail.aspx?sku=ISO%2fIEC+17025%3a2005
The American Association for Laboratory Accreditation
The American Association for Laboratory Accreditation (A2LA) describes itself as a nonprofit, non-governmental, public service, membership society. They are an independent, non-profit accreditation service. A2LA offers programs for the accreditation of inspection bodies, proficiency testing providers, reference material producers and product certification bodies. Their motto is, "A better world through accreditation."
See: https://www.a2la.org
National Institute of Standards and Technology
The US Department of Commerce, National Institute of Standards and Technology (NIST), National Commission on Forensic, Science Subcommittee on Accreditation and Proficiency Testing published a document titled: Critical Steps to Accreditation. The document outlines some of the requirements of a quality management system. According to the report (p.1):
"Accreditation helps to ensure both ongoing compliance to industry standards and continual improvement of an FSSP's operations. Accreditation assesses an FSSP's capacity to generate and interpret results. Accreditation criteria are based on accepted industry standards and applicable international standards."
The NIST report (pp. 2-3) recommends the following steps towards accreditation:
- Written procedures for evidence (security/control/handling)
- Written reports
- Technical and administrative review of reports and supporting records
- Testimony monitoring
- Note taking
- Technical procedures
- Training program
- Proficiency testing
- Corrective and preventive action process
The full NIST report can be found at this link:
http://www.ascld.org/wp-content/uploads/2016/01/Views-Doc-Critical-Steps-to-Accreditation.pdf
Quality Manager
A key person in any organization that implements ISO's 17020 and 17025 is the designated quality manager. The quality manager is tasked with ensuring that the management system is maintained and that the requirements are adhered to.
Audits
Audits are sometimes a source of consternation for managers. Below is an excerpt from ISO/IEC 17025 regarding audits:
4.14 Internal audits
4.14.1 The laboratory shall periodically, and in accordance with a predetermined schedule and procedure, conduct internal audits of its activities to verify that its operations continue to comply with the requirements of the management system and this International Standard. The internal audit program shall address all elements of the management system, including the testing and/or calibration activities. It is the responsibility of the quality manager to plan and organize audits as required by the schedule and requested by management. Such audits shall be carried out by trained and qualified personnel who are, wherever resources permit, independent of the activity to be audited.
NOTE The cycle for internal auditing should normally be completed in one year.
Audit Nonconformities
Audits are a time-consuming and sometimes difficult, albeit necessary, process. Accrediting organizations perform audits to monitor compliance among their member-agencies. According to the ANAB, the top ISO/IEC 17025 non-conforming audit findings in 2015 were:
5.6 Measurement traceability
- Missing procedures
- Lack of document traceability to national standards
- Labs using other non-accredited labs for equipment calibration
4.6 Purchasing services and supplies
- Incomplete procedures
- No procedure for purchasing calibration services
4.15 Management reviews
- Incomplete procedures - lack of scheduling of management reviews
- Failure to record actions arising from management review or time scales
4.14 Internal audits
- No predetermined schedule, failed to address all elements of IOS/IEC 17025
- No follow-up, No close-out of corrective action
5.2 Personnel
- Lack of training and education records
- Competency testing - authorization to perform work
- Lack of training plans
5.5 Equipment
- Equipment records not maintained
- Instructions not readily available
- Log books
4.13 Control of Records
- Lab not following it's own procedures
- Data not recorded as required
- Corrections to data not properly recorded
4.1.5 Organization and Management
- Lack of required policies
- Failure to document responsibilities of the Quality Manager
4.3 Document Control
- Failure to control documents
- Revision identifiers obsolete or absent
- Lack of document review
5.4 Test and Calibration Methods and Method Validation
- Lack of procedure or incomplete procedure
- Lack of validation records
The list of nonconformities above provides a sobering reminder to quality managers that there is LOT to think about regarding compliance with the accreditation requirements. For criminal and civil case purposes, opposing counsel might use non-conforming audit findings to attack the credibility of work-product originating from a lab.
The Scientific Working Group on Digital Evidence
In 2017, The Scientific Working Group on Digital Evidence (SWGDE) produced a document titled: MYTHs and FACTs about Accreditation for Digital and Multimedia Evidence Labs. Some information from that document (pp. 5-7) is provided below:
MYTH: "Accreditation is unaffordable."
MYTH: "Everyone can afford accreditation."
FACT: If an organization does not currently employ personnel to write and enforce quality assurance policy, there may be a significant cost associated with the accreditation process. There is a financial cost to be paid to the accrediting body (application fee, on-site visit(s), and annual fee).
All accreditation related costs vary based upon the size of the laboratory and the choice of implementation (p.5).
MYTH: "Accreditation will require more personnel resources."
FACT: Personnel involved in quality assurance procedures will vary based on the size of the laboratory. Larger laboratories may require personnel dedicated to working within a quality system, whereas smaller laboratories may not require dedicated personnel. There are successful approaches for small and one-person laboratories to implement quality systems (p.5).
MYTH: "The accreditation process can be completed within a few months."
FACT: The process of developing and implementing a quality management system by a laboratory that can meet accreditation requirements may take 1- 3 years depending on the size and resources of the laboratory (p.5).
MYTH: "Accreditation means a laboratory or work product is perfect."
FACT: The work product produced by non-accredited laboratories may be as good or better than accredited laboratories. factors, such as human error or faulty equipment, exist regardless of whether a laboratory is accredited. Accreditation requires that laboratories have processes to document, address, and correct problems (p.6).
MYTH: "Accreditation creates unnecessary work."
FACT: The accreditation process can identify areas for increased efficiencies, quality control, and promote consistency within the laboratory. Additional documentation that is required in accredited laboratories can improve repeatability and reproducibility within the laboratory and foster communication across laboratories (p.7).
Considerations
Initial accreditation and the maintenance of accreditation involves a commitment of personnel, time, and money. Some of the questions to consider regarding accreditation include the following:
- Is accreditation required by a sponsoring or governing organization?
- What is the budget impact of the accreditation process?
- What are the application fees?
- What are the Inspector site-visit costs?
- What are the time and labor costs for in-house personnel throughout the process?
- What are the follow-up and ongoing costs?
- Who will be assigned as Quality Manager?
- Will the assignment be a full-time position or a subordinate duty?
- Who else in the organization must be actively involved in the accreditation process?
- What policies need to be devised, written, published and enforced?
- Who will author, revise and monitor policies and procedures?
- What will the process be for non-compliance and who will be involved in the administration?
- How many person-hours will be needed to complete the accreditation process?
- Over what period of time will the accreditation process occur?
- Will successful accreditation act as a protective shield or mitigator in the event of civil lawsuit against the organization?
- In the event of failure to obtain accreditation, will the reputation of the organization suffer?
- Will the organizations' failure to accredit work as an aggravating factor in future civil lawsuits against the organization?
- Is a "de minimis" policy practice preferred because it opens the organization up to fewer legal attacks?
- Will accreditation improve the lab's profit-margin? (Cost/benefit analysis; applicable to for-profit labs)
- Could the lab simply obtain and follow the regulations without enduring the formal accreditation process?
As investigative caseloads increase and budgets shrink, digital forensic lab managers are finding less time and money to conduct the administrative duties required by accreditation. Some are eschewing accreditation altogether while arguing that their employees are trained and certified, their equipment and software tools are validated, and consequently, overall lab accreditation is an unnecessary additional burden.
The decision about whether or not to pursue accreditation is important and sometimes controversial. Some argue that digital forensics examiners should be trained and certified, but it is not necessary to accredit digital forensics labs. Critics of lab accreditation also posit that court cases are not won or lost based on lab accreditation but instead upon the individual work, training, certification and expertise of the examiner. Proponents of accreditation believe that the process helps to improve the work-product and improves the quality of lab operations.
Additional Information Sources re: Accreditation
ANAB
What Sets ANAB Apart for Forensic Accreditation? https://www.anab.org/forensic-accreditation
Forensic Focus
ISO 17025 For Digital Forensics – Yay Or Nay? https://articles.forensicfocus.com/2018/01/24/iso-17025-for-digital-forensics-yay-or-nay/
A2LA
American Association for
Laboratory Accreditation Frequently Asked Questions
https://www.a2la.org/
=-=-=-=-=-=-=-=-=-=-=-=
No comments:
Post a Comment
Thank you for your thoughtful comments.