By: James H. Nichols

CLSI EP18, Risk Management Techniques to Identify and Control Laboratory Error Sources, provides guidance for risk management activities that include risk analysis (failure modes and effects analysis, or FMEA) and risk monitoring (failure reporting, analysis, and corrective action systems, or FRACAS) based on best practices.² It is a revised guideline that expands risk management beyond the focus of unit-use devices in the original document. FMEA and FRACAS are quality tools that can be used to identify and control potential causes of risks, which constitute an integral part of a well run quality management system. EP18 provides a risk management approach to preventing harmful events in the clinical laboratory using a step-by-step explanation of techniques as adapted from other industries. It recommends a quality management system for in vitro diagnostic test systems that is based on expert opinion, practical to implement, and applicable to various devices and settings, so that sources of failure (potential failure modes) are identified, understood, and managed. This quality management system will assist device manufacturers, users, regulators, and accrediting agencies in assuring correct results. EP18 addresses regulatory considerations (e.g., principles and accountability), recommends the development of a partnership between users and manufacturers, provides a source-of-failures matrix, and suggests approaches to quality monitoring and identification of the problems. The source-of-failures matrix is an appendix table in FMEA format with a comprehensive list of preanalytical, analytical, and postanalytical potential sources of error or failure modes. While this table is intended for manufacturers to ensure that these sources of potential failure have been considered and, where appropriate, controls are put in place, clinical labs will find this table useful in assessing risk in their lab when implementing a new device or troubleshooting complaints. EP18 has completed the CLSI voting process and is currently in press as an approved second edition guideline.²

 

EP22, Presentation of Manufacturer’s Risk Mitigation Information for Users of In Vitro Diagnostic Devices, provides guidance to manufacturers on the establishment and disclosure of information they might choose to share with users regarding the scope and effectiveness of device risk mitigations intended to prevent production or release of inaccurate patient test results. Manufacturers of in vitro diagnostic devices use risk management principles to develop, characterize, and improve their products and to control risks that might be encountered when using devices. Often, manufacturers give instructions to users about procedures to follow to verify that device components are working as intended. However, most manufacturers do not provide users sufficient information about device features that mitigate risk, nor do they provide information about how such features are judged as effective risk mitigations. EP22 gives general guidance to manufacturers about the scope of risk-mitigation information that would be helpful to users to create and implement a quality control plan to meet established goals for quality and to comply with regulatory and accreditation requirements. The document also suggests a format for communicating that information. The manufacturer information should include a description of the device feature or recommended user intervention intended to mitigate the risk, the targeted failure mode, and how the design mitigation feature or recommended user intervention works to ensure the quality of patient test results. EP22 is currently being distributed for delegate comment and vote as a proposed guideline together with EP23.

 

EP23, Laboratory Quality Control Based on Risk Management, provides guidance based on risk management for laboratories to develop quality control plans tailored to the particular combination of measuring system, laboratory setting, and clinical application of the test. EP23 describes good laboratory practice for developing a quality control plan based on manufacturer’s risk mitigation information, applicable regulatory and accreditation requirements, and the individual healthcare and laboratory setting. Although the manufacturer is responsible for quality in design of its measuring system and reagents, the laboratory and, ultimately, the laboratory director are accountable for the quality of test results. To establish effective test quality control, laboratories must process an array of information (regulatory requirements, manufacturer-provided information, the laboratory’s environment, and the medical applications of tests performed) through a risk assessment process. This process identifies weaknesses in the measuring system and environment that are assessed in conjunction with the probability for error, the effectiveness of control processes built into the measuring system, and the laboratory’s tolerance for risk in consideration of the clinical use of a laboratory result. EP23 provides guidance to laboratories for establishing a quality control plan. Once developed, the quality control plan is monitored for effectiveness and modified as unanticipated failure modes or underestimated risks of error are discovered, or as particular control procedures are no longer required once sufficient objective data demonstrating reliable performance have been established. The advantages and limitations of a variety of quality control measures are discussed to help the laboratory develop a quality control plan that is appropriate for its particular measuring system, laboratory, and clinical environment.   

 

The U.S. Clinical Laboratory Improvement Amendments of 1988 (CLIA ’88) require laboratories to follow manufacturer recommendations for quality control and for moderate- or high-complexity tests, require laboratories to conduct quality control at two clinically relevant concentrations at least once every 24 hours (with the exception of blood gases and coagulation, which require more frequent quality control every 8 hours). While the analysis of liquid controls that mimic patient specimens can meet these regulations, performance of liquid quality control on some unit-use and single-test kits will consume the entire test without providing assured quality of the very next test. For a pregnancy test or urine dipstick, for instance, analysis of quality control on one test consumes the entire kit and doesn’t prove quality with the very next kit, even from the same lot of tests. In addition, liquid quality control increases the cost of testing and does a poor job at detecting specimen clots, bubbles, or interfering substances (e.g., hemolysis or drugs) that may occur with a single specimen. 

 

Newer device and test kit configurations incorporate other types of quality control processes. Some of these control processes may only evaluate a portion of the total testing procedure, while other control processes may better evaluate the entire test system. For the glucose meter, electronic controls on the device only determine that the electronics of the test system are working; they do not assess the viability of the chemical components of the test in each strip. On the other hand, pregnancy tests also contain an internal control. This internal control is a separate antigen-antibody reaction that can evaluate the chemistry, the specimen, and the analysis of the test. Internal pregnancy controls are sensitive to the volume of specimen applied to the test, the viability of reagent and kit storage or reactivity, appropriate test timing, and even visual acuity of the operator. This type of internal control process may be better than a liquid control in detecting specimen integrity for errors from clots, sample viscosity, and interfering substances that could affect a single specimen, since the internal control is analyzed with each sample while the liquid controls are only performed once a day on a random sampling of test kits. 

 

The CLSI EP18, EP22, and EP23 documents provide guidance to assess the risk of a test failure, consequences of not detecting the hazard, and an appropriate quality control mechanism to minimize risk. To develop a quality control plan, clinical laboratories need information from manufacturers to understand the various failure modes for a test, probability of a failure occurring in their laboratory, and what the manufacturer has engineered into the device in order to minimize risk.

 

Risk for particular errors may be greater in some laboratories than others. For instance, use of a test in a central laboratory may have fewer hazards when conducted by medical technologist staff with significant laboratory testing experience and low turnover compared to the frequency of failures encountered with the test performed in a busy emergency room by nurses and clinicians with high staff turnover and little laboratory experience. In addition, the tolerance for errors may be greater for a test where medical action may not occur for several days compared to a test in a critical care environment where immediate treatment decisions are made. As such, a quality control plan will depend on the manufacturer’s information, the testing environment, and the clinical application of the test. Quality control plans will therefore vary between laboratories and clinical settings. 

 

EP18 provides a framework of risk management principles based on ISO14971 described in a manner that clinical laboratories can understand. EP22 is directed to manufacturers for providing information on device risks and control mechanisms to laboratories purchasing their devices. EP23 provides guidance to laboratories for processing the provided information through a risk assessment and developing a customized quality control plan for their laboratory and patient population that will detect and prevent errors from occurring. 

 

Why should manufacturers comply with the EP22 guidance, since information about device risks could be used against a company by their competitors?  Manufacturers invest significant amounts of money to engineer processes into their devices that reduce the risk of harm to patients and provide operational advantages to the laboratory. Such information about features within a product that offer safety and other advantages over currently marketed devices is itself a marketing tool that vendors should want to publicize. The information described in EP22 simply summarizes the data about device features that manufacturers already have on-hand in a simple format that laboratory consumers can use to assess the effectiveness of the control feature and determine the role of that feature in the laboratory’s quality control plan.

 

A quality control plan developed by a clinical laboratory as described in EP23 must necessarily meet national and local laws governing laboratory testing. In the United States, CLIA ’88 mandates that controls must be performed daily, but it allows for some flexibility in those control processes. For instance, internal controls may substitute for external liquid controls under certain circumstances.  Currently, the Centers for Medicaid and Medicare Services allow laboratories to perform an evaluation to compare internal controls to external liquid controls when a laboratory wants to substitute internal controls in lieu of external controls to satisfy daily CLIA’88 requirements.³ Yet, this equivalent quality control comparison is not based on a scientific premise that either control process will reduce risk to a clinically acceptable level. The quality control plan developed through the CLSI EP23 risk assessment provides a more scientific foundation for laboratory directors to determine the appropriate control processes for their methods. The EP23 quality control plan provides the list of hazards considered by the laboratory and the rationale for control processes to control each risk. Once implemented, this quality control plan forms the basis for quality monitoring and future improvement. As unexpected device failures or complaints occur, the source of the error is determined, and the quality control plan can be reviewed and modified as needed to prevent similar occurrences in the future. The quality control plan provides a justification for minimizing certain hazards in the laboratory and a management plan for future improvement. Such a quality control plan provides a more objective approach for laboratory accreditation and inspection than current regulations that mandate a one-size-fits-all type of control across the diversity of laboratory instruments and tests. Quality control plans based on risk management may provide a foundation for more scientifically based revisions to future laboratory quality regulations. 

 

1. “Medical devices – Application of risk management to medical devices. ISO 14971:2007” (Geneva, Switzerland: International Organization for Standardization, 2007).

 

2. “Risk Management Techniques to Identify and Control Laboratory Error Sources – Approved Guideline, Second Ed.: EP18-A2 (Wayne, PA: Clinical and Laboratory Standards Institute, 2009).

 

3. “Clinical Laboratory Improvement Amendments – Equivalent Quality Control Procedures, What are they and when can I use them?” Brochure #4 (Baltimore, MD: Centers for Medicare and Medicaid Services, 2003). Available online at http://www.cms.hhs.gov/CLIA/downloads/6066bk.pdf. 

 

James H. Nichols, PhD, DABCC, FACB is chairholder, CLSI EP23 Subcommittee, and professor of pathology at Tufts University School of Medicine. He is medical director, clinical chemistry at Baystate Health (Springfield, MA). He can be reached at james.nichols@baystatehealth.org.