Risk management for IVDs, Part 1: Planning and documenting the risk management process

Donald M. Powers, PhD, is president and principal consultant of Powers Consulting Group (Pittsford, NY) and is a member of IVD Technology's editorial advisory board. He can be reached at powers@ frontiernet.net [5].

Historically, IVDs have been treated as low-risk medical devices on the grounds that physicians do not rely on a single piece of information when making diagnoses or treating patients. However, this perception is changing. For example, a recent FDA report on medical device safety states that up to 80% of medical decision-making is now guided by the use of laboratory tests.¹ A leading IVD manufacturer claims that 70% of patient information comes from IVD analysis of blood and other body fluids performed in centralized laboratories.² Furthermore, AdvaMed (Washington, DC) and the European Diagnostic Manufacturers Association (EDMA; Brussels) argue that the information from IVD testing has been greatly undervalued, a position supported by recent analyses of the value of laboratory testing.^3–5

While hard data to support such beliefs may be lacking, the FDA Web site has been listing an increasing amount of medical device reporting (MDR) data from IVD manufacturers; the number of Class II IVD recalls, which imply a significant risk to health, has now surpassed Class III IVD recalls; and Class I recalls, which were previously rare in the IVD industry, are occurring more often. Regardless, it would be difficult to argue that IVD assays play an insignificant role in medical decisions and thus present no risk to patients. Furthermore, risk management is not limited to deaths or serious injuries. The burden is on IVD manufacturers to determine and control the potential for any harm through an objective process.

How can IVD manufacturers determine the true risk of their products and comply with regulatory requirements for managing risk? This series of articles will explore the application of medical device risk management principles to IVDs and attempt to place IVD risks in the proper perspective. A systematic risk management process is described in ISO 14971, a voluntary standard developed jointly by the International Organization for Standardization (ISO; Geneva) and the International Electrotechnical Commission (IEC; Geneva).⁶ Part 1 of this series covers regulatory requirements, general risk management principles, and planning and documenting a risk management program.

Regulatory Aspects

ISO 14971 has been adopted by most countries as the preferred standard for complying with risk management requirements for medical devices. FDA recognizes the standard and participates in the technical committee that developed ISO 14971. In the European Union, a declaration of conformity to ISO 14971 creates a presumption of compliance with the IVD Directive's essential requirements pertaining to risk management. Australia, Canada, Japan, and several other countries have adopted the ISO 13485 quality system, which requires risk management. Given ISO 14971's flexibility, logical framework, and strong support from the regulatory community, adopting a different risk management approach would be a questionable strategy.

While risk analysis is the term that FDA uses to describe one of the design control requirements in the quality system regulation (QSR), the theme in regulatory and quality circles today is risk management and its integration into the quality management system.⁷ According to ISO 14971, risk analysis is only the first step in a total product life cycle risk management process. Risk management is a much broader concept, and a recent guidance from the Global Harmonization Task Force (GHTF) stresses its integral role in the quality management system.⁸

The difference between the risk analysis required in 1996 and today's risk management expectations is more an evolution of terminology than an escalation of requirements. When FDA drafted the QSR, risk analysis and risk assessment were the terms used, often interchangeably, to describe the activities that IVD manufacturers undertake to minimize the risk of using their products. In the meantime, ISO defined risk concepts more precisely for medical devices. The QSR's true regulatory intent is revealed in its preamble, which indicates that FDA never intended manufacturers to stop the risk management process after only the first stage.

While the risk management requirements in the European IVD Directive are often thought to be more demanding, in reality they are not very different from the expectations laid out in the QSR preamble.

When their risk management program is audited, many IVD manufacturers still point to a risk analysis document prepared during product development as their only evidence of risk management. Nearly 10 years have passed since FDA first published the QSR. The third edition of ISO 14971 with an expanded set of IVD guidelines is nearing completion. And the GHTF has published a final guidance on integrating risk management into the quality system.^{8, 9} A review of the basic requirements of a risk management program for IVDs is therefore timely.

Risk Management Process

Figure 1. A schematic representation of the risk management process. (Source: ISO/DIS 14971:2005).

For the purpose of complying with regulatory requirements, only risks from exposure to health, safety, or environmental hazards are considered relevant. While business risks caused by project delays, inefficient manufacturing processes, or dissatisfied customers are important to IVD companies, they are of little interest to FDA. If a manufacturer addresses business risks in the same risk assessment, the documentation should make it clear that they are considered secondary to any health or safety risks. Ideally, IVD manufacturers should address business risks separately.

According to ISO 14971, a systematic risk management process for medical devices, including IVDs, involves the following four main stages (see Figure 1):

• Identifying hazards inherent in the use of IVD products, and estimating the risks of harm to patients, laboratory workers, and the environment.

• Evaluating the acceptability of such risks against criteria established by company management.

• Reducing risks to a level that complies with the company's risk acceptability policy, and verifying that the risk controls are effective.

• Monitoring internal and external product experience for new hazards, increased risks, and the possibility that society's tolerance for risks has changed.

The first three stages integrate logically into the flow of a product design and development process.¹⁰ The last stage of risk management is ongoing throughout the life of a product, and overlays the entire quality system architecture. The vigilance required to sustain risk management involves almost every organization in an IVD company, and it is a never-ending responsibility of top management.

The GHTF member countries view risk management as a necessary part of an effective quality management system. In its recent guidance document, GHTF takes the position that integrating risk management efforts into the quality management system is necessary to ensure that such efforts will be coordinated and all identified risk issues will be brought to closure.

Communicating Risk Concepts

Failures to identify and control risks can often be attributed to a lack of understanding of basic risk concepts, and miscommunication among departments. Both the ISO standard and the GHTF guidance make a strong case that IVD manufacturers should use a common risk vocabulary to communicate risk concepts effectively within their organizations. Speaking the same language can also help when explaining the company's risk management activities to an external auditor or FDA investigator.

IVD manufacturers need to understand the potential of their assays to cause or contribute to patient harm. This determines the level of risk, which depends on the severity of the possible harm and the probability that actual harm will occur. An important concept clarified in the new revision of ISO 14971 is that harm can only occur when a patient is placed in a hazardous situation, which means exposure to the hazard. For IVDs, this generally means that an incorrect result (the hazard) must be reported to a physician or other healthcare provider in a position to act on the result (the hazardous situation). It may also mean that a critical result was not able to be delivered when needed.

An IVD assay has inherent risk if physicians rely on the results for medical decisions. Obviously, IVD assays that fail to meet performance specifications can be hazardous. However, a state-of-the-art assay that conforms to all performance specifications can also generate hazardous results. Given the biological nature of many IVD assay components, state-of-the-art specificity is rarely capable of perfect discrimination between positive and negative specimens, and some well-established quantitative methods operate at the limit of their accuracy. For example, the National Institute of Standards and Technology (NIST; Gaithersburg, MD) commissioned a report that quantified the effects of misdiagnosis from incorrect calcium results from state-of-the-art clinical test methods. Such misdiagnoses resulted in $119 million in extra costs to the U.S. healthcare system, in part due to unnecessary medical procedures performed in response to falsely abnormal (i.e., potentially hazardous) results.¹¹

On the other hand, an incorrect result may be a hazard, but it may not necessarily present a significant risk. The likelihood that a patient will be exposed to harm may be reduced by the ability of the IVD instrument itself, or laboratory workers or even physicians, to detect hazardous test results. According to ISO 14971, IVD manufacturers must formulate a preestablished risk acceptability policy that enables management to decide whether risks from incorrect or delayed results are acceptable. Factors such as patient benefits, applicable regulations and standards, technology state of the art, and current values of society dictate the acceptability of the risks.

[6]

Table I. Definition of risk concepts. (Click to enlarge.)

Because of the nuances involved in risk decisions, care should be taken to document risk assessments in clear and explicit language. Imprecise terminology can make it difficult to determine what the risk analysis team actually concluded. For example, conventional risk analysis summary tables often have a column with the heading “probability of occurrence” or, simply, “occurrence.” But occurrence of what? Potential failure modes? Hazardous situations? Harm? Such terms are not synonymous: not every failure leads to a hazardous situation, and not every hazardous situation leads to harm. The documentation should clearly convey what the risk analysis team concluded by using standardized risk management terminology (see Table I [6]).

Risk Management Planning

Like all quality management activities, risk management activities must be planned, and the plan must encompass an IVD product's entire life cycle. However, despite this explicit requirement of ISO 14971, IVD companies that claim to follow the ISO risk management process often cannot produce a satisfactory plan during an audit. Some companies mistakenly regard their standard operating procedures as their risk management plan, even though procedures do not include all of the essential elements of a plan.

According to the GHTF guidance, a risk management plan should address the following:

• Approaches to be used in determining acceptable levels of risk.

• Roles and responsibilities for risk management activities.

• Reviews of risk management results scheduled at appropriate intervals.

• Inputs from risk management to the quality management review process.

In planning risk management activities, IVD manufacturers should start by defining the information needed to show that all potential hazards have been identified and assessed, all unacceptable risks have been mitigated to an acceptable level, and the effectiveness of all risk controls has been verified. ISO 14971 requires this information in a risk management report. The GHTF guidance also suggests a summary table format that captures the results of individual hazard and risk evaluation activities, and provides traceability of risk-control measures to the product design requirements and verification and validation activities.

A risk management file is an essential requirement of ISO 14971. The format and content of the risk management documentation should facilitate its use throughout the product life cycle. IVD manufacturers should design and create the documents with this end in mind. For example, change-control decisions, complaint and failure investigations, CAPA case evaluations as well as reportability decisions for MDRs and Corrections and Removals need to refer to an up-to-date risk analysis. In turn, the outputs of such processes should feed into the next iteration of the risk analysis.

Once the information required for the risk management file is defined, manufacturers can develop an efficient strategy to obtain the information and maintain it. IVD manufacturers can then plan the risk management activities to ensure they are comprehensive, and that risk controls are traceable to initiating hazards.

In selecting the appropriate risk analysis approach, common sense should prevail. Not every assay requires the same depth of analysis. IVD assays are often closely related variations on a basic design with only one component varying from one assay to another (e.g., an antibody or enzyme to change the specificity). The risk management plan for a new IVD product can leverage experience from similar products. IVD manufacturers should explain the extent to which they will use related risk assessments in the risk management plan for the product. Manufacturers can use a screening strategy to eliminate low-risk issues to streamline the risk management process. Once the patient hazards are identified, IVD manufacturers can use prior knowledge about hazards from the assay design or the manufacturing process to eliminate low risks from further consideration.

While a formal risk management plan may not be an explicit FDA requirement, incorporating risk management planning into design and development planning makes good business sense. Such a plan identifies specific risk management tasks, the resources needed to perform them, the relationships to other tasks, and the responsibilities for completing each task. A risk management plan is a logical component of every new IVD product development plan.

Risk Assessment Reviews

The GHTF recommends that IVD product design and development reviews should include risk assessment results (i.e., risk analysis and risk evaluation). Therefore, design-review procedures need to define the risk assessment tasks that will be performed at the different stages of design and development. For such design reviews, reviewers with the breadth and depth of experience to assess design decisions concerning risk acceptability are needed.

For example, early design and development reviews will focus on hazard identification, risk estimation, and the needs and requirements for risk control measures. Reviews at later stages of design and development will shift their focus to implementing risk-control measures and evaluating residual risk.

Verification and validation reviews will address the effectiveness of risk-control measures. The final design review will include an evaluation of overall residual risk after completing an evaluation of all single identified hazards. If the residual risk is still too high, a risk/benefit analysis can be performed to determine whether the medical benefits outweigh the remaining risks.

Future Topics

The next article in this series will focus on risk analysis and risk evaluation, which make up the risk assessment phase of risk management. The following articles will cover risk control and the postproduction monitoring stages of risk management, and point out effective ways to integrate risk management into an existing quality management system. These articles will also highlight common gaps found in the risk management programs of IVD companies.