|Regulations & Standards|
With the traceability requirements of the European Union's IVD Directive looming ahead, a group of stakeholders with diverse interests in clinical laboratory measurements met in a Paris suburb this June to form the Joint Committee for Traceability in Laboratory Medicine (JCTLM). The participants discussed short- and long-term strategies to enable manufacturers to comply with the traceability requirements of the directive, which goes into effect on December 7, 2003, and to prioritize activities to close the gaps among the various reference systems used by clinical laboratories.
Donald M. Powers, PhD, is president of Powers Consulting Services (Rochester, NY), an independent IVD regulatory and quality consulting firm. He can be reached at email@example.com.
Although a new European regulatory requirement may have triggered the meeting, the theme of the meeting was achieving global comparability of clinical laboratory results. At the end of the meeting, JCTLM formed two working groups to address the immediate needs of quantitative IVD assay manufacturers.
The outcome is encouraging to manufacturers that have been working diligently to identify suitable reference systems for their assays, and to implement changes to their value-assignment processes. Without any official guidance, manufacturers have been on their own to figure out what the traceability requirements mean for their products and how to comply within the short remaining time frame. If these activities continued without coordination, the desired comparability of patient results over time and space would not be achieved, since there is often more than one reference method and reference material for the same analyte, and they do not always agree. JCTLM clearly articulated and accepted the need for international consensus on a single reference system for each analyte.
Requirements and Issues
According to the IVD Directive, "the traceability of values assigned to calibrators and/or control materials must be assured through available reference measurement procedures and/or available reference materials of a higher order."1 Since many manufacturers thought traceability was merely a European euphemism for accuracy, they are quickly learning a lesson in metrology.2–4
The metrological definition of traceability concerns the application of calibrator and control values that can be related to national or international standards through an unbroken chain of comparisons.5 This definition does not mean the same thing as demonstrating an acceptable recovery of the value that is assigned to a standard, as many manufacturers had assumed. In addition, the traceability requirement applies only to calibrators and controls with target values, and does not extend to patient results.
The ability of manufacturers to establish the metrological traceability of an assay calibrator hinges on not only whether a reference material or reference measurement procedure is available that qualifies as "higher order," but also whether a suitable reference laboratory can be found to perform the analysis. A reference system for a given analyte will ideally consist of a certified reference material, a recognized reference measurement procedure (both of higher order), and an accredited reference measurement laboratory.
However, international consensus on the criteria for differentiating higher order from lower order has not yet been reached, and even the meaning of "available" is being debated. For example, while certified reference materials (CRM) from the National Institute of Standards and Technology (NIST; Gaithersburg, MD) should meet the strict criteria laid down in ISO 15194—the international standard that defines requirements for reference materials—debate continues over whether less rigorously prepared materials, such as the World Health Organization's standards for biological quantities, will qualify.6 With all this uncertainty over which reference materials and reference methods are qualified and available, the danger remains that manufacturers will make hasty choices and be forced to undergo another round of changes when the international consensus turns out to be different.
The good news for manufacturers is that a plan and a coordinating body, the JCTLM, have finally been established. The bad news is that there is no chance that the necessary infrastructure will be in place in time to meet the EU deadline.
Meanwhile, there is still a danger that enthusiastic metrology organizations may take the lead and focus on developing new reference systems, instead of on establishing viable reference systems from reference materials and methods that are already available. Maintaining a strong industry presence in JCTLM is the only way to prevent something like that from happening.
Working Group 1
The scope of Working Group 1 is reference materials and reference measurement procedures. This group has been charged with reviewing the lists of reference systems that various organizations have compiled, identifying those systems that qualify as higher order, and confirming their availability. Since this working group first has to agree on criteria, it will not complete this task until November 2002 at the earliest.
Since there are no entries yet on the list of available reference systems that JCTLM must maintain, the industry might reasonably take the position that none are "available" at this time, and manufacturers may CE mark their assays without demonstrating traceability to any higher-order reference systems. However, CRMs for many well-defined analytes such as glucose, cholesterol, and the electrolytes are already available from NIST and the Institute for Reference Materials and Measurements (Geel, Belgium), and they are widely used throughout the IVD industry.
The International Federation of Clinical Chemistry and Laboratory Medicine (Milan, Italy) has also published a series of enzyme reference methods based on ISO 15193, the international standard that defines requirements for reference measurement procedures.7 Manufacturers should quickly agree on a short list of qualified reference materials and methods, leaving JCTLM to sort out the more complex biochemical analytes.
Once consensus is reached on additional reference materials and methods, manufacturers will need time to establish traceability, obtain any necessary regulatory approvals, implement changes to labeling, and notify their customers. If there is a bias to the true values, as defined by the reference systems, then manufacturers will need to decide whether to recalibrate their assays to remove the bias, or maintain the bias and disclose it in their product labeling. For some analytes, such as HbA1c, traceability to the clinical database that established the effectiveness of the assay is more important than analytical truth. It would not be unreasonable to allow manufacturers at least two years lead time from the time JCTLM reaches and publishes its decision, so that manufacturers can implement traceability properly.
If the European Commission accepts the JCTLM plan, it would indeed give manufacturers a short reprieve while the working group sorts out what is available. Similarly, when the quality system regulation was adopted in the United States, FDA stayed enforcement of the design control provisions for one year in order to allow manufacturers time to gain the necessary experience. In any case, manufacturers will need to document in their products' technical files the basis for each decision to establish or not establish traceability to an external reference.
Manufacturers should also note that the unavailability of an external reference material or method does not let them off the traceability hook. If a suitable external reference is not available, good assay-design principles require that calibration be traceable to an anchor—usually an internally developed reference preparation. If such an anchor is also not available, the manufacturer may face a compliance issue during its next FDA inspection.
What about reference systems that are developed for more analytes later on, and how should JCTLM prioritize which ones to address first? The initial proposal was to prioritize the systems based on the medical importance of the analyte. While this approach has a certain logical appeal, the external quality assessment (EQA) and proficiency testing (PT) providers pointed out that it would mean devoting resources to reference systems for analytes for which method comparability has not been an issue, and ignoring those that do not agree well in the surveys. The final consensus was to start with analytes with comparability problems in the EQA/PT surveys. This agreement demonstrated the value of having all the key stakeholders present in order for a consensus process to work efficiently.
Working Group 2
The scope of Working Group 2 is reference measurement laboratories, which includes defining an accreditation process, identifying existing qualified reference laboratory networks, and determining the need to establish additional ones. The assumption is that networks of accredited laboratories will be needed to perform the reference methods, assign calibrator and control values, and confirm value assignment.
Manufacturers may set up their own laboratories according to two ISO standards: ISO 17025, a standard for calibration laboratories, and ISO 15195, a standard specifically written for laboratories performing calibration of IVD systems.12,13 Of course, maintaining a reference measurement laboratory that is accredited to these standards is an expensive proposition. Alternatively, manufacturers can outsource the work to an accredited laboratory. Only a few such laboratories exist today for a limited number of analytes, mostly in Europe. The level of interest among U.S. laboratories in performing such services for industry is not clear.
If networks of reference laboratories do become established, the problem that EQA/PT providers face in verifying the accuracy of laboratory methods might be solved. This solution requires a paradigm shift.
Since nearly all routine methods today are commercial systems and kits, a method's accuracy could be verified more efficiently at the manufacturer level instead of at the level of the end-user laboratory. One of the reference laboratories can measure method bias on authentic patient specimens, with the reference-method value defining truth. The PT surveys would then verify that the routine laboratories fall within the expected statistical distribution, since the manufacturer will have precertified the accuracy of the peer group mean.
Overcoming Pitfalls and Distractions
The JCTLM has been self-appointed to undertake its mission. Since the JCTLM has no official status in the European Union or any other part of the world, its authority comes solely from a memorandum of understanding among the stakeholders, who are represented by international organizations whenever possible (see sidebar).
This approach will work if care is taken to avoid potential pitfalls. One danger lies in overextending the JCTLM's scope beyond urgent and immediate needs. Another is allowing one stakeholder to dominate the work program.
For example, the metrologists seem anxious to commission a long-range plan to develop reference materials for all existing clinical analytes. While this objective might be a desirable long-term goal, it would be a very expensive undertaking and a distraction from the task at hand. Fortunately, industry representatives were outspoken at the JCTLM inaugural workshop and reminded everyone that manufacturers must focus now on meeting the legal requirements of the IVD Directive by December 7, 2003.
Another potential distraction also surfaced during the JCTLM meeting. EQA providers have a special, if not yet well-defined, surveillance role under the IVD Directive, and they are very interested in having stabilized survey samples that behave like patient samples when analyzed using routine clinical laboratory methods.
However, because of the problem of matrix effects, EQA providers cannot tell whether a result that does not agree with the reference value is due to an inadequate method or to the survey sample. EQA providers are therefore pushing for the development of commutable reference materials, commutability being another metrological term describing samples that behave the same as patient samples on a method. Commutable survey samples have been the Holy Grail for U.S. PT providers since the Clinical Laboratory Improvement Amendments of 1988 turned proficiency testing into a policing tool and created a perceived need to verify the accuracy of laboratory methods. Despite the obvious enthusiasm of the EQA providers and the metrologists for solving this problem, it is not necessary for compliance with the traceability requirement and would only be another distraction for JCTLM.
While all of the above proposals may sound like pipe dreams, most are doable with today's technology. However, implementation will require a significant amount of capital investment, and that is the limiting factor for what JCTLM can accomplish. JCTLM exists purely in a coordinating role. With no budget, no support staff, and no infrastructure, all of its work must be performed by its members. Thus, the work program of JCTLM will be self-limiting: only what can be justified as important to the members will get done.
Industry should be willing to pay for help in determining which reference materials and measurement procedures qualifying as "higher order" are already available. Individual manufacturers will pay for the services they receive from CRM producers and reference laboratories, and some manufacturers will probably be willing to help fund limited development activities related to the specific products they manufacture—particularly if they solve significant standardization problems. But in general, since new reference methods and materials, reference laboratory networks, and the development of commutable reference materials will have to be justified on the basis of improving patient care, the sponsors will have to look to government and private sources for funding.
1. "Council Directive 98/79/EC of the European Parliament and of the Council of 27 October 1998 on In Vitro Diagnostic Medical Devices," Official Journal of the European Communities L331 (7 December 1998).
2. DM Powers, "Traceability of Assay Calibrators: The EU's IVD Directive Raises the Bar," IVD Technology 6, no. 4 (2000): 26–33.
3. N Greenberg, "Calibrator Traceability: The Industry Impact of the IVD Directive's New Requirements," IVD Technology 7, no. 2 (2001): 18–27.
4. L Siekmann, "Measurement Traceability in Clinical Chemical Analyses," IVD Technology 7, no. 1 (2001): 41–58.
5. In Vitro Diagnostic Medical Devices—Measurement of Quantities in Samples of Biological Origin—Metrological Traceability of Values Assigned to Calibrators and Control Materials, ISO FDIS 17511 (Geneva: International Organization for Standardization, 2002).
6. In Vitro Diagnostic Medical Devices—Measurement of Quantities in Samples of Biological Origin—Description of Reference Materials, ISO FDIS 15194 (Geneva: International Organization for Standardization, 2002).
7. In Vitro Diagnostics Systems—Measurement of Quantities in Samples of Biological Origin—Preparation of Reference Measurement Procedures, ISO FDIS 15193 (Geneva: International Organization for Standardization, 2002).
8. Proceedings of the Workshop on Measurement Traceability for Clinical Laboratory Testing and In Vitro Diagnostic Test Systems, ed. Ellyn S Beary (Gaithersburg, MD: National Institute of Standards and Technology, 2001).
9. "NIST Workshop Sets Course on Measurement Traceability," IVD Technology 7, no. 1 (2001): 12–18.
10. DM Powers, "Calibration Traceability: NIST Workshop Takes a Major Step Forward," IVD Technology 7, no. 1 (2001): 26–30.
11. René Dybkäer, "Joint Committee on Traceability in Laboratory Medicine (JCTLM): A Trail-Blazing Endeavour," IFCC News, [on-line] (New York, NY: International Federation of Clinical Chemistry and Laboratory Medicine, 2002 [cited 24 June 2002]); available from Internet: http://www.ifcc.org/news/mar2002/ifccnewsmar02.htm.
12. General Requirements for the Competence of Calibration and Testing Laboratories, ISO 17025 (Geneva: International Organization for Standardization, 1999).
13. In Vitro Diagnostic Medical Devices—Measurement of Quantities in Samples of Biological Origin—Requirements for Reference Measurement Laboratories in Laboratory Medicine, ISO FDIS 15195 (Geneva: International Organization for Standardization, 2002).
Copyright ©2002 IVD Technology