|Regulations & Standards|
The sequencing of the human genome in 2001 brought about rapid development of technologies to measure multiple genes, which resulted in the creation of gene arrays. These tests, and multianalyte protein marker panels, constitute a new type of advanced diagnostic that is rapidly entering the clinical laboratory.1 As a result of the vast information that was made available from the Human Genome Project, great expectations developed that we would be able to diagnose, target, predict, and treat a vast array of diseases more effectively than ever before. A drive to identify those genes and genetic pathways associated with disease produced hope that genetically informed approaches to diagnosis and treatment were imminent.
A recent publication by Francis Collins (NIH) and Margaret Hamburg (FDA), acknowledges these tremendous scientific advances.2 The authors state that there are many challenges ahead to bring these advances to practical reality, including taking steps forward in “basic, translational science and regulatory science.”2 It is likely that this “regulatory science” is aligned with FDA’s past thinking when the second version of the FDA Guidance on In Vitro Diagnostic Multivariate Index Assays (IVDMIAs) was released in July 2007.3
In addition, the emergence of more complex Laboratory Developed Tests (LDTs) under CLIA have proceeded so quickly that FDA is now considering IVDMIA tests in their overall review of LDTs. The IVD industry awaited a final IVDMIA Guidance Document; however, comments by Jeffrey Schuren, Director, CDRH, in June 2010 make it clear that FDA will reconsider its enforcement discretion policy over LDTs. IVDMIAS are likely to be considered within the broader context FDA has in mind for LDTs and the risk-based review process. An FDA-sponsored meeting held last July aired stakeholder views on the impact of increased regulatory oversight of LDTs, including IVDMIAs.
FDA appears concerned about the technological nature of these multi-component tests. Other concerns are how results are analyzed, the robustness of the evidence to demonstrate analytical and clinical performance, and the consistency of inter-laboratory results.
IVDMIAs are comprised of multiple components (typically DNA, RNA, or protein) analyzed using a unique, proprietary algorithm, often transparent to the lab performing the test and to the medical professional that orders it. The majority of those already approved by FDA and commercially available have a clinical application or “intended use” in the area of oncology. With the LDT initiative at FDA, new IVDMIAs made available through a CLIA lab as LDTs exempt from the standard or de novo 510(k) process are likely to come under increased regulatory scrutiny. An examination of these tests provides evidence of FDA approval criteria that set a high standard for analytical robustness and clinical utility.
The first IVDMIA, the MammaPrint system, made by Agendia Inc., is a qualitative IVD test service performed in a single lab outside the United States using a 70-gene expression profile of fresh frozen breast cancer tissue samples to assess a breast cancer patient’s risk for distant metastasis. FDA approved MammaPrint in February 2007 under de novo classification procedures. Because the test was the first of its kind, an FDA approval order was issued. Approval meant it was intrinsically safe and effective-not comparatively safe and effective, as indicated by a traditional 510(k) clearance. Upon approval, FDA issued a “special controls” guidance (21 CFR 866.6040) conferring a de facto Class II status to all genomic breast cancer prognostic tests, thereby facilitating a regulatory review path via a 510(k) rather than the more rigorous premarket approval process.4 Whether this regulatory pathway remains in place after FDA issues new guidance on LDTs in 2011 is an open question.
The MammaPrint system was approved for use in women 61 years of age or younger with early-stage invasive breast cancers (<5 centimeters) that are node-negative and hormone-receptor-positive or -negative.
The test is a custom array chip (Agilent Technologies) using an oligonucleotide microarray platform that assesses the mRNA expression of 70 genes. Analysis is a multistep process: isolation of RNA, DNAse treatment, amplification and labeling, cRNA purification, hybridization to the MammaPrint microarray, scanning the microarray, data acquisition, and analysis. Results are provided as a score that is interpreted to assign the breast cancer low or high risk for metastasis.
MammaPrint was developed based on basic genomic research conducted at the Netherlands Cancer Institute and initially published in 2002.5 The test met multiple analytical laboratory criteria, and the clinical application of the 70-gene array was demonstrated in several published reports.6,7,8,9
The clinical study data submitted in the 510(k) indicated that at five years, low-risk patients have a 95% probability of metastasis-free survival, whereas high-risk patients have a 78% probability of metastasis-free survival.
Following the initial 2007 approval, a subsequent 510(k) clearance supported testing using fresh tissue rather than the originally cleared, fresh-frozen breast cancer tissue, and a separate 510(k) allowed for the addition of 5-year prognostic data for breast cancer patients 61 years of age or older to the original intended use.
In April 2010 the manufacturer recalled MammaPrint because of laboratory-management issues and reporting errors. The manufacturer was required to comply with FDA’s correction and removal regulations.10 The FDA recall notice stated that, during a 6-month period, approximately 15% of MammaPrint results over-reported the chance of metastasis risk as 29% risk of recurrence instead of 10%. No patient injuries were reported during or after this time period; however, FDA expressed concern that this over-reporting of risk, if used by physicians in therapy decisions, might have resulted in patients receiving unnecessary therapy.
Subsequently, FDA approved other IVDMIAs under the de novo 510(k) classification procedure. In August 2008, the AlloMapTest, made by XDx Expression Diagnostics, met 510(k) requirements to emerge as a multigene blood test for use in heart-transplant patients (21 CFR 862.1163, cardiac allograft gene-expression profiling test system). AlloMap Molecular Expression Testing was approved as an IVDMIA single-lab test service, assessing the gene-expression profile of RNA isolated from peripheral blood mononuclear cells (PBMC). AlloMap Testing is intended to aid in the identification of heart transplant recipients with stable allograft function who have a low probability of moderate or severe acute cellular rejection (ACR) at the time of testing in conjunction with standard clinical assessment. The test is for use in patients who are 15 years of age or older and are at least two months (≥55 days) since transplantation. It is performed on blood samples processed to prepare the PBMCs, which are lyzed to release RNA that is stabilized, purified, and converted into cDNA. This cDNA is then mixed with gene-specific primers and probes. The expression of each gene is measured by amplification and fluorescence detection using a qRT-PCR.
The AlloMap test is a panel of 20 gene assays, 11 informative and nine for normalization and/or quality control. A mathematical classifier combines the measured gene expression values into a single AlloMap score between 0 and 40. Each score is associated with a negative predictive value (NPV) and a positive predictive value (PPV), and each value indicates the probability that the patient does not have (or does have) current rejection.
The AlloMap test was developed starting with approximately 25,000 to 30,000 human genes. DNA microarrays were used to discover 252 candidate genes for which the quantity of RNA in blood samples was related to rejection to create the candidate gene-expression markers of rejection. qRT-PCR confirmed 68 of the candidate genes and was used to develop the 20-gene AlloMap gene-expression panel.
The XDx Reference Laboratory conducted numerous precision studies to demonstrate the robustness of the AlloMap test. The clinical validation of the AlloMap test used patient samples and clinical data obtained during the Cardiac Allograft Rejection Gene Expression Observational (CARGO) study. Nine U.S. heart transplant centers enrolled 737 patients who contributed 5,834 blood samples and associated clinical data.11 An initial clinical experience at three medical centers was published in 2006, and the data confirmed the efficacy and performance of the AlloMap test.12
A comparative effectiveness study, the Invasive Monitoring Attenuation through Gene Expression (IMAGE) study, compared clinical outcomes of patients managed with AlloMap to clinical outcomes of patients managed with endomyocardial biopsy. U.S. heart transplant centers enrolled 602 patients who were at least six months post-transplant. The results showed that AlloMap was not inferior to endomyocardial biopsy with respect to clinical outcomes when used to monitor stable, asymptomatic heart transplant patients.13
Tissue of Origin Test
In July 2008, the Tissue of Origin Test, made by Pathwork Diagnostics, was cleared. This microarray RNA profiling test is to be used on clinical, formalin-fixed, paraffin-embedded (FFPE) biopsy tissue to aid in the classification of the origin of the tumor tissue. In June 2010 a second clearance introduced a different specimen and specimen-preparation method, and the algorithm for analysis of the expression data to create a diagnostics report and interpretation.
The test uses microarray technology by Affymetrix Inc. and advanced analytics to measure the gene-expression patterns of challenging tumors, including metastatic, poorly differentiated, and undifferentiated cancer. It is intended to measure the degree of similarity between the RNA expression patterns in a patient’s tumor tissue with the RNA expression patterns in a database of fifteen known tumor types that were diagnosed according to then-current clinical and pathological practice.
Analytically the test was shown to be robust through test protocols including interlaboratory reproducibility, limit of detection, and specificity studies with potentially interfering substances, etc. Clinical validation was performed to assess the predictive capability of the FFPE test in determining the tissue of origin of poorly differentiated, undifferentiated, and metastatic FFPE tumor specimens compared with tissue of origin within the 15-tissue panel. The average percent agreement ranged from 72% (gastric) to 96.5% (breast), and overall agreement with available diagnosis was 88.5%. Published reports show good clinical sensitivity (87.8%) and specificity (99.4%) in frozen tumor tissue.14,15
In September 2009, the OVA1 test (submitted to FDA by Vermillion and later acquired by Quest Diagnostics), was the first de novo 510(k)-cleared IVDMIA that was protein based (21 CFR 866.6050, ovarian adnexal mass assessment score test system). The test was launched in March 2010 by Quest Diagnostics Inc. The test combines into a single score the results of five protein biomarkers that change due to the presence of ovarian cancer. It is indicated for women who are older than 18 years, who have an ovarian adnexal mass present for which surgery is planned, and have not yet been referred to an oncologist. The OVA1 Test is an aid to further assess the likelihood that malignancy is present when the physician’s independent clinical and radiological evaluation does not indicate malignancy.
The five serum biomarkers that comprise the test are prealbumin, apolipoprotein A-1, transferrin, beta-2-microglobulin, and CA125, and they are measured using standard immunoassays. The results are analyzed with software to produce a single result ranging from 1 to 10 to classify the likelihood that a woman’s pelvic mass is cancerous or benign.
The OVA1 test was developed in collaboration with academic medical centers testing more than 2500 clinical samples.16 Extensive analytical studies were done to evaluate repeatability and reproducibility of the OVA1 test result and each of the five component proteins. These studies supported the 510(k).
In early 2007 Vermillion began a multicenter prospective clinical trial to demonstrate the clinical performance of the OVA1 test. Clinical specimens were collected at 27 sites, and test performance was determined based on 516 evaluable subjects who underwent surgery to remove a documented ovarian tumor and for whom a pathology result was available. After surgery, the specimen was examined by a surgical pathologist per routine procedures. The ability of physicians to predict malignancy without the OVA1 test was compared with the ability of physicians and the OVA1 test via dual assessment to predict malignancy. With dual assessment, 80% of cancers missed by clinician impression alone were detected, and the sensitivity and negative predictive value were each more than 90%.
Other IVDMIAs are available as laboratory developed tests (LDTs). These tests have not gone through the FDA approval or clearance process. In select cases, such as with the Oncotype DX breast cancer assay from Genomic Health, test performance is supported by numerous published articles.17, 18 Oncotype DX is another multigene expression test, and its application is predicting the likelihood of chemotherapy benefit as well as recurrence in early-stage breast cancer.
The challenge to scientifically develop and clinically validate an IVDMIA is not for the faint of heart. The applicable 510(k) summaries and published articles on the IVDMIAs reviewed in this article indicate that extensive analytical and prospective clinical testing is required for FDA approval or clearance. Many of these new tests are conducted on tissue rather than on standard serum or plasma samples, thereby requiring new and detailed sample-stability testing protocols. In addition, these tests employ algorithms for the interpretation of analyte measurements to provide a single result, and the analytical process is not fully transparent, thereby posing a regulatory risk. Furthermore, clinical validation requires prospective clinical trials with endpoints comparable to those described in well-designed pharmaceutical trials (e.g., patient clinical status or outcomes), not studies designed as direct “a to b” methods comparison.
For the near term, IVDMIAs will fall under the long-standing risk-based approach for IVDs. This will maintain a level playing field and keep alive the hope that IVDMIAs will not be singled out for excessive scrutiny compared to other single-component IVD tests of comparable risk.
For the long term, when FDA elucidates new regulatory discretion over LDTs, many IVDMIAs are likely to be affected as well. Any new test indications for use that address risk of disease recurrence or the selection of patients that will benefit most from a drug treatment will be viewed as higher risk and will be subject to greater regulatory requirements.
Many IVD developers came away from the July 2010 FDA public meeting on LDTs believing that the CLIA route to new IVDMIA introduction presents more unknowns, given FDA’s commitment to issue a new LDT guidance in 2011. In these new pronouncements, FDA is likely to take a risk-based approach, which has been the precedent. It will allow FDA to triage new product reviews and compliance activities accordingly.
New IVDMIAs will continue to emerge as translational science progresses. These more advanced diagnostics will provide better tools for diagnosis, monitoring, and prognosis of cancer and cardiovascular and other disease, to help manage these complex maladies. Any new FDA “regulatory science” must complement this forward march, not inhibit it, while ensuring safety and effectiveness of new multiparameter diagnostic tests.
1. JC Venter et al., “Sequencing the Human Genome,” Science 291(5507):1304-1351, 2001.
2. MA Hamburg and FS Collins, “Perspective: The Path to Personalized Medicine,” New England Journal of Medicine, July 11, 2010.
3. FDA Guidance on In Vitro Diagnostic Multivariate Index Assays (IVDMIAs). July 26, 2007.
4. Guidance for Industry and FDA Staff---Class II Special Controls Guidance Document: Gene Expression Profiling Test System for Breast Cancer 21 CFR 866.6040. May 9, 2007.
5. LJ van ‘t Veer, H Dai, MJ van de Vijver, et al. “Gene expression profiling predicts clinical outcome of breast cancer”, Nature, 415(6871):530–536, 2002.
6. MJ Van de Vijver, YD He, LJ van’t Veer, et al., “A gene-expression signature as a predictor of survival in breast cancer.” New England Journal of Medicine, 347(25):1999–2009, 2002.
7. AM Glas, A Floore, LJ Delahaye, et al., “Converting a breast cancer microarray signature into a high-throughput diagnostic test,” BMC Genomics 7:278, 2006.
8. M Buyse, S Loi, L van’t Veer, et al., “Validation and clinical utility of a 70-gene prognostic signature for women with node-negative breast cancer.” Journal of the National Cancer Institute 98(17):1183–92, 2006.
9. BS Wittner, DC Sgroi, PD Ryan, et al., “Analysis of the MammaPrint Breast Cancer Assay in a predominantly postmenopausal cohort,” Clinical Cancer Research 14(10):2988-2993, 2008.
10. Code of Federal Regulations, Title 21, Part 7, Enforcement Policy.
11. MC Deng, HJ Eisen, MR Mehra, et al., “Noninvasive discrimination of rejection in cardiac allograft recipients using gene expression profiling,” American Journal of Transplantation 6:150-160, 2006.
12. RC Starling, M Pham, H Valantine, et al., “Molecular testing in the management of cardiac transplant recipients: initial clinical experience,” Journal of Heart and Lung Transplantation Dec. 25(12):1389-95, 2006.
13. MX Pham, JJ Teuteberg, AG Kfoury, et al., “Gene-Expression Profiling for Rejection Surveillance after Cardiac Transplantation,” New England Journal of Medicine 362(20): 1890-1900, 2010.
14. FA Monzon, CI Dumur, “Diagnosis of uncertain primary tumors with the Pathwork tissue-of-origin test,” Expert Review of Molecular Diagnostics 10(1):17-25, 2010.
15. FA Monzon, F Medeiros, M Lyons-Weiler, et al., “Identification of tissue of origin in carcinoma of unknown primary with a microarray-based gene expression test,” Diagnostic Pathology 5(3):1-9, 2010.
16. ET Fung, “A recipe for proteomics diagnostic test development: The OVA1 Test, from biomarker discovery to FDA clearance,” Clinical Chemistry 56:327-329, 2010.
17. S Paik, S Shak, G Tang, et al., “A multigene assay to predict recurrence of tamoxifen-treated, node-negative breast cancer,” New England Journal of Medicine 351(27):2817-2826, 2004.
18. K Albain, W Barlo, S Shak, et al., “Prognostic and predictive value of the 21-gene recurrence score assay in postmenopausal women with node-positive estrogen receptor-positive breast cancer on chemotherapy: a retrospective analysis of a randomized trial,” Lancet Oncology 11(1):55-65, 2010.
Katie M. Smith, PhD is principal at Katie Smith Consulting (San Diego). She can be reached at firstname.lastname@example.org.