Ideal Samples for Validating Next-Generation Sequencing-Based Plasma Circulating Tumor DNA Assays

Over the past decade, liquid biopsy-based testing, especially using next-generation sequencing (NGS) to detect circulating tumor DNA (ctDNA) in plasma, has been increasingly adopted in clinical practice for cancer screening, diagnosis, therapy selection, and treatment response monitoring. This is due to the advantages it offers over tissue biopsy: less invasive nature, lower cost, real-time information on the state of the tumor and, in some cases, the ability to overcome the issue of tumor heterogeneity.¹ However, only two NGS-based plasma ctDNA tests have been approved by the U.S. Food and Drug Administration (FDA) to date.² These tests can identify single nucleotide variants (SNV), small insertions/deletions (indels), and structural alterations in cancer-related genes, as well as assessing tumor mutation burden (TMB) and microsatellite instability (MSI).² The test results may have an immediate impact on the patient’s disease management for early cancer detection, staging, early relapse detection, real-time monitoring of therapeutic efficacy, and detection of therapeutic targets and resistance mechanisms. However, to become a part of routine clinical care, NGS-based plasma ctDNA assays must be thoroughly validated to demonstrate their analytical validity, clinical validity, and clinical utility as required by the FDA’s guidance for Medical Devices.³

Recently, the Association for Molecular Pathology (AMP) and the College of American Pathologists (CAP) jointly published recommendations to facilitate assessment of the analytical and clinical performance of plasma ctDNA assays.⁴ Among the seven recommendations for clinical laboratory testing of ctDNA, emphasis was put on validation of the assay performance characteristics. These include sensitivity, specificity, positive predictive value, negative predictive value, accuracy, and concordance, not only as a whole of the assay but also for each variant or at least each variant class, including SNV, indel, copy number variation (CNV), SV, or mutational signature (MSI, TMB, or homologous recombination deficiency). Similarly, the validation of limit of detection (LoD) of the ctDNA assays should be performed for each variant or variant class. It was also emphasized that appropriate controls should be included to remove potential sources of assay interference and sources of result interpretation error, such as variants from clonal hematopoiesis of indeterminate potential (CHIP) and the presence of germline variants. Moreover, laboratories were recommended to define and describe orthogonal method confirmations as well as comparison to tissue-based specimens. 

To carry out a successful validation, laboratories need to have a thoroughly laid-out plan and prepare all needed components, such as reagents, instruments, software, operators, locations, and most importantly validation samples, in advance. When preparing validation samples, clinical specimens are the recommended first choice, but the following considerations should be kept in mind. 

For Validation of Sensitivity and Specificity

When putting together all required tests recommended by the AMP/CAP guidelines plus necessary replicates recommended by BloodPAC,^4,5 a large number of clinical specimens will be required in order to accumulate enough variants of each variant class for validating their individual sensitivity and specificity. The amount of ctDNA needed to accomplish even one aspect of the validation will be huge and unachievable with a single patient specimen, in which ctDNA accounts for only 0.1–10% of the total circulating cell-free DNA (cfDNA), with normal plasma cfDNA levels ranging from 10–100 ng/ml.⁶ Therefore, pooled clinical specimens may be needed to provide enough ctDNA for all aspects of the validation.  

For Validation of LoD

Another challenge of ctDNA assay validation is that the allele frequencies (AFs) of variants in the clinical specimens may not be at the same level. Therefore, not all variants present are informative, especially for validation of LoD, which requires assessing variants with AFs around the targeted LoD. Again, accumulating enough variants of each variant class at a similar AF for LoD validation requires a large number of clinical specimens.  

For Validation of Concordance

Concordance studies of plasma ctDNA with tumor tissue biopsy have a few inherent limitations, which may not be resolved using the selected sample types. 

• • Tissue biopsy and liquid biopsy may not be collected at the same time-points, which can result in different variant profiles due to the dynamic nature of a tumor that is constantly evolving under selective pressure from different clinical interventions. In contrast, ctDNA is easily obtainable through collection of multiple specimens corresponding to clinically important time points, such as at baseline diagnosis, after surgical resection of a tumor, or at progression. 
  • Tissue biopsy usually goes through a formalin fixed paraffin embedded (FFPE) process, which can cause DNA damage and introduce extra genomic alterations not existing in liquid biopsy. 
  • Tissue biopsy is usually collected at a single anatomic location, which may fail to capture a global picture of disease, thus missing the genomic profiles of tumor cells present at other sites due to the heterogeneous nature of the disease or possible metastasis. ctDNA may be able to better capture disease heterogeneity, since any genomic variants originating from all disease sites across the body can be sampled via liquid biopsy.⁵ 

The above challenges clearly demonstrate how critical appropriate samples are in ctDNA assay validation. While all other components of the validation are relatively easy to assemble, sourcing samples takes most of the time and effort. Thus far, the lack of appropriate types or numbers of clinical specimens has been the major limitation hindering ctDNA assay validation. These limitations inherent in clinical specimens can be mitigated with well characterized contrived samples. It is anticipated that ideal contrived samples with the following characteristics will greatly facilitate the validation of the NGS-based plasma ctDNA assays. 

• The contrived sample will have a normal human genomic background with an appropriate number of spike-in/knock-in variants in cancer-related genes sufficient for statistically significant validation study. 
• The DNA of the contrived sample will have a similar profile to the plasma ctDNA. 
• An FFPE format of the unfragmented contrived sample will be also available to act as a tissue biopsy concordance control. 
• Wild-type (WT) controls without the spike-in/knock-in variants in ctDNA format and FFPE format will be available to act as background variant controls. 
• The spike-in/knock-in variants in the contrived samples, both the plasma ctDNA format and the FFPE format, will include SNV, indel, CNV, SV, and mutational signatures (MSI and TMB), and each class of variants will be added at an appropriate number for statistical validation purposes. 
• The spike-in/knock-in variants in the contrived sample will have a panel of at least 5 dilutions around the targeted LoD⁵, which will be titrated according to the variant class, usually higher for CNV and SV. Therefore, LoDs for each variant class can be identified with a single contrived sample, which is created with appropriate numbers of each variant class for statistical analysis. 
• The contrived samples should be readily available and exhibit consistent performance among batches so that re-validation of the plasma ctDNA assays should be comparable to the original ones when the same contrived samples are used. 

Contrived samples or reference materials have been recommended for use in NGS-based plasma ctDNA assay validation guidelines from AMP/CAP and BloodPAC.^4,5 While ideal contrived samples or reference materials mimicking the patient samples and fulfilling all the requirements of ctDNA assay validation are yet to be created, a few applicable ones are already commercially available depending on your validation goals. At LGC Clinical Diagnostics, we have created several ctDNA reference materials (with more under development) to target different cancer types, including pan-cancer, solid tumor, lymphoma, and myeloid malignancies, or different study purposes, such as ctDNA mutation profiling, minimal residual disease monitoring, TMB scoring, and methylation screening. The newest version of ctDNA reference material, to be released soon, has 93 variants, including 43 SNVs, 22 indels, 12 CNVs, 10 SVs, and 6 MSI biomarkers.

Learn More

You can learn more about those ctDNA reference materials, download the poster.

New in 2023!

Take a look at the Seraseq^® products that were launched this year by clicking here, or on the button below.

References: 

1. Armakolas, A., Kotsari, M., Koskinas, J. Liquid Biopsies, Novel Approaches and Future Directions. Cancers 2023, 15, 1579. https://doi.org/10.3390/cancers15051579. 
2. https://www.fda.gov/medical-devices/in-vitro-diagnostics/nucleic-acid-based-tests. 
3. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/in-vitro-companion-diagnostic-devices. 
4. Christina M. Lockwood, Laetitia Borsu, Milena Cankovic, et al. Recommendations for Cell-Free DNA Assay Validations: A Joint Consensus Recommendation of the Association for Molecular Pathology and the College of American Pathologists. Articles in Press. The Journal of Molecular Diagnostics. DOI: https://doi.org/10.1016/j.jmoldx.2023.09.004. 
5. Godsey JH, Silvestro A, Barrett JC, et al. Generic Protocols for the Analytical Validation of Next-Generation Sequencing-Based ctDNA Assays: A Joint Consensus Recommendation of the BloodPAC's Analytical Variables Working Group. Clin Chem. 2020 Sep 1;66(9):1156-1166. doi: 10.1093/clinchem/hvaa164. PMID: 32870995; PMCID: PMC7462123. 
6. Fleischhacker M, Schmidt B. Circulating nucleic acids (CNAs) and cancer—a survey. Biochim Biophys Acta (BBA)-Reviews on Cancer. 2007;1775:181–232. https://doi.org/10.1016/j.bbcan.2006.10.001.