Assay Performance - Analytical Validation

Assay Performance (Analytical Validation)

An assay for a biomolecular analyte destined for submission to the FDA must be analytically validated prior to moving to clinical validation. The process essentially establishes the performance characteristics of the assay (e.g., sensitivity, specificity, and precision) based on its context of use. Assays are imperfect in that they rarely distinguish a disease from a normal state with 100% accuracy. To determine the extent of overlap requires the construction of Receiver Operator Characteristic (ROC) curves; the area under the ROC determines the accuracy of the test (the capacity of the test to distinguish a disease versus normal state). This information is necessary to establish "cut points" for the assay, which are required by physicians to make clinical decisions on differing health states. (a ROC, used to set cut points, is essentially a plot that captures true positive rate against the false positive rate of an assay.)

Obviously in a perfect test, there would be no overlap between results from healthy patients versus those with disease, as there would be a simple bi-modal distribution. However, in the real world biomarker/diagnostic assays and sophisticated mathematical approaches are often required to evaluate the overlap between populations. This step is achieved by establishing the relationship(s) (and trade-offs) between the sensitivity and specificity of the assay by analyzing the area under the ROC curve. This approach enables setting cutoff points based on understanding the trade-offs between these two parameters (e.g., increasing sensitivity while decreasing specificity). This aspect of analytical validation when combined with clinical data ultimately drives how to best employ the test in clinical decision making. 

An important aspect of determining cut-points to support clinical decision making is determining the positive and negative predictive values for the assay. In other words, what is the likelihood of a false positive or false negative result, and what does that ultimately mean in terms of impact. The answer lies not just in the specificity and sensitivity of the assay but also depends on the defined and declared context of use ("fit-for-purpose”) of the assay. For example, false positives are often viewed in terms of impact on quality of life for the patient—and increasingly on the cost impact. For example, if a test has low specificity, perhaps due to confounding pathologies, the elevated false positive rates can produce controversial results such as currently observed with PSA screening. False negatives can be even more damaging in that the disease state may be misdiagnosed, and in case of acute incidents such as a myocardial infarction, may not enable predicting a short-term cardiovascular crisis. 

Analytical validation of a biomarker/diagnostic assay is complex, and the FDA requirements must be fully understood relative to the regulatory pathway chosen (PMA or 510(K)) along with the clinical context of use (FDA Biomarker Qualification Program). How will the data from the assay be used to make clinical decisions? For example, for molecularly based assays, it is critical to establish the normal range of results for an assay in a normal disease-free population. If the assay is quantitative, its linearity must be established; the linear range must be known and documented as this will define the range of values that the analyte can measure; and the precision (the ability to obtain the same result from multiple tests) must be demonstrated and any deviations understood and documented. Overall, analytical validation is a complex and often difficult barrier in biomarker development, but all requirements must be met before moving a biomarker/diagnostic assay forward to qualification and/or clinical validation (depending on the context of use).

Preforming to the Standard

Assay Development & Performance