Factors including a flawed regulatory approach and false marketing by test developers led to a troubled roll-out of SARS-CoV-2 serology testing this spring, officials at the US Food and Drug Administration have acknowledged.
In a recent commentary in the New England Journal of Medicine, Jeffrey Shuren, director of the Center for Devices and Radiological Health (CDRH) at FDA, and Timothy Stenzel, director of the FDA’s Office of In Vitro Diagnostics and Radiological Health, recounted the agency’s decisions around SARS-CoV-2 serology testing and how an overly lax regulatory stance led to test misuse and confusion within the medical community and population at large.
Serology tests look for the presence of host antibodies against disease, which can indicate whether a person has been exposed to an infectious agent and potentially whether they have developed a measure of immunity against it.
Development of serology testing for SARS-CoV-2 began to ramp up in the early months of the pandemic. On March 16, 2020, FDA announced that it would be substantially loosening its regulations for SARS-CoV-2 serology tests, noting that it “does not intend to object to the development and distribution by commercial manufacturers or development and use by laboratories of serology tests to identify antibodies to SARS-CoV-2,” provided that test makers have validated their assay and notified the FDA of the assay. Additionally, test developers would need to include with the assay notice that the test has not been reviewed by the FDA, that false negatives and false positives are possible, and that the results should not be used alone to determine infection status.
Test developers who intended for their serology assays to be used alone to determine infection status and/or who were developing tests for use in CLIA-waived environments were still required to take their tests through the FDA’s Emergency Use Authorization process.
A primary use for serology testing was for public health efforts tracking SARS-CoV-2 infections and looking at the prevalence of the virus in various communities. More problematic was the use of such tests to diagnose infections in specific individuals. Because of the relatively low prevalence of the virus in many of the populations where such testing was being done, false positive results were common, limiting the usefulness of this kind of testing while potentially giving people false assurance that they had been infected and recovered from infection.
Shuren told 360Dx that, in fact, the agency did not expect serology tests to be used for diagnosing COVID-19 given that serology testing is not typically used for this purpose. “There are some circumstances in which they have been used, but typically they don’t play much of a role,” he said.
Serology tests “are much more for surveillance purposes to see how many people may have been infected,” Shuren said, noting that early in the pandemic “there were a lot of open questions, such as do people develop immunity if they have SARS-CoV-2, and then how long does that immunity last for?”
FDA saw serology testing as key to helping answer these kinds of questions, he said. “So, the [March 16] policy was really intended to allow those tests to get out onto the marketplace more quickly to help answer those questions.”
The policy, however, led to a raft of serology tests — some with questionable performance characteristics — that, as Shuren and Stenzel put it, “flooded” the market following the agency’s March 16 announcement.
“One of the early problems we observed … was that developers were claiming a certain performance, sensitivity and specificity, but we were getting user complaints that the tests weren’t performing to that level,” Stenzel said. “So, there were issues with either the validation of these tests and/or the performance of at least some of the tests.”
Additionally, they noted, in early April, “government officials began touting the potential usefulness of [serology] tests for reopening the economy, and insurance coverage was provided for uses not supported by science and not in keeping with the limitations that the FDA had laid out.”
“We didn’t expect to see what happened at different levels of government, in the promotion of those tests,” Shuren said. “At the federal level, state level, local level, you had people who were latching on and promoting it for opening up the economy, and that started to drive a lot of the demand.”
On April 17, FDA communicated to healthcare providers that some serology test-makers were falsely claiming their products were approved or authorized by the agency, and on May 4, FDA changed its policy to require serology test manufacturers to take their tests through the Emergency Use Authorization process. It also provided performance recommendations for specificity and sensitivity for serology tests.
According to the NEJM commentary, as of February 1, 2021, FDA “had removed listings for 225 tests from our website, issued 15 warning letters, and placed 88 firms on import alert for violations.”
Shuren said that the warning letters FDA issued with regard to serology test were all for tests being sold outside the CLIA lab setting, mainly for tests being promoted for point-of-care or home use against the agency’s policy.
Ultimately, the agency concluded that the March 16 serology guidelines were a mistake. “Knowing what we know now, we would not have permitted serology tests to be marketed without FDA review and authorization, even within the limits we initially imposed, Shuren and Stenzel wrote. “Although other factors may have driven unauthorized products to flood the marketplace, our March 16 policy allowed it to happen.”
The authors also provided a number of recommendations for how test evaluation and other aspects of pandemic management might be better handled in the future. For instance, they highlighted FDA’s collaboration with the National Cancer Institute to evaluate the performance of serology tests as an approach that might be useful both in future pandemics and as part of the routine regulatory process.
Shuren said that the NCI arrangement allowed FDA to have “a mechanism for at least a postmarket evaluation,” of the tests allowed onto the market through the March 16 pathway.
The agency has also used the process to inform premarket evaluations, he added.
“We think the NCI evaluation has really been an effective approach for providing that kind of independent evaluation of the serology tests and really determining which ones are accurate and which ones are not,” Shuren said, adding that the experience suggested there is value to having that capability within the federal government either through a body like NCI or select outside labs doing such evaluations on the government’s behalf.
“If that is done right, it could also help speed the evaluation of novel tests and the availability of novel tests that are shown to be accurate and reliable,” Shuren said, noting that this could in the case of future outbreaks help speed needed diagnostics to market.
He highlighted the example of South Korea and its rapid rollout of diagnostic testing for SARS-CoV-2.
“South Korea had established that capability as part of their pandemic planning,” he said. “That was something the US had not done before, and one of our suggestions coming out of this is we should really be thinking about having the US government in a position to do that.
“FDA doesn’t do it. We are reviewing the evidence for a test, not evaluating the test. But building that capability elsewhere could be very helpful,” Shuren said.
This story first appeared in our sister publication 360Dx.