How Probable Cases Changed Through the COVID-19 Pandemic

When SARS-CoV-2 (the virus that causes COVID-19) first emerged, public health officials across the US had some quick work to do. First, they needed to learn enough about the virus to understand how and when to test for its presence. Next came manufacturing needs, and the distribution of tests. Data pipelines had to be built, to funnel diagnostic test results up from labs, and all of this needed to happen as fast as humanly possible.

What health officials also knew was that they needed a way to tell if someone probably had COVID-19—a way to account for the situations of virus spread among individuals who weren’t being tested. So, in April 2020, the CDC adopted a definition for probable COVID-19 cases. The definition was based on exposure and symptoms, and it was an attempt to signal to state health officials that there could be one standardized way to interpret a probable case.

In those early months, probable cases comprised only a small fraction of the national case count, likely because the probable case definition relied on contact tracing and symptom tracking, which were difficult for states to perform at scale. Then, as antigen tests became more widely available and able to quickly identify COVID-19 with a high degree of certainty, the CDC expanded the probable case definition to make room for antigen test results, even in the absence of exposure and symptoms. It was only with this expanded definition, adopted in August 2020, that probable cases made a dent in the national case volume.

As we’ve seen across the range of COVID-19 metrics, though, just because there was (some) federal guidance didn’t mean states were following it. With probable cases, how and when states chose to incorporate the CDC’s guidance varied. By now, most states do use a probable case definition that includes antigen positive individuals, but there are still a few outlier states using different or unclear definitions. It is this veneer of uniformity— these quiet definitional discrepancies and the varying journeys states took toward similar conclusions—that this piece will explore.

Current definitions: confirmed, probable, and suspect cases of COVID-19

To track any infectious disease, you need what’s called a “surveillance definition,” a set of criteria used to measure the spread of a disease on a large scale (rather than making individual clinical diagnoses). The Council of State and Territorial Epidemiologists (CSTE) sets the surveillance definitions used by CDC for many infectious diseases, and COVID-19 was no exception: The organization has issued two iterations of definitional guidelines for COVID-19 cases since the pandemic began, in April 2020 and August 2020, that were adopted by the CDC shortly thereafter. The current guidelines call for states and the CDC to track three categories of COVID-19 cases—confirmed cases, probable cases, and suspect cases. The CSTE recommends that states track confirmed and probable cases publicly, while suspect cases are meant as an internal category for disease surveillance.

Confirmed cases

Confirmed cases of COVID-19 are the most straightforward: When a PCR test comes back positive, the individual is declared a confirmed case. PCR tests are often referred to as “gold standard tests” for COVID-19 because of their high sensitivity, which means PCR tests are able to pick up even very small amounts of viral RNA, even early in an infection. There is a low chance for false positives.

Probable cases

Probable cases can be identified in three main ways. 

The primary way currently uses antigen testing, which is generally known to have lower sensitivity than PCR or other viral RNA detection tests. A person with a low level of viral infection might get a false negative result. However, antigen tests are highly specific tests, meaning that false positive results are unlikely. Because these tests are cheaper than PCR tests and produce results much more quickly, they can be used to rapidly identify individuals with high levels of virus who are likely to infect others.

The second way to identify a probable case is through symptoms (clinical criteria) and likely exposure (epidemiological linkage). In other words, if an individual is experiencing certain symptoms known to be associated with COVID-19, and they were in a situation where they had known exposure to a probable or confirmed case, they could be classified as a probable case without a test result or with a test result pending.

Finally, deceased individuals with COVID-19 listed on a death certificate as the presumed cause of death or significant condition contributing to death and without a positive laboratory result can be classified as a probable case. This is different from being classified as a probable COVID-19 death—though the COVID-19 case definitions are often used by jurisdictions to determine COVID-19 deaths as well. 

The diagram below illustrates the specific combinations of criteria that the CSTE says should be used to classify a case as probable in the absence of a laboratory test result:

Suspect cases

In the hierarchy of case classification, suspect cases represent individuals with the lowest level of evidence that they are presently infected with COVID-19. These individuals may have a positive antibody test—indicating past infection, but probably not a current infection. The CSTE recommends that states and the CDC keep track of these individuals internally but not include them in public COVID-19 case counts.

What changed? The evolution of case definitions

As mentioned, the current CSTE case definition is the second iteration, an update on the original one from April 2020. While the definition of a confirmed case did not change with the updated version, the definition of a probable case was adjusted to reflect the growing understanding of COVID-19’s clinical presentation, patterns of spread, and changing availability of lab tests.

To understand the changes, it’s helpful to know how CSTE case classifications work. Each case classification is made up of a set of criteria: laboratory criteria, clinical criteria, epidemiological factors, and the vital records criteria. This can all be combined to standardize what it means to identify a confirmed or probable case of COVID-19.

Laboratory criterion changes

Between April and August 2020, the most significant changes were to the laboratory criterion: 

1. Changes in how the criterion can be used: Beginning in August, laboratory evidence on its own—absent any other criteria—indicated a probable case. (The original April 2020 definition required clinical or epidemiological criteria to be satisfied, as well, in order to be considered a probable case.) 

2. Changes to the criterion itself: Antibody tests were removed as laboratory evidence, leaving only antigen tests. The internal (not public) suspect cases category was added as a new category for patients who tested positive on antibody tests.

As antigen tests became more prevalent, they proved to be highly reliable at indicating active COVID-19 infections. Meanwhile, unlike both antigen and PCR tests, antibody tests identify prior rather than current infections. They detect the presence of specific antibodies, and according to the CDC, it can take one to three weeks after the infection for your body to produce detectable antibodies. For that reason, antibody tests are not appropriate for diagnosing probable or confirmed cases of COVID-19.

Epidemiological and clinical criteria

The epidemiological linkage criteria were adjusted to reflect a better understanding of COVID-19 transmission:

• The updated definition does not require close contact with someone who is sick with a COVID-19-like illness and linked to a confirmed COVID-19 case, reflecting an understanding that asymptomatic transmission of COVID-19 is common.

• In addition, travel to an area outside of the US with sustained, ongoing community transmission of COVID-19 was removed as a necessary epidemiological linkage factor, as most of the United States itself could be described as having sustained ongoing community transmission.

Definitions of clinical criteria were also modified as COVID-19 symptoms became better understood. Most notably, in the original probable case definition, the loss of taste and smell needed to be present with other symptoms, whereas the updated definition counted the loss of taste and smell (olfactory disorder) as a COVID-19 symptom on its own. 

In addition, the symptoms listed in the original definition were almost exclusively respiratory. However, as more and more people presented with gastrointestinal symptoms and extreme fatigue, these additional symptoms were also added to the list of possible COVID-19 symptoms. 

How did these changes affect the data?

The Covid Tracking Project’s work has focused on understanding and contextualizing COVID-19 data, so we wanted to take a closer look at the probable cases states reported. Namely, we wanted to see what, if any, impact the CSTE’s definitional changes made to the data that was reported. Our research was driven by these questions: 

• We knew when CSTE updated the case definition for probable cases, but when did states actually adopt the updated definition?

• Did the updated definition have an impact on the number of probable cases that were being reported?

• Were those effects the same across states?

Our analysis looked at data gathered during our year of daily data collection, which stopped on March 7, 2021, and shows that as of March 7, 2021 the reporting of probable cases was still very uneven across the country. Six states—California, Iowa, Maryland, Missouri¹, Nevada, and the Northern Mariana Islands—did not report probable cases. Among states that did report probable cases, most converged on using the updated case definitions in the fall of 2020, but there’s regional variation in how the definitions have been implemented and interpreted. 

States’ adoption of CSTE-02 definitions

Of the 55 states and territories we tracked, 36 had been providing separate counts of confirmed and probable cases as of March 7, 2021.² (When we previously wrote on this topic, in June 2020, we identified only 25 states providing separate counts.)

Of the remaining states, seven reported combined confirmed and probable cases, six states reported confirmed cases only, and six states reported cases with an unclear definition.

Of the 36 states that report confirmed and probable cases separately, 35 used the updated (CSTE-02) definition as adopted by the CDC in August 2020. One state, Vermont, uses a custom probable case definition. Most states that switched to the updated definition moved in September and October 2020, though two of those states, Delaware and Mississippi, still display the original case definition on their dashboards. (We reached out to officials at Delaware’s Department of Health and Social Services, who confirmed that they are using the updated definition and said they will be updating the language. A spokesperson from Mississippi’s Department of Health said they weren’t aware the previous case definition was on the state’s website but confirmed use of CSTE-02.)

Trends in probable case reporting

It’s important to remember that the number of COVID-19 cases will never be a measurement of the number of infections, in part due to the fact that many people across the country, and particularly people in communities hit hard by COVID-19, might not have had easy access to PCR or antigen tests, or might have decided against seeking one. Still, probable case numbers aren’t insignificant. Given the trends we’re seeing from states that do separately report probable and confirmed cases, we can assume that in the six states reporting confirmed cases only, large numbers of would-be probable cases aren’t making it into any public-facing official count. Indeed, our analysis shows that starting in the fall of 2020, an overall increase in the number of probable cases appears in almost every state. 

Our analysis also looked at whether the proportion of probable cases as a share of total reported cases changed over the course of the pandemic. An increase in the proportion of probable cases would indicate that states were more efficiently tracking probable cases compared to summer of 2020. If the proportion of probable cases did not increase in the total number of cases, that means the increase in the number of probable cases was simply an artifact of increased cases as a whole.

We looked at the 21 states and territories that reported probable and confirmed cases separately and for which we’ve collected continuous data from August 5, 2020—the day the updated CSTE definition was issued—until March 7, 2021. We found a growing share of probable cases, by as much as 20x in a few states:

Differences in interpretation and use between states

There’s no denying that the decision to count antigen positive test results as probable cases contributed to the growth of probable numbers, but the extent to which antigen tests alone drove that growth isn’t easy to parse. Given the varying definitions across states, it’s especially difficult to interpret this data. In some states, a probable case of COVID-19 translates to an antigen positive test result unequivocally; in others, broader symptomatic and epidemiological criteria has played a role in defining the probable count.

(A quick word of caution, too: It’s important to remember that with many antigen tests happening outside of traditional healthcare settings, the data reporting pipelines might not be set up to actually send all antigen test results to state officials.)

There are nine states that report both probable cases and either antigen positive individuals or antigen positive tests—and report them all smoothly enough and for a long enough period that we can spot meaningful trends.³ Out of those nine states, five—Arkansas, Connecticut, Tennessee, Maine, and Virginia—have probable cases that very closely track trends in antigen positive test results. 

Our suspicion is that in these states, officials are identifying probable cases only (or almost entirely) through antigen tests, and we suspect these decisions were made because the scale of the pandemic grew too large to track symptomatic and epidemiological criteria in a meaningful enough way.

Four additional states—Florida, Georgia, Indiana, and Louisiana—have notes on their websites about using only antigen tests to identify probable cases. And one more state, Massachusetts, decided to use only antigen tests and death certificates on September 2, 2020, cutting cases that only met the clinical and epidemiological criteria out of its count. A note from the state about the definitional shift provided some insight into their decision: “In order to provide a single set of consistent data for tracking COVID-19 in Massachusetts, today’s data includes only probable cases identified through antigen testing or death certificates. These criteria are the most objective and able to be applied over time.” 

That day, we noticed a sharp decrease in several data points for Massachusetts; the state’s overall case count dropped by more than 7,000.

There are four states—Kentucky, Ohio, South Carolina, and Texas—where trends in antigen positives do not closely track probable case counts. In three of those four states, all except South Carolina, the antigen test numbers have largely been lower than probable cases.⁴ There are a handful of possible explanations for these discrepancies. It is possible that some of the difference could be due to sourcing between test and case metrics: for example, states could be only reporting electronically transmitted antigen tests but also including faxed reports in probable case counts. But most likely, this difference reflects use of the symptomatic and epidemiological criteria—as Texas officials confirmed it did in their state.

You might think that the decision to track only antigen-identified probable cases would result in an undercount of cases overall—especially given that removing the symptomatic and epidemiological criteria initially led to such a large drop in Massachusetts. But probable cases grew as a share of total cases by a comparable amount in these two groups of states between November 1, 2020 and March 7, 2021—approximately doubling in both the states where probable cases don’t track with antigen positives and in the states where they do.

The near identical growth in both groups leads us to believe that state testing strategies played an important role in shaping probable case counts: The states that dropped the symptomatic and epidemiological criteria likely have strong enough antigen testing programs to pick up on those individuals through testing, anyway. Identifying probable cases from just contact tracing and symptom exposure was a laborious task for state health departments, especially at scale, so with the rise in the availability of antigen tests, health departments had a more efficient method. 

This research serves as a reminder that even if all states were to be using standardized definitions, comparisons would still be fraught due to differences in infrastructure and testing strategies. 

Probable cases are real COVID-19 cases

Like so many COVID-19 data points, the probable cases metric is complex, and to interpret it accurately requires an understanding of its definition, practical application, and evolution. This is all especially important in a context where the difference between the number of cases and the number of positive PCR tests has become a common source of confusion and of disinformation—with some alleging that counting probable cases inflates case numbers. 

The reasons for tracking probable cases aren’t obvious, nor are they consistent across states, but the differences stem from legitimate decision-making rooted in epidemiology, not from anything nefarious or negligent. Given the high specificity of COVID-19 antigen tests, it’s entirely reasonable for states to be counting these positive test results as probable cases. In fact, in many countries, an antigen positive constitutes a confirmed COVID-19 case.

The share of probable cases in the US’s overall COVID-19 case count didn’t grow because state health officials were itching to count just anybody. It grew because as the country’s understanding of COVID-19 and capabilities around testing for it evolved, so too did the ability to paint a fuller picture. There is no reason to be concerned that probable case counts are inflated; if anything, both confirmed and probable case counts, largely at the mercy of inadequate testing, underestimate the true scale of COVID-19 infections in the United States. 

Additional research: Michal Mart, Jessica Malaty Rivera

Graphics: Dave Luo, Kara Schechtman, Lauran Hazan