The “Good” Metric Is Pretty Bad: Why It’s Hard to Count the People Who Have Recovered from COVID-19

In November, The Covid Tracking Project stopped reporting recovery figures for the United States as a whole, and today we are also removing many, though not all, of the state-level “recovered” values from our website. 

Why remove the one “good” metric? It’s not that we want to report only bad news: It’s that we want, above all, to provide accurate and meaningful information. Unfortunately, when it comes to recovery data at both the national and state levels, accurate and meaningful information is hard to come by. 

There are several reasons behind our decision to remove this data from our website. First, several states and territories, including large states like California and Florida, don’t report any kind of recovery data, and it doesn’t make sense to report a national total that excludes so much of the country. The summed national recovery figure we were reporting was certainly far lower than the true number of people who have recovered from COVID-19, making the already bleak pandemic look even bleaker than it is. 

A second and critical reason is that “recovered” has no standard definition, and states report it in many different ways. It’s therefore misleading to compare one state’s reported number of recovered people to another state’s, or to add those numbers together to derive a national total, or to label them all “recovered.”

Just as importantly, many people who have had COVID-19 and have lived to tell the tale—and who are often categorized as “recovered”—don’t consider themselves to have actually recovered. COVID-19 can have many long-term health consequences, and none of the definitions for counting people who have “recovered” from COVID-19 account for latent or ongoing health issues that can be caused by COVID-19. Children who develop Multisystem Inflammatory Syndrome because of COVID-19 and “long-haulers” who continue to suffer worrying symptoms months after first falling ill with COVID-19 are often included in recovery statistics, since not all pandemic-burdened public health departments have the resources to do the individual follow-up investigations that they would ordinarily do for an infectious disease. Moreover, when public health offices do conduct individual case investigations, many COVID-19 patients do not respond to inquiries, leaving case investigators in the dark about the process of convalescence—the sometimes slow and always individual voyage back to health. Determining how many people have recovered from COVID-19, then, is currently more like trawling with a net than fishing with a pole: Every attempt dredges up a lot of scaly things we don’t want.

Here’s a deeper dive into the muddy waters of COVID-19 recovery definitions. 

No direct guidance from the CDC 

The CDC has not provided an official definition of what it means for a COVID-19 patient to recover in the sense of returning to a pre-COVID-19 state of health, but it does provide some guidance on when COVID-19 patients no longer need to be isolated. Here’s the CDC criteria for “discontinuation of Transmission-Based Precautions” for persons with COVID-19:¹

In this guidance, the CDC recommends using a “symptoms-based strategy,” which calls for releasing people with mild cases of COVID-19 from isolation ten days after their illness began if and only if their symptoms have improved without the need of medication.² Those who are asymptomatic or immunocompromised should still be tested to see if they have any remaining virus in their system, but they should consult with local health experts instead of relying only on negative test results.

A key aspect of these CDC guidelines is that they are aimed at controlling infection, not at judging health. This is an important distinction, especially given the demonstrable long-term health effects of COVID-19 that extend beyond the respiratory symptoms stated in the guideline. Many states use these CDC definitions to inform their own reported data of how many people have “recovered” from COVID-19, which means that states are really reporting the number of people who are no longer infectious, not the number of people who have clinically recovered. Using the word "recovered" as synonymous with "non-infectious" is common in epidemiology. For example, in the SIR model frequently used by infectious disease researchers to predict the spread of disease, the “S” stands for “Susceptible” people, the “I” stands for “Infected” people, and the “R” stands for either “Recovered” or “Removed”—that is, people who can no longer give others the disease. However, the usage of the term doesn’t capture the full spectrum of health issues experienced by former COVID-19 patients. 

No consistent definition of “recovered” 

As with many other COVID-19 metrics, different US jurisdictions rely on different definitions for reporting recoveries. An investigative journalism project at the University of Illinois Urbana-Champaign covered this lack of standards back in July, as did the Wall Street Journal in October, but the situation is constantly changing. Some states and territories have still not adopted the CDC’s mid-July recommendation to primarily take symptom improvement into consideration when estimating how many people are no longer infectious, some states have begun tracking recovery data for “probable” cases of COVID-19 identified by rapid antigen tests, and several states that once reported recovery data have recently stopped.

Here, we provide a table of current state and territory recovery definitions for states that report some kind of recovery data. (To see the exact sources of the data itself, as opposed to the definitions, see the Data Sources page on our website.) Our analysis relies exclusively on publicly available documents from official websites. 

Among the 48 jurisdictions that have reported a version of a “recovered” value, available definitions generally fall into one of these categories: 

• Days since diagnosis/onset: This category of definitions bases “recovery” on a certain number of days—generally between 14 and 30—after a positive test result or symptom onset where the patient has not died. For example, the state of Pennsylvania reports that it calculates “recovered” statistics as follows: “Individuals who have recovered is determined using a calculation, similar to what is being done by several other states. If a case has not been reported as a death, and it is more than 30 days past the date of their first positive test (or onset of symptoms) then an individual is considered recovered.” Eighteen jurisdictions have provided definitions with similar criteria based on a certain amount of time since the start of a person’s illness. This is the most common type of recovery definition among US states and territories.

• Symptom improvement: These definitions, which often resemble (or are exactly equivalent to) the CDC’s multi-level guidance for releasing patients from isolation, include information about whether a patient’s COVID-19 symptoms have improved. For example, Maine has defined recovery as “a person [who] has met the ‘released from isolation requirements’ defined by the Federal CDC. The requirements are: at least 1 day (24 hours) have passed since recovery, defined as resolution of fever without the use of fever-reducing medications and improvement in symptoms; and, at least 10 days have passed since symptoms first appeared.” Fourteen jurisdictions have provided definitions with similar criteria that take into account whether a person’s symptoms have improved.

• Hospital discharged: This “recovered” definition simply refers to people diagnosed with COVID-19, hospitalized, and then discharged from the hospital. For example, Virginia describes this as “the total number of confirmed COVID-19 cases who have been hospitalized and discharged.” States whose recovery data is defined by hospital discharge in our dataset do not provide any other recovery metric. However, this definition produces a sizable undercount of the people who might be considered to have recovered from COVID-19, since the majority of COVID-19 patients are not hospitalized. Moreover, it comes with ambiguities: Some patients might be diagnosed with COVID-19 after being admitted, but might not have been hospitalized for COVID-19 symptoms. States also differ on whether this metric includes only lab-confirmed cases of COVID-19 or also suspected or probable cases. Ten jurisdictions have provided definitions with similar criteria. 

• Unclear recovery definition: These states report a “recovery” figure but do not include any publicly available definitions. Eleven jurisdictions have not provided any definitions that we could discover.³

Five states use some combination of the three main types of recovery definitions we identify above: we have listed those states in both definition categories.⁴

Eight remaining jurisdictions have never provided any recovery statistics.⁵ This could be due to many factors, including a lack of understanding of the disease or the difficulty of collecting relevant data, as many cases experience mild symptoms or no symptoms at all. Washington, for example, has never reported recovery data, and according to the Seattle Times, the Washington Department of Health “doesn’t track how many people have recovered … because so little is known about what recovery looks like.” Similarly, Rhode Island writes that it estimates recovery figures but does not publish them due to the lack of a standard definition: “We build our estimates from positive test results, hospital admissions, and characteristics of the disease documented since the beginning of the epidemic. RIDOH doesn't publish these estimates widely because we publish only those statistics that states and countries use to compare themselves with one another, and that the State may use to guide public health policy.” Rhode Island also attributes difficulties in reporting this figure to the fact that many of those who were infected were not tested. 

Six states that once reported recovery statistics have stopped, often citing their difficulties with collecting complete or reliable data. (In our table, states who once reported recovery figures are flagged in a column titled “Stale Metrics.”) Oregon, for instance, ceased reporting recovery figures on October 9, stating in their announcement that while the state once estimated recoveries by counting people “alive 60 days after onset of illness,” it subsequently decided that this definition “did not factor in people who experience prolonged illness or lasting effects from COVID-19.” Oregon health officials stopped making follow-up calls to individual COVID-19 patients on May 1, and the state now reports only the recovery figures resulting from those spring follow-up calls. Delaware, similarly, said on December 7 that it had stopped reporting recovery data because the data “included only infected persons who reported to DPH that they had reached the end of their isolation period” and because it did not include “those who do not respond to, or complete, case investigation by contact tracers.”

It became clear to us months ago that reporting this incomplete patchwork of unlike statistics at the national level would be a distortion. We believe that under the current lack of standardization and complete reporting, the total number of people in the US who have actually recovered from COVID-19 cannot reasonably be inferred.

No single path to recovery

While we collect thousands of data points about COVID-19, these numbers cannot capture the varied experiences of the 22 million people who have tested positive in the US to date. Since March, more than 371,000 people have died, and different individuals who are said to have “recovered” based on states’ definitions may be in dramatically different states of health. One study found that people with severe cases of COVID-19 continued to suffer related health problems three months after being discharged from the hospital. Most or even all such cases would likely be considered recovered by CDC and US state definitions. 

Similarly, the impressive group of researchers born out of the Body Politic support group for “long COVID” patients points out in a summary of its patient-led research that “recovery is volatile, includes relapses, and can take six or more weeks.” The group asked people who had tested positive for COVID-19 to define “recovery” for themselves—partly because of the lack of a clinical definition, and partly as a way to honor the lived experience of people who actually got the disease. In their report, “What Does COVID-19 Recovery Actually Look Like?” the researchers wrote that in the future they hope to create a standardized definition of recovery based on the type of symptoms and the severity of the illness. 

How should US states and territories report recoveries?

As we approach the one-year anniversary of the first COVID-19 diagnoses in the United States, states and territories have reported more than 22 million detected cases. Few US public health departments have had the capacity to follow up on each case and assemble an accurate picture of how many people in a given jurisdiction have genuinely recovered. Good recovery data for the first year of the US pandemic is—and will likely remain—impossible to produce. But going forward, there are reasonable steps that states and territories can take to make the data they can collect more useful to the public. 

Ideally, public health departments would have sufficient time, money, and personnel to conduct case investigations of every individual COVID-19 patient in order to obtain specific symptoms-based information about the real-life course of recovery from COVID-19, and ideally every COVID-19 patient would participate fully in these case investigations. But in pandemic conditions, this approach has proved unworkable, as Oregon and many other states have discovered. 

Given the realities public health departments are facing, we believe that jurisdictions should begin by carefully choosing labels that better represent the figures they are reporting. Labels such as “Released from Isolation” or “Inactive Cases” are more specific—and therefore better—than labels such as “Recovered” or “Presumed Recovered,” which are much less helpful. Notably, Puerto Rico is using the term “convalecientes” (“convalescent”), a term it has borrowed from the Department of Veterans Affairs. “Convalescent” means that cases are “in recovery,” which crucially indicates that their return to full health is in progress and not yet completed. We believe that making this distinction would further clarify the meaning of this figure, as well as recognize the long periods required for many COVID-19 patients to return to normal life and acknowledge the painful experience of COVID-19 long-haulers. 

Recovery data and The COVID Tracking Project

The COVID Tracking Project has already removed the misleading and incomplete national figure for “recovered” patients from our website, API, and CSVs. For all the reasons explored in depth above, today we are also removing all recovery values except those defined as “hospital discharges” from our website. Of the forty-eight jurisdictions in our dataset that currently report some kind of “recovered” data, eight are reporting hospital discharges: Arizona, Colorado, Connecticut, Kansas, New Jersey, New York, Rhode Island, and Virginia. The hospital discharge data we have been reporting for these eight states in the recovered field will remain in the outcomes category on our website and will be relabeled “hospital discharges,” while the “recovered” values for the other forty states will be removed from our website. All values will remain in our CSVs and API, however, since it can be useful to track a single state’s recovery data over time, and we will continue to collect all data in the recovered field of the API. In two weeks, we will also be moving “hospital discharges” from the recovered field to their own field in the API. 

We urge the media and members of the public to remember that most “recovered” metrics are estimates rather than precise figures, are defined and calculated differently in different jurisdictions, and are often an indication that a person may no longer be infectious rather than that a person is in good health.

Interactive map by Alice Goldfarb Many people at The COVID Tracking Project contributed to the research and data-compilation efforts that made this story possible. We would like to thank Jennifer Clyde, Elizabeth Eads, Rebecca Glassman, Kate Hurley, Nicole King, Daniel Lin, Michal Mart, Barb Mattscheck, Daria Orlowska, Kara Schechtman, Erika Thomson and other contributors for their tireless work on this and many other data quality efforts.