Test Positivity: So Valuable, So Easy to Misinterpret

In an ideal world, we would know exactly how many COVID-19 infections there are in the United States at a given time—a measurement known as disease prevalence, which is usually portrayed as a percentage of the population.

Although it’s hard to measure COVID-19 prevalence, we can look at the number of “confirmed cases,” a figure that corresponds to the number of viral diagnostic tests that come back positive. By dividing this number by the total number of reported tests and multiplying the result by 100, we get a test positivity rate, also known as percent positivity. US states and territories report tests using several different units, so it’s important to use the same unit in the numerator (positive tests) and denominator (total tests) of this equation.

CDC test positivity formula

Test positivity is a confusing statistic because it’s affected by two factors: viral prevalence and testing strategy. When analyzing test positivity, it’s important to look at the full picture to figure out which one of these factors is driving the change in the number.

Let’s consider a state where a group of 100 people have been tested for COVID-19.

Out of these 100, only three people received a positive test result.

In this case, considering the sample of people who were tested, we would say the test positivity in this particular population is 3 percent.

But testing is limited in this state, so it’s impossible to test 100 percent of a population. By zooming out and looking at the entire population of the state, we can see how the sample we selected is not an accurate representation of the larger population.

Test positivity reflects the percentage of people tested who have the virus, but not necessarily the percentage of people who have the virus in the entire population. That means test positivity only shows a portion of reality. In this example, if we had tested a different sample of people, the test positivity in our imaginary state would have varied anywhere from 1 to 16 percent, even though the prevalence is the same.

The choice of who gets tested is based on state- or county-specific criteria, but is often made based on how sick people appear to be, which in turn influences test positivity. If a state only tests people who have clear symptoms of the virus, it will likely have a higher test positivity than one that is also testing asymptomatic people.

Comparing States or Regions

Just as test positivity can paint a biased picture of a specific state’s outbreak, it’s also imperfect when used to compare two different states or regions. Let’s look at two states with the same test positivity calculated based on the same period of time.

Even though the sample of people tested in both states shows the same test positivity, the overall prevalence of the virus—something we don’t see in the test results—can be very different.

In this extreme example, State A has over three times as many infections as State B, even though both have 6 percent of tests coming back positive. This could be explained by differences in testing strategy—who is getting tested.

Recent college reopenings provide a concrete example of why it’s important to understand the limitations of test positivity. At the University of Notre Dame, a local news source accurately reports, test positivity is “fewer than half a percent” when the fall semester started and 12,000 people were tested. However, the publication also states that, a few days later, “15 new infections out of only 30 tests” were found, which represented a “50 percent positive rate.”

Although there is technically nothing wrong with this calculation, it is misleading to imply that the test positivity changed from <1 percent to 50 percent. The original sample had the purpose of testing everybody, regardless of their symptoms, upon their arrival to campus, with thousands of tests performed. In contrast, only 30 tests were performed in the second dataset. It’s also unclear if the 30 tests performed were on individuals with symptoms or if these individuals had recent contacts of the virus. Even if calculated correctly, the test positivity numbers are misleading when compared to one another, since two very different sets of people were tested in each situation.

While this is an extreme example, it’s important to understand the limitations of the test positivity metric—and to be precise about why the percentage is changing. Increases in test positivity are concerning for different reasons. It could be because different people are being tested, and more tests should be performed to find all of the missing cases and have a better handle on the outbreak. Or it could be because there is an actual rise in cases. Declines in test positivity could mean the virus is waning, or it could mean the outbreak size is the same and something is changing about who is being tested, where tests are being performed, and how many tests are being utilized.

At The COVID Tracking Project, we strive to provide a transparent national dataset that contextualizes the limitations of our statistics. In the absence of federal data standards, calculating test positivity responsibly requires extreme care with and transparency about testing units and labels. As a reminder to our readers, The COVID Tracking Project does not calculate test positivity rates and will not do so until we are confident in our ability to communicate precisely about these complex issues in our visualizations. We urge caution when relying on any (governmental or non-governmental) test positivity calculation that does not transparently and prominently address the question of inconsistent testing units across jurisdictions. Even if calculated correctly and consistently, it’s critical not to overinterpret what this valuable metric can tell you about the pandemic.

Júlia Ledur is a visual journalist and illustrator who leads the visualization team at The COVID Tracking Project. She is a former graphics reporter at Reuters.

@juledurg