Counting COVID-19 Tests: How States Do It, How We Do It, and What’s Changing

Since The COVID Tracking Project started compiling COVID-19 data in March, one question has been at the core of our mission: how much testing has each state and territory done? 

Although case counts remain in the foreground in most reporting about COVID-19, the “total tests” metric does just as much of the heavy lifting. Without total tests, case figures mean very little, because total tests can help us understand not just how many people are testing positive, but how much states are doing to understand and combat the spread of COVID-19. Depending on the units in which they are provided, total test counts can serve as the numerator for “testing coverage”—the percentage of a state’s population that has been tested—or as the denominator for percentage positive rate, which in conjunction with other measures, helps us understand an outbreak’s severity.

It is also a metric that may, at first glance, seem extremely simple to define: how complicated could it be to calculate total test results for a state? As it turns out, very complicated. It did not take us long for us to realize that, because the federal government has failed to provide data standards, total testing data would be one of the most ambiguous of all the data points we tracked; there are at least three different units that can be used to count totals, each with its own pros and cons. 

From the beginning of the pandemic in the United States, in the absence of any federal guidance, states and territories have approached the problem of choosing a unit in different ways. Their health departments, caught in a hectic situation where the implications of data definitions were shifting every day, ended up changing or adding caveats to their methods as some data collection paradigms became unfeasible or less useful. But for a long time, states did not document those changes, failing to provide clear definitions on their public health websites.

Given the substantial lack of clarity and consistency in total test results definitions between states, The COVID Tracking Project has until now operated on a simple principle when it came to assembling a national number: we have taken whatever we could get from states. In the early months of our work, this usually meant summing a state’s figures for individuals receiving positive and negative results; even when states directly provided a figure for total tests, they often didn’t define them clearly. For that reason, we took the definitions provided by states with a grain of salt and usually continued to use positive+negative in our main total test results field (totalTestResults in the API).

But now, six months into the pandemic, the vast majority of states are finally providing total test results figures with clear definitions. As of August 11, we have identified only one out of the 56 states and territories we track (Pennsylvania) that does not provide explicit totals, and only nine states and territories that do not clearly explain how they count total tests.

While we are still far from having a national standard on how to count tests, these clearer definitions mean we can start switching states, one by one, from using calculated positive+negative totals to using explicitly reported total tests figures in our main totalTestResults API field and our Total Test Results figures on our website. To support this change, we are launching a new policy about which units of total tests we prioritize in that column and are making more evident which units we are using in each state.

Because we are rolling out these modifications gradually, you will see some movement in both state and territorial totals and our topline national numbers for the totalTestResults API field, as well as the Total Test Results figures on our website. We will keep you posted whenever we make a change to the way we count a state’s test figures on a forthcoming page summarizing everything we know about state and territorial test units, in our API and CSVs, and within our website’s display of our data for each state and territory. (Each state and territory has its own page on our site, linked to from the main Our Data page. Information about these changes will appear in the test results field title and notes section on each state page and the main page.)

Today, we are putting out the first change to match this priority order: we are switching Colorado’s totals to prioritize a figure called total test encounters. Our team is only aware of two other states providing this number right now—Virginia and Rhode Island—and we hope to switch to using it for them soon, too. These numbers will also be accessible soon in a new totalTestEncountersViral field in our API.

What are “testing encounters,” and what makes the method such a useful way to count total tests? The simple definition used by Virginia, Rhode Island, and Colorado is that testing encounters are the number of people tested in one day. It’s not immediately obvious why this method of counting would be superior to reporting total tests the way most states currently do it, either by counting all specimens tested, or by counting unique people tested, a figure Colorado and Rhode Island also provide and Virginia did before May. 

But small changes in the way a state reports total tests can make a big difference in how the numbers can be used. Our team has determined that, to best estimate the testing capacity of both a state and the nation, testing encounters are the best figure, followed by specimens, and lastly, unique people. Read on to learn more about each of these ways of counting tests and what we are doing to align our total test results data with this new policy.

Three ways of counting total tests

When states started providing definitions for explicit totals, most states reported their total testing in units of either specimens or unique people. 

When states count total tests in unique people, they tally the number of individuals in the state who have been tested at some point. This usually means that states control for two kinds of duplication in their figures: 

• First, states reporting unique people aim to control for multiple swabs per person—an individual can be swabbed multiple times when they go for a COVID-19 PCR test, usually via the nose and mouth. If individuals have multiple swabs taken, they are still counted as one person tested, even though multiple samples were collected and tested. This is usually done as replication to control for false results. 

• Second, states adjust for multiple tests per person across time so that if someone takes two tests, months apart, they are still recorded as one person tested.

States have often preferred to count tests in unique people for two reasons. First, it naturally matches units with the “Cases” figures that states provide, which are ideally fully deduplicated in the same way. Second, unique people can capture total testing coverage, or what percentage of the population has met testing criteria and has been tested at least once. This was an especially useful statistic early on in the pandemic, when it was important to track how much states were scaling up testing capacity to cover more ground in order to discover outbreaks.

However, the unique people method has come to take on a major drawback. Since the beginning of the pandemic, states’ testing strategies have shifted towards surveillance rather than measuring testing reach. And for surveillance, unique people tested is not an appropriate metric, because while it captures how many people have been tested over time, it can’t accurately capture how many people are being tested now.

A core aspect of surveillance is repeat testing, especially of highly exposed populations such as healthcare workers or workers in high-density workplaces. But daily increases in unique people tested will not measure the true amount of repeat testing states perform, because they exclude the growing number of people who are being tested for a second or third time. For example, if a healthcare worker tested negative in March, they would not be counted in a unique people figure if they tested negative again yesterday. For that reason, the figure should not be used to represent how many tests a state is running each day, which is a common state reopening indicator and also named by the White House as a key metric for states to use in reopening plans. 

By contrast, when states count total tests in specimens, they record the number of individual samples collected and tested, including in the count both kinds of duplication that the unique people metrics control for: repeat testing of individuals and multiple swabs per person. An individual tested once in March and again in August would be recorded as having at least two specimens taken. And, if the individual had multiple swabs taken on one or both of those occasions, the total number of specimens would be even larger.

Because the specimens metric captures repeat testing and the number of samples collected, it is the purest measure of testing capacity, directly counting how many tests a state is running each day. The number of specimens tested is almost always larger than the number of people tested for these reasons.

But the expansive nature of this measure works both ways. If a state’s testing sites frequently take two or more specimens from each person and that state reports the total tests in units of specimens tested only, it might seem as though the state is testing more people than it is. States, and individual testing sites within states, have different policies about how many swabs they take to conduct a test, resulting in a lot of variation. It is hard to compare testing coverage across states, or fully understand it within states, using specimen numbers only. 

That’s where testing encounters come in. When states report their total figures in testing encounters—the number of people tested per day—they capture many of the strengths of the specimens and unique people metrics. Figures reported in testing encounters provide a more accurate daily picture than unique people, because they include multiple tests for people who go in for repeat tests on different days. And like totals reported in specimens, totals reported in testing encounters can be used to measure testing capacity over time. But since testing encounters figures count the number of people tested per day rather than samples collected for testing, they do not inflate the counts like specimens figures can, so we can understand what percentage of the population is being tested per day.

Although the phrase “testing encounters” is unfamiliar, its definition just describes the way we talk about how many times people have been “tested for COVID-19” in everyday life. If an individual had been tested once a week for a month, she would likely say she had been tested four times, even if she had been swabbed seven times (counted as seven tests if we count in specimens), and even though she is just one person (counted as one test if we count in unique people). In this case, that commonsense understanding is also best for the data.

Up until this week, our project has mostly chosen to prioritize unique people units when we decide which of a state or territory’s numbers to report as our main “total tests” figure for that state or territory. This is largely the legacy of our initial choice, made in the early days when few states directly reported total tests, to produce total test counts by summing positive and negative counts, which were mostly available in units of people. 

From this week on, we will be giving testing encounters first priority, followed by specimens (which measure testing capacity effectively), followed by unique people. In the very near future, we will also be rolling out changes to the way state and territory data is displayed on our website that will allow us to provide a richer view of the data for the jurisdictions reporting in multiple units.

How states are reporting now

Today, only Colorado, Virginia, and Rhode Island appear to be explicitly reporting total tests in testing encounters, but we suspect that many other states are likely reporting a number similar to testing encounters, labeling them as specimens or people. 

The process of deduplicating tests to arrive at a true “unique people” metric is extremely complex, particularly given the patchwork of testing sites and commercial and public labs state and territorial public health departments receive data from. For many states, the process has proven impossible: some states gave up trying to report tests in unique people back in the spring and switched to specimens, and several others currently reporting in people publicly note the difficulty or impossibility of consistent deduplication.

We also suspect that many states and territories currently reporting tests in specimens may also be reporting a figure that is actually closer to testing encounters. Several major commercial labs have options to pool nasal/oral samples, and we suspect these may be counted as either one or two specimens depending on the state. 

The goal of our project is to provide the most comprehensive, highest-quality state-level data on COVID-19 in the United States. We believe that this is also the goal of the many public health authorities working to process the often extremely complex data streams available from testing laboratories. State and territorial public health authorities—and their reporting systems—have been under unprecedented pressure for months, and we understand that many highly dedicated departments have been unable to maintain their initial attempts to deduplicate tests with precision. 

Given all of these challenges, we would recommend that states and territories take the step of providing full documentation about their reporting units and deduplication strategies, including the reality of reporting attempts that may not have gone according to plan. Our outreach team has been in contact with many state and territorial public health authorities this week, and continues to attempt to clarify exactly how each one reports its tests. 

Ultimately, though, it would be best if all states converged around a single measure of total tests, so we can get a consistent national testing number. Today, because of the federal government’s failure to provide guidance, the inconsistencies in testing data across states and territories means that the country lacks a consistent, accurate way to measure national testing capacity. The best we can do right now is to aggregate data from states reporting in specimens and states reporting in people, despite the incompatibilities of those figures. If more states currently reporting only in people or specimens were to embrace testing encounters as a method of reporting tests, we would be much closer to assembling a precise picture of US testing capacity.

Correction: Although this blog post announced an expected increase in our national testing numbers, stored in the totalTestResults field,  due to switching Colorado’s time series from positive + negative to directly reported test encounters in the state’s totalTestResults. Although we had planned to update the national figures at the same time as the first state, we did not actually update the national totalTestResults API field until September 17, 2020.