Something Is Wrong with Testing Data in the Great State of Texas

Here at the COVID Tracking Project, we track the daily testing numbers published by the Texas Department of State Health Services (DSHS). Specifically, we track the difference each day in the value of total PCR tests—what Texas calls “molecular tests”— completed, as reported by the state on its dashboard. This calculation results in the same value as the state reports for “Daily Molecular Tests” on other pages of the dashboard, though the state reports it visually as a seven-day average, as we often do.

Since August 2, this daily tests number has plummeted. Although the state was averaging more than 60,000 new tests per day at the end of July, that number fell by half by August 12. The state then reported a record number of new tests on August 13, which we’ll cover more later. 

At the same time as Texas’s decline in reported tests, the state’s reported number of COVID-19 cases has not declined nearly as much. As a result, the state’s official seven-day positivity rate—calculated by dividing new cases by new tests—has shown a steep rise, peaking at 24.5 percent on August 12 before dropping in a single day to a still-high 16.1 percent on August 13. A nearly 10% fluctuation is unexpected for any seven-day averaged value, but especially for something as important as the positivity rate.

The other data we compile from the state, however, doesn’t seem to support the narrative suggested by such a precipitous rise in positivity rate between August 2 and 12. New cases have declined, although some of that fall could be explained by a reduction in testing. But the number of people with COVID-19 who are so sick they require inpatient admission at a hospital has fallen steadily from its late-July peak. Declines in both cases and hospitalizations suggest that the outbreak in Texas is easing. But as outbreaks slowed in other hard-hit states, such as New York and Arizona, we saw positivity rates decrease, not increase, as has happened in Texas. So what’s going on in Texas? Is the state really completing dramatically lower numbers of tests, or are we seeing some kind of reporting problem?

Tests pending assignment

After closely reviewing the downloadable data files the Texas DSHS publishes, we have concluded that the number Texas reports for total cumulative tests conducted is the sum of two figures: the number of tests it has matched to counties and another field for tests that have been reported to the state, but are not yet assigned to a county. (You can find the code we used in our analysis here.) The state calls this category “pending assignment.” 

Between August 2 and August 13—a date range coinciding with a DSHS systems upgrade—the number of tests in the “pending assignment” bucket plummeted. As this has happened, the number of daily new reported tests has also fallen. Then on August 13, the state added a net total of 85,542 tests to the “pending assignment” category, resulting in the state’s largest single-day increase in reported tests. KXAN in Austin has reported that the August 13 influx was the result of correcting a coding error at a major hospital.

This correlation between new total tests and tests “pending assignment” has led some journalists and observers to the conclusion that something in this transfer of tests from the “pending assignment” category to the various counties is leading a large number of tests to be thrown out, not merely assigned to a county.

As you can see in the chart above, over 100,000 tests were removed from this category on three days in the last week, part of a large drawdown since August 2. The state then added over 85,000 tests to "pending assignment" on August 13, the same day the state reported its largest ever daily increase in tests.

Importantly, the downward pressure on the state’s testing numbers produced by drawing down tests "pending assignment" may be counterbalanced if all of those tests are assigned to counties—but that may not be happening.

The chart below shows the number of new tests reported by the state at three levels: 

1. all reported tests (what we report as our topline number for the state)

2. tests pending assignment, and 

3. tests attributed to individual counties. 

First, the number of total new tests reported has declined dramatically(blue). Additionally, the number of new tests attributed to counties has skyrocketed (green), reaching a seven-day average of over 120,000 on August 12. If you only looked at individual counties, or took the total of all individual counties, it would look like Texas was conducting more COVID-19 tests than ever. The true story is somewhere in the middle. Compared to the end of July, the daily number of new tests attributed to counties has increased, but not by as much as new tests "pending assignment" has decreased (orange)—this raises the question, are all the tests leaving "pending assignment" being accounted for? Ultimately, what we see is a plummeting total new tests reported figure and suspicious fluctuation in statewide positivity rates.

As you can see below, in Bexar, Dallas, Fort Bend, Harris, and Tarrant Counties (the five counties with the greatest number of tests reported to date), the number of new tests reported has increased dramatically since the end of July. This creates a phenomenon in which positivity rates at the individual county-level are decreasing, while they are increasing at the state-level. However, it is unclear whether the “new” tests assigned to counties are really new tests or whether they are simply older tests being reallocated from the previous “pending assignment” designation.

To complicate matters further, when Texas reported a record number of new tests on August 13, it actually lowered the seven-day average for new tests in four of the five counties shown here, as the majority of these new tests were added in the "pending assignment" category and not associated with a county at all.

Our main question: has Texas been dropping some of the tests removed from "pending assignment" from the cumulative statewide count, or are they all assigned to counties? The former would mean that the recent slowdown in testing may be in part a reporting artifact, rather than a real decline. If the latter is true, it means that all recent county-level data may be unreliable—as are the gating criteria/metrics used by local governments, schools, and health systems that are built from these data—because it appears that many of the “new” tests assigned to counties are actually from a backlog of tests "pending assignment" that may include tests from weeks or even months ago.

New systems

This analysis is also complicated by the fact that DSHS moved to a new electronic test reporting system on August 2, designed to “enable incoming lab results to be processed faster.” This new system, whose adoption coincides exactly with the beginning of the changes we observe, could be responsible for the recent swings in "pending assignment" tests and new daily tests. As the number of total tests stored in "pending assignment" is reduced—it is currently at 476,276, down from more than 1.1 million at the beginning of August—we may find that the number of new reported tests begins to normalize and align better with our expectations. Right now, however, the system is erratic, and movement of tests into and out of the "pending assignment" bucket seems to be causing huge swings in the state's topline testing numbers and confusing the public and health authorities alike.

What Texas can do

Testing and positivity rates are a crucial tool used by states and localities to assess the trajectory of the virus in their area. Right now, the data reported by Texas DSHS—both statewide and at the county level—is unusable for that purpose. While a period of disruption when working through a backlog might be understandable and justified as DSHS moves to a new system, communication from state authorities has been lacking. Unlike other states that have experienced electronic reporting issues and been forthcoming about technical challenges, Texas is not saying anything about why these changes are occurring or whether the data is accurate. There had been no official communication from state authorities until August 13, when Gov. Greg Abbott noted in a press conference that DSHS was bringing in a “special data team” to investigate the rising positivity rates, but he did not bring any additional clarity to the situation beyond suggesting that testing demand has declined. 

To start, DSHS can answer some fundamental questions about their data:

1. What is the technical definition for tests that are "pending assignment"? In a statement to the Texas Tribune on August 13, DSHS clarified that situations like incomplete addresses lead to tests being categorized this way until further research is done to determine which county to assign them to, but more information on how these situations are dealt with is needed.

2. What is the process by which tests are moved from “pending assignment” to individual counties?

3. Are tests ever removed from "pending assignment" but not attributed to a county, thus decreasing the total number of tests conducted statewide? If so, how many, and why have they been removed (e.g., duplication, antigen/antibody tests, non-residents)?

Due to its size and its substantial COVID-19 outbreak, Texas plays a major role in our nationwide dataset. We’ve seen testing decline at a national level in recent weeks, and the apparent testing decrease in Texas has been a big part of that (though not the only culprit). Texas should be increasing testing to suppress its outbreak. Along with this, the public—and officials throughout Texas—needs a full explanation for the behavior of the positivity rate curve in recent weeks, including an explanation of which tests are recent and which are from weeks ago. Without transparent documentation of what is really happening with testing in Texas, the state’s data remains too murky to rely on for decision-making at any level of government.