As North Dakota’s Deaths Metrics Diverge, We’re Switching to a Less Backlogged Measure of Fatalities

Over the past week, a gap has opened between the number of COVID-19 deaths that North Dakota's health department reports and The COVID Tracking Project’s counts of deaths in the state. On November 30, North Dakota’s health department dashboard reported 927 cumulative COVID-19 deaths, but The COVID Tracking Project recorded 680. And where North Dakota reported a steady rise in deaths from November 23 onward, our numbers stayed at 674 for seven days. While North Dakota's deaths from the pandemic appeared to have nearly stopped in our dataset, we can see from the state’s other deaths metric that they are in fact rising rapidly.

If you scroll down past North Dakota’s dashboard to its detailed breakdowns, you’ll see the reason for the discrepancy: Caught between two conflicting federal examples for death reporting, North Dakota reports not one death metric, but four.

North Dakota’s four death metrics as reported on their state health department COVID-19 data website on November 30, 2020.

Most US jurisdictions have chosen just one method of reporting deaths, but a few report multiple COVID-19 deaths metrics, giving our researchers a choice to make: Which figure should we use? Informed by guidance from the National Center for Health Statistics and the World Health Organization, we originally decided to count people with COVID-19 listed on their death certificate whenever we had to make a choice. In North Dakota, this metric was listed only at the bottom of the state’s dashboard, but for most of the year, it closely matched the state’s more prominently featured deaths figure—deaths among confirmed COVID-19 cases—as it should. Over the past month, however, reporting backlogs have made the death certificate method of counting deaths fall far behind the state’s own main deaths figure, leading us to reevaluate our policy. 

It’s clear that a reporting backlog is to blame for most of the widening discrepancy between figures: The share of people with lab-confirmed COVID-19 who died in North Dakota and whose death certificates are not yet filed has doubled since the beginning of November. Today, that figure stands at 114 people whose death certificates are pending. 

To better reflect the reality of rising deaths in North Dakota, we are changing our policy for counting deaths in the state to the other main paradigm for tracking deaths: counting deaths among cases. We made this decision to deviate from the federal standard guided by two principles: reflecting the reality of the situation in North Dakota and matching what the state reports as its most prominent figure. In other states reporting multiple deaths metrics, where reporting the number of individuals with COVID-19 listed on their death certificates doesn’t violate those principles, we’ll continue to do so for now. If we see signs of reporting pipelines buckling in other states—such as stagnating numbers in the deaths metrics we track while other deaths metrics continue to rise—we will switch over to deaths among cases for those states as well. 

For a more detailed breakdown of what we believe is happening in North Dakota and our policy in other states, keep reading.

The North Dakota death certificate backlog

Most states’ methods for reporting deaths can be grouped into one of two categories: They measure either deaths among individuals who had COVID-19 when they died (called “deaths among cases”), or deaths among those with COVID-19 listed on their death certificate (“deaths due to COVID-19”). 

You can read all about each method’s merits and drawbacks in our previous blog post on the subject. But to understand what’s going on with North Dakota, the important thing to know is that the two ways of counting deaths work on different timelines. Counting deaths due to COVID-19 is a more epidemiologically meaningful metric than deaths among cases, but it’s slower. Even in normal circumstances, it can take up to 10 days from the date of death for a medical examiner to file a death certificate. When reporting systems come under unprecedented pressure from surges of COVID-19 deaths, large backlogs can accumulate.

That’s what appears to have happened in North Dakota over the past few months: The state has fallen behind on writing death certificates for individuals dying during its surge. North Dakota explicitly reports the number of COVID-19-positive individuals who have died but who have not yet been issued death certificates in a category called “Pending Death Records,” so we can see the patterns in the backlogs:

Before mid-September, the number of individuals that North Dakota reports as having pending death records rarely exceeded 10. But when deaths began spiking in the state, the pending death records number also grew—at first proportionately to the rising deaths, and then disproportionately, causing a backlog. By late November, the number of individuals pending death certificates greatly outpaced growth of those with death certificates: In the past 10 days the former grew by 19 people, while the latter grew by 72 people. 

Because we did not include COVID 19-positive individuals in our count until they had a death certificate filed, our deaths metric didn’t include these 72 people. However, North Dakota itself counts these individuals as COVID-19 deaths in its most prominent official metric for deaths on its dashboard—called ”cumulative deaths”—which represents all individuals with lab-confirmed cases of COVID-19 who died. 

We would prefer to be able to apply the standardized definition of deaths due to COVID-19 to all jurisdictions that provide a choice. But in North Dakota, where the counts of deaths due to COVID-19 are so different from what we know is actually happening—and where the state itself is choosing to highlight a different figure—standardization takes a back seat to our desire to reflect reality.

Starting on December 1, our count in North Dakota tracks deaths among cases of COVID-19, more closely aligning with the state’s own prominent “cumulative deaths” metric. There will remain a small difference between our reporting and North Dakota’s: Following the lead of the CDC, we’ll be including deaths among individuals with probable cases of COVID-19 as well as lab-confirmed cases in our counts, unlike North Dakota; as of today, our topline figure will thus include six more individuals than North Dakota’s, because we’re counting the people who died with no positive lab test but who had COVID-19 listed on their death certificate (one route to probable case classification, according to the Council of State and Territorial Epidemiologists, or CSTE, COVID-19 case definition).

This change in policy caused an increase of 241 deaths to the topline number of deaths we recorded in North Dakota on November 30. (The increases are distributed over the entire time series, so the daily differences are much smaller.)

What about other states?

Among the remaining states we track, most choose to provide just one figure, either deaths based on the death certificate or deaths among cases. When reporting pipelines clog up in these states, there’s not much we can do except report what the state does. But besides North Dakota, three other states also give us a choice of deaths metrics: Colorado, Arkansas, and Minnesota. 

For now, in Arkansas and Colorado, we’re sticking with reporting deaths due to COVID-19 in those states. In both of those states the reporting of deaths due to COVID-19 is steady, and both states feature deaths due to COVID-19 on equal or higher footing on their dashboard as deaths among cases. (In Minnesota, the state’s definitions are not clear enough for us to determine key details about the metric representing deaths due to COVID-19. We have requested clarification from the state’s public health department.) 

Going forward, we’ll continue to evaluate these policies on a state-by-state basis, preferring deaths due to COVID-19 unless there are compelling reasons not to. One side effect of this approach is that we’ll no longer be standardizing to the federal government’s definition by default. Instead, we’ll be trying to reflect the reality in each state as best as we can, bearing the standard in mind as an ideal. But that doesn’t bring us too far from the norm: Deaths data reporting is already silently unstandardized across every other state, with health departments making the same kinds of calls when they decide which figures to post publicly. 

Notes for close readers of deaths data

If you’d like to access a full history of North Dakota’s death certificate figures, you can still do so, even though they’re no longer in our API. The historical data from North Dakota’s “Deaths Due to COVID-19” figure is archived through our screenshots initiative, which captures historical data for every state. And you can obtain provisional counts of deaths due to COVID-19 from the National Center for Health Statistics, which has direct access to all filed death certificates across the country. 

Careful observers may also have noticed that while North Dakota’s “Pending Deaths” figure accounts for most of the rises in deaths over the past month, they don’t fully explain the cumulative difference in deaths between our old figure and the new one. An additional 127 deaths come from a category of “deaths that list something other than COVID-19 as the official cause of death,” which are counted in a “deaths among cases” total, since they represent people who are all COVID-19-positive, but are not counted in a measure of individuals with COVID-19 listed on their death certificate. 

In reality, it’s likely that some of the people listed in that category actually do have COVID-19 listed on their death certificate—just on a different part of it than those in North Dakota’s official count. North Dakota counts “Deaths Due to COVID-19” as individuals with the virus listed on part I of their certificate, which records the immediate and underlying cause of death. Individuals in the “deaths that list something other than COVID-19 as the official cause of death” category could have had COVID-19 listed on part II, which records any other independent significant causes that contributed to death.

CDC guidance on how to handle part I and part II of the death certificate is inconsistent between cases and deaths: According to CSTE’s official COVID-19 case definition, which the CDC follows in its official COVID-19 deaths tracker, both part I and part II of the death certificate may be consulted in assigning probable case status, but the NCHS strongly encourages listing COVID-19 on part I in its death guidance.

As a result, even within the “death certificate” method of counting, states make different decisions about which parts of the death certificate to consult. These ambiguities in how to interpret the death certificate do not leave room for states to count individuals dying of causes completely unrelated to COVID-19, as some have claimed about comorbidity. As we have noted many times, COVID-19 deaths figures are likely a significant undercount, no matter the method used for counting and reporting. In a future blog post, we’ll explore the key differences in how states treat death certificates.

Additional credits: Research credit to Rebecca Glassman