Note: We have a new assessment system for state data quality.
To help members of the public understand how well each state is performing, we have assigned each state a data-quality grade based on our assessment of the completeness of their reporting. Our original state grades were based on the presence of the most basic categories of data, and failed to capture our current understanding of “completeness,” so we have updated our grading system accordingly.
Our new state grading system uses 16 separate factors broken into 5 categories to calculate a letter grade for each state and territory we track. (We use “state” throughout to refer to states, territories, and the District of Columbia.)
Reporting
These factors evaluate how well states format and publish their COVID-19 data.
- Is the state’s official COVID-19 website the best source that exists for that state’s consistent, reliably updated data?
- Does the state format its COVID-19 data in a machine-readable way?*
(*) Extra credit factor: the score for this factor can increase a state’s grade, but is not required to achieve an A.
Testing data completeness
These factors measure whether a state is publishing complete basic testing data.
- Is the state reporting the total number of positive test results?
- Is the state reporting the total number of negative test results?
- Is the state reporting the total number of tests conducted?
Patient outcomes
These factors evaluate whether a state is reporting on COVID-19’s effects on patients—and healthcare systems—in their state.
- Is the state reporting how many patients are hospitalized with COVID-19?
- Is the state reporting how many patients with COVID-19 are being treated in ICUs?
- Is the state reporting how many patients with COVID-19 are on ventilators?
- Is the state reporting how many patients have recovered from COVID-19?
Demographics
These factors evaluate whether a state reports basic COVID-19 data (case count and deaths) for the demographic categories identified as most immediately useful by epidemiologists, data scientists, and reporters we’ve consulted with.
Please note: We ask states to distinguish between racial and ethnic categories to improve the usefulness of the resulting data. You can read more about that distinction on the project blog.
- Is reported data broken down by patients’ pre-existing conditions?
- Does the state break down reported COVID-19 cases into racial categories?
- Does the state break down reported COVID-19 cases into ethnic categories?
- Does the state break down reported COVID-19 deaths into racial categories?
- Does the state break down reported COVID-19 deaths into ethnic categories?
Other
These are two factors identified to us as immediately meaningful and relevant for researchers and the public that didn’t fit into the above categories, but form part of our grading factors.
- Does the state report hospital capacity?
- Does the state report its medical data in the format of line lists?
Please also note that this is not a grade for the testing effort itself, but for the comprehensiveness and regularity of each state’s reporting.