How is COVID-19 diagnosed?
For most of history, medical diagnoses have been determined by comparing patient symptoms (again, these are human-observable impacts on a patent, usually constituting natural language nouns and adjectives) with lists that doctors together agree define a particular condition. Recently, this qualitative methodology has been slowly amended with quantitative measures as they have become available: e.g., pulse, blood pressure, EEG and EKG, blood oxygen content, “five part diff” (which quantifies different kinds of blood cells), CT, MRI, blood sugar levels, liver enzyme activity, lung and heart pumping volume, viral load, and now DNA and RNA sequencing of tissues and pathogens. These latter tools have become particularly important in genetically tracking the spread of #SARS-CoV-2, because by following the sequence around the world you can sort out at the individual case level where it came from. And then simply being able to specifically detect viral RNA to provide a diagnosis is important because COVID-19 symptoms (other than fatality rate) are quite similar to that of the seasonal flu. Beyond differentiating COVID-19 from “influenza like illness”, new tools are being brought to bear that enable near real time quantification of viral RNA, which enables estimating viral load (number of viruses per sample volume), and which in turn facilitates 1) understanding how the disease progresses and then 2) how infectious patients are over time. These molecular assays are the result of decades of technology improvement, which has resulted in highly automated systems that take in raw clinical samples, process them, and deliver results electronically. At least in those labs that can afford such devices. Beyond these achievements, novel diagnostic methods based on the relatively recent development of CRISPR as a tool are already in the queue to be approved for use amidst the current pandemic. The pandemic is serving as a shock to the system to move diagnostic technology faster. We are watching in real time a momentous transition in the history of medicine, which is giving us a glimpse of the future. How are all these tools being applied today?
(Note: My original intention with this post was to look at the error rates of all the steps for each diagnostic method. I will explain why I think this is important, but other matters are more pressing at present, so the detailed error analysis will get short shrift for now.)
Recapitulating an explanation of relevant diagnostics from Part 1 of this series (with a slight change in organization):
There are three primary means of diagnosis:
1. The first is by display of symptoms, which can span a long list of cold-like runny nose, fever, sore throat, upper respiratory features, to much less pleasant, and in some cases deadly, lower respiratory impairment. (I recently heard an expert on the virus say that there are two primary ways that SARS-like viruses can kill you: “Either your lungs fill up with fluid, limiting your access to oxygen, and you drown, or all the epithelial cells in your lungs slough off, limiting your access to oxygen, and you suffocate.” Secondary infections are also more lethal for people experiencing COVID-19 symptoms.)
2. The second method of diagnosis is imaging of lungs, which includes x-ray and CT scans; SARS-CoV-2 causes particular pathologies in the lungs that can be identified on images and that distinguish it from other respiratory viruses.
3. Thirdly, the virus can be diagnosed via two molecular assays, the first of which uses antibodies to directly look for viral proteins in tissue or fluid samples, while the other looks for whether genetic material is present; sophisticated versions can quantify how many copies of viral RNA are present in a sample.
Imaging of lungs via x-ray and CT scan appears to be an excellent means to diagnose COVID-19 due to a distinct set of morphological features that appear throughout infected tissue, though those features also appear to change during the course of the disease. This study also examined diagnosis via PCR assays, and found a surprisingly high rate of false negatives. It is not clear from the text whether all patients had two independent swabs and accompanying tests, so either 10 or 12 total tests were done. If 10 were done, there are two clear false negatives, for a 20% failure rate; if 12 were done, there are up to four false negatives, for a 33% failure rate. The authors observe that “the false negative rate of oropharyngeal swabs seems high.” Note that this study directly compares the molecular assay with imaging, and the swab/PCR combo definitely comes up short. This is important because for us to definitively diagnose even the number of serious cases, let alone start sampling the larger population to track and try to get ahead of the outbreak, imaging is low throughput and expensive; we need rapid, accurate molecular assays. We need to have confidence in testing.
How does “testing” work? First, testing is not some science fiction process that involves pointing a semi-magical instrument like a Tricorder at a patient and instantly getting a diagnosis. In reality, testing involves multiple process steps implemented by humans — humans who sometimes are inadequately trained or who make mistakes. And then each of those process steps has an associated error or failure rate. You almost never hear about the rate of mistakes, errors, or failures in reporting on “testing”, and that is a problem.
Let’s take the testing process in order. For sample collection the CDC Recommendations include nasopharyngeal and oropharyngeal (i.e., nose and throat) swabs. Here is the Wikipedia page on RT-PCR, which is a pretty good place to start if you are new to these concepts.
The Seattle Flu Study and the UW Virology COVID-19 program often rely on home sample collection from nasal and throat swabs. My initial concern about this testing method was motivated in part by the fact that it was quite difficult to develop a swab-PCR for SARS-CoV that delivered consistent results, where part of the difficulty was simply in collecting a good patient sample. I have a nagging fear that not everyone who is collecting these samples today is adequately trained to get a good result, or that they are tested to ensure they are good at this skill. The number of sample takers has clearly expanded significantly around the world in the last couple of weeks, with more expansion to come. So I leave this topic with a question: is there a clinical study that examines the success rate sample collection by people who are not trained to do this every day?
On to the assays themselves: I am primarily concerned at the moment with the error bars on the detection assays. The RT-PCR assay data in China are not reported with errors (or even variance, which would be an improvement). Imaging is claimed to be 90-95% accurate (against what standard is unclear), and the molecular assays worse than that by some amount. Anecdotal reports are that they have only been 50-70% accurate, with assertions of as low as 10% in some cases. This suggests that, in addition to large probable variation in the detectable viral load, and possible quality variations in the kits themselves, human sample handling and lab error is quite likely the dominant factor in accuracy. There was a report of an automated high throughput testing lab getting set up in a hurry in Wuhan a couple of weeks ago, which might be great if the reagents quality is sorted, but I haven’t seen any reports of whether that worked out. So the idea that the “confirmed” case counts are representative of reality even in hospitals or care facilities is tenuous at best. South Korea has certainly done a better job of adequate testing, but even there questions remain about the accuracy of the testing, as reported by the Financial Times:
Hong Ki-ho, a doctor at Seoul Medical Centre, believed the accuracy of the country’s coronavirus tests was “99 per cent — the highest in the world”. He pointed to the rapid commercial development and deployment of new test kits enabled by a fast-tracked regulatory process. “We have allowed test kits based on WHO protocols and never followed China’s test methods,” Dr Hong said.
However, Choi Jae-wook, a medical professor of preventive medicine at Korea University, remained “worried”. “Many of the kits used at the beginning stage of the outbreak were the same as those in China where the accuracy was questioned . . . We have been hesitating to voice our concern because this could worry the public even more,” Mr Choi said.
At some point (hopefully soon) we will see antibody-based tests being deployed that will enable serology studies of who has been previously infected. The US CDC is developing these serologic tests now, and we should all hope the results are better than the initial round of CDC-produced PCR tests. We may also be fortunate and find that these assays could be useful for diagnosis, as lateral flow assays (like pregnancy tests) can be much faster than PCR assays. Eventually something will work, because this antibody detection is tried and true technology.
To sum up: I had been quite concerned about reports of problems (high error rates) with the PCR assay in China and in South Korea. Fortunately, it appears that more recent PCR data is more trustworthy (as I will discuss below), and that automated infrastructure being deployed in the US and Europe may improve matters further. The automated testing instruments being rolled out in the US should — should — have lower error rates and higher accuracy. I still worry about the error rate on the sample collection. However, detection of the virus may be facilitated because the upper respiratory viral load for SARS-CoV-2 appears to be much higher than for SARS-CoV, a finding with further implications that I will explore below.
How is the virus spread?
(Note: the reporting on asymptomatic spread has changed a great deal just in the last 24 hours. Not all of what appears below is updated to reflect this yet.)
The standard line, if there can be one at this point, has been that the virus is spread by close contact with symptomatic patients. This view is bolstered by claims in the WHO Joint Mission report: “Asymptomatic infection has been reported, but the majority of the relatively rare cases who are asymptomatic on the date of identification/report went on to develop disease. The proportion of truly asymptomatic infections is unclear but appears to be relatively rare and does not appear to be a major driver of transmission.”(p.12) These claims are not consistent with a growing body of clinical observations. Pinning down the rate of asymptomatic, or presymptomatic, infections is important for understanding how the disease spreads. Combining that rate with evidence that patients are infectious while asymptomatic, or presymptomatic, is critical for planning response and for understanding the impact of social distancing.
Two sentences in the Science news piece describing the Joint Mission report undermine all the quantitative claims about impact and control: “A critical unknown is how many mild or asymptomatic cases occur. If large numbers of infections are below the radar, that complicates attempts to isolate infectious people and slow spread of the virus.” Nature picked up this question earlier this week: “How much is coronavirus spreading under the radar?” The answer: probably quite a lot.
A study of cases apparently contracted in a shopping mall in Wenzhou concluded that the most likely explanation for the pattern of spread is “that indirect transmission of the causative virus occurred, perhaps resulting from virus contamination of common objects, virus aerosolization in a confined space, or spread from asymptomatic infected persons.”
Another recent paper in which the authors built an epidemiological transmission model all the documented cases in Wuhan found that, at best, only 41% of the total infection were “ascertained” by diagnosis, while the most likely acertainment rate was a mere 21%. That is, the model best fits the documented case statistics when 79% of the total infections were unaccounted for by direct diagnosis.
Finally, a recent study of patients early after infection clearly shows “that COVID-19 can often present as a common cold-like illness. SARS-CoV-2 can actively replicate in the upper respiratory tract, and is shed for a prolonged time after symptoms end, including in stool.” The comprehensive virological study demonstrates “active [infectious] virus replication in upper respiratory tract tissues”, which leads to a hypothesis that people can present with cold-like symptoms and be infectious. I will quote more extensively from the abstract, as this bit is crucially important:
Pharyngeal virus shedding was very high during the first week of symptoms (peak at 7.11 X 10^8 RNA copies per throat swab, day 4). Infectious virus was readily isolated from throat- and lung-derived samples, but not from stool samples in spite of high virus RNA concentration. Blood and urine never yielded virus. Active replication in the throat was confirmed by viral replicative RNA intermediates in throat samples. Sequence-distinct virus populations were consistently detected in throat- and lung samples of one same patient. Shedding of viral RNA from sputum outlasted the end of symptoms. Seroconversion occurred after 6-12 days, but was not followed by a rapid decline of viral loads.
That is, you can be sick for a week with minimal- to mild symptoms, shedding infectious virus, before antibodies to the virus are detectable. (This study also found that “Diagnostic testing suggests that simple throat swabs will provide sufficient sensitivity at this stage of infection. This is in stark contrast to SARS.” Thus my comments above about reduced concern about sampling methodology.)
So the virus is easy to detect because it is plentiful in the throat, which unfortunately also means that it is easy to spread. And then even after you begin to have a specific immune response, detectable as the presence of antibodies in blood, viral loads stay high.
The authors conclude, rather dryly, with an observation that “These findings suggest adjustments of current case definitions and re-evaluation of the prospects of outbreak containment.” Indeed.
One last observation from this paper is eye opening, and needs much more study: “Striking additional evidence for independent replication in the throat is provided by sequence findings in one patient who consistently showed a distinct virus in her throat as opposed to the lung.” I am not sure we have seen something like this before. Given the high rate of recombination between strains in this family of betacoronaviruses (see Part 1), I want to flag the infection of different tissues by different strains as a possibly worrying route to more viral innovation, that is, evolution.
STAT+ News summarizes the above study as follows:
The researchers found very high levels of virus emitted from the throat of patients from the earliest point in their illness —when people are generally still going about their daily routines. Viral shedding dropped after day 5 in all but two of the patients, who had more serious illness. The two, who developed early signs of pneumonia, continued to shed high levels of virus from the throat until about day 10 or 11.
This pattern of virus shedding is a marked departure from what was seen with the SARS coronavirus, which ignited an outbreak in 2002-2003. With that disease, peak shedding of virus occurred later, when the virus had moved into the deep lungs.
Shedding from the upper airways early in infection makes for a virus that is much harder to contain. The scientists said at peak shedding, people with Covid-19 are emitting more than 1,000 times more virus than was emitted during peak shedding of SARS infection, a fact that likely explains the rapid spread of the virus.
Yesterday, CNN joined the chorus of reporting on the role asymptomatic spread. It is a nice summary, and makes clear that not only is “presymptomatic transmission commonplace”, it is a demonstrably significant driver of infection. Michael Osterholm, director of the Center for Infectious Disease Research (CIDRAP) and Policy at the University of Minnesota, and always ready with a good quote, was given the opportunity to put the nail in the coffin on the denial of asymptomatic spread:
"At the very beginning of the outbreak, we had many questions about how transmission of this virus occurred. And unfortunately, we saw a number of people taking very firm stances about it was happening this way or it wasn't happening this way. And as we have continued to learn how transmission occurs with this outbreak, it is clear that many of those early statements were not correct," he said.
"This is time for straight talk," he said. "This is time to tell the public what we know and don't know."
There is one final piece of the puzzle that we need to examine to get a better understanding of how the virus is spreading. You may have read about characterizing the infection rate by the basic reproduction number, R0, which is a statistical measure that captures the average dynamics of transmission. There is another metric the “secondary attack rate”, or SAR, which is a measurement of the rate of transmission in specific cases in which a transmission event is known to have occurred. The Joint Mission report cites an SAR in the range of 5-10% in family settings, which is already concerning. But there is another study (that, to be fair, came out after the Joint Mission report) of nine instances in Wuhan that calculates the secondary attack rate in specific community settings is 35%. That is, assuming one initially infected person per room attended an event in which spread is known to have happened, on average 35% of those present were infected. In my mind, this is the primary justification for limiting social contacts — this virus appears to spread extremely well when people are in enclosed spaces together for a couple of hours, possibly handling and sharing food.
Many missing pieces must be filled in to understand whether the high reported SAR above is representative globally. For instance, what where the environmental conditions (humidity, temperature) and ventilation like at those events? Was the source of the virus a food handler, or otherwise a focus of attention and close contact, or were they just another person in the room? Social distancing and eliminating public events was clearly important in disrupting the initial outbreak in Wuhan, but without more specific information about how community spread occurs we are just hanging on, hoping old fashioned public health measures will slow the thing down until countermeasures (drugs and vaccines) are rolled out. And when the social control measures are lifted, the whole thing could blow up again. Here is Osterholm again, from the Science news article covering the Joint Mission report:
“There’s also uncertainty about what the virus, dubbed SARS-CoV-2, will do in China after the country inevitably lifts some of its strictest control measures and restarts its economy. COVID-19 cases may well increase again.”
“There’s no question they suppressed the outbreak,” says Mike Osterholm, head of the Center for Infectious Disease Research and Policy at the University of Minnesota, Twin Cities. “That’s like suppressing a forest fire, but not putting it out. It’ll come roaring right back.”
What is the age distribution of infections?
The short answer here is that everyone can get infected. The severity of one’s response appears to depend strongly on age, as does the final outcome of the disease (the “endpoint”, as it is somewhat ominously referred to). Here we run smack into another measurement problem, because in order to truly understand who is infected, we would need to be testing broadly across the population, including a generous sample of those who are not displaying symptoms. Because only South Korea has been sampling so widely, only South Korea appears to have a data set that gives some sense of the age distribution of infections across the whole population. Beyond the sampling problem, I found it difficult to find this sort of demographic data published anywhere on the web.
Below is the only age data I have been able to come up with, admirably cobbled together by Andreas Backhaus from screenshots of data out of South Korea and Italy.
