Day: 22 September 2020

There is now a lot of interest in testing for coronavirus. The ‘test-and-trace’ strategy in the UK has been hugely controversial with a lot of complaints about people missing tests. But there is also a lot of confusion about how reliable testing is – Does it overestimate the number of cases? Does it tell people wrongly about their status? Are there people who are told they are not infected but they actually go about and spread the disease? Conversely, how many people are told wrongly they have the disease and as a result, their life is completely turned upside down unnecessarily?

There are at least three broad types of tests for the coronavirus. The first one uses swabs from people’s throat and nose to detect the virus itself; they are based on a molecular technique called PCR. This detects viral RNA and uses a method of “amplification” by which small amounts of RNA which are difficult to detect are “amplified” by a lab procedure to reach the detection level. This test tells us who is currently infected.

The second one is based on blood tests and detects the presence of antibodies – molecules that our body uses to fight off the virus. This test gives information on how many people have been exposed to the disease in the past. The third group looks for symptoms like infection of lungs and concentrates on people who had a particularly bad case of the coronavirus infection.

This blog is about statistics, not about molecular biology, so it suffices to say that all tests are pretty accurate and are continuously being evaluated and improved. At the same time, none of the tests is 100% effective and so we need to look at the consequences of potential misreadings.

There are two kinds of errors that are associated with testing. Firstly, the test might produce a negative result when a person is in fact infected. In other words, a person who in reality is infected by the virus is told “all clear”. There are many reasons for this. The swab test, for example, only can pick up the virus during a certain period following infection – not too early and not too late. Also, it is not a pleasant test – the swab spatula need to go far into the throat or nose. If not inserted properly (particularly if you are doing it yourself!), it will not pick up the viral particles.

In statistics, such an error is called a “false negative” result.

Secondly, the test can produce a positive result when a person is not infected. They perhaps went to get tested because they had fever and cough, but it was caused by an “ordinary” common cold. However, the test swab was for some reason contaminated with the viral RNA, or there was a problem with the procedure. This is usually less likely, but not impossible, as discussed recently in The New York Times.

We call it a “false positive” result.

It is important to stress at this point that the problems do not in any way undercut the importance of testing. However, it is important to understand the limitations and think about ways to deal with them.

Let’s look at some numbers. Suppose that on a certain day 200,000 people were tested and 3,000 positive results returned. These numbers are not too dissimilar to what is happening in the UK at present (18th September 2020: 233,199 tests were carried out with 3,395 positives).

	Infected	Not infected	Total
Positive	?	False positives: ?	3,000
Negative	False negatives: ?	?	197,000
Total	?	?	200,000

We are now asking, how many people who went to get tested were “true” positives – not how many people in the whole population carried the virus which is a different question.

To calculate this we also need to have an idea of how good the tests are and so how likely the “false positives” and “false negatives” are. We currently do not have firm estimates, but we know that the “false positives” are not very likely, but “false negatives” could be more probable. Let’s assume first that the odds for both “false positives” and “false negatives” are 1:999. In other words that the (conditional) probability of the test to produce a wrong result is 0.1% in each case – very low indeed.

Then, a simple calculation shows that:

	Infected	Not infected	Total
Positive	2,803	False positives: 197	3,000
Negative	False negatives: 3	196,997	197,000
Total	2,806	197,194	200,000

What does this mean? Firstly, the number of people “really” infected (2,806) is similar to but slightly lower than the 3,000 reported – good so far. Secondly, the number of “false negatives” is very low (3/2806=0.1%). However, there are quite a few “false positives” – in fact, 6% (197/3000) of all positive tests are “false”.

We have made quite optimistic assumptions about the accuracy of the test, particularly for the “false negative”. Changing the odds to 2:8 just for these entries (there is some evidence that the best rate at which we can detect the virus is about 80% if tested 3-4 days after infection and 1-2 days after symptoms), we get:

	Infected	Not infected	Total
Positive	2,803	False positives: 197	3,000
Negative	False negatives: 350	196,650	197,000
Total	3,153	196,847	200,000

The number of “false positives” stays the same (roughly, as we are doing some rounding off), but the number of “false negatives” shoots up. We now expect to have more “true” cases (3,153) than detected and to include a sizeable proportion of “false negatives”.

Why do we bother? The consequences of these two “false” test results are a bit different. If a person is told they are COVID-positive, they will need to self-isolate. It might also mean that their family, or friends, or contacts, will need to isolate as well. Or, if this happens at school, or at a care home, then the whole class or a home population will be affected, albeit for a relatively short time. If this is based on a wrong diagnosis, it can be a nuisance and it can lead to economic and social hardship, but there are hardly any epidemiological consequences.

In fact, if at all, they seem to be positive rather than negative. Most people who go to get tested have some symptoms or have been in contact with those who have symptoms. Flu symptoms are similar to the ones for COVID-19 and so the self-isolation might break the flu transmission as well. Maybe this is one reason why Australia and New Zealand apparently have been recording fewer influenza cases in their Winter season (June-September 2020) than in other years (masks and social distancing contribute as well).

The consequences of “false negatives” are more serious, as a person that carries the coronavirus infection will get a green light to carry on normal life. This could lead to further infections and, if a person is involved in a superspreader event, could result in a large outbreak.

In this post I only looked at the efficiency of testing, thinking exclusively about people who decided or were told to be tested. Thus, I only really looked at the PCR/swab tests.

---

There is a whole group of people who are infected – and infectious – but who for one reason or another are not tested. Some of them will self-isolate if they have symptoms. But many will simply carry on, unwillingly – perhaps sometimes carelessly – spreading the virus. In the first wave (February-June) we think there were about 8 such asymptomatic and untested people for every positive tested case. It is believed that with better testing this multiplier is now 2-3. As with the “false negatives” talked about above, it is very important to “catch” as many of them and to reduce the potential for transmission. But, this is a story for another blog post.

---

For the mathematically curious, this is the set of equations involved in the calculations:

   

where

	Infected	Not infected	Total
Positive		False positives:	3,000
Negative	False negatives:		197,000
Total			200,000