Projected Risks and Benefits of Increased Vaccination Rates

The United States is currently debating, and in some cases, implementing a series of different Covid-19 vaccine mandates.  Passions are rather high on the topic; I thought that it would be a good idea to attempt to quantify the expected health risks and benefits associated with greatly increasing the rates of vaccination in the country prior to the winter.  I take it for granted that health is not the only good to be considered overall, and considerations of liberty and self-determination are also at play.  But here, I am just trying to isolate and quantify the public health aspect of the question alone, in the service of people trying to find the right prudential balance.

The Risks

Supposing you were to forcibly vaccinate every remaining unvaccinated person in the country, excepting those people with valid medical reasons for an exemption (such as a history of severe allergic reactions to vaccination, etc.).  What would be the downside?

Vaccines are not perfectly safe, and vaccinating a large number of people will inevitably have side effects, up to and including the death of some people.  Here, I am going to estimate the number of deaths only, and not attempt to quantify the amount of detriment to health that severe but non-lethal side effects will have.  I will do the same thing for the health effects of Covid on the other side of the scales.  The reasoning behind this is that a lot of different kinds of health problems tend to behave very similarly in this way: that the severity of outcomes occur in a natural distribution curve.  That is, a small number of people will die, a larger number of people will have severe but non-lethal outcomes, and the largest number of people will have moderate to mild outcomes.  

Given that distributions of outcomes tend to be shaped like a normal distribution curve of some type, it is often the case that comparing just the most easily measured of a possible range of outcomes will still give a valid comparison, rather than laboriously comparing most serious to most serious, least to least, and so forth.  And for both vaccinations and disease, deaths are the least ambiguous outcome and one where we are most sure of the numbers.  So that's why I'm going to compare just death rate to death rate, as a proxy for the entire health burden of vaccination versus disease.

So how many deaths would one expect from a massive vaccination campaign of about 124 million people (about the number of people remaining to be vaccinated)?  Well, I know of three real and dangerous side-effects that have been seen from the vaccines: anaphylactic shock (a complication possible from any vaccine), myocarditis, mostly seen from the mRNA vaccines, and a rare type of blood clotting in women seen from the adenovirus vaccines.

Anaphylaxis

Incidence of anaphylaxis post-vaccination are pretty rare (about 3-5 per 1 million doses), and given that anaphylaxis is normally very treatable with on-site application of steroids, I have struggled to find any cases of deaths from anaphylaxis verifiably caused by the vaccine.  I did find reports of one in India, one in England, and one in Vietnam, so I believe it *does* happen.  I think it would be hard to justify a death rate of anything higher than 1 death per 10 million people vaccinated, though, so let's very pessimistically put this number at 12 people dead from anaphylactic shock.

Myocarditis

Incidences of myocarditis from the vaccines are rare enough, but since the myocarditis is also generally a pretty mild form when it does happen, deaths are even rarer.  No person in the United States has been found to have died from myocarditis post-vaccination so far, out of about 200 million people vaccinated.  Europe has seen 5 people die from myocarditis or pericarditis after about 200 million people were vaccinated, but it's unsure if all of those were actually related to vaccination or were coincidental.  So a good number here is probably something like 1 per every 100 million people vaccinated.  For our scenario, let's round that up to 2 people dead.

Blood Clots

For the unique form of blood clotting associated with the adenovirus vaccines, the incidence of deaths is higher.  Three people in the United States have been verified to have died from blood clots caused by the Johnson & Johnson vaccine.  Death rates in other places where usage of adenovirus vaccines are much more prevalent have been estimated at 7 per million people vaccinated.  Thus far, only 8% of Americans have been vaccinated with the Johnson & Johnson vaccine . . . and I have to think that this percentage would *have* to decrease, especially among those women who are most susceptible to this kind of blood clotting.  In fact, I would suspect that doctors would strongly urge women in the susceptible groups to get the mRNA vaccines instead, of which we have more than plenty supply for everybody.

So I'm going to assume that the J&J vaccine accounts for only 4% of the total vaccinations in our hypothetical campaign, which would then work out to it causing something like 35 deaths.

Other Unknown Causes

It's difficult to account for the pure unknown.  In this case, I find there is little need to do so.  Statistics has methods of characterizing when a sample of a certain population is sufficient to give you statistical certainty that you are going to have a representative result from that sample.  And those methods are all *entirely pointless here*, because the number of people to be vaccinated in this theoretical vaccination campaign is *smaller than the total number of people already vaccinated*--the "statistical sample" is larger than the rest of the population.

I don't know if there is a word for statistical "super significance", but the data we already have from vaccinating a majority of the population should classify for that if anything does.

There is no reason to believe at this point that any new surprises are going to arise from a new massive vaccination campaign.  But hey! let's imagine something comes up anyway.  We will arbitrarily double the number of projected deaths we have come up with so far, adding another 49 deaths from some mystery side effect--let's say something that effects small children only maybe.

Final Range

Now let's take the number we have come up with, and multiply it by 2 for a fudge factor--maybe all of the deaths we have looked at so far have only caught 50% of the true deaths caused by vaccination, for some reason I can't right now imagine.  Then let's round up, and we have a total of 200 deaths caused by this vaccination campaign.  This is probably an absurd number, as it is already arbitrarily quadrupled over a number that was already unrealistically pessimistic.  But let's go with it for now as an upper bound, or the pessimistic side of our projection.

What's the optimistic side of this projection?  Well, the total number of confirmed deaths from Covid vaccines so far in the United States is 3.  Those were all from the J&J vaccine, and it's reasonable to think that if you are vaccinating a smaller group of people and you just avoid giving the J&J vaccine to vulnerable women, then nobody will die.  Further, it is thought that now that we understand that this rare blood clotting is a risk, we understand also the warning symptoms that proceed the stroke it causes.  It's possible that none of those three people would have died if doctors had known what to do and had taken preventative action that is now on the standard warning label for the J&J vaccine.

So it's quite reasonable to think that the total death toll of vaccinating 124 million people would be exactly zero.

Consequently, the final range for deaths caused by vaccination we can project to be something between 0 and 200 people.

Benefits

Now, what are the benefits of this massive vaccination campaign?  To answer that question, we would have to know how many people we expect to die if nothing special is done and people just continue to be vaccinated by choice, slowly (currently happening at the rate of about 500,000 people per day).  Keeping in mind that vaccination takes 6 weeks to reach full effect, the current rate of vaccination will only reach about 15 million people before it's too late to do any good for this winter season--*if* the current rates hold up, which I find doubtful since we are currently "burning through" the only-somewhat-hesitant crowd.  That's only about 10% of the total people you could vaccinate--probably not enough to make a substantial difference in the total illness and death.

This is a very difficult thing to estimate, full of different kinds of uncertainty.  I'm going to try several different ways of estimating and compare the results of these different approaches.

Summer to Winter Comparison

In 2020, there was a summer surge in infections and deaths, followed by a winter surge in infections and death.  Importantly, the social restrictions in place between those two time periods were largely the same, and the levels of vaccination were the same in both cases: effectively zero, as vaccinations didn't start having a real population-level effect until after the primary winter season was over.

In 2021, we have also had a summer surge, and it is inevitable that we will also have a winter surge.  The social restrictions in place between the two surges will be the same, I assume: effectively nothing.  Likewise, the total level of vaccinations will be roughly the same--new vaccination rates having dwindled to their lowest numbers ever just before the summer surge and having only just sluggishly risen to their current tepid numbers now as we are near the end of it.  There will be about a 10-15% increase in meaningful vaccinations at the start of the winter surge compared to the start of the summer surge, but that will be only a second order effect on the total numbers (probably, more on this later).

So here is one reasonable approach to estimate the number of people who will die this winter from Covid: take the ratio of the number of people who died from Covid in the winter compared to the summer of 2020, and multiply the number of people who have died from Covid in the summer of 2021 by that ratio.  Then, make some adjustment for the increased levels of vaccination: say, subtract 20% to account for the 15% increase in vaccination levels.  The result will be the projected number of people who will die this winter.

I've done this in a spreadsheet here: Winter Death Estimates.  The result that this gives you is about 450,000 deaths projected for this winter.

Percentage of the Remaining Vulnerable

I think 450,000 is extremely pessimistic, so let's try a different approach.  Let's try thinking about how many people are even left to infect, and how many of those people might plausibly fall sick this coming winter season.

The reason I think the previous number is pessimistic is that if you start calculating up the total number of people who have been either previously infected or vaccinated, you start to come up with large percentages of the population--things like 88%.  This is very close to levels where we could consider herd immunity in play.  If we actually reach an effective herd immunity, then R levels could drop below 1.0 and we could have close to *zero* spread of the disease this winter--that is somewhat plausible.

Now, figuring how close we are to herd immunity this is an inexact science for several reasons, of which there are three primary ones.  

The first primary reasons is that there is always a large body of people who are infected but not diagnosed--the asymptomatic majority.  What the proportion is between the known infections and these unknown infections, however, is still a matter of some conjecture.  Something like 50% to 75% of the total Covid infections are these "occult" infections, and since you don't know that number, you don't know how much immunity is really out there.  

The second major source of uncertainty here is that we don't know for sure how much existing immunity--either from previous infections or from older vaccinations--is going to prevent spread of the virus this coming winter.  Both older vaccinations and older infections may hold up well to prevent serious illness and death in those who have them--but as for holding up for preventing spread of the disease, that is far less certain.

Finally, herd immunity doesn't work as well in a fragmented society.  If there are large populations of people who are less vaccinated than other people *and* they spend a lot of time in close proximity to each other (say, for example, school children), then it won't matter if the population as a whole has a high rate of immunization, the disease will still be able to spread robustly through that proportion of the population which is less vaccinated.  So, overall vaccination numbers don't tell you the whole story as far as limits on disease spread.

*But* let's attempt to do this calculation anyway.  The population of the United States and the number of vaccinated people are easy numbers to obtain, as is the number of individuals who have recovered from diagnosed Covid.  To calculate the remaining vulnerable people, you have to assume a certain ratio of diagnosed Covid to non-diagnosed Covid, and the lower that ratio, the more vulnerable people are still around.

Beyond that, you have to determine what you think the attack rate this coming winter will be: what percentage of vulnerable people will come down with the disease.  This is an extremely difficult question to settle.  Given that the Delta variant is about twice as infective as original covid, with an R0 of 5-6, traditional models for infectious diseases gives you a very high mark for what the final attack rate will be--anything above 3 and the final attack rate is typically reckoned in the mid-to-high 90's.  But you can't just take Delta's R0 since one assumes that, first of all, not all people who could possibly get sick from Covid eventually will actually get sick precisely this winter, and secondly, that the interference of large amounts of immune people will have *some* oppressive effect on spread, and hence final attack rate.

The Range

In the absence of far too much information, I have put as an upper-end-of-plausibility an attack rate of 75%.  For reference, the attack rate of just the 2020 winter season was about 3% of the total population, so this rate is *remarkably* higher than what we have ever seen in a season before.  The fact that we will be seeing a holiday season for the first time, however, both with a very infectious version of Covid very much about *and* no social distancing to speak of, makes this attack rate maybe not entirely fantastical.

So by taking 75% of the remaining vulnerable population as a worst-case estimate, and maximizing the size of the remaining population by *minimizing* the number of people who are already immune, I come to a maximum death rate for this winter of 672,000 people.

On the lower end, I assumed a higher number of existing immune people, and assumed an attack rate of  1.5%, which is assuming that the disease spreads half as vigorously among the remaining vulnerable than it did among the vulnerable from last winter (probably an extremely questionable assumption given how robustly it has spread during this summer, but let's go with it).  These assumptions lead to a projected maximum death rate of only about 3000 people.

This is, to say the least, an extremely broad range of possibilities.

Equalizing Death Rates

The final method of coming up with a winter death rate is as follows:

We know that Covid is capable of killing up to 3297 people for every million persons in a population.  We know this, because that is currently the total death rate for the State in the union that currently has the highest number (Mississippi).  Now, when people look to explain why Mississippi and other states have had their death rate totals be so high, they normally have been looking to its comparatively low vaccination rates compared to other states.  That, however, is an over simplification.  In fact, while Mississippi *has* had a lower vaccination rate than other states, it's not all *that* much lower.  Meanwhile, Florida *also* had an extremely bad summer and has ended with a very high death rate, and *its* vaccination rates overall weren't bad.

The real reason behind these high death rates is not the overall vaccination rates.  Rather, because Covid spreads among interconnected network of vulnerable people, the high death rates in the states that have suffered them this summer has been primarily due to large numbers of unvaccinated people *living in close communities* with each other.  Primarily we are talking here about close-knit black and Hispanic communities with low vaccination rates.

So then the next realization you can have is that, when Thanksgiving and Christmas come around *everyone* becomes a close connected community.  This is a major--perhaps *the* major--reason why these seasons are replete with illness.  Consequently, one could expect that when the holiday season comes, all of the other States will be vulnerable to disease spread among the unvaccinated in the same way that the other States already had disease spread among the unvaccinated.  Unvaccinated people who were previously unconnected because there were not living in close community with other unvaccinated people will all of a sudden be spending time, cheek by jowl, with each other over the holidays.

The hypothesis then is, take the current highest death rate among the States (3297 people per million of population), and assume that all of the other States will rise to equal that level over the holidays as unconnected unvaccinated become connected.

This will probably *underestimate* the death rates because it will assume that nobody will rise *above* that existing death rate (and I'm sure that more people will die from Mississippi, making that assumption untrue).  But it also probably *overestimates* the death rates because there are a number of states that are *way* beneath that level and seem unlikely to go that far in a season.  But let's assume that the over and under estimates somehow balance each other out and see what we get.

This calculation is also on the spreadsheet, and it comes out to about 365,000 deaths.

Final Calculation of Benefits

The final calculation of the benefits of a massive vaccination campaign, in terms of lives saved, is therefore a number of extremely broad range.  The vaccines have been holding up in terms of preventing death from Covid at something around the 95% rate, so you only need to adjust those final death numbers down a little bit to get total lives saved.  In reality, if you did somehow manage to vaccinate *all* the eligible people, you'd save more than 95% of the potential lives, since you'd *seriously* cut into infections as well as prevent deaths.  Actual prevention of death would probably rise to something closer to 99% of the people who would have otherwise died.  But let's be pessimistic again at call it just 90% of people saved.  

In calculating the final range of people who might possibly die, I also refuse to consider the 672,000 figure from the "Remaining Vulnerable" method.  I don't know what's actually wrong with that calculation, but instinct tells me that it is not plausible.  So I'm only going to use 3000 for the bottom range and 450,000 for the top range.  Taking 90% of that tells me that the projected benefit of an extremely aggressive vaccination campaign getting close to 100% of eligible people before winter could result in between 2,700 and 400,000 lives saved.

Risks vs. Benefits

Is this analysis worth anything?  With such a wide range of possible outcomes, is there any take-away action we can recommend?  That is beyond the scope of this post, but I will point out two things:

1. The risks of extremely aggressive vaccination at its most pessimistic is more than an order of magnitude less than the benefits at *its* most pessimistic.  There is therefore no realistic scenario, at all, in which an extremely aggressive vaccination program will not provide more benefit than harm, at least at the level of physical health.
   
   
2. The range of possible outcomes is for projected deaths this winter is extremely broad, but I think definitely realistic.  I would be greatly shocked if deaths this winter were less than 3000 people, and I would be shocked if they exceeded 450,000 people. 
   
   And this means, I think, that numbers in the middle range are very appropriate projections for a State to plan for.  I think that 100-200,000 is a very realistic possibility, though I'm very hopeful that the number will be closer to 20-50,000.  In general, if there are very few downsides (which we have shown is true, comparatively), the State *ought* to plan for the worst.  So I would advocate for a way of thinking in which we assume that 100-200,000 deaths will happen this winter if we do nothing.

Dying "with" vs. Dying "from" Covid, pt. 3

One issue with the previous analysis is that it applies only to deaths that were purely coincidental to Covid infections.  Another set of deaths, it might be argued, that overly inflate the total death toll of Covid are those deaths in which Covid did, in fact, act as the proximate cause, but which were deaths of very weak or sickly old people who were probably going to die anyway--in a matter of days, weeks, or months.

For these deaths, the timing issue would not arise.  In fact, these deaths would technically still be attributable to Covid, since it was Covid that, so to speak, pushed these people over the edge.  So the previous analysis would not be able to separate out these deaths from other Covid deaths, whereas really the deaths *should* be more attributed to the extreme age or weakness of the people who died.

I don't think there's any question that deaths of this kind occurred and were included in the official Covid tally.  Can we get some sort of estimate, however, on the magnitude of this effect?

Average Death Rates

Here, it is good to look at the average yearly death rates.  In the United States, these vary from year to year, but not by a whole lot.  They are also relatively constant throughout the year aside from a yearly noticeable peak during flu season.

Given the relative stability of this yearly average death rate in the U.S., we should be able to eliminate at least some of these types of deaths of the extremely weak.  The hypothesis is that these people would have died soon anyway without Covid, and it just happened to be Covid that was the last straw.  If this were the case, then we should see that these deaths, though included in the official Covid death tally, would *not* have increased the total deaths that occurred in the year above the yearly average.  In other words, suppose 300,000 people are supposed to have died from Covid in 2020, but half of those people were people who were most likely going to die that year anyway.  We should then expect the total number of deaths from all causes in 2020 to have risen above the average only by 150,000, not by 300,000.

This analysis has been done (and continues to be done) by the CDC on an ongoing basis.  The report may be found here:

Excess Deaths Associated with COVID-19

Currently, the CDC is estimating that since Covid began, we have had an average of about 760,000 excess deaths above what is statistically expected given recent history.  Meanwhile, the official Covid death count is only at about 620,000 (for the time period at which the CDC data on that page was most recently updated).  This means that there seems to be an *excess* of deaths resulting from Covid, *above* what the official tally reveals--something on the order of 20% or so.

Other Analyses

The CDC is not alone in coming to this conclusion.  The Wall Street Journal has done several excellent statistical analyses of  U.S. and world data on excess deaths more than once and has come to similar conclusions (though with slightly higher estimates for the underreporting of Covid-19 deaths at 35%):

The Covid-19 Death Toll Is Even Worse Than It Looks

Covid-19 Disrupts Years of Health Progress in U.S.

Yet other statistical analyses have put the toll even higher: Estimation of excess mortality due to COVID-19 by the IHME puts the real toll of Covid in the U.S. at about 57% higher than the official tally based on excess deaths.

What Explains the Extra Deaths?

So why would there be more deaths from Covid than actually reported?  The first explanation some people might want to gravitate towards is: social measures taken to halt Covid have had bad side effects on the population.  For example, people who *should* have gone to the hospital with a heart condition were afraid to and hence died at home rather than being treated as they would otherwise have been.

The problem with this theory is that if you look at the graph of excess deaths, they very clearly track exactly with Covid infections--as Covid cases go up, the excess deaths go up, and as they go down, the excess deaths go down.  The excess deaths do *not* track with the rigor of societal restrictions, which were most strict in the very earliest phase of the pandemic, but eased off before the Summer '20 surge and were even more eased just before the disastrous Fall / Winter '20 season.  

Therefore plausible reasons to explain these excess deaths have to find a cause that's correlated with Covid infections.  I have three theories that I would like to propose:

1. In regions in which medical resources became strained due to Covid outbreaks, care of other patients suffered as well due to total lack of resources.
   
   
2. Many people who died from Covid were the elderly, in nursing homes, and with other comorbidities.  It's quite possible that many of those people died in such a way that their comorbidities were blamed rather than Covid, but that their deaths would not have happened for more than a year without Covid.  (My opinion is that this represents the bulk of the difference between official tally and actual death toll).
   
   
3. As I have pointed out several times, Covid has a significant second method of killing people, which is attacking the heart.  We know for sure that Covid is to blame for at least a few heart attacks that otherwise came out of the blue, even in people who were otherwise only very mildly sick from Covid or even completely asymptomatic.  Myocarditis has been found associated with Covid in otherwise healthy people who were asymptomatic at extremely high rates: up to 25% in one study of athletes with Covid.
   
   It is quite possible that a number of people over the past year have dropped dead from heart attacks directly caused by Covid, but without that cause ever having been discovered.  In fact, I believe we have some indirect evidence of this in studies which have shown that reports of heart attacks have inexplicitly risen in regions with high Covid occurrences.

What Do these Excess Deaths Mean?

That large number of deaths have indubitably occurred since Covid began and have been rising and falling largely in step with the rise and fall of Covid infections makes it exceedingly difficult for the theory that the official death tally for Covid is an overstatement.  It makes it essentially impossible to claim that most of these people who died were on death's door already.  To continue to maintain that the official death tally is a gross overstatement requires some other explanation, which I have not yet heard and which I can't even imagine currently.

This does not mean, however, that the death tally can't be put into a certain amount of perspective.  While the people who died were certainly not all on death's door, nevertheless most of them have been quite old and vulnerable in other ways.  It *is* quite possible that many and possibly the majority of them had only 3-5 years left anyway, or maybe 10 at the outside.  Given the age of the primary victims of Covid, this is something that is necessarily true.

If you wanted to be morbidly precise and weigh out everyone's life in a balance, then it is possible to take the age and comorbidity statistics that we have and figure out what the approximate toll of Covid has been in terms of total man-years of life lost.  This would certainly be a legitimate way to minimize the impact of Covid, though a rather ghoulish one, in my opinion.

If you were to do that, however, it would only then be fair to add into the balance the serious illness as well.  There are many vulnerable people who were sickened by Covid and did *not* die, but nevertheless were left with significant, long lasting recoveries and put into a permanently weakened state: weaker heart, shredded lungs.  These people are not counted yet on Covid's death toll, but have *certainly* lost total years of their life due to their battle with Covid.  This will be a harder toll to add up than just the people who died, but *if* you go down the route of counting years of life lost, then you certainly need to add those future lost years in as well.

Conclusion

It is not possible to maintain that the official death toll for Covid is vastly overstated.  By virtue of coincidental deaths, it might be as much as 10% overstated, but by virtue of excess deaths we observe, it is very likely to be something like 20% *under*-stated.  It *is* true that the majority of these deaths were elderly or otherwise vulnerable people, and you can make of that fact what you want based on how much value you place on the lives of the elderly.

Dying "with" vs. Dying "from" Covid, pt. 2

For the first method of data analysis, I note that the official Covid death tally is surmised to be composed of two series of numbers: the people each day who die of some random cause but only happen to be infected with Covid, and the people each day who actually die of Covid.  And *both* of these series of numbers will be related to another series of numbers: the number of people each day who are diagnosed with Covid.  However, the two types of people who die each day will each have  a *different* relationship to this number.

For the people who die of some other completely unrelated cause, the number of those people--who just happen to also have Covid--will be directly related to how many people currently have Covid in the population.  If a lot of people happen to have Covid at some time, a lot of people who die *at that time* will also happen to have Covid by coincidence.  If few people happen to have Covid at that time, few people will die coincidentally also having Covid.  So if you plotted the number of people who have Covid at any particular time on the same graph as the number of people who die "with" Covid at any particular time, the second graph will be a mirror of the first graph (but smaller).

The same thing is true of people who die "from" Covid--*except* for the important fact that this graph would be not only mirrored, but also time shifted.  It takes some time after you are diagnosed with Covid to actually die of Covid.  So if a lot of people at a particular time are diagnosed with Covid, then *later on* a lot of people will die from Covid--but not right away.

This time dependency represents a difference between the two types of people that we are surmising compose the total official death tally of Covid.  We should then be able to separate out roughly how many people fall into each category by doing a time-dependent analysis.

My Analysis

Here was my approach, using publicly available datasets and a custom Python program:

I assumed that the number of "deaths with" (the coincidental deaths) included in the official death tally was some fairly constant percentage of the total deaths (seeing as I couldn't think of any good reason for this to change over time).  I also assumed that the number of these deaths over time would be directly proportional to the number of Covid cases at the time.  I could therefore generate a time series that represented those deaths by taking the time series number of confirmed cases per day and scaling it down until the number of deaths it represented equaled a given percentage of the total official death tally.

I made this target percentage (the percentage of deaths in the official tally which are "spurious") a variable so that I could generate multiple time series of spurious (or coincidental) deaths per day corresponding to any target magnitude of this effect I wanted.

For each iteration of my run, I would generate the "spurious" deaths that would correspond to a given magnitude.  I then subtracted these deaths from the official tally.  The hypothesis of this particular run would be that the remaining deaths were the deaths caused "by" Covid, and should therefore match the Covid infection curve, but with a time delay.  I then scaled these deaths up to match the infection curve and found the best time delay which caused the death and infection curves to match.

By doing this for a target "spurious" death percentage of 0%, 10%, 25% and 50%, I figured I could see which rate of "deaths with" resulted in the best final match between time-shifted deaths and the original infections.  That is, the closer my arbitrary percent of "deaths with" ended up being to reality, the better the remaining deaths would correspond to the infections that actually caused them.

The result was as following (orange is scaled up deaths, blue is infections):

As you can clearly see, assuming that "deaths with" Covid account for either 0% or 10% of the total deaths results in a perfectly reasonable final death curve that matches the causal infection curve pretty nicely.  However, the further you increase this number above 10%, the worse the match becomes.

Periods of Rapid Infection Growth

The most telling part of these curves are the sections in which infections are increasing rapidly--primarily at the start of the Fall/Winter surge of 2020 and the current Summer surge of 2021.  The reason these diverge so strongly is that when you have infections very rapidly rising, you can start getting large differences between the infections and time-delayed deaths.  That is, you see large numbers of infections two weeks into one of these very rapid surges, whereas the deaths have not moved at all.  These time periods are extremely hard to explain using the "deaths with" hypothesis--if the infections are rising rapidly, why are coincidental deaths not also rising rapidly?  And you can see this divergence visually in my analysis by the big dips in the resulting death graph compared to infections during those periods.

You can see this problem already starting to emerge even in the 10% graph, as is clear in this blowup focusing in on the start of the Summer '21 surge:

That specific downward divergence problem only gets worse and worse as the hypothesized percentage of "spurious" deaths increase (as do other problems as well).  For this reason, I think that the 10% hypothesis has already slightly overshot the reality of how many coincidental deaths there actually are.  I would therefore put 10% as the upper cap on how much of the official death tally could be caused by purely coincidental deaths.

Another Important Factor: Amount of Time Shift

Another important thing to consider is how much the death graph had to be shifted back in time to match up with the infection graph.  Because removing spurious deaths takes deaths away from the left side of the death curve, in order to make the resulting curve match up with the infection curve, I had to increase the amount of time shift each time I increased the percentage of total deaths that I deemed spurious.

For the hypothesis that 0% of the total death tally is spurious, I had to shift the deaths back 20 days to get them to match up with the infections properly.  I had to increase this a few days for each subsequent graph, all the way up to 30 days of time shift or the graph where I assume 50% of the total death tally is spurious.

Here it is important to note that the average time-to-death from infection has been established independently based on case studies, and it's normally given at something in the range of 18 days.  This also argues against positing that the total percentage of spurious deaths goes very far above 0%--it's another way that the hypothesis results in unrealistic data the larger this percentage gets.

Some Closing Comments on this Analysis

1. Just to comment in case someone was confused: yes, there is a clear divergence between deaths and confirmed infections at the beginning of the graph.  This is a known issue caused completely by the fact that at the beginning of the pandemic we had very poor testing, meaning that the actual amount of Covid infection was far higher than what appeared by the number of confirmed Covid cases.  This hasn't been a problem since mid-last year.

2. One objection might be made, suppose there were other causes of overreporting aside from purely coincidental deaths?  This analysis doesn't rule those out per se, however given how well the time-shifted deaths matches the infections (when scaled), those causes of overreporting would have to be somehow time-matched to actual Covid deaths.  That is, the overreporting would get worse when *actual deaths from Covid* go up (not just Covid infections) and get better when these deaths go down.  I have not yet been able to think of a cause of overreporting that would be proportional to correct reporting in such a way.

3. Finally, I should note that this analysis will only catch overreporting of deaths due to coincidental Covid infections.  It would not catch any *underreporting* of Covid deaths.  Most causes of underreporting that you might think of would actually be time-matched with the actual deaths: for example, suppose elderly people with severe comorbidities who died of heart attacks due to stress on their system caused by Covid were sometimes thought to have died just from the heart attack, because it was known that their hearts were weak already.  In this case, a certain percentage of deaths actually caused by Covid could be put down as "just heart attacks" by whomever recorded their death. 

This could happen on a regular basis a certain percentage of time and it would not show up as an anomaly on this kind of a comparison graph, since the deaths are just scaled up to match with the infections anyway.  That would be one time-matched factor causing deaths to be *underreported*, and others could also be easily thought of.

This means that this particular analysis does not offer any sort of cap on how much the official death tally might be under-representing the actual death toll of Covid.  More on this point in Part 3.

Conclusion

The hypothesis that a significant portion of the official tally of Covid deaths are actually coincidental and result from some other cause is consistent with the timing of those deaths, but only if the total proportion of coincidental deaths is held at about 10% or below.  Meanwhile, the possibility that there might be signficant *undercounting* of Covid deaths for other reasons is still, at this stage, a possibility.

Dying "with" vs. Dying "from" Covid, pt. 1

You can still find people claiming, nowadays, that the official death toll from Covid is an overstatement of the actual deaths caused by Covid.  The rules for reporting deaths as "from" Covid, they say, are far too broad, and many people who are dying of other causes but simply happen to have Covid as well are being counted in Covid's official death toll.

There have been several serious flaws in this argument from the beginning, in my opinion.  First, proponents of this theory have frequently misread official guidelines for diagnosis or misapplied guidelines made at one level of government to local hospitals.  It's been a confusing set of changing guidelines, and unfortunately the tendency has been to jump on any rule change or guideline that supports this theory and publicize it widely, while ignoring rules or guidelines that don't.

Second, and more importantly, proponents of this theory have typically overemphasized the rules and down-played the common sense that actual humans writing death certificates bring to the table.  In my experience, there is quite a lot of interpretation according to common sense when it comes to the medical field.  Neither doctors nor nurses typically spend a lot of time robotically applying the exact written rules without regard for what they think is likely the right thing to do.  So I think it likely that common sense and good judgment is going to eliminate a lot of obviously wrong diagnosis.

Note that I am consistent in applying this principle.  At some point, the official rules for reporting adverse effects of a vaccination were updated to include guidelines specifically for Covid, and these rules are ridiculously strict--you are supposed to report any serious side effect after a Covid vaccination, whether you think it is linked to the vaccination or not ("regardless of causality", see text here: Reporting Adverse Events Following Vaccination).  This is explicitly different from the normal vaccination side effect reporting rules.  And yet I've never assumed that this rule has been followed completely, which--if it were--would imply that 100% of adverse side effects from the vaccines were being reported.  I believe the percentage is pretty high--but I also believe that despite the official rules, you will still get a lot of doctors applying common sense and saying, "no, I don't think that adverse effect is related" and not reporting something.  This, in my experience, is how the medical field operates most of the time.

However, I admit that these reasons are not super convincing.  Basically, how well you trust the numbers boils down to how much you trust the average hospital reporter to apply common sense.  And it's reasonable to have greater or less trust in these people, depending on your experiences and knowledge of the field.

So the question is, do we have any better, objective way of determining how often people's deaths are attributed to Covid purely because of coincidence?  And at this late stage in the pandemic, we do.  We now have a lot of data to work with and we can do some "forensic analysis" to get an idea of how often this happens.  We probably can't get a precise percentage, but we will be able to put some bounds of plausibility.

In Part 2 and Part 3, I will demonstrate two different ways of quantifying how often this kind of coincidental death occurs.

Pandemic of the Unvaccinated

Here is a heat-map showing percentage of Black population in each county in the U.S.:

Original Source: https://www.reddit.com/r/MapPorn/comments/gdwty5/percentage_of_black_population_in_the_us_by_county/

I have highlighted areas of particularly dense concentration in green.

Here is a heat-map showing percentage of Hispanic population in each county in the U.S.:

Original Source: https://en.wikipedia.org/wiki/File:2010_US_Census_Hispanic_Population_by_County.svg

I have highlighted most of the areas of particularly dense concentration in purple.  For reasons I'll explain in a bit, I've left out California.  I've also highlighted one county in Oregon that has a particularly low concentration of Hispanics.

Now here is a heat-map showing the current spread-rate of Covid per county.  I have transferred those same highlights from the previous two maps to this map (by hand, in Paint, so excuse the crudity of my model):

Original Source: https://twitter.com/EricTopol/status/1426549915581251590/photo/2

My highlights from the previous maps are covering the bulk of the high-transmission counties.  So then the last piece of data is the vaccination rates of Black and Hispanics:

Original Source: https://kff.org

These vaccination rates are only national averages, and are thus not the whole story.  Vaccination rates of Blacks in the south seem particularly bad, when I look at them.  In Florida, they're terrible--high 20s if I remember correctly.

So obviously, I'm driving at a pattern here.  My hypothesis is that a very significant portion of this surge in Covid is being driven by Black and Hispanic communities with low levels of vaccinations.  Remember: diseases don't actually spread geographically so much as they spread through socially connected networks of people.  Therefore, it doesn't matter that Florida actually has an above-average vaccination rate if there exists within Florida a community of socially connected people having a shared low vaccination rate.  In this case, Covid will spread through that specific community at a rate that is high concomitantly with their lack of vaccination.

A few anomalies on the combined map

There are a few things on the Covid heatmap that might seem a bit anomalous, given my hypothesis.

1. California and Wyoming maybe seem a little reversed compared to what you would expect.  California should maybe have more Covid spread based on its high percentage of Hispanics, and Wyoming seems anomalously high, maybe.
   
   But California also has had one of the strictest anti-Covid regimes of any State in the Union, fairly consistently from early on in the pandemic.  I think we can maybe see the result of this type of policy in the heatmap.  Other places where I might see the result of public policy are Virginia--which has an interesting clear demarcation from North Carolina, and also has been more Covid-cautious in its public policies--and New York, which became much more Covid-cautious after early disaster.
   
   And as for Wyoming, I don't think it actually fits that badly with the hypothesis--its most infected county is, after all, also its county with the highest percentage of Hispanics, and the state overall does have a fair share of Hispanics.
   
   I also suspect that if you really dug into the statistics (if you could get them), you would probably find that there was an inverse correlation between social "class" and low vaccination rates, as well.  I do know that Wyoming is ranch-heavy and therefore hires an awful lot of migrant worker, and I suspect vaccination rates among them are *quite* low.
   
   
2. There is that one area in Oregon which has a very low percentage of Hispanics, but a very high Covid transmission rate.  This, it turns out, is the exception that proves the rule.  This area is comprised of two counties: Douglas and Josephine counties.  And although those two counties may not have high percentages of Hispanic populations, for whatever reason they are considerably less vaccinated than other counties around them:
   
   

   

   Vaccination Rate per 10,000: Taken from the Oregon Health Authority COVID-19 Site

   So this just highlights that the problem here isn't race or ethnicity per se: the problem is vaccination rate among socially connected persons.

So, what we have been told is in fact correct: this is--now--a pandemic of the unvaccinated.

What does this imply?

I think the implications of this reality are pretty straightforward: the highest priority for ending this pandemic in the United States should be increased vaccination, and the area where this most needs to happen is in Black and Hispanic communities.  How we increase vaccination in these communities . . . I have no idea myself.

Furthermore, I think we need to be particularly concerned with the vulnerable people in those communities: the elderly, the sick, and the immunocompromised.  Greater effort should now be exerted, I believe, to seek out those individuals for vaccination.

One thing that has puzzled me recently is how the death rate from Covid compared to infections has not really decreased *that* much since vaccination.  Data is still sketchy on this, but my initial estimations put it at 1/2 to 1/3rd what it used to be before the vaccines.  That's better--but it doesn't match up very well with the great efficacy we have been seeing in the vaccines preventing hospitalization and death, *and* the relatively high rates of vaccination among those most in danger.  If 80% of the elderly are vaccinated and the vaccines are 95% effective at preventing death, then you wouldn't expect the death rate among the elderly to drop just by 1/2 or 2/3rds--it should be a lot more.

But if the spread is happening primarily in communities in which vaccination is low, this now makes a lot more sense.  In fact, deaths from nursing home residents have fallen drastically as a percentage of overall Covid deaths since vaccinations (see New COVID-19 Cases and Deaths Among Nursing Home Residents Have Dropped Since Vaccinations Began).  But not all elderly and infirm live in nursing homes; plenty of them live at home with family among these vulnerable communities.  I believe it very likely that the bulk of the deaths from this latest surge of Covid are coming from these people: elderly, hesitant unvaccinated parents of hesitant unvaccinated children.

So none of this gives us a way forward, per se.  But I think it *does* give us a focal point and I would like to start hearing more discussion about how we are going to solve this specific problem.

Efficacy, Effectiveness, and the Prevalence of Vaccinated among the Hospitalized: Part 2

Now let's look at an example case in which the efficacy of the vaccines in preventing infection and hospitalization has seemed to be inadequate.  It has been reported (see This 900-person delta cluster in Mass. has CDC freaked out—74% are vaccinated) that an outbreak of Covid in Barnstable County, Massachusetts was a key datapoint in the CDC reversing its recommendation on mask wearing for vaccinated people.  The CDC report on the outbreak is available here, and the key worrisome facts about this outbreak is that a full 74% of the people who became sick were vaccinated, 4 out of the 5 people who were hospitalized were vaccinated, and the Ct values (i.e., roughly how many particles of virus were found from nasal pharygeal swabs of the infected) of the vaccinated and non-vaccinated were "similar".

These numbers do make it seem, on the face of it, that the vaccines aren't doing very much to limit infection spread among the vaccinated.  However, as we saw in the previous post, it is possible for those overall numbers to be misleading, especially if there is a chance that a significant proportion of those people who became infected or hospitalized were immunocompromised.  And it turns out that this is likely to be the case.

The nature of the outbreak in Barnstable County

According to the CDC paper, the outbreak in Massachusetts was the result of "multiple summer events and large public gatherings were held in a town in Barnstable County, Massachusetts, that attracted thousands of tourists from across the United States".  The Massachusetts Department of Public Health, when they interviewed people associated with the outbreak found that people "reported attending densely packed indoor and outdoor events at venues that included bars, restaurants, guest houses, and rental homes."  But the CDC paper also contains this line in the discussion on limitations of its findings at the end of the paper: 

Third, demographics of cases likely reflect those of attendees at the public gatherings, as events were marketed to adult male participants; further study is underway to identify other population characteristics among cases, such as additional demographic characteristics and underlying health conditions including immunocompromising conditions.

What sorts of events are "marketed to adult male participants"?   It turns out that Barnstable County, MA, is the most popular summer vacation destination on the East Coast for LGBT+ vacationers.  It has the highest rate of gay marriage in the entire country.  In the middle of July, they host something called "Bear Week", which is essentially a week-long party for gay men.  During this period, the town population increases 20-fold (from 3,000 people to as high as 60,000) as a result of out-of-town LGBT+ vacationers.  (Wikipedia: Providencetown, Massachusetts)

Neither the CDC report nor the Massachusetts July Covid Update (July 30, 2021 | Update: COVID-19 Cluster in Provincetown) say how many of those who fell sick were gay men.  However, the Massachusetts report *did* show that a full 89% of the people who were sick were male, mostly young.  While Covid has been shown to impact men more seriously than women if they get sick, both genders will get sick at approximately the same rate.  For the natural balance to be upset to that degree, something like 728 of the 934 people who got sick (or 78%) would have to have been gay men.

The CDC report does mention a fairly small number of the patients (30) from the outbreak whom they had already confirmed had an HIV diagnosis--but this was mentioned as being a *preliminary* finding, and they only confirmed by cross-indexing the Massachusetts index of people registered with HIV.  Given that most of the people who got sick were from out-of-town, and given that registering as HIV positive is optional, this number is certain to be too low.

In fact, it's been found in random surveys that more like 1 in 5 of young gay men who frequent bars have HIV, and about half of those people don't even know it yet: (cf: 1 in 5 Gay/Bi Men Have HIV, Nearly Half Don't Know).  So this means that something more like 146--not 30--of the patients in the outbreak are likely to have been immunocompromised.

How many vaccinated people in Barnstable should we have expected to be hospitalized?

The answer to this question depends very much on how many of the out-of-town vacationing gay men were fully vaccinated.  I would *think* that this number would be very high.  If I were a gay man looking to party for a week in crowded settings with other gay men, I would want to make sure I were vaccinated ahead of time.  So I think that my estimate of 90% vaccination rates from Part 1 seems very reasonable.  I admit; this is a guess.  But let's assume this is true for now.

In Part 1, the various multipliers for hospitalization that we came up with work out to a 450x greater likelihood of hospitalization for immunocompromised vaccinated individuals compared to immunocompetent vaccinated individuals.  If you applied this multiplier to the 146 likely immunocompromised people in Barnstable, then you would expect to see about 80 vaccinated patients hospitalized from that group for every single patient hospitalized outside that group.  The actual proportion of vaccinated to unvaccinated was 4 to 1.  So based on my numbers from Part 1, the vaccines seem to be working more effectively than I would have expected.

The difference is so great, in fact, that I suspect my assumptions from Part 1 are wrong.  In particular, I think that my assumption from the South African Novavax study--that the vaccines would be not effective at all for the HIV positive--might be incorrect.

If you expect 80 people to be hospitalized, but only 4 are, this translates to a vaccine efficacy of about 95%.  And indeed, this is roughly what the CDC has been reporting--that the vaccines remain something like 95-96% effective in preventing hospitalization.  So I see the numbers of hospitalized in Barnstable, even given how many immunocompromised were likely in that population, as tending to confirm the efficacy of the vaccine--as far as it goes.  There will still be more immunocompromised people being hospitalized from Covid than immunocompetent, but if you compared immunocompromised to immunocompromised only, the vaccinated will have roughly the same comparative advantage against the unvaccinated.

How many immunocompromised would we have expected to get sick?

So much is good for the issue of hospitalization.  However, the CDC did not look at the Barnstable results and say that the vaccines weren't preventing hospitalizations: they worried (apparently) that the vaccines weren't preventing infection and spread.  So let's look at the reported numbers of the symptomatic infected more closely.  

I estimated in Part 1 that immunocompromised people were twice as likely to catch a disease (at least, symptomatic disease) in the first place compared to the immunocompetent.  This, however, was just taking into account their diminished capacity to fight off a disease rapidly.  In the case of the Barnstable County outbreak, you also have a large group of people that includes (most likely) many immunocompromised people who are also engaging in much higher risk behavior: packing themselves into crowded bars and restaurants for a week-long party.  In this specific scenario, what effectiveness of the vaccine for the immunocompromised would have to obtain in order for the vaccinated to make up 74% of the infected?

I have put together another section of the spreadsheet (link again here: Efficacy vs. Effectiveness) that attempts to model this scenario.  The key assumptions for the Barnstable County outbreak that I am making are as follows:

• 50,000 visiting LGBT+ vacationers.
• 90% of visiting LGBT+ vacationers are vaccinated (compared to 69% of the locals, which was the value reported).
• A baseline vaccine efficacy of 80%
• A reduced vaccine effectiveness for HIV positive people of 50% the baseline (i.e., 40% in this case)
• A 2x multiplier for HIV positive people to come down with symptomatic Covid if they are infected.
• An 8x "exposure rate" for the vacationers compared to the locals to account for the crowded party activities.

Given these assumptions, I can come up with a percentage of the vaccinated among the total infected of 73%, which is right in line with what actually happened.  In other words, it is not necessary to conclude that vaccine effectiveness at preventing disease spread has fallen very far at all in order to see large numbers of vaccinated become ill, in this specific scenario.

VERY Important caveat on interpreting statistical results of models

If you are not very experienced with this sort of analysis, you might mistakenly think that this result is amazingly accurate and therefore must reflect reality.  That I should be able to so accurately reproduce the real-life numbers with some reasonable inputs into a model might seem proof-positive that the immunocompromised are the real reason for the vaccinated making up 74% of the infected people from the Barnstable County outbreak.  But if you thought this, you would be very wrong.

In reality, I had to tweak many of the inputs in this model in order to come up with a percentage very close to the observed percentage.  I constructed the spreadsheet, put in some initial numbers, and then tweaked those parameters that I thought could be realistically tweaked, until my end result finally said "73%".

Since none of my parameters are outlandish, but all could potentially be realistic and true, it is correct to say that my theory (that immunocompromised are almost completely the cause of the scary proportion of vaccinated individuals from Barnstable) is consistent with the available data.  "Consistent with" is a very different statement from "proof of", and it is important to be aware of the difference between a study claiming one and a study claiming the other.

Sensitivity Analysis

It is for this reason that whenever I do a rough-model like this, I provide the link to the spreadsheet, and I encourage anyone who reads my estimates to look at the spreadsheet and make changes--play around with different parameters to see what sorts of result are generated with the new numbers.

When you do this formally and rigorously, this sort of thing is called "sensitivity analysis": you systematically change each parameter--individually and in groups--and determine which parameters are the important ones that actually make a difference on the outcome.  Good statistical packages can do this for you automatically nowadays--though you have to be sure you include all of the relevant parameters as inputs to the program!  "Garbage In, Garbage Out" is still a very true dictum.

Understanding the dynamics of which parameters make a difference to the outcome can do several things for you:

First, it allows you to see what aspects of the problem are more important to get clarity on.  For example, in my toy model, I found that although I accounted for the difference between the local population and the vacationing population, I really didn't need to bother.  The vacationing population is so much bigger, the effects of the local population on the outcome doesn't really matter.  This tells me that the "69% vaccinated" rate of Barnstable County that was reported in the CDC paper and in a lot of news outlets, is really irrelevant. 

Second, it allows you to see how reliable your result is.  In the case of my toy model, I am able to see that my result is not very reliable at all--I have some excessively sensitive parameters that are also too much of a raw guess on my part.  The key number here that makes all the difference is the vaccination rate of the vacationers.  I set that at 90%; if instead you set if lower (to 80%, for example), you have to set the effectiveness of the vaccine way down (to somewhere around the 40% level) in order to still end up in the neighborhood of 74% vaccinated among the infected, if you keep all the other parameters the same.

Likewise, the proportion of HIV positive individuals among the vacationers has a huge impact on the result, and my proportion of 1/5 is taken from a single study of general trends, not any sort of specific survey of this particular population.

Therefore, what this model really proves is only that the events of Barnstable County are currently capable of multiple interpretations.  If we want to know what is really going on in this outbreak, we need more information.  In particular, the prevalence of both vaccination and HIV in the vacationing population are very important for a correct interpretation: both of which might be very difficult to obtain at this point.

Bottom Line

The bottom line conclusion of this must be that we cannot make firm conclusions about the real-world effectiveness of the vaccines from the Barnstable data as it has been reported to us so far.  We would need a far better knowledge of other variables at play--other risk factors--in order to know which variables aside from vaccination effectiveness may have caused different people to end up infected or in the hospital.

This conclusion is especially true in the case of the Barnstable data given that the outbreak there occurred under circumstances far from normal for the national population in ways that are materially relevant to disease spread and vaccine efficacy.  But the conclusion is also true for a lot of other data that has been bandied about by many people.  In general, the prevalence of vaccinated among the hospitalized is a very bad statistic on which to make conclusions.  There are far too many confounding variables that are in play--far too many risk factors which dramatically change the likelihood of hospitalization independently of vaccination status--for this bare statistic to be of any use without a whole lot of other data about those people.

The gold standard for judging the effectiveness of a vaccine is the ability to compare vaccinated people versus unvaccinated people when you are able to control for all other variables.  You want to compare vaccinated sick old men with unvaccinated sick old men, vaccinated teenage girls with no health problems with unvaccinated teenage girls with no health problems, vaccinated middle-aged gay party-goers with HIV to unvaccinated middle-aged gay party-goers with HIV.  And so forth, and so on.

Simply comparing the raw numbers of hospitalized vaccinated to hospitalized unvaccinated people--with no differentiation--is going to be comparing apples to oranges with a vengeance.

Efficacy, Effectiveness, and the Prevalence of Vaccinated among the Hospitalized: Part 1

There has been a lot of angst, recently, over how many vaccinated people are coming down with Covid, even to the extent of being hospitalized.  A lot of people are coming to the conclusion that vaccine effectiveness has been waning--either because the new Delta variant is escaping suppression, or because the vaccines are losing effectiveness over time, or both.

There is some truth to those fears, for sure.  However, I believe that neither fear plays as much a role in uncomfortable numbers of vaccinated people getting sick as a lot of people think.  A third factor that you need to consider when you look at the number of people who are vaccinated who are also getting sick or being hospitalized is the difference between vaccine "efficacy" and vaccine "effectiveness".

I am going to explain what the difference is between these terms and why it matters in Part 1.  In Part 2, I am going to look at an important case study in which this distinction might be very important.

Efficacy vs. Effectiveness

These two words--"Efficacy" and "Effectiveness"--are technical terms in immunology.  Briefly, "efficacy" refers to how well a vaccine reduces disease in an ideal, properly balanced clinical trial.  "Effectiveness" refers to how well a vaccine reduces disease in the real world, given a distribution of recipients that is limited by real world constraints rather than the artificial constraints of a clinical trial.

When a vaccine (or any other medication, for that matter) is tested in a clinical trial, the goal is to identify how well the vaccine works compared to non-vaccination, all other things being equal.  A well-designed clinical trial will identify all characteristics of a subject that might have an effect on the outcome of the trial.  The trial runners will then balance the trial vs. the placebo groups so that an equal proportion of each characteristic appears in each group.  That is, both the placebo and trial groups should contain the same proportion of elderly to young people, of each race and gender, of sickly vs. healthy people, and so forth.  If there is a certain population that is more (or less) susceptible to the disease in your trial, and you do not balance that population properly between your trial and placebo groups, then your final results will be biased by the properties of this particular population.

If a clinical trial is properly balanced, then you can take the difference between how many people get sick in the vaccinated group vs. how many get sick in the unvaccinated and calculate your vaccine *efficacy*.

In the real world, however, there is no one balancing out the vaccinated vs. the unvaccinated populations.  People get vaccinated or not for reasons other than balanced, random chance.  Therefore it is quite possible (and actually inevitable) that, in the real world, the group of people who are vaccinated will have different characteristics from the group of people who are unvaccinated.  This means that you should not expect that the difference between the vaccinated and the unvaccinated who get sick in the real world to reflect the same efficacy as was found in the clinical trials.  What you get from this calculation is the *effectiveness*, and this can be skewed from the "efficacy" number for a lot of reasons.

Risk Averse Behavior

The most critical reason effectiveness can be skewed from efficacy (or at least the reason that *I* think is the most important) is the very nature of human behavior relating to risk.  Simply put, it is pretty obviously true (if you think about it), that those people who are more naturally at risk from a disease will choose to take a vaccine against that disease at a much higher rate compared to people who do not feel similarly at risk.  Given that this is true, you should expect, in the real world, that the group of all vaccinated people will contain many more naturally at-risk people that the group of all unvaccinated people.

How large of an effect will this have?  Well, it could actually have quite a large effect, depending on the specifics.  I think there are, broadly, two different types of person this distinction applies to, which I will now describe.  Then I will try to estimate the magnitude of this effect with some reasonable guesses and a spreadsheet.

At-Risk of Exposure

One type of person is more at risk of catching Covid because of an occupation.  This type of person may have a job that brings him into close contact with a lot of potentially sick people on a regular basis.  The classical example here is health care workers; they are obviously exposed to Covid (at least potentially) a lot more than other people are.  Now, some early studies showed that health care workers weren't getting sick all that more often than other people (see Prevalence of SARS-CoV-2 Infection Among Health Care Workers in a Tertiary Community Hospital)--but those studies were done during a time in which rather extreme Personal Protective Equipment (PPE) practices were in place.  Nowadays, practice has tended to relax a lot further and we are looking at close to business-as-usual behavior.  So I think it is fair to say that healthcare workers will very likely be exposed to a lot more Covid virus than other people.

For this type of person, both natural immunity and vaccination would be expected to work as well for them as for anyone else.  However, given that they will be exposed so much more, the vaccination will be given much more opportunity to fail for these people than for others.

At-Risk of Infection

Another type of person is not necessarily more exposed to Covid, but might be more naturally at risk of catching the disease when they are exposed.  The most important group here, I think, is the immunocompromised.  This is a larger group of people than most realize: it includes the HIV positive, people on a whole array of immunosuppresive drugs, people undergoing cancer treatment, and people with Down syndrome--to name just a few.  Randomized surveys have shown that a full 2.7% of Americans at any given time have been diagnosed as immunosuppressed. (Prevalence of Immunosuppression Among US Adults, 2013).  And I think this number might be understating the total effect, because there are certain conditions that typically don't get you outright diagnosed as "immunocompromised", but which actually do have some compromising effects on immunity.  The two most important of these conditions are age and obesity, both of which are known to exert some suppressing force on immunity (see Impact of Obesity and Metabolic Syndrome on Immunity) and which are together the two most important "comorbidities" for Covid.

How much effect immunocompromising conditions might have on the effectiveness of vaccines is not completely known--but it might be rather severe.  The Novavax study in South Africa included a cohort of known HIV positive test subjects in its vaccine trial.  The sample size was not large--which makes its results here uncertain because of lack of statistical power--but insofar as the results are reliable, they indicate that the Novavax vaccine was not at all effective in the HIV cohort (Efficacy of NVX-CoV2373 Covid-19 Vaccine against the B.1.351 Variant).  The trial found the vaccine pretty effective for other people, but did not see any effectiveness for the HIV positive--in fact, slightly more vaccinated HIV subjects came down with Covid than unvaccinated HIV subjects.

Immunocompromised also have a two-fold problem: not only are they more prone to catching a disease, since their bodies are not well equipped to fight a disease once it gains a foothold, they are much more likely to be hospitalized from a disease once they catch it.

How much of an effect could these special cases have?

But, you might say, even if the vaccines don't work very well for the immunocompromised, that's only roughly a 3% portion of the population.  That couldn't mess up the numbers that badly, could it?  Yes, it could.  Covid attacks people very unevenly; the more vulnerable people aren't just a little bit more likely than everyone else to suffer badly from Covid, they are far more likely to suffer badly from Covid.  This uneven distribution of adverse effects means that even a pretty small population of very vulnerable people can have a very substantial effect on the makeup of the total hospitalized and dead.

To illustrate this, I have put together this spreadsheet with what I considered simplified but still reasonable numbers, here: Efficacy vs. Effectiveness

In this spreadsheet, I setup a group of 1 million people, and then determine who gets sick and who gets hospitalized based on various parameters.

Here are the assumptions I make for this spreadsheet:

• The average person will have a 1% chance of catching Covid.
• 5% of the population will be in "high exposure" occupations, which I set to 4x the regular exposure to the virus.
• 3% of the population will be immunocompromised.
• The immunocompromised will be 2x as likely to catch the disease compared to everyone else (a guess, but I think reasonable based on my experiences with an immunocompromised daughter).
• The vaccine will be 88% efficacious for normal people.  
• It will be 0% efficacious for immunocompromised people.  (This is consistent with the Novavax trial results even if not proven by them.)
• 5% of unvaccinated people with functional immune systems who catch Covid will end up hospitalized
• The vaccine will be 96% effective at keeping people who come down with Covid from being hospitalized.
• 40% of immunocompromised who catch Covid will end up being hospitalized.
• 80% of people in high-exposure occupations will be vaccinated.
• 90% of immunocompromised people will be vaccinated.
• 60% of everyone else will be vaccinated.

These are all reasonable assumptions.  They may not be quite accurate and some of them are fairly bald guesses, but nothing here is outlandish and nothing contradicts anything I actually know to be true.

With these assumptions, what you will see as far as numbers of actual infections and hospitalizations is that 1394 people who are vaccinated will become infected, whereas 4140 who are unvaccinated will be.  This translates to an effectiveness of approximately 66%.  Meanwhile, 218 vaccinated people will be hospitalized, whereas 228 unvaccinated people will be--which makes it seem as if the effectiveness of the vaccine at preventing hospitalization is basically 0%.

And remember--these results are assuming a very robust protection from the vaccine for 97% of the population: 88% protection against infection and 96% infection against hospitalization over and above that protection.  So we have a seemingly paradoxical result that even with a vaccine that is highly effective, as many vaccinated as unvaccinated are seen to be suffering badly from the disease.

Bottom line conclusion: It is very easy for important sub-groups of your population to completely throw off your bottom-line numbers if you do not interpret the results correctly.  So make sure you understand the sub-groups of your population and account for their characteristics in any study that you do.

In the case of how many people are vaccinated yet also hospitalized, we need to pay careful attention to what other risk factors these people have.  It is not necessarily the case that high numbers of vaccinated people in the hospital implies inefficacy of the vaccine.

Decrease of Vaccine Efficacy against the Delta Variant?

There have been some worrisome headlines recently to the effect that the vaccines are not very effective against the Delta variant of Covid.  Here's one example reporting the "39%" number that Israel has been reporting:  https://www.cnbc.com/2021/07/23/delta-variant-pfizer-covid-vaccine-39percent-effective-in-israel-prevents-severe-illness.html.

I'd like to put my oar in and give an opinion on what I think may be going on.  I'd like to stress that this opinion is less well formed than some of my other opinions, so if anyone has contrary evidence, I'd be happy to hear it so that I can revise what I think.

To explain what I think is happening, first I need to explain some basics--(be warned that this is a dramatic oversimplification):

How immunity works

I think maybe "Immunity" is an unfortunate word, because it has an absolute feel to it: as if whoever is "immune" has an absolute invulnerability to a disease.  This isn't how immunity to disease works, however.  It's important to realize that immunity is the result of specific cellular functions and therefore operates in a specific way that can offer various degrees of protection.

Let's start at the beginning: when a virus first enters the body of an individual who has never before encountered one of its kind.  At this time, it enters a cell, replicates and spreads to other nearby cells.  Fairly soon, it will start to encounter cells that form part of the body's innate immune system--but successful viruses are able to spread an multiply fairly well for a time before triggering a powerful system-wide immune response.  So for a time, the virus is able to spread exponentially from cell to cell without a lot of opposition.

The *timing* is important here: how fast does the body mount an immune response (elevated temperatures, increase immune cell production, etc.), compared to how fast can the virus replicate throughout the body?  In a typical case of first-time Covid, the virus is replicating throughout the body ("incubating") for maybe 3-7 days before the body's defenses really begins noticing things and starts to kick-in with the defense.  This is why Covid can spread so quickly throughout a population--those last few days of non-defended replication can make a person a font of virus particles for other people before they feel ill themselves.

In addition to mounting a systemic immune response, the human body also begins producing a virus-specific response: neutralizing antibodies.  These are tailor-made particles of just the right shape and properties to stick to the invading viruses and de-activate them (mostly, as we know, by gumming up the spike protein and thus preventing the viruses from entering cells in order to replicate).  The initial production of these antibodies, however, takes some time to ramp up--it's something like 7-14 days before we see any of these of note.

Typically, the combination the innate immune system and neutralizing antibodies will be enough to clear the system of the virus before very long.  The person can be sick for some time afterwards (for reasons I'm not clear on and I don't know if anyone *else* is clear on either).  One thing that has been seen for Covid, however, is that people can test positive for Covid for some time after they have been mostly free from symptoms but that these people very rarely spread Covid to other people.  After the body starts producing neutralizing antibodies, those virus particles that are shed by the sick person tend to come out pre-coated with those antibodies--meaning they don't have much capability to infect others.

How immunity continues after illness or vaccination

Once the body has fought off the disease, the innate immunes system settles down.  However, those specific neutralizing antibodies are still produced at a high level for some time--the body remains "alert", as it were, to the presence of the disease.  This doesn't mean that there is some magic shell around the body repelling the virus, however: if another copy of the virus makes it into your nasal mucus membranes, they will again begin to reproduce.  Immunity isn't some chemical property that infuses all of your cells--it's a "herd immunity" of a collection of cells.  So infection will begin to spread again just fine.  The presence of the antibodies, however, means that the infection will not get very far--some cells will be infected, but enough replicated virus particles will become coated with antibodies that the colony of infection will not spread robustly, but rather will die out.

How quickly this happens depends on several factors: how innately good the virus is at spreading, how many antibodies are being produced by the host, and how effective those antibodies are at impeding the growth of the virus.  For some diseases, the antibodies we produce in response to the virus are incredibly effective at stopping the spread of the virus, so that even a small presence of those antibodies are sufficient to very rapidly halt the progression of a viral infection from cell to cell.  The medical field calls immunity to diseases of this sort "sterilizing immunity", because it is as if the body is completely impervious to a disease after getting it.  In reality, there is still some very small level of infection and replication inside the body if the particular virus gets in--it's just that the spread is *so* small that it is not noticeable and produces no relevant results.

With other diseases, however, the neutralizing antibodies are not quite as effective.  And this starts to matter after a while, because the body doesn't keep up production of neutralizing antibodies for a specific disease at a high level, permanently.  It always keeps some around, but it is normal for the levels of these neutralizing antibodies to drop over time.

What the body relies on, after some time has passed, is the ability to ramp up production of these neutralizing antibodies again.  The immune system has a memory mechanism, by which once it has produced antibodies to a particular virus, it can remember and ramp up production of that same antibody if ever the virus comes back.

All that has been said here applies essentially the same for illness and for vaccination.  The bodily responses are not identical, but the basics here are the same.

The Dynamics of the Immune Response

How your body responds to a virus when you have been reinfected after you have developed an immunity therefore depends on several circumstances.

First, what are the remaining levels of neutralizing antibodies in your system?  If they are currently still high, the virus may never have a decent shot at colonizing your body a second time and the infection will rapidly die, likely without you even noticing it.  Since you have a bunch of antibodies in your system, while the infection is progressing inside you, you are less likely to be infectious as virus particles you expel will tend to be coated in antibodies.  So you could test positive for Covid, but not be a risk to anyone around you.

If your levels of neutralizing antibodies are lower, however, the virus might start successfully spreading throughout your body, up until the point where your body takes notice of it and decides to start activating the innate immune response and ramping up production of the antibodies it already knows how to produce.

How sick you get and how infectious you will be to others now depends on timing.  It becomes a race between the virus becoming established and the body ramping up production of the neutralizing antibodies.  Factors for how this will end up include how healthy your immune system is and how effective your antibodies are against the new viral invader; so the outcomes can range widely.

What I think is happening with Delta and the vaccines

I believe that the dramatically decreased reported effectiveness of the vaccines against the Delta variant are principally due to this dynamic and not principally due to the genetic drift between Delta and original Covid.  It has been shown that neutralizing antibodies produced against original Covid do not work quite as well against Delta as against the original--but I believe this translates into a decrease of vaccine effectiveness something like from 95% to 88%, not all the way down to 39%.

Instead, what I believe is happening is that immunized individuals are getting reinfected with Covid after their neutralizing antibodies have waned somewhat.  There is then a period of time in which the virus is able to colonize their bodies before immune memory kicks in.  This triggers both the emergence of symptoms (which are what get noticed) and rapid uptick of production of neutralizing antibodies.

This is why Israel has been seeing a large increase in infections, but not a correspondingly large increase in hospitalizations or deaths.  In fact, I believe the symptoms that are being reported in Israel as "symptomatic Covid" are primarily immune response symptoms, very similar to what the vaccinated experience on their second dose of the vaccine.

I think evidence for this theory is reflected in the different experience that the U.K. has been having with Delta and the vaccines.  They have been seeing an effectiveness against symptomatic Covid with Delta in the vaccinated more like the 88% range.  But the crucial difference here is that the U.K. pursued a strategy in which they significantly delayed the second dose of the vaccine.  Therefore, most of those fully vaccinated individuals in the U.K. have not had the second dose until fairly recently.  Therefore their levels of neutralizing antibodies are still rather high, and the Delta variant is thus dealing with individuals who do not need a ramp-up time to fight it off.

What does this mean for infectiousness?

Does this mean that Covid is going to be able to spread well among people who were vaccinated back in December / January?  Is the time that the virus is able to reproduce in the body of people whose neutralizing antibody levels have waned somewhat enough time where it can then also spread to other people.

I think the answer to this question must be, yes, at least a bit.  I have a hard time thinking that this won't increase somewhat the ability of the virus to spread from vaccinated individuals.  However, that doesn't mean that I think that this will be a major factor.

The issue is, again, the timing.  While it has been shown that in some cases, Covid can incubate and spread as quickly as in a single day, this is not very common.  More commonly, a person becomes significantly infectious 3-5 days after becoming infected themselves.  Meanwhile, we have carefully watched antibody levels during the vaccine trials, and we see from that that whereas it takes the body a full 7-14 days to start making any antibodies at all after the first shot of vaccine, the second shot of the vaccine produces an almost immediate spike of antibody production--within a single day.

I therefore think that it is likely that in the large majority of cases, a vaccinated individual with lower levels of antibodies will begin producing antibodies and thus become mostly non-infectious before the virus gets a chance to truly blossom.  Presence of symptomatic Covid in vaccinated individuals is therefore usually a sign of delayed but operational immunity, and not necessarily a danger of infectiousness.