Why have epidemiological forecasts been so wrong and what to do about it

If we look at the forecasts, we got from epidemiologists initially in the Covid-19 pandemic it has turned out that they have massively wrong. While tragic the number of people who has died in this pandemic has been much lower than forecasted.

The reason given by epidemiologists then is that that is because of interventions – lockdowns. But then you made the wrong kind of forecast – you forgot to forecast what would happen IF lockdowns were implemented.

Furthermore, how do you explain the numbers in South Korea, Taiwan and Japan? There were no lockdowns (until recently) and we haven’t seen a massive death told, which was forecasted by the kind of epidemiological models used for example by the epidemiologists at the Imperial College in the UK.

Similarly, in Sweden with no lockdown, which as the only European country did not have a lockdown. Despite of that the death toll in Sweden has not in general been higher than in other Western European countries. It should of course be mentioned that Sweden’s Covid-death toll has been higher than in the other Nordic countries, but also here government epidemiologists massively overestimated the number of deaths and the need to hospitalizations even after claiming to take the lockdown effects into account.

And the Covid-19 pandemic is not the only pandemic where epidemiologists have been wrong – on the upside.

They were wrong (generally) very wrong on HIV/Aids, Ebola, Swine flu and SARS.

The death toll from these pandemics never reached the levels predicted by leading epidemiologists and there never was the kind of “super spike” in number that standard epidemiological models seem to predict.

We need to take human behavior and technological progress into account

There are probably (at least) two reasons why the standard epidemiological models tend to be wrong in my view.

First, of all as economists have pointed out since the HIV/Aids pandemic standard epidemiological models ignore human behavior. People dislike being sick and hence will change behavior if they know that will reduce the likelihood of being infected with Aids or Covid-19.

The literature on economic-epidemiologic goes back at least to work done by among others University of Chicago economist Tomas Philipson in the 1990s on HIV/Aids and the present pandemic has spurred a lot of new research on this topic. Another example is professor at Cambridge University Flavio Toxsvaerd’s excellent work on what he has termed ‘equilibrium social distancing’.

Furthermore, numerous new studies clearly confirm the thesis that human behavior is very important in terms of mitigating the spread of a virus like Covid-19. American economist William Luther in a recent paper shows that people reacted to the news of the spread of Covid-19 before US States started to implement lockdown.

The same goes for Sweden where there has been no government mandate lockdown and for example school have remained open, but people has nonetheless significantly changed behavior.

The change in behavior before government mandate lockdown has also been shown in a very good blog post by Catarina Midões from the research institution Bruegel. She shows that Google searches for ‘restaurants’ was way down in a number of European countries well before lockdowns were imposed by governments around Europe. The graph below from Midões’ blog post illustrates this well.

A second reason why epidemiological models tend to overestimate the death toll from pandemics is that they ignore technological progress – or rather medical progress. The Aids epidemic of course is a prime example of this, but it is also likely to be the case with Covid-19.

Here we have to see medical progress in a broad sense – it could for example be that we become better at protecting the people most at risk – for example the elderly or the obese – and we get better at treatments. This does not have to be major medical breakthrough but gradually nurses and doctors as they learn small things will adjust their treatment of Covid patients and that on its own will gradual reduce mortality.

This is similar to the critique economists have had for centuries against environmental doomsayers like Thomas Malthus, Paul Ehrlich and Greta Thunberg. The world is not coming to an end as humans adapt all the time and are highly innovative.

Concluding, one can say that epidemiological forecasting today is done like weather forecasting, but it should be done a lot more like economic forecasting – you can’t change the weather by carrying an umbrella, but you can change the course of a pandemic by practicing social distancing.

If there are a pandemic people will react to that – during the present pandemic people are practicing social distancing and extra hand washing without government intervention. We just all want to reduce risk – at the lowest cost.

However, there likely is a more fundamental problem – economists would call it a Public Choice problem or a principle-agent problem

Epidemiologists don’t necessarily have the incentives to be right

Most epidemiologists are government employees – either working for the health authorities or government-funded research institutions.

They are generally not paid to be right. They are mostly paid to do reporting and research – not accurate forecasting.

Furthermore, a government-funded epidemiologist will not be rewarded for making a too optimistic forecast, but will likely receive a lot more funding and attention if they are making doomsday forecasts.

I am not claiming this is done on purpose, but incentives work – also in research and economists would make the exact same mistakes if they worked in the same ‘reward-system’.

However, if we compare epidemiological forecasting to macroeconomic forecasting there is one crucial difference and that is competition.

There are not one or two economists making forecasts on the US or Euro zone economies – there are a many. That means that forecasts can be compared.

Furthermore, we have the financial markets to tell a story. In February the global equity markets started telling the story that the global economy would take a major hit.

If macroeconomic forecasters had ignored that information then they would have been too optimistic. Similarly, now – markets are telling us that the recovery will be quite fast.

That is challenging macroeconomic forecasters making very gloomy forecasts – they have to explain their model assumptions and why they believe markets are wrong.

Markets might indeed be wrong from time to time, but they are unbiased and we also know that it is very hard to find anybody who consistently can beat makes – so if you are a professional macroeconomic forecaster forecasting something very different from what markets are “predicting” then your clients and others will surely question whether your forecast is off.

Furthermore, economists argue in public all the time about their forecasts, which means that policy makers and investors get a very good impression of different scenarios for the outlook for the economy. I haven’t seen a lot of debate among epidemiologists – at least not in public – about how we should expect the Covid-19 pandemic to develop.

It is, however, also well-known that government institutions such as Finance Ministries and the IMF make fairly bad forecasts as they often are politicized (see here and here). For that reason, these forecasts are mostly ignored by market participants.

To me it seems like most epidemiological forecasts are quite similar to economic forecasts made by Finance Ministries – and even worse because Finance Ministries’ forecasts will always be compared by the media to independent forecasts.

We don’t see this to nearly the same degree in epidemiological forecasting as there really isn’t a private market for pandemic forecast – at least not yet.

So to repeat:

1) epidemiological forecasting is not done in an institutional framework where precision is rewarded enough and where doomsday forecasts will get more attention and ‘middle-of-the-road’ forecasts.

2) There isn’t enough competition – there are simply too few epidemiological forecasters and too little competition.

If Covid-19 and other pandemics stay with us and continue to shock major parts of the global economic system going forward then this will change as it will become profitable to do good epidemiological forecasting. (In fact, there might be a business idea here and if you are interested in this drop me a mail: LC@mamoadvisory.com)

However, at the present good epidemiological forecasting isn’t especially rewarding for epidemiologist. Alarmist forecasts on the other hand are.

Please note that I am not claiming economists make good forecasts or that macroeconomic forecasting isn’t often very wrong. It is.

However, macroeconomic forecasting continues to develop and adjust in the market place and being right will yield great economic rewards, which pushes forecasters to do a better job.

We need a market for forecasting pandemics

However, we have something a lot better than economists at doing forecasting – and that is markets.

Financial markets being the ‘wisdom of crowds’ is unbiased and generally (weakly) efficient. That means they are a quite good guide for the direction the economy will take.

This is also why I for years have argued that policy makers should utilize so-called prediction markets to make decisions and why I for example have advocated that monetary policy should be focused on market inflation expectations rather than model-based forecasts.

Generally, we need prediction markets to help policy makers make the right decisions based on unbiased market forecasts.

That mean there would be great public benefit from having ‘global warming’ markets and a ‘pandemic market’

This of course also goes for economic policy – we need prediction markets for unemployment and real and nominal GDP. We already have ‘inflation markets’ given the existence of inflation-linked bonds. I have long argued this – see for example here.

Returning to epidemiological forecasting my point is not that ‘epidemiologists are bad” and “economists are good”. My point is that they operate in different incentive structures.

That said, forecasting economic growth over the coming 2-3 years is something “we” are used to and macroeconomic forecasting has been around forever. Each new shock is different, but not completely different. We have a lot of experience in forecasting macroeconomics.

On the other hand, forecasting the development in a pandemic means you start from scratch every time. That of course is very different and much harder than observing the same kind of shock over and over again.

But the incentive structures do help. Furthermore, the fact that we have seen a lot of “black box models” during this crisis doesn’t help. We need openness and transparency regarding model assumptions. And we need competition rather than ‘forecasting monopolies’

This is not a critique of epidemiologists, but a critique of the overall way forecasting is done and mostly how it is used to shape policies and it is a critique very similar to things I have said about economic forecasting for years.

Concluding, the world more than ever needs good epidemiologists, but more than that we need good epidemiological forecasts and that means that we as societies need to ensure that we have ‘markets’ for epidemiological forecasting rather than forecasting monopolies.

——

Contacts:

Lars Christensen, LC@mamoadvisory.com, +45 52 50 25 06.

See my profile at my Danish speaker agency here.