Prediction markets and government budget forecasts

Recently I have had a couple of posts (here and here) on biases in the forecasts of policy makers and why central banks and governments should use prediction markets to do forecasting instead of relying on in-house forecasts that might or might not be biased due to for example political pressures.

Anybody who have studied the forecasts of government agencies are well aware of the notorious biases of such forecasts.  Jeffrey Frankel in a recent paper “Over-Optimism in Official Budget Agencies’ Forecasts” documents strong biases in government growth and budget forecasts. Here is the abstract:

“The paper studies forecasts of real growth rates and budget balances made by official government agencies among 33 countries. In general, the forecasts are found: (i) to have a positive average bias, (ii) to be more biased in booms, (iii) to be even more biased at the 3-year horizon than at shorter horizons. This over-optimism in official forecasts can help explain excessive budget deficits, especially the failure to run surpluses during periods of high output: if a boom is forecasted to last indefinitely, retrenchment is treated as unnecessary. Many believe that better fiscal policy can be obtained by means of rules such as ceilings for the deficit or, better yet, the structural deficit. But we also find: (iv) countries subject to a budget rule, in the form of euroland’s Stability and Growth Path, make official forecasts of growth and budget deficits that are even more biased and more correlated with booms than do other countries. This effect may help explain frequent violations of the SGP. One country, Chile, has managed to overcome governments’ tendency to satisfy fiscal targets by wishful thinking rather than by action. As a result of budget institutions created in 2000, Chile’s official forecasts of growth and the budget have not been overly optimistic, even in booms. Unlike many countries in the North, Chile took advantage of the 2002-07 expansion to run budget surpluses, and so was able to ease in the 2008-09 recession.”

Hence, the conclusion from Frankel’s paper is clear: We can simply not trust government forecasts! His solution is to set-up independent budget institutions as in Chile. Sweden and Hungary as in recent years set-up similar Fiscal Policy Councils. Obviously this is much preferable to the politicised forecasts that we see in many countries (and from international institutions such as the EU and the IMF!), but I strongly believe that budget forecasts based on prediction markets would be much better. The easiest thing would be to let the central bank of the country set-up a prediction market for key macroeconomic numbers which are relevant for the conduct of monetary and fiscal policy and then all government institutions would use these numbers in the conduct of policy.

Robin Hanson reaches a similar conclusion.

Please fasten your seatbelt and try to beat the market

Scott Sumner and other Market Monetarists (including myself) favour the use of NGDP futures to guide monetary policy. Other than being forward-looking a policy based on market information ensures that the forecast of the future development is not biased – in the market place biases will cost you on the bottom-line. Similarly, I have earlier suggested that central banks should use prediction markets to do forecasting rather rely on in-house forecasts that potentially could be biased due to political pressures.

A common critique of using “market forecasts” in the conduct of monetary policy is that the market often is wrong and that “herd behaviour” dominates price action – just think of Keynes’ famous beauty contest. This is the view of proponents of what has been termed behavioural finance. I have worked in the financial sector for more than a decade and I have surely come across many “special” characters and I therefore have some understanding for the thinking of behaviour theorists. However, one thing is individual characters and their more or less sane predictions and market bets another thing is the collective wisdom of the market.

My experience is that the market is much more sane and better at predicting than the individual market participants. As Scott Sumner I have a strong believe in the power of markets and I generally think that the financial markets can be described as being (more or less) efficient. The individual is no superman, but the collective knowledge of billions of market participants surely has powers that are bigger than superman’s powers. In fact the market might even be more powerful than Chuck Norris!

Economists continue to debate the empirical evidence of market efficiency, but the so-called Efficient-Market Hypothesis (EMH) can be hard to test empirically. However, on Thursday I got the chance to test the EMH on a small sample of market participants.

I was doing a presentation for 8 Swedish market participants who were on a visit to Copenhagen. I knew that they had to fly back to Stockholm on a fight at 18:10. So I organized a small competition.

I asked the 8 clever Swedes to write down their individual “bets” on when they would hear the famous words “Please fasten your seatbelts” and the person who was closest to the actual time would win a bottle of champagne (markets only work if you provide the proper incentives).

“Fasten your seatbelts” was said at 18:09. The “consensus” forecast from the 8 Swedes was 18:14 – a miss of 5 minutes (the “average” forecast was 8 minutes wrong). Not too bad I think given the number of uncertainties in such a prediction – just imagine what Scandinavian winter can do to the take-off time.

What, however, is more impressive is that only one of the 8 Swedes were better than the consensus forecast. Carl Johan missed by only 1 minute with his forecast of 18:08. Hence, 7 out 8 had a worse forecast than the consensus forecast. Said in another way only one managed to beat the market.

This is of course a bit of fun and games, but to me it also is a pretty good illustration of the fact that the collective wisdom in market is quite efficient.

I showed the results to one of my colleagues who have been a trader in the financial markets for two decades – so you can say he has been making a living beating the market. The first thing he noted was that two of the forecasts was quite off the mark – 14 minutes to early (Erik) and 14 minutes to late (Michael). My colleague said “they would have been dead in the market”. And then he explained that Erik and Michael probably went for the long shot after having rationalized that they probably would not have any chance going for the consensus forecast – after all we were playing “the-winner-takes-it-all” game. Erik and Michael in other words used what Philip Tetlock (inspired by Isaiah Berlin) has called a Hedgehog strategy – contrary to a Fox strategy. “Foxes” tend to place their bets close to the consensus, while “hedgehogs” tend to be contrarians.

My colleague explained that this strategy might have worked with the “market design” I had set up, but in the real world there is a cost of participating in the game. It is not free to go for the long shot. This is of course completely correct and in the real market place you so to speak have to pay an entrance fee. This, however, just means that the incentive to move closer to the consensus is increased, which reinforces the case for the Efficient-Market Hypothesis. But even without these incentives my little experiment shows that it can be extremely hard to beat the market – and even if we played the game over and over again I would doubt that somebody would emerge as a consistent “consensus beater”.

From a monetary policy perspective the experiment also reinforces the case for the use of market based forecasting in the conduct and guidance of monetary policy through NGDP futures or more simple prediction markets. After all how many central bankers are as clever as Carl Johan?

PS Carl Johan works for a hedge fund!

PPS if you are interested in predictions markets you should have a look at Robin Hanson’s blog Overcoming Bias and Chris Masse’s blog Midas Oracle.

UPDATE: See this fantastic illustration of the Wisdom of the Crowd.

Update 2: Scott Sumner has yet another good post on EMH.

%d bloggers like this: