Yet another argument for prediction markets: “Reputation and Forecast Revisions: Evidence from the FOMC”

I am already spamming my readers today so this will not be a long post. But take a look at this working paper – “Reputation and Forecast Revisions: Evidence from the FOMC” by 

Peter Tillmann. Here is the abstract:

“This paper investigates how FOMC members revise their forecasts for key macroeconomic variables. Based on a new data set of forecasts from individual FOMC members between 1992 and 2000 it is shown that FOMC members intentionally overrevise their forecasts at the first revision and underrevise at the final revision date. This pattern of rationally biased forecasts is similar to that of private sector forecasters and is consistent with theories of reputation building among forecasters. The FOMC’s shift towards more transparency in 1994 had an impact on how members revised their forecasts and intensified the tendency to underrevise at the later stage of the forecasting process. The tendency to underrevise, i.e. to smooth forecast revisions, is particularly strong for nonvoting members of the committee.”

HT George Farnon

Benn & Ben – would prediction markets be of interest to you?

Benn Steil from the Council on Foreign Relations has an interesting comment on the Federal Reserve’s forecasting performance. I don’t really want to discuss Benn Steil’s views, but rather the fed research he quotes.

Here is Steil:

“The Fed studied its own staff’s forecasting performance over the period 1986 to 2006. It found that the average root mean squared error—or the deviation from the actual result—for the staff’s next-year gross domestic product (GDP) forecasts was 1.34, compared with 1.29 by what the Fed describes as a “large group” of private forecasters. That is, the Fed’s predicting performance was worse than that of market-watchers outside the Fed. For next-year CPI forecasts, the error term was 1.03 for Fed staff, and only 0.93 for private forecasters. The Fed’s conclusion? In its own words, its “historical forecast errors are large in economic terms.”

I have unfortunately not be able to locate the research quoted by Steil so if anybody out there can locate it please let me know. I have the feeling that the research is rather old – and as such Steil’s story is not really “breaking news”.

Anyway what can I say? The Fed is not able to beat the “consensus forecast”. That is not really surprising. That does not show that the Fed economists in anyway are incompetent. It just shows that the “market” or the wisdowm of the crowds is better at forecasting than the Fed. In fact the “consensus” will most of the time beat any professional forecaster.

So the relevant question that Steil should ask is why is the Fed doing forecast instead of leaving it to the market. The Fed of course should set-up a prediction market rather than relying on in-house forecasts – especially when the market clearly is better at forecasting than the skilled economists at the Fed.

By the way contrary to what Steil implies I don’t think we can say anything about whether the Fed should be trusted or not based on the Fed’s forecasting performance. In fact if the Fed consistently was able to beat the market then I guess the market would pretty fast adopt the Fed forecast. There is a lot of reason to be skeptical about the Fed, but the “average” forecasting performance of the Fed’s staff is not one of them. I have personally been doing a lot of forecasting over the years and I would never claim that I am better at forecasting that the “crowd” so this is not a critique of the Fed economists, but rather an endorsement of the market.

See my previous posts on the use of prediction markets in the conduct of monetary policy.

Robin Hanson’s brilliant idea for central bank decision-making
Prediction markets and government budget forecasts
Please fasten your seatbelt and try to beat the market
Central banks should set up prediction markets

PS Mr. Steil might be interested in noting that market expectations for medium-term inflation still is well below 2%. Contrary to what Mr. Steil seems to think US monetary policy is overly tight! Unfortunately neither Benn nor Ben seem to care much about market expectations…


Update: George Farnon alerted me to this article: Federal Open Market Committee forecasts: Guesses or guidance? It is yet another argument for prediction markets…the Fed would never dare…

Why did the A’s stop winning? Scott has the answer

I have been watching Moneyball. It is a great movie, but unlike Scott Sumner and my wife I have actually no clue about movies. However, economics play a huge role in this movie. So that surely made me interested. It is of course very different from Michael Lewis’ excellent book Moneyball, but it is close enough to be an interesting movie even to nerdy economists like myself.

If I was not blogging about monetary policy and theory then there is a good chance I would be blogging about what Bob Tollison called Sportometrics – the economics of sports. It combines two things I love – sports and economics. But why bring Moneyball into a discussion about money and markets? Well, because the story of the Oakland A’s is a pretty good illustration that Scott Sumner is right about the Efficient Market Hypothesis (EMH) – even when it comes to the market of baseball players. So bare with me…

Any American male knows the story about the Oakland A’s but for the rest of you let me just re-tell the story Michael Lewis tells in Moneyball.

The story about the Oakland A’s is the story about the A’s’ general manager Billy Beane who had the view that the market was under-pricing certain skills among baseball players. By investing in players with these under-priced skills he could get a team, which would be more “productivity” than if he had not acknowledged this under-pricing. Furthermore as other teams did not acknowledge this he would increase his chances of winning even against teams with more resources. It’s a beautiful story – especially because theory worked. At least that is how it looked. In the early 2000s the Oakland A’s had much better results than should have been expected given the fact that the A’s was one the teams in the with the lowest budgets in the league. The thesis in Moneyball is that that was possible exactly because Billy Beane consistently used of Sabermetrics – the economics of Baseball.

Whether Lewis’ thesis correct or not is of course debatable, but it is a fact that the Oakland A’s clearly outperformed in this period. However, after Moneyball was published in 2003 the fortune of the Oakland A’s has changed. The A’s has not since then been a consistent “outperformer”. So what happened? Well, Billy Beane was been beaten by his own success and EMH!

Basically Billy Beane was a speculator. He saw a mis-pricing in the market and he speculated by selling overvalued players and buying undervalued players. However, as his success became known – among other things through Lewis’ book – other teams realised that they also could increase their winning chances by applying similar methods. That pushed up the price of undervalued players and the price of overvalued player was pushed down. The market for baseball players simply became (more?) efficient. At least that is the empirical result demonstrated in a 2005-paper An Economic Evaluation of the Moneyball Hypothesisby Jahn K. Hakes and Raymond D. Sauer. Here is the abstract:

Michael Lewis’s book, Moneyball, is the story of an innovative manager who exploits an inefficiency in baseball’s labor market over a prolonged period of time. We evaluate this claim by applying standard econometric procedures to data on player productivity and compensation from 1999 to 2004. These methods support Lewis’s argument that the valuation of different skills was inefficient in the early part of this period, and that this was profitably exploited by managers with the ability to generate and interpret statistical knowledge. This knowledge became increasingly dispersed across baseball teams during this period. Consistent with Lewis’s story and economic reasoning, the spread of this knowledge is associated with the market correcting the original mis-pricing.”

Isn’t it beautiful? The market is not efficient to beginning with, but a speculator comes in and via the price system ensures that the market becomes efficient. This is EMH applied to the baseball market. Hence, if a market like the baseball market, which surely is about a lot more than making money can be described just remotely as efficient why should we not think that the financial markets are efficient? In the financial markets there is not one Billy Beane, but millions of Billy Beanes.

Every bank, every hedgefund and every pension fund in the world employ Billy Beane-types – I am one of them myself – to try to find mis-pricing in the financial markets. We (all the Billy Beanes in the financial markets) are using all kind of different methods – some of them very colourful like technical analysis – but the aggregated result is that the markets are becoming more efficient.

Like Billy Bean the speculators in the financial markets are constantly scanning the markets for mis-priced assets and they are constantly looking for new methods to forecast the market prices. So why should the financial markets be less efficient than the baseball market? I think Scott is right – EMH is a pretty good description of the financial markets or rather I haven’t seen any other general theory that works better across asset classes.


PS there is of course also the possibility that Billy Beane was just lucky“The sample is simply not big enough” (“Peter Brand” in the movie about why his theory initially did not work).

PPS  In Moneyball the movie the term “Winning streak” is used. That is a bit of a turn off for anybody who has studied a bit of sportometrics. There is not such a thing as a winning streak or a “hot hand” – at least that can not be proved empirically.

PPPS Moneyball is not really about Billy Beane, but rather about Paul DePodesta. In the Moneyball Paul DePodesta is renamed Peter Brand.

PPPPS I have no clue about baseball and find it rather boring…that must be the ultimate disclaimer.

%d bloggers like this: