August 4, 2016

The Price for Bad Economic Forecasting

Last winter I met with one of America's better known governors, a man whom I think has the qualities to become a good president. When I introduced myself as an economist he said:

"You know why God made economists? To make astrologers look good!"

The first time I heard that joke I was in graduate school. It was told to me by a political scientist, whose discipline is of unknown academic origin and merit. Therefore, back then, almost 20 years ago, I was a bit annoyed at the joke. Not so when Governor X told it to me. In fact, I am the first to admit that the profession of economics has gone awry and marginalized itself from much of the current political and public policy world. 

One reason for this is the almost manic obsession with econometrics. Multivariate regressions and recursively soluble equation systems have been elevated to the same truth-sayer status as once the Oracle of Delphi. In fact, despite the restricted applications of these quantitative methods their status in economics has contaminated other disciplines, such as political science. A while ago I had the opportunity to visit a seminar for graduate students in political science. Of the long list of papers presented, a clear majority relied on various levels of regressions to validate the author's conclusions. 

There is nothing wrong, inherently, with statistical analysis. On the contrary, it is essential to the daily work of millions of professionals around the world, in economics, finance, marketing, not to mention engineering and the natural sciences. Even legal practitioners and scholars rely on statistics on a regular basis. 

The problem comes when statistical analysis dictates the type of analysis a scholar can, or even should, do. This is where economics is today. Today, a graduate track in economics is little more than a long agonizing exercise in correlative statistics. Training in economic theory and methodology is increasingly marginalized, and it is practically unthinkable that a student is granted six weeks to study the concepts of probability and uncertainty and their application to macroeconomic analysis (the best class I took in graduate school!). 

Theoretical and methodological deterioration in economics is not just a problem for students of the discipline. It is very much a problem for people outside of economics - and outside of academia. Let me offer two high-impact examples. 

The first goes back to 2013. In 2012 the Greek economy was bombarded with austerity policies imposed on them by the European Union (EU), the European Central Bank (ECB) and the International Monetary Fund (IMF). The spending cuts and tax increases accelerated the Greek recession into a full-scale depression from which the country still has not recovered - and probably will not recover in the foreseeable future. 

I chronicled the Greek disaster in my book Industrial Poverty, as well as other episodes of similar policies (not the least of which took place in Sweden in the 1990s). My main conclusion was that austerity, if done right, can actually work - but when done with the main purpose to save the welfare state, as is almost always the case, the only result is that the welfare state survives at the expense of the private sector

This is exactly what austerity did to Greece. For every punch the welfare state took, the private sector had a leg broken. It was not the case, as some austerity advocates seem to believe, that the private sector would somehow come out thriving from the austerity episodes. Policies designed to save the welfare state in a crisis will first and foremost save the welfare state. 

Because of the accelerated deterioration of the private sector, it was impossible for me to see how austerity policies could do anything but kick the Greek economy deeper down into the depression hole. And that is exactly what happened. 

Not everyone agreed, though. Based on sophisticated econometric analysis the IMF predicted, back in 2012, that more austerity would have less negative effects on the economy. In effect, they forecasted small negative macroeconomic effects from big austerity programs. There was no theory in their forecasts to explain how this could be possible, yet they rolled out their forecasts and convinced the Greek government to implement yet another austerity package. 

The effect was quite different than what the IMF economists had predicted. In fact, the gap between the "map" and "reality" was so large that IMF's chief economist Olivier Blanchard and his colleague Daniel Leigh found it necessary to publish an unprecedented Mea Culpa:
This paper investigates the relation between growth forecast errors and planned fiscal consolidation during the crisis. We find that, in advanced economies, stronger planned fiscal consolidation has been associated with lower growth than expected, with the relation being particularly strong, both statistically and economically, early in the crisis. 
Obviously. Early in the crisis there is a lot of both welfare-state programs and private-sector activity to tear down by means of austerity. Later on, when a crisis has solidified its grip on an economy, private-sector activity is already crippled and entrepreneurs, consumers and investors are keeping their economic commitments at a minimum. Anyone who has read the first paragraph of the chapter "Sundry Observations" in Keynes's "General Theory" knows why and what the consequences are of this behavior. 

Then for the real admission of error from Blanchard and Leigh:
A natural interpretation is that fiscal multipliers were substantially higher than implicitly assumed by forecasters. The weaker relation in more recent years may reflect in part learning by forecasters and in part smaller multipliers than in the early years of the crisis.
 It takes some guts to write this, so kudos to the two IMF economists for doing it. The problem is that (and yes, I am audacious enough to point this out) if they had exercised better macroeconomic judgment; if their reliance had been at least as much on theory and methodology as on econometrics; then the Greek economy would have been far better off today than it is. A rough estimate puts blame on the IMF's erroneous forecasts for about one third of the Greek youth unemployment. 

And that is exactly where this becomes a problem for all of us, not just econo-nerds like me. When economists make grave forecasting mistakes and politicians trust them (which at the end of the day is what economists want, right?) middle-class families lose their incomes, young people find no jobs, people are hurled from a comfortable life into poverty.  

Now... you would have hoped that the IMF would have learned from its Greek mistakes. As far as I know, their chief economist Olivier Blanchard is an intellectually honest man whose forecasting errors are attributable to conventional economic thinking more than anything else. If anything, he has cleaned up the acts and done his best to prevent the same mistake from happening again. 

Yet the forecasting practice itself does not seem to have benefited all that much. Consider this story from British Daily Express on July 20:
Officials are facing a humiliating climbdown after claiming Britain would face a recession if the country opted to leave the European Union. But now the IMF expects the British economy to grow by 1.7 per cent this year and 1.3 per cent in 2017. ... IMF managing director Christine Lagarde, said Brexit would be “very very bad” for Britain but latest forecasts now suggest otherwise. 
Evidently, British consumers and businesses responded much more quietly and confidently to Brexit than many, among them the IMF, had predicted. The big question, though, is how this obvious point could escape economic forecasters. Voters who support Brexit believe that among the available alternatives it is the best one. They do not turn around, walk out of the polling station and suddenly fear doom and gloom. Their economic response to the Brexit question will be the same as their electoral response.

In other words: the same level of confidence inspires a person in several of his actions. 

The forecasting disaster regarding Greece was based on a somewhat different error than in the Brexit case. On Greece, the IMF slavishly followed conventional econometric modeling techniques without considering the fact that they were playing with variables and events that simply could not be squeezed into a standard macroeconomic model. Austerity is almost by definition a rare event, and its primary impact is on the institutional structure of the economy. Normally, that structure remains unchanged; changes to it affect economic activity differently depending on the nature of the change. 

Econometricians are not good at modeling institutional changes to the economy. This is not a point of criticism, just a matter of fact. The reason is plain and obvious: in order to be able to forecast economic activity using standard quantitative techniques you need some kind of quantitative historic background. When institutions do not change, no such background exists. When changes are rare, the quantitative material you have to work with is inadequate for the level of rigorous solutions required in econometrics. 

It does not stop there, though. In order to produce a forecast, standard modeling economists need to have an economy that "keeps together". It is not possible to have an economy that, for example, accelerates into instability and very large swings in growth, unemployment and other key variables. Such outcomes of a forecast, if allowed by a model, are not deemed "confident". Yet when rare changes happen to economic institutions, those scenarios are the ones that would be the most likely.*

The Brexit forecasting error appears to be of a similar nature, at least in part. I am inclined, though, to believe that this particular forecast was contaminated by political preferences. That does not diminish the methodological error on behalf of those economists who put it out there. On the contrary, if they had been more broadly minded in their forecasting work they could have insulated themselves against such political pressure.

IMF's economists are not the only ones making big forecasting mistakes, far from it. A quick sweep through forecasts of the U.S. budget deficit a year before the Great Recession began, and comparison of those forecasts to what actually happened a year later, shows that the average forecast got the deficit wrong by 289 percent. That said, the IMF has a prominent position in the global economy and in global politics. For that reason alone they deserve a place in the spotlight, especially when it is for less honorable reasons.

Hopefully, the errors made by the IMF economists will be a lesson learned for economists in general. At some point I do hope that practitioners of the economics profession, especially within academia, will begin to listen - and to reform their discipline. 

The world needs it.

---
*) I purposely avoid the term "probable". To see why, please consult A Treatise on Probability by John M Keynes or Epistemics and Economics by George L S Shackle.

No comments:

Post a Comment