Ed Lotterman: Economic studies warrant skepticism

May 17, 2013 

Politicians in particular and the public in general should be careful about supporting policies based on one or two economic studies. The latest lesson is the recent brouhaha over a flawed study by Harvard professors Kenneth Rogoff and Carmen Reinhart.

The best lesson to take from the study is not about its immediate assertion that high national debts slow economic growth. It is rather that while the discipline of economics can provide valuable insights, it is still highly limited in what it can prove or disprove.

The controversy is over the 2010 journal article, "Growth in a Time of Debt," in which they argue that historical data demonstrate that increasing levels of government debt slow economic growth, with a particular bend point when debt hits 90 percent of gross domestic product. While they examined 44 countries for periods across 200 years, these conclusions are drawn largely from a group of 20 countries from 1946-2009.

The paper was highly influential, cited by U.S. Republicans favoring spending cuts and Europeans favoring such cuts combined with tax increases - all with the goal of reducing debt. But there were criticisms of it from the start.

Then last month, graduate students at another school, unable to replicate the results claimed by Rogoff and Reinhart, found the authors had made a typographical mistake that excluded a few key numbers when averaging a column in a spreadsheet. Moreover, the Harvard professors had chosen to not use some available data for one country that, when included, changed the outcome. And then there was the question of how the countries were "weighted."

Correct for the error, add the unused data and it turns out there is no dramatic drop-off in growth at the 90 percent of GDP threshold. Rogoff and Reinhart acknowledged the spreadsheet error, defended their exclusion of certain data and the weighting technique and argued they still thought their general assertion was correct. The polemic goes on.

But the study had serious problems even before the error and excluded data were discovered. And these problems pop up with distressing frequency in other economic research.

The first was the assumption that if high debt were correlated with slow growth, the debt must be the factor slowing the growth.

One can also make plausible arguments turning the assertion upside-down: that slow growth causes high debt levels. And there may be an unmentioned third factor causing both - World War II. Several of the 20 countries, including the United States, U.K., New Zealand and Canada, all fought in the war. All emerged with very high national debts compared with GDP.

And all suffered a post-war slump for a year or two as millions were demobilized from the military and production shifted to civilian goods. Here you have several data points, nations and years, with high debt and slow growth. But was the debt causing the slow growth in those transition years of 1946 and 1947? Clearly not.

Another fundamental flaw was the implicit assumption that nations are homogeneous and that the average of the experiences of many nations of widely varying sizes and economic systems can serve as a valid predictor of what may happen to a specific large nation like ours.

This is a flawed extension of statistical analysis in the natural sciences. In trials of a new medicine, it is better to have 10,000 participants than 1,000. One can control the experiment: Some people got the medicine, and some did not. This number was cured, and that number was not. Yes, people may vary in age, weight, nutrition and other health factors, but such variations can be controlled for.

Logic might seem to indicate it is similarly useful to study 20 nations instead of three or four. But are nations like New Zealand and the Netherlands really part of the same cohort?

Half a century ago, prosperity in these small nations depended very highly on the pace of economic growth in their dominant trading partners, the U.K. and Germany respectively.

If their booms and busts were driven in large part by demand elsewhere, does examining their debt-growth relationships really provide any guidance for the largest nation in the world, one whose currency still is the dominant one used for international trade and reserves?

Also, the 90 percent threshold identified in the study was a variation of the fallacy of false precision. Why exactly 90 percent? Why not 83 or 102 percent?

This specific number was not identified via some analysis. Rather, researchers began by setting up arbitrary intervals of debt-to-GDP percentages, 0 to 30, 30 to 60, 60 to 90, 90 and above.

They then found no relationship in the first two brackets but did as one passed from the third to the fourth, again while using the errors subsequently identified.

But other researchers have now repeated the study using different arbitrary brackets, i.e. 20-40-60 or 25-50-75, and have not found any distinct bend point.

It is somewhat like the error often made about Social Security being a scam because it was set up with a retirement age of 65 when life expectancy was under 63. But that expectancy was so low largely because of high infant mortality.

Similarly, finding an average of 90 percent debt-GDP for 20 nations does not mean every nation at that threshold also experienced slow growth. Some may have been sluggish at 50 percent, and some did just fine over 120 percent.

Write economist Ed Lotterman at ed@edlotterman.com.

Idaho Statesman is pleased to provide this opportunity to share information, experiences and observations about what's in the news. Some of the comments may be reprinted elsewhere in the site or in the newspaper. We encourage lively, open debate on the issues of the day, and ask that you refrain from profanity, hate speech, personal comments and remarks that are off point. Thank you for taking the time to offer your thoughts.

Commenting FAQs | Terms of Service