Theory vs. Evidence: Unemployment Insurance edition

The argument over "theory vs. evidence" is usually oversimplified and silly, since you need both to understand the world. But there is a sense in which I think evidence really does "beat" theory most of the time, at least in econ. Basically, I think empirical work without much theory is usually more credible than the reverse.

To show what I mean, let's take an example. Suppose I was going to try to persuade you that extended unemployment insurance has big negative effects on employment. But suppose I could only show you one academic paper to make my case. Which of these two papers, on its own, would be more convincing?

Paper 1: "Optimal unemployment insurance in an equilibrium business-cycle model", by Kurt Mitman and Stanislav Rabinovitch

Abstract:

The optimal cyclical behavior of unemployment insurance is characterized in an equilibrium search model with risk-averse workers. Contrary to the current US policy, the path of optimal unemployment benefits is pro-cyclical – positively correlated with productivity and employment. Furthermore, optimal unemployment benefits react nonmonotonically to a productivity shock: in response to a fall in productivity, they rise on impact but then fall significantly below their pre-recession level during the recovery. As compared to the current US unemployment insurance policy, the optimal state-contingent unemployment benefits smooth cyclical fluctuations in unemployment and deliver substantial welfare gains.

Some excerpts:

The model is a Diamond–Mortensen–Pissarides model with aggregate productivity shocks. Time is discrete and the time horizon is infinite. The economy is populated by a unit measure of workers and a larger continuum of firms...Firms are risk-neutral and maximize profits. Workers and firms have the same discount factor β...Existing matches [i.e., jobs] are exogenously destroyed with a constant job separation probability δ...All worker–firm matches are identical: the only shocks to labor productivity are aggregate shocks...[A]ggregate labor productivity...follows an AR(1) process...The government can insure against aggregate shocks by buying and selling claims contingent on the aggregate state...The government levies a constant lump sum tax τ on firm profits and uses its tax revenues to finance unemployment benefits...The government is allowed to choose both the level of benefits and the rate at which they expire. Benefit expiration is stochastic...

Paper 2: "The Impact of Unemployment Benefit Extensions on Employment: The 2014 Employment Miracle?", by Marcus Hagedorn, Iourii Manovskii, and Kurt Mitman

Abstract:

We measure the aggregate effect of unemployment benefit duration on employment and the labor force. We exploit the variation induced by Congress' failure in December 2013 to reauthorize the unprecedented benefit extensions introduced during the Great Recession. Federal benefit extensions that ranged from 0 to 47 weeks across U.S. states were abruptly cut to zero. To achieve identification we use the fact that this policy change was exogenous to cross-sectional differences across U.S. states and we exploit a policy discontinuity at state borders. Our baseline estimates reveal that a 1% drop in benefit duration leads to a statistically significant increase of employment by 0.019 log points. In levels, 2.1 million individuals secured employment in 2014 due to the benefit cut. More than 1.1 million of these workers would not have participated in the labor market had benefit extensions been reauthorized.

Some excerpts:

[W]e exploit the fact that, at the end of 2013, federal unemployment benefit extensions available to workers ranged from 0 to 47 weeks across U.S. states. As the decision to abruptly eliminate all federal extensions applied to all states, it was exogenous to economic conditions of individual states. In particular, states did not choose to cut benefits based on, e.g. their employment in 2013 or expected employment growth in 2014. This allows us to exploit the vast heterogeneity of the decline in benefit duration across states to identify the labor market implication of unemployment benefit extensions. Note, however, that the benefit durations prior to the cut, and, consequently, the magnitudes of the cut, likely depended on economic conditions in individual states. Thus, the key challenge to measuring the effect of the cut in benefit durations on employment and the labor force is the inference on labor market trends that various locations would have experienced without a cut in benefits. Much of the analysis in the paper is devoted to the modeling and measurement of these trends. 

The primary focus of the formal analysis in the paper is on measuring the counterfactual trends in labor force and employment that border counties would have experienced without a cut in benefits...The first one...allows for permanent (over the estimation window) differences in employment across border counties which could be induced by the differences in other policies (e.g., taxes or regulations) between the states these counties belong to. Moreover, employment in each county is allowed to follow a distinct deterministic time trend. The model also includes aggregate time effects and controls for the effects of unemployment benefit durations in the pre-reform period...The second and third models...reflect the systematic response of underlying economic conditions across counties with different benefit durations to various aggregate shocks and the heterogeneity is induced by differential exposure of counties to these aggregate disturbances. 

These two papers have results that agree with each other. Both conclude that extended unemployment insurance causes unemployment to go up by a lot. But suppose I only showed you one of these papers. Which one, on its own, would be more effective in convincing you that extended UI raises U a lot?

I submit that the second paper would be a lot more convincing. 

Why? Because the first paper is mostly "theory" and the second paper is mostly "evidence". That's not totally the case, of course. The first paper does have some evidence, since it calibrates its parameters using real data. The second paper does have some theory, since it relies on a bunch of assumptions about how state-level employment trends work, as well as having a regression model. But the first paper has a huge number of very restrictive structural assumptions, while the second one has relatively few. That's really the key.

The first paper doesn't test the theory rigorously against the evidence. If it did, it would easily fail all but the most gentle first-pass tests. The assumptions are just too restrictive. Do we really think the government levies a lump-sum tax on business profits? Do we really think unemployment insurance benefits expire randomly? No, these are all obviously counterfactual assumptions. Do those false assumptions severely impact the model's ability to match the relevant features of reality? They probably do, but no one is going to bother to check, because theory papers like this are used to "organize our thinking" instead of to predict reality.

The second paper, on the other hand, doesn't need much of a structural theory in order to be believable. Unemployment insurance discourages people from working, eh? Duh, you're paying people not to work! You don't need a million goofy structural assumptions and a Diamond-Mortensen-Pissarides search model to come up with a convincing individual-behavior-level explanation for the empirical findings in the second paper.

Of course, even the second paper isn't 100% convincing - it doesn't settle the matter. Other mostly-empirical papers find different results. And it'll take a long debate before people agree which methodology is better. 

But I think this pair of papers shows why, very loosely speaking, evidence is often more powerful than theory in economics. Humans are wired to be scientists - we punish model complexity and reward goodness-of-fit. We have little information criteria in our heads.


Update: Looks like I'm not the only one that had this thought... :-)

@Noahpinion history of thought: impetus for writing the empirical papers came from "unconvinced" referees at the QJE on optimal one in 2011

— Kurt Mitman (@SorryToBeKurt) May 23, 2016

Also, Kurt has a new discussion paper with Hagedorn and Manovskii, criticizing the methodology of some empirical papers that find only a small effect of extended UI. In my opinion, Kurt's team is winning this one - the method of identifying causal effects of UI on unemployment using data revisions seems seriously flawed.