Misuses of empirical econ

Tyler Cowen has a new post in the ongoing blog discussion about the value of empirical economics. Most of Tyler's post is about the potential misuse of empirical economics. He writes:

The political process does not select for humble versions of empiricism.  Those end up with virtually no political influence, whereas some of the more dogmatic form of empiricism may find some traction.

Absolutely true. Ideologues just pick and choose results that support them. If 100 studies show the impact of immigration on labor markets is small, and one George Borjas study says it's large, the anti-immigrant people will wave around the Borjas study. (And then the anti-empiricists will say "See? No one can really know who's right!")

Tyler:

A lot of the bias in empirical methods comes simply from which questions are asked/answered. Post Trump and De Vos, I see plenty of commentators and researchers reporting “vouchers don’t raise test scores” and virtually no “vouchers increase parental satisfaction.” Is that empiricism? In isolation, maybe. In terms of reflecting the broader spirit of science, not so much. It is also not humility.

This is certainly true. I'm not sure focusing on certain questions and forgetting about others signals a lack of humility. Groupthink, maybe, but not arrogance.

Tyler:

I find a very common pattern among both researchers and commentators.  They first form...judgments about social systems, based on overall views of history, current politics (too much)...They then view very particular empirical debates through the broader lenses they have chosen.  For instance, views on politics used to correlate with views on the interest elasticity of money demand.  Today views on politics correlate with views on minimum wage elasticity, and so on.

Yep, this is certainly going on all over the place. There's evidence for ideologically motivated reasoning in econ research, though I think the evidence doesn't show that much motivated reasoning. The problem is almost certainly worse for us commentators, and worst of all for politicians. In general, the less people know about how econ research works, the more they seem to pick and choose the results they like based on which ideology it seems to support. Stopping people from doing this is a Sisyphean task for those of us commentators who are dedicated to a more dispassionate analysis of the facts, and even we aren't always the good guys either.

BUT, consider the other approaches to understanding the world. There's formal theory. There's intuition formed from exposure to theory ("thinking like an economist"). And there's intuition formed from exposure to casual observation and stylized facts (which Tyler calls "relatively general empirical judgments").

I'd argue that all of these are equally or more likely to be misused by commentators, ideologues, and politicians in exactly the same way Tyler describes empirical results being misused. People who want to justify fiscal austerity will wave around DSGE models that support fiscal austerity. People who want to kill the minimum wage will use theoretical intuition to claim that minimum wage hurts employent ("Demand curves slope down, DUH!", etc.). People who want industrial policy will use stylized facts - which Tyler calls "broad empiricism" - to point out that development successes like Korea and Japan usually have a lot of industrial policy in their past. And so forth.

The point is: Ideologues gonna ideologue. No economic analysis method will completely or even mostly put a stop to motivated reasoning. The hope for empirical economics is that a weak signal of reality is better, over the long term, than no signal at all. Over time, the hope is that the solid studies push out the shaky ones, that economists' natural rationality and desire to know the truth inches the academic consensus toward a better correspondence with the facts.

I'm optimistic - I think the empirical revolution in econ is no fad. Yes, there will be setbacks, as hot new techniques are over-applied, and as prominent results fail to replicate. But even better techniques will be developed, meta-analyses will weed out spurious results, and armies of smart, fair-minded grad students will comb through methodology sections and data sets to separate the wheat from the chaff. Eventually, theory will follow the weak but insistent tug of evidence - theories that don't fit the facts so well will gradually fall into disuse, while those that explain the most solid results will inch into the limelight. Classes will teach these more popular theories, and the next generation of econ majors will learn intuition that corresponds just a little more closely with observable fact. Eventually, people who rely on the "broad empiricism" of casual observation will start noticing the stylized facts that agree with their new, better theoretical intuition. Thus, progress will crawl forward, bit by bit, never reaching truth, but always headed in more or less the right direction.