Trust only 1/3 of the results from Pscyhology, but 2/3 of the results from behavioral economics

Dan Ariely (psychologist and behavioral economist) was forced to retract a well-cited paper based on fraudulent data. Ariely, a frequent TED Talk speaker and a Wall Street Journal advice columnist, cited the study in lectures and in his New York Times bestseller The (Honest) Truth About Dishonesty: How We Lie to Everyone — Especially Ourselves. We have blogged about Ariely's work and want to warn readers that we may have retract the blog posts, pending the outcome of an internal investigation.  

Professor Ariely's problems are part of a larger "replication crisis." It began when the journal Science found that only 1/3 of the psychology experiments underlying the most cited papers in the field of psychology  could be replicated.  A bigger and more recent study finds much the same thing.  [Behavioral Economics does better than psychology, as 2/3 of the results can be replicated.]

Contributing to the crisis is what is called HARK, Hypothesis After Results Known.  Due to the pressure to publish statistically significant results, researchers do science backward.  They begin by looking for a statistically significant result (p hacking).  Then, they specify an hypothesis to explain it.  Now, good practice requires that researchers register hypotheses before testing them.  Note that a similar problem plagues non-experimental data as well (regression fishing).

We regularly blog about deviations from the rational actor paradigm, and include some of the more useful results in our textbook, notably loss aversion (losses hurt more than gains help), and hyperbolic discounting (overweighting the present).  To the extent that these findings cannot be replicated, we could be wrong.

For the sixth edition, we will do our best to determine whether these Behavioral Economics results can be trusted.  For now, place a 2/3 weight on them being right.  

It’s Not the Incentives, Stupid?

One of the consistent themes on this blog is the power of incentives to motivate behavior. Well, here’s an article from Dan Ariely (behavioral economist and author of Predictably Irrational) in the most recent issue of Wired UK magazine discussing how monetary incentives might not increase the quality of activity, only the quantity. Ariely and a couple of co-authors conducted some experiments that had subjects completing tasks that required creativity, attention, memory and concentration. The subjects were split into three groups that were offered different levels of bonuses for doing well: a day’s pay, two weeks’ pay, and five months’ pay. It turns out that the first two groups performed about the same while the last group (the high bonus group) performed the worst.

If our tests mimic the real world, then massive bonuses clearly don't work. They may not only cost employers more but also discourage executives from working to the best of their abilities. The financial crisis, perhaps, didn't happen in spite of the bonuses, but because of them.

Dan Ariely Interview

John Heins, publisher of Value Investor Insight interviews Dan Ariely, the author of Predictably Irrational and the forthcoming The Upside of Irrationality (drops June 1). He concludes with a comment on the impact on the financial crisis on his work.

Do you think the financial crisis has made the book more popular?
It’s hard to find comfort in this, but there’s no question the financial crisis has been a good thing for behavioral economics. A few years ago people assumed markets drove out all this irrationality we were seeing. That assumption isn’t nearly as prevalent today.