When his firm decided to introduce a Christmas menu into their  restaurant chain, one of my students decided to test the profitability  of the change by introducing it in only half the restaurants in his  territory. By comparing sales changes at these restaurants (the  "experimental group") to changes at restaurants that did not introduce  the menu (the "control group"), he concluded that the change did little  to increase overall sales, despite the apparent popularity of the menu.
This  inference was possible only because my student constructed what  economists call a difference-in-difference estimate of the change. The  first difference is before vs. after introduction of the menu; the  second difference is between the experimental and control groups.
The  difference-in-difference methodology controls for other unobserved  factors that might have accounted for the change. The FTC has released a  number of studies following up on merger enforcement decisions to try  to figure out whether they did the right thing. For two consumated oil  mergers, FTC economists Dan Hosken and Chris Taylor (
article) and John  Simpson and Chris Taylor (
article) found  that prices in cities affected by the merger did not increase relative  to prices in control cities. In the time series graph below, the three  lines represent gas prices of the experimental city (Louisville)  relative to gas prices in three control cities (Chicago; Houston;  Arlington, VA) for the Marathon Ashland gasoline merger. The vertical  line represents the date of the merger. By comparing prices before and  after the merger, we see the merger had no effect, or that the FTC was  correct to let it through without a challenge.

FTC  General Counsel (now Commissioner) 
Bill  Kovacic coined the term "Enforcement R&D" to describe the  practice of government agencies following up on their decisions to  improve policy (
article).
I  would like to hear stories from readers about how, or if, their  organizations test  decisions. 
NOTE:  This post is copied from our old, almost defunct blog, Management R&D.