Do the Results of Experiments Extend to Field Situations?
Ultimately, we want to develop ideas that have use in the field. It is nice to know that there is a way to cure laboratory rats of cancer, but the ultimate question is whether the technique works with people. How then can we take ideas developed in the lab and validate them in the real world of economics and business? There are several methods.
- One method is parallelism, by which the investigator checks the robustness of a result by gradually adding layers of complexity, bringing the experiment closer and closer to field conditions. A related technique is test bedding, in which a field institution is recreated in the lab. Cal Tech's Charlie Plott has done a great deal of work in these areas, most recently test bedding some ideas for future FCC auctions.
- Another method is to literally move the experiment out of the lab and run it in the field. Google's Principal Scientest and former Professor of Economics David H. Reiley has done some very interesting work along these lines, running auctions designed for scientific purposes on the internet.
- Another method is to look for common regularities across lab and field environments. Common regularities increase confidence that regularities discovered in the lab are valid in the field as well. Some of our work in arbitration fits here. Ohio State's John Kagel has identified field institutions that appear to compensate for the winner's curse, the latter being an auction phenomenon most directly observed in the lab.
- Another less direct method is to see whether an idea developed in the lab can, at least in theory, be generalized. New York University's Efe Ok and Columbia University's Rajiv Sethi have rigorously modeled some ideas about interdependent preferences that were initially developed in the lab.
As with medical research, it is likely that some of the ideas that show promise in the lab will succeed in the field and some will fail. But this sort of 'hit-and-miss' is in the nature of all research work (a sure thing is, pretty much by definition, not research).