This article was originally published by Stanford Social Innovation Review on Summer, 2011 with the headline: New School Economics
I like Dean Karlan. I like his work. Our Mulago Foundation funds his organization, Innovations for Poverty Action (IPA). We do whatever we can to get others to fund IPA. Disclaimers out of the way, here's a two-sentence summary of Karlan's More Than Good Intentions: This book is a gem. Anyone serious about aid, philanthropy, or impact investing should read it, maybe a couple of times.
More Than Good Intentions lays out a new approach to exploring and testing solutions to the thorny problems of global poverty. Yale University professor Karlan and his coauthor, IPA project associate Jacob Appel, have produced a book that is very readable, hugely useful, and often entertaining. Metrics geeks looking for a technical manual will be disappointed; those of us looking for a practical way to understand what works will not be.
Karlan and his colleagues at IPA are part of a new movement in development economics, a movement spearheaded by Esther Duflo and Michael Kremer and represented by a small army of researchers all over the world. As Karlan puts it, their work consists of a "two-pronged attack" on the problem of f nding the best solutions to poverty: 1) using rigorous evaluation methods akin to clinical research to test poverty solutions, both old and new; and 2) understanding problems and interpreting results using the lens of behavioral economics.
Karlan and Appel believe that understanding what works for poverty alleviation programs boils down to one deceptively simple question: "How did people's lives change with the program, compared with how they would have changed without it?" The primary—but not only—tool that Karlan et al. use to answer that question is the randomized controlled trial (RCT). In an RCT, a pool of subjects is randomly divided into intervention and control groups; the former gets the interventions and the latter does not. The two groups are fundamentally alike—both are measured before and after, and the impact is the difference between what happened to the intervention group and to the control group. RCTs are not new. The novel element here is the systematic and creative application of RCTs to test poverty solutions in the real world.
RCTs have their flaws, and there has been an understandable backlash against them. They can be expensive and complicated; perfect control groups are a myth; and results are too often too broadly interpreted. Yet Karlan is not doctrinaire about RCTs. He simply believes that you should measure from the beginning, measure the right thing, get good quality numbers, and make a case for what would have happened without you. One of the best examples in the book doesn't involve an RCT, but instead a "natural experiment" in Kerala, India, in which areas without cell phone service served as controls for a study of how fishermen used their phones to find where to get the best price for their catch and increase their profits.
Both the work and book benefit enormously from the application of behavioral economics, which goes beyond the narrow utilitarianism of classical economics to examine how real people make decisions. Behavioral economists assume that we don't operate on the basis of simple cost/benefit calculations, but have many different priorities, and that what may at first seem irrational often is not. Karlan and Appel use this approach as a tool to interpret results, make predictions, and come up with new ideas and hypotheses. In doing so, they draw on the strengths and flaws common to all of us and provide a respectful picture of the poor—not as some faceless other, but as us in different circumstances.
Given the overall clearheadedness of the book, one thing that puzzled me was the way that Karlan pulls his punches on microcredit. He reports that women entrepreneurs in Sri Lanka were often worse off after taking loans; that the poorer entrepreneurs in South Africa showed no effect from loans; and that even those entrepreneurs who did profits on consumer goods rather than reinvesting in their businesses. That said, he comments brightly that "it does not mean that … the enormous amount of enthusiasm [microcredit] has generated is necessarily misplaced." Well, what exactly does it mean? Perhaps he's just being nice, but if this methodology is as powerful as he'd have us believe, he should have something a bit more definitive to say about microcredit.
Still, More Than Good Intentions is a relentless and honest effort to find out what works and why. We really need what these new school development economists are providing. We've done far too many things that didn't work for far too long. But it is not enough to show what works: The one missing element in Karlan and Appel's fine book is a discussion of what it takes to turn research findings into real change. In a sector that does not yet channel resources toward impact, all that we learn about the behavior of the poor will be wasted unless we learn how to change the behavior of government bureaucrats, NGO executive directors, and the people who run foundations.
