This article was originally published by Stanford Social Innovation Review on March 11th, 2014 with the headline: GiveDirectly? Not So Fast
GiveDirectly is the current flavor of the month, and every couple of days someone asks us what we think of it. For those of you behind on your news feed, GiveDirectly does unconditional cash transfers -- it sends money via mobile payment straight to the poorest people in Kenyan villages. It's a brilliant approach to cutting out corruption and waste: 93 cents of every donated dollar goes to recipients as cash, and those recipients reliably spend it on food, housing, health care, education, and business investment. GiveWell lists GiveDirectly as one of its three "Top Charities" and recently gave the organization $5 million to do a lot more of the same.
The GiveDirectly team is really impressive, and we like approaches to development that give poor people the power of choice. We also like the cheekiness of simply handing out cash, and it was fun to hear the GiveDirectly crew talking about their work on NPR's "This American Life" (equally fun to hear Heifer International pinned to wall for its lack of -- almost contempt for -- impact data on the same show). But is GiveDirectly's model, as Slate put it: "the best and simplest way to fight poverty"?
No. It's an experiment -- an important one, but an experiment nonetheless. We hope we're wrong, but our hunch is that it is more of a 1-year reprieve from deprivation than a cost-effective, lasting "solution to poverty."
Poor people are poor because they don't have money, and so we think unconditional cash transfers should be judged primarily by how much money recipients are making a few years out from the windfall. GiveDirectly's work in Kenya is too new to know that, but cash-transfer enthusiasts like GiveWell point to other studies of other cash-transfer programs and predict a slam-dunk.
We looked at those studies, and we're puzzled as to why GiveWell's analysts, who rightly prize impact and cost-effectiveness, chose GiveDirectly as one of their "Top Charity" trinity. In the most relevant, longer-term study that GiveWell cited -- a program working with unemployed youth in Uganda -- recipients of an initial $382 grant had a 41 percent greater monthly income 4 years out. This sounds like a big return on the donor's dollar.
It's not. Working out the cost-effectiveness of income-generating funding can be confusing, and we at Mulago find it useful to benchmark grants by calculating the amount of additional income over 3 years, divided by the amount of grant money it took to generate it: the income bang for the donor buck. Of the baseline income of those youth, 41 percent turns out to be $11 per month. By that calculation ($11 per month x 36 months ÷ $382), the unconditional cash grant produced $1.03 of additional income over 3 years per donor dollar, essentially a wash.
By contrast, One Acre Fund, which works with the same population in Western Kenya, spends about $120 for roughly the same $380 in additional 3-year income. KickStart and Proximity Designs, which distribute subsidized irrigation equipment, generate substantially more than $10 over 3 years per donor dollar. VisonSpring, which helps people recover lost livelihoods by supplying reading glasses, turns a donor dollar into $60 of additional income. It can be hard to compare the results of academic studies to data generated by growing, evolving field organizations, but all of these business-minded NGOs spend substantial amounts on monitoring and evaluation, and have credible methodologies.
With these results in mind, the 1-year results of a randomized controlled trial (RCT) on GiveDirectly's work are illuminating. On the basis of that study, GiveDirectly reports a 28 percent income return on a $500 grant. By our benchmark analysis, if this return is consistent over 3 years (0.28 a year x $500 x 3 years = $420 total), then the income return per donor dollar on a $500 grant is less than $1. Ouch. Here's the kicker, though: That's gross income, not profit. When you factor in additional expenses, well, as the study's authors said, "We do not observe a significant increase in estimated profits from self-employment." Profits may materialize in the next few years, but really, who knows?
The other findings of the Kenya RCT left us underwhelmed, including:
- Food security: A the start of the study, 64 percent of people didn't have enough food for the next day; at the end of the year, 57 percent still didn't. Kids were still hungry: The average kid went from 4 days a year without food to 2.5 days. This isn't food security; it's a little less food insecurity. And since the net income gains are minimal, it's hard to be sanguine about lasting effects.
- Education: No apparent effect on outcomes, at least not yet.
- Health: Ditto.
- Assets: Some people -- 23 percent -- use the money to replace their thatch roof with a metal one. Enthusiasts are citing the replacement as a significant savings of money, since people must replace thatch every 1-2 years at a cost of $55 (an average of about $37 a year). However, a metal roof costs $400 (All of these cost figures come from the RCT report). That means it takes a decade for people realize any savings, and by that time they're looking to replace the damn thing anyway! It's no wonder that 77 percent of folks chose to do something else with the money.
It was interesting to see that $1 put into livestock yielded $2 in revenue -- maybe Heifer is on to something. In contrast, $10 invested in non-agricultural enterprise yielded only $11 in revenue. It may be that the average rural Kenyan isn't aware of or doesn't have access to great investments.
We like the evidence that people are happier and less stressed, and it's intriguing to speculate that improved decision-making enabled by lower stress levels may yield perceptible long-term benefits. But we worry that stress levels may revert to the status quo ante after a year or two. In the Uganda study, initial psychosocial benefits didn't last.
The excitement about unconditional cash transfers is perhaps more a reflection of the sorry state of development aid overall than of the real impact of the cash itself. And while the transfers are posed as a replacement of other subsidies, it is still the case that for people to make the most of the cash, they must be able to purchase high-quality, affordable products and services -- such as water purification, education, health care, fortified foods, and farm extension services -- that are typically not available in rural areas unless someone else subsidizes them.
Choice is a good thing; the notion that poor people must simply take whatever we feel like giving them is odious and wrong. However, a blind belief that "people always know best" ignores the reality that when you don't have access to high-quality education, information, and products, it can be hard to make optimal decisions and take effective action. The poor don't spend the cash on stupid things; they just may not have access to great things.
We'll be delighted if the work in Kenya eventually shows cost-effective and lasting health, education, and income benefits. GiveDirectly -- whose own claims are more modest than what we see in the media -- should continue its investigation of how, when, and for whom unconditional cash transfers will have the most impact. What we've seen so far, though, leaves us skeptical, and the hype remains well ahead of the impact.
