The other day, I read an excellent article by two social entrepreneurs who shut down their family planning project in Ghana when they realized it wasn’t working and never would. How and why they pulled the plug makes for fascinating reading, but what really struck me was their take on why it is so rare for ineffective NGOs to go out of business:
“In the nonprofit world, organizations are accountable to their donors, not their beneficiaries. If you can’t secure funding, it’s game over. Nonprofits are therefore only held accountable for their effectiveness as far as it affects their ability to obtain further funding. With a compelling story and a strong fundraising team, projects can continue—even scale—without evidence that they work or, even when there is evidence that they don’t.”
In the for-profit world, firms are accountable to their customers. If customers don’t value their products, the firm doesn’t make a profit. If the firm doesn’t do better, it goes out of business and investors lose their money. Both firms and investors are accountable to the customer. Provide value or die.
The nonprofit world doesn’t work like that. The customers—we presume to call them “beneficiaries”—pretty much have to take what they’re given. There’s no meaningful way for them to signal whether the “product” has value for them or not. There are no direct consequences for a failure to benefit the beneficiaries.
Meanwhile, the investors—funders of all stripes—are even more insulated from consequences. We can fund whatever ineffective crap we want. Blindly funding stuff that doesn’t work won’t sully your image as a generous champion of the poor (or the environment, or whatever). Your reputation—the one thing funders really care about—doesn’t suffer.
This accountability vacuum has made a muddle of the social sector. Ineffective nonprofits survive, even thrive. Great ones have little no fundraising advantage. People like me never get fired for lack of impact. Money doesn’t flow efficiently toward those best able to create change. Excellent work is insufficiently rewarded—if even recognized—while lousy work is tolerated. Talent we desperately need goes elsewhere. We accomplish a fraction of what we might.
Only funders can fix this. Our fundamental job is to hold nonprofits accountable for impact on behalf of those who don’t have a voice. Because the customers can’t, we have to ensure the value of the product. We have to hold nonprofits—and by extension ourselves—accountable for impact.
The way to do that is fund only those nonprofits that persuasively measure their own impact. In my experience, the vast majority of nonprofits that do a good job of measuring impact are in fact effective, and those that work hard at it do a pretty good job. That they take the trouble to measure, on its own, means that they not only care about impact, but they’re generating the information they need to get better at what they do. Weeding out organizations that don’t make a serious effort to measure their impact is a great first step.
None of this means that you should hector and hassle the doers. Anyone trying to make the world a better place deserves your respect and gratitude. It means: You ask them for the right stuff, not a bunch of dumb stuff. You make it clear that your job is to be a good partner in the assurance of impact, that you’re trying to help them do it that much better, and that you’ll work to get them the resources they need to do it. You support the efforts of those who are trying to do a good job, and respectfully decline the proposals of those who don’t. You decline to fund stuff that doesn’t work, but you’re not a jerk about it.
Helping them do a good job of measuring impact and assuring that you’re doing a good job is not easy, but it’s conceptually simple. Here are the four things it takes to capture real impact:
Mission: Know what you’re setting out to accomplish. Call out the end outcome that matters most: “Save kids’ lives in West Africa,” “make students literate in rural India, or ”get clean water to families in the Sahel.” The inability to set that out clearly and concisely is a big red flag.
Metric: Figure what doable measures best capture your end-outcome. For the above that might be child-mortality rate, grade-level fluency, and per-person consumption of clean water, respectively. Friendly reminder: “people reached” and “lives touched” are not meaningful metrics.
Change: Get high-quality numbers that can show changes in terms of those metrics. This means getting a baseline and an endline with the right interval between them—because there’s nothing worse than when people forget to get a baseline—getting a big enough sample size and using proven survey and sampling methods.
Counterfactual: How much of that change was due to your efforts? Your impact is what happened with you minus what would have happened without you, and figuring out what would have happened without you—the “counterfactual”—is often the trickiest part of measuring impact. You have to compare the results you got with something useful: Occasionally (very) the status quo ante is good enough, but usually you have to design in a good comparison (often called a control) from the outset, perhaps even employing techniques like randomization.
Mission, Metric, Change, Counterfactual. That’s it. The simplest approach is just to ask the organization for their take on these four things. If they can’t give you something that makes sense, don’t fund them. Stage matters: An early-stage organization may not have much data yet (what’s important is that they can tell you how they’ll get it, and how they’ll know if they’re making progress), while organizations poised to scale should already have rigorous evidence of impact. Minimize the hassle: If they are persuasively reporting on progress in terms of “the four things,” then a lot of the other metrics we bug them about don’t matter. Reward the organizations doing a good job of it, help the ones that are demonstrably trying, and dump the ones that aren’t.
Over the last 20 years, I’ve seen an inspiring growth in the number of high-integrity nonprofits and funders who do hold themselves accountable for impact. By and large, funders can master the fundamentals of evidence, and when they take it seriously, they do it effectively. However, there are still way too few to drive the fundamental shift we need—It has to become the norm. If you’re a funder and you’re not making a good-faith effort, you’re not doing your job. You’re part of the problem. You’re letting everyone down: yourself, the organizations you fund, the taxpayers who subsidize your giving, and—most important by far—the people we’re all trying to serve.
If you don’t have the time, resources, and interest to hold the nonprofits you fund accountable for impact—to be a true advocate for those we serve—then you should follow a funder who does. If you can’t bring yourself to at least do that, then perhaps this game isn’t for you. You’re making things worse.
I’ve come to believe that if the funding pool was half its size but meaningfully driven by impact, we’d accomplish a lot more than we do now. High-impact organizations would attract a lot more money while spending a lot less on fundraising, because funders would be chasing after them, the way investors chase potential paydays. Low-performing nonprofits would either improve or die. High-performing organizations could justify—and be supported for—paying people better, thus attracting more talent to the sector. In a world where resources flow efficiently toward those best able to create change, this whole business of funding would become more interesting and exciting, and more smart people would want to jump in. Over the long run, we’d end up with way more money creating way more impact.
This is why, while we do want more money to come into philanthropy, it has to be driven by impact. I want to pry money out of all those donor-advised funds (DAFs) as much as anyone, but it has to be aimed at impact. From the Giving Pledge to HalfMyDAF, there are a lot of admirable efforts to get more money flowing (although you Giving Pledgers need to actually cough up the dough). However, it doesn’t matter if the hose gets bigger if you don’t pay attention to where it’s pointed. Instead of watering the garden, you might end up soaking the living room furniture.
There are some heartening reasons to believe that things could change, despite the general lack of accountability that funders enjoy. Over the past few years, we’ve managed to attach reputational risk—and virtue—to things like diversity, decolonization, and local leadership—and funders changed how they go about their business. It’s high time we do the same for impact. It astonishes me that we haven’t done it already: Impact in the lives of the neglected and overlooked is the ultimate form of justice and inclusion. Don’t let them down. Do your job.
