It has become the holy grail of philanthropy: measuring the impact of a charitable contribution. Donors don’t want their money misused or used less effectively than it could be, but how can impact be measured?
A new rating system, ImpactMatters, is aiming to do just that. It rates similar nonprofit groups across an array of areas, all with an eye toward impact. The first wide-scale batch of 1,078 ratings were released on Friday, ahead of the holiday giving season.
Donors want an answer to a principal question: “Which nonprofit is spending their money wisely?” said Elijah Goldberg, executive director of ImpactMatters. “We wanted to understand the impact in a quantitative, rigorous way. We went out to solve this problem.”
ImpactMatters, which is based in Manhattan, looks at how much good an organization achieves per dollar. For example, a group that provides a meal for $2 when the cost in the area is $4 will get a higher rating than a similar group that provides a meal in that area for $5.
There are already well-established charity rating systems. But Mr. Goldberg, who founded ImpactMatters with Dean Karlan, a Northwestern University economics professor who taught Mr. Goldberg at Yale, said the other ratings did not measure impact as maximizing a donor’s dollar.
GuideStar collects nonprofit groups’ tax forms, known as 990s, and makes searching for financial information easy. Charity Navigator uses the amount of money a charity spends on overhead as a crucial metric in rating a charity’s effectiveness.
GiveWell takes a more exacting approach, naming top charities based on how much good an additional dollar will do. Then there are crowdsourced sites like Great Nonprofits.
All have their defenders and detractors. How much a charity spends on overhead is important but tells a limited story. A small list of top charities may isolate ones that do the most good, but some donors may want a larger set of options.
ImpactMatters is trying to highlight an apples-to-apples comparison of organizations in a sector, like food or health care. It contends that its approach will direct more money to the charities that have the most impact in their fields while pushing laggards to step things up.
Several philanthropic advisers questioned the usefulness of an impact measurement focused on individual organizations when problems like hunger require many different approaches.
“Giving for impact is critical, but it’s not the only driver for philanthropy,” said Dianne Chipps Bailey, national philanthropy strategy executive at Bank of America. “The most interesting question in measuring impact today is getting beyond this framework of efficiency and effectiveness to embrace advocacy, and shifting fundamental attitudes to make commitments that will endure over a long time.”
This requires many nonprofit groups to take on the same issue — say, hunger in a single neighborhood — from different directions, Ms. Bailey said.
“If you’re just looking at the cost of each meal provided, you’re missing the broader point about why that meal should be provided,” she said. “Why isn’t it being prepared in affordable and safe housing that is the person’s own?”
A Wells Fargo/Gallup Investor and Retirement Optimism study released on Wednesday interpreted impact differently. It found that the biggest current motivation for affluent individuals to give was the fractured political climate’s impact on services. Forty percent said they were driven by the political climate to give more; wealthy donors in the next-largest group, 26 percent, said they were motivated to give more by economic conditions.
“The clients we deal with are allocating more dollars to complex issues,” said Beth Renner, national director of philanthropic services at Wells Fargo. “With complexity comes difficulty achieving success.”
Finding a simple way to rate charities that are dealing with complex problems is challenging. ImpactMatters said it had abandoned a more involved framework and replaced it with a five-star rating system.
“Moving one at a time, we realized we were never going to get to philanthropy writ large,” Mr. Goldberg said. “But we could use the same tools and get to more philanthropists using publicly available data without the one-on-one core analysis.” The current rating system is much less labor intensive than the earlier one.
Mr. Goldberg said the goal was to help donors at least find the top nonprofit groups in eight areas where impact could be measured: veterans, clean water, homelessness, health, poverty, hunger, education and climate change.
“What we did was introduce a benchmark for each outcome,” he said. “In some ways, our estimates are better because we do 300 food banks and soup kitchens at once.”
In the first batch, 59 percent received five stars, 28 percent four stars, 13 percent three stars and just one a single star, for reporting improprieties. (Two-star ratings, which will appear in future releases as ImpactMatters rates more charities, are reserved for groups that have haven’t reported enough information.)
The number of high marks raises the issue of inflated ratings. Are the ratings worth the effort if most nonprofits score a five?
“It’s reflective of the fact that these groups are doing good,” Mr. Goldberg said. Plus, ImpactMatters still has tens of thousands of charities to rate.
For organizations on the receiving end, a high ranking is welcome and can help them stand out.
Sightsavers, which works internationally to help prevent blindness and advocate for people who are blind, received five stars. Caroline Harper, the British organization’s chief executive, said that it had long worked on self-evaluation but that the rating gave it outside validation.
“We’re not as well known as I’d like us to be,” she said. “It’s just another thing that builds our reputation.”
That has already happened with D-Rev, which develops medical devices like prosthetic knees and lamps to cure jaundice in newborns. It is already seeing the benefit of the earlier iteration of the ratings, which were given a limited release.
“It grew our audience,” said Sara Tollefson, D-Rev’s director of impact. “We have new donors around the world. People have reached out to me since.”
Dianne Calvi, chief executive of Village Enterprise, which works with entrepreneurs in Africa, was cautious in her praise of the rating system, even though her organization received five stars.
“It’s hard for a rating system to be perfect,” she said. “Donors have to figure out where to put their money. Otherwise, they’re just listening to the nonprofit and don’t have a way to compare it to another nonprofit.”
The nonprofit groups that have received fewer than five stars are worried. Liz Plachta, who founded Ruby’s Rainbow to help young adults with Down syndrome, said she didn’t understand why her organization had received four stars. She said she worried that the rating would hurt her fund-raising and ability to support programs for teenagers with Down syndrome seeking higher education.
“I guess a four out of five is O.K., but we’re so passionate about what we do and very little goes to overhead,” she said. “We rely on people to believe in us and know that we’re authentically doing what we do. I don’t want someone looking at that and saying, ‘Why are they a four and not a five?’”
For most donors, though, impact is just one measure. Ms. Bailey said having an objective source was a better option.
“Most high-net-worth philanthropists, 71 percent, are relying on the organizations they’re giving to,” she said, “to tell them if they’re having impact.”