Philanthropy Has an Impact Problem That Technology Can Help Solve

David Goodman participated on the Vendor Plenary Panel at the 2016 Technology Affinity Group (TAG) annual conference, where he first discussed his views on data, evaluation, and the sometimes over-emphasis on "impact" in philanthropy.

While my background is in research and evaluation, I’ve spent a great deal of time working with foundations and nonprofits to build their capacity to understand and use research and evaluation. I’m very excited to be able to bring my experience to the philanthropic sector at a time when there is a growing realization that it can benefit from the expertise of researchers and evaluators from other fields or disciplines.

This is a good thing. It gets me out of bed in the morning.

Yet, as much as I am excited about the emphasis on data, measurement, and impact, there is also a part of me that wants to pause – just for a moment – to talk about the disproportionate emphasis on “impact” alone.

Of course we all want to make an impact – a real and lasting impact. There’s nothing wrong with that, but too many foundations seem to be looking for quick answers, and are bypassing the critical work to ensure that data and metrics are used (and evaluations are conducted) appropriately and meaningfully.

The (over)emphasis on impact has compelled many foundations to contract with consultants and external evaluators to conduct impact evaluations. We are essentially creating an “impact industrial complex” where consultants are getting paid handsomely to assess the impact of foundations that are not yet ready to do so. As a result, these foundations will have a tough time showing that any evidence of impact is the result of the funded work and not the result of any other outside factor.  

Don’t get me wrong. There is excellent evaluation work being done in philanthropy, but for most foundations challenges to meaningful evaluation remain persistent – but not insurmountable.

Here are three of the most critical challenges we face – and how technology helps us address each of them.

Capacity – The lack of foundation capacity is the biggest hindrance to meaningful evaluation in philanthropy. And when I say “capacity” I mean more than just having a dedicated researcher or evaluator on staff. Capacity is also about having the time and resources to find, understand, and use data and metrics. It’s having the ability to reconcile all the varied and – in some cases – contradictory information and guidance that exists. And it’s about possessing the systems and technology to collect, analyze, visualize, and share data in an effective and efficient way.

The reality is that many foundation staff are already spread too thin. They often wear multiple hats – hired as a grants manager, they may also do program work, and, in some cases, be called upon to conduct research or evaluation in order to help inform the foundation’s work or assess a grantee.

Thankfully, technology can help address these challenges and help build capacity of foundation staff in the process. Technology can help collect, merge, and analyze data from different grantees or with external datasets. It can give you the ability to slice and dice data for different perspectives. And the right technology can make the data much more accessible – no longer held in the heads of program or grants officers. Technology has the potential to alleviate the burden running an evaluation program poses.

Preparation for Impact – There is a lot of very good evaluation work being done. But there is also a lot of confusion, misinformation, and conflicting opinions swirling around. The problem is that too many foundations are jumping right to impact and not doing the critical work to prepare for assessing impact.

The Center for Effective Philanthropy Benchmarking Foundation Evaluation Practices Report tells us that less than only 20 percent of foundations actually do this important developmental and formative evaluation work. Developmental evaluation allows you to develop a framework that tells you what outcomes to target and what data to use, what analyses to examine, and what methods are appropriate. And implementation evaluation can tell you if your model is accurate or if components might need to be changed or revised based on its implementation. The process allows you to reduce the uncertainty around your work. You can be confident that the impact story your data are telling you is accurate.

Again technology is crucial to this work. It can allow you to monitor and track components of your framework – outcomes, outputs, indicators – and make sure that they “behave” as intended. Technology allows you to capture and categorize a great deal of data on programs, contexts, participants, among other data points. It helps you examine that data in a way that instills confidence that your results weren’t influenced by outside factors.

Sharing of data – I believe that the that the goal of philanthropy is to create the biggest impact possible. Maybe you’ve heard me say this before: our job is to turn one success story into 1,000 similar success stories. We do this through appropriate use of data, analysis, sharing, and replication. And technology makes this all possible while also reducing the burden in doing so.

Technology provides the ability to find other funders, nonprofits, and organizations that are doing similar work or working in similar contexts. It allows you to leverage their knowledge, expertise, and data. This can help reduce the burden on both parties and alleviate the duplication of efforts. Technology also gives you the critical ability to compare data and metrics across contexts, in different settings, and identify those factors that might be driving impact or deterring from it. This will allow you to replicate the work and ultimately expand your impact and achieve more success and overall sustainability.

The challenges aren’t trivial, but with the will, determination, and intelligent use of the right technology, they aren’t insurmountable. We can't afford to ignore the critical work up front that prepares us for success.

David Goodman, Ph.D., is the Director of Impact at Fluxx. In his role, he engages with both grantmakers and nonprofits to find ways to better use data, metrics, and research to help them achieve greater impact and sustainability. David has served as the Director of Evaluation and Learning at the Houston Endowment, the Managing Director of Data Governance at Teach for America, and as Researcher at Edvance Research & Research Specialist at the Texas Education Agency.