We came across an opinion piece in in the Chronicle of Philanthropy by Kelly Campbell and Matt Forti of the Bridgespan Group in which they make a number of arguments for conducting rigorous outcome evaluations of nonprofit programs. While the piece concludes with a statement about the value of ongoing assessment in the service of continuous program improvement, our sense was the it mainly emphasizes the external value of evaluation. Often, the goal of such outwardly focused work is the production of evidence about program outcomes; either to satisfy existing funders or to interest new ones.
We believe that it’s time fornonprofit organizations to push back against such a conception of outcome evaluation. First, of course, evaluation is difficult and expensive to do and few funders are willing to pay for it. Good evaluation– the kind that really documents the efficacy of a program model– is very expensive. Second, without a broad-based, large N study, preferably involving randomized control groups, outcomes are often inconclusive. Lastly, most programs, at best, have only very modest effects. The people nonprofit organizations serve– at risk youth, substance using adults, the homeless, etc.– live in devastating situations that may have been a lifetime in the making. The challenges they face are deeply woven into the fabric of society as a whole, that is to say they are structural, and a single program, even a very well-run and comprehensive program, is not likely to make a substantial and measurable difference for most of its participants.
What this means is that single case (or single organization) outcome evaluations, done with limited time and dollar resources, are not likely to show much change. Is this a reason to abandon evaluation? Of course not. But it does provide a rationale for shifting questions away from those that ask whether an organization has achieved a set of specific outcomes, and towards those that ask, in a more open-ended fashion, what it has acheived, for whom, and under what circumstances. Those are the important things to look at. People, communities and indeed whole societies do change, but incrementally and sometimes, only imperceptibly. Effective organizations use evaluation not to justify their work– does a dance program for inner city kids really need to be justified– but to deepen their understanding of what they do and how it affects those they serve. Doing so shows that they’re in it for the long haul and that in and of itself, ought to interest funders enough to invest.
{ 2 comments… read them below or add one }
While I sympathize with your reasons for suggesting a non-RFP approach, here are a couple of problems with the details of your alternative:
1. It would take even more time for the small oufit to do than doing an RFP. Really!
2. It’s still a paper exercise and the professional proposal writers at the big shops will still win.
3. It will take more time and skill for the client to handle ten of these than ten proposals. This is a big consideration for them.
4. I think there’s a better solution: offer to fly three good staff into the (potential) client’s office, at your expense, in return for a 3hr meeting with their project staff; prepare for this minimally by having your team read a paper trail re their organization, and the RFP; and wing it. Show you’re fast learners and flexible designers/responders/helpers in person.
Just a thought.
Thinking on one’s feet is key as is showing up and showing some enthusiasm. Everyone ought to want to work with clients that value such things. Let the Beltway Bandits have all the stodgy foundations and non-profits. I think we might give your idea a try!