Earlier this year, Usable Knowledge conducted a survey of 250 nonprofit groups designed to learn how they use evaluation in their organizations and how they perceive funder support for it. The survey was administered online between April 15th and May 15th of this year and covered calendar 2014. Although the response rate was small, the organizations that did complete the survey represent groups of all sizes (under $500,000 to over $16 Million) and a broad range of nonprofit sectors (human services, environmental advocacy, youth, employment and training, etc.). About a third had one or more staff members dedicated to evaluation (the rest used a senior staff member for evaluation) and a little more than half reported hiring an external evaluator during the study period.
Over the course of the next few weeks we will be releasing some of the findings from the study as we prepare a final report. Here are the key findings so far:
- Nonprofits report that they use evaluation to assess program implementation, participant satisfaction, and program outcomes.
- Nonprofits are proactive in using evaluation internally for program improvement, new program planning, in proposals for new funding, and in communicating results via conference presentations and writing.
- While respondents reported that there funders require evaluation, less than half indicated that they support evaluation financially or in other ways. Many in fact were uncertain as to how their evaluation reports are being used.
Beyond simple output or compliance related questions, the organizations that responded indicated that the use evaluation to assess program outcomes. All indicated that evaluation was at least somewhat important in helping them find out whether participants benefited as expected from their programs (90% stated this was a very important use for evaluation data) and 90% told us that they use evaluation to assess whether their programs achieved their funding goals (76% indicated this was a very important use for evaluation). Organizations also indicated that they use evaluation to assess implementation and participant satisfaction.
Moving Beyond Assessment
Respondents also reported using evaluation in forward-looking ways. Large majorities reported that it was at least somewhat important to making modifications to existing programs (90%) and planning new ones (79%). Ninety-five indicated that they use evaluation results in new proposals and nearly as many stated that they use evaluation when they present their work at conferences or and articles.
Funders and Evaluation
Although respondents see significant value in evaluation, 90% of funders require it, less than half of the survey’s respondents indicated that there funders really supported. Forty percent report that the statement “our funders generally include funds for evaluation in their grants” somewhat describe their funders (only 15% agreed that it very much describe their funders). Similarly only 35% agreed that there funders supporting evaluation in other ways. While 56% told us that the statement “our funders like to meet with us to discuss evaluation results” somewhat describe their funders (17% stated it very much describe them), 74% were uncertain as to whether and how funders actually use their evaluation reports.
If you’d like to receive a chart with these findings, please click here.
The survey included a number of qualitative items designed to encourage respondents to explain their responses to the closed-ended questions. We were particularly interested in understanding how nonprofits use evaluation proactively– in modifying existing programs and planning new ones– and asked survey participants to provide specific examples from their practice. To our knowledge, the data we collected represents the only materials available on these questions and we look forward to analyzing it in the coming weeks. If you’d like to be informed when this analysis is ready, please subscribe by entering your email at the top of the side-bar.