Coro New York is a leading provider of leadership development training through its fellows program and through its work with emerging leaders in community development and education. Following a strategic planning process in 2011, the organization set community transformation as one of its desired long-term impacts. In 2013, Coro asked Usable Knowledge to conduct an evaluation of its Neighborhood Leadership Program (NLP) designed to improve the leadership capacity of local business improvement districts (BIDs) in the City of New York and to help BID staff manage their relationships with the city. NLP consisted of a series of trainings, held over a five to six month period. Half the trainings directly address leadership skills. The other provided an opportunity for participants to interact with and learn from city officials. [Read more…]
In our last post on measuring downstream impacts, we offered a language for talking about impacts. In this one, we discuss the value of qualitative methods for assessing them. Where to begin? First of course, you need to be prepared to wait for the impacts to happen. How long depends upon the kinds of impacts you’re trying to achieve. Referring back to the table in our last post, depending on the size of the school and the intensity of the intervention, it may be possible to observe changes in school climate in just a few months, though whether they will be sustained over time is another question (and one that is often neglected). Many substance abuse, employment and criminal justice metrics look at changes six months post program. Impacts such as community change on the other hand may take years to observe. The evaluation team needs to be prepared to follow-up several months, or possibly several years, after the program has concluded and this waiting has to be built into the project timeline and budget. [Read more…]
If you follow a common approach to logic modeling used in the human services and other sectors, your three right most columns contain your project’s short-term, medium-term and long-term outcomes. We prefer to call short-term outcomes effects; medium-term outcomes, outcomes; and long-term outcomes, impacts. Many programs follow a similar approach. Short-term outcomes include the immediate results of the intervention and have to do with how people and systems have been affected by a program. Medium term outcomes generally describe what people are doing differently, whereas long-term outcomes get at the impacts of the changes. The table below provides some examples: [Read more…]
Several months ago we wrote about post-training surveys. You know, those surveys you get following a training session that ask about your experience. We’ve had several requests for more information on the topic and have decided to provide a sample survey you can download to use as a guide. The survey was created for a client that received a grant to provide basic, legal services to poor New Yorkers living with mental illness. It’s underlying goal was to advocate for individuals facing administrative proceedings aimed at reducing or eliminating their public benefits. Without such benefits, the organization’s clients would have been at risk for losing their homes. One facet of the program was the provision of training for mental health advocates designed to help them help those they worked with successfully navigate and prevail in the administrative hearings process.
There are two key issues in a post training survey, how did participants respond to the trainer, and what did they take away from their experience. The former is pretty straightforward. If you download the sample survey, you’ll find items that are usable in several different types of training programs. Assessing the takeaways from training is a bit more challenging. In the sample survey we kept it simple, asking whether participants felt they would be able to help clients navigate the benefits hearing process. Of course it’s possible they knew how to do this beforehand but earlier in the survey we asked about the overall appropriateness of the training. [Read more…]