This post is the last in our series about hiring an evaluation firm. Just to recap, we’ve suggested in our previous posts that while technical expertise is absolutely critical to a successful evaluation, soft skills, including flexibility and a strong client services orientation can make or break a project. Many organizations seeking a consultant issue a formal RFP, or request for proposals, believing that it creates a level playing field on which firms can pitch their approaches and compete for business. Typically, the RFP describes the program in question, suggests some high level evaluation questions, and specifies the form the response should take, and the criteria upon which responses will be evaluated. It may offer a budget or budget range. It is important to recognize however that not all evaluation projects begin with an RFP. Indeed there are good reasons to not employ an RFP process. What are they?
First, many firms, particularly smaller ones, often do not have the resources necessary to put together proposals on spec. Credible responses require a considerable effort to create and the best small firms may be too busy managing existing projects to respond. More important however, even the most well written RFP response will do little to help you evaluate a consultant based on the selection criteria we mentioned earlier. Certainly you’ll be able to gauge responders’ writing ability and perhaps get an idea of their technical competence, but you won’t learn much about the soft skills that are so essential to making an evaluation project a success. You won’t learn much, for example, about an evaluator’s flexibility, his or her ability to problem solve or willingness to go the extra mile necessary to make your project a success. Yet as we have said, organizations get the most value from an evaluation when it becomes a collaborative endeavor involving give-and-take with a trusted advisor.
For these reasons, we recommend you consider setting aside a formal RFP model and select your evaluator more holistically. One approach involves a letter of interest process. It differs from the RFP model in that it asks prospective consultants to share their perspectives on your project— their ideas and approaches— while de-emphasizing the kind of detailed evaluation plan that often changes anyway as projects move forward. Beyond asking for an overview of their approach however, ask responders to describe the challenges they anticipate and their strategies for dealing with them. Ask them to draft a list of questions for your team. Their responses will be a good indicator of their flexibility and thoughtfulness. In reviewing the letters of intent you receive, first ask yourself whether the firms’ approach aligns with the project as you see it. Are you, in other words, on the same page? Assuming that the firm’s history suggests the technical competence necessary to do a credible job, the key question is this– do you get the sense that working with them will add value to your organization and help it advance its strategic goals? Do you feel they will make effective thinking partners? Would you enjoy working with them? Again, while it is critical that you verify their methodological competence, these kinds of soft skills are essential to getting the most value out of an evaluation project. Jane Davidson of Genuine Evaluation has penned a more pointed and somewhat irreverent take on this approach which you’ll find here.
Whether you choose to try the letter of intent approach outlined above, or decide to go the route of a traditional RFP, make sure your solicitation includes the following items.
| A brief history of your organization and of the program in question | This should be no more than a single paragraph especially if your organizations website provides a good background about your work. |
| The goals of the evaluation work | See Part I above. If you have started to develop evaluation questions you should include them here. |
| A detailed outline of the program | What are the programs main components? How far are you in terms of implementation? Who are the key participants? How many people are there in each participant group? Where are they located? |
| A budget range | This is key. A given program can be evaluated in many different ways. Firms responding to your solicitation need to get some idea of the scope of work you had in mind. |
| Expectations (if any) about the role internal staff will play in the work | Will staff assist with data collection? To what degree? Will they themselves be interviewed, etc.? |
| Expectations about reporting | Are you expecting a formal written report or will a series of shorter summary reports be satisfactory? |
| Due dates and contact information | Due dates are obvious enough but responders may also wish to contact you with follow-up questions about the project. Their questions may be administrative (clarifications about your process) or substantive (questions about the program design). You may choose to delegate these two types of inquiries to different people on you staff. |
You should give responders at least three to four weeks to prepare their proposals, less if you are contacting firms you already know about for whom you have solid email addresses or telephone numbers, and more if the proposal is being released publically. It generally takes at least a few days for publically released solicitations to get noticed by potential responders.
We hope you have found this series useful and more important that it has given you some new perspectives on how to locate and select an appropriate firm for your next evaluation project. If you have any thoughts you’d like to share on our ideas, please don’t hesitate to leave us a comment. We’ll try our best to respond right away.
