In our last post, we talked about how to word open-ended questions. Why is this important? Because if done correctly, you can capture rich qualitative data from people using less expensive survey methods. We also talked about answer piping– taking the responses from one question and porting them into the text of another question. The value of answer piping is that it allows each survey participant to answer questions that are personally relevant to them which, crucially, engages them and inspires rich reflective responses. It looks something like this:
Now imagine something even cooler.
We were recently asked to design a study that involved following a group of 90 professionals over time as they developed a new approach to working with their clients. As with any new program, we anticipated that they would, at least at first, experience challenges implementing the new approach. We were interested in learning whether and how, over the course of the program, they would deal with these challenges. With 90 participants, it wasn’t possible to conduct multiple interviews with everyone, yet we wanted to understand how each person addressed the issues he or she faced. We could have asked each person, at the end of the project, what challenges they anticipated at the start, but over the 18 months it was active we were concerned that they would not accurately recall their initial concerns. How can you make this work in a survey?
Shortly after the program gets underway, participants will receive an online survey that asks, among other things, what they believe their biggest challenge will be implementing the new techniques they are learning. Our research team will review these responses and enter them into a respondent database. At the end of the program, participants will get a second survey seeded with their responses to the “biggest challenge” question. When they log in to complete the survey their individual response to the question will be drawn from this database and fed into a survey item that will take the form:
“When the program began, you were concerned that A, would be your biggest challenge. Was A in fact the biggest challenge you faced? If it was, how did you address it?”
“A” of course is the challenge they identified in the first survey. With this system, our team will be able to create the kind of engagement with program participants that comes close to what is possible in an interview based study, but at a much lower cost.
How Can You Use This?
Anytime you have prior information about a respondent you can pipe it into a survey question to make the question particular and relevant to that respondent. If you are surveying principals across a school district you can include the name of their school. If you are surveying parents you can include their child’s name, his or her teacher’s name, or any other information you have that helps the respondent engage and focus on the question you are trying to get answered. What do you need? An online survey that respondents log into, a database with the information you want to seed, and a survey system that can manage the two. We’re not sure that all of the free online systems have this kind of functionality, but if your project requires that you collect this kind of detailed information from respondents it may pay to locate a platform that allows it. Including open-ended questions doesn’t quite let you have your qualitative cake and eat it but it can come close.