Surveys Don't Drive Insights

A survey is a great tool that helps companies better understand consumers and develop a wide range of analyses.

A common misconception is that surveys drive insights. With numerous free survey services available and a growing set of panel providers, it’s easy to think that a simple input will yield a great output. For the output to be accurate and useful, there’s much more that goes into a survey. The setup and process of a survey—from design and sampling to team buy in to data review and analysis—are all critical to achieving successful outcomes.

How critical is the setup and process? That all depends on how important it is to get the answers right or not. Because it can appear so easy and isn’t, it’s worth explain what goes into making them valuable.

As part of our consulting work at Stax, we frequently field surveys to support our commercial due diligence and corporate strategy engagements. But surveys are just one of many tools. Over the past 20 years, Stax has conducted thousands of consumer and B2B engagements, gathering exponentially vast sets of data points. Based on this experience, we have identified vital components to develop a survey and deliver in-depth, actionable insights very quickly.

• Preliminary Research Prior to Questionnaire Design—To ask relevant questions, the research team needs to have knowledge about the company or customers, products/services, competition, and industry dynamics. Survey designers should use a mixture of industry research, online content and social media reviews, and a few in-depth interviews targeting select consumers and experts to refine and confirm the initial research hypothesis. At Stax, we pair secondary research teams and industry knowledgeable consultants with our dedicated survey team to prepare for client discussions.
• Teaming with Your Client—You need to engage with your team and client about what’s the need to know, nice to know, and what would be actionable. Are we trying to find out why something is happening (using open ended questioning)? Or are we trying to quantify solutions (options for a price/value package)? Who is involved, what can be acted upon if we find answer X, Y, or Z, and what are the nuances we need to take into account? Are there any specific relevant data points regarding the target company or market that need to be addressed or can contribute to a better-refined survey instrument?
• Questionnaire Design—Survey questions should be clear, concise and tied directly to the objectives of the engagement. The questions should be framed in a format that can provide the most meaningful insights while limiting respondent fatigue. A set of screening and qualifying questions should be included to limit participation to only relevant respondents and filter out individuals who do not meet the necessary requirements. Branching and skip logic should be utilized to ensure that respondents are only seeing relevant questions and no unnecessary screens. More interactive question types such as maximum difference scaling or choice-based conjoint analysis can also be deployed to generate more statistically accurate insights.
• Sample Plan & Design—It is important to ensure that the sample used for analysis is representative of the target population and limits sample bias—because there is always some bias. It is possible to oversample specific segments after a representative sample is established. This is done to acquire more detailed data using separate/distinct surveys with specific questions in order to prioritize data quality relative to top questions.
• Survey Monitoring and Results—Once the data is collected, inconsistent data points from the sample should be removed by researchers to verify the quality of responses. This includes identifying and eliminating professional survey takers who are often paid for their answers, as well as inattentive and/or inconsistent responses. At Stax, we developed a methodology and process to mitigate the influence of professional survey takers.
• Iterations—You need time to review results, think and discuss. A common opportunity is deeper dives into interesting observations. Researchers can also segment respondents into natural groupings and conduct targeted analysis to uncover more valuable insights.

Surveys are an extraordinarily valuable tool for generating deep customer insights quickly. They work as standalone efforts and even better when integrated with macro data, externally generated data and internally generated data. Like all good things, a little process and preparation helps you go faster and deeper to generate insights. By understanding that the entire process is what generates insights and not just the survey itself, you’ll ensure the output is meaningful and impactful.