We UX designers often learn about the “ideal” way to do things. But let’s be real: In our day-to-day work, we don’t always get the chance to follow that perfect design flow. Today I want to talk about surveys and when it’s ideal to use them. Using an example survey, we will break down the creation of surveys in general and how can they be effective.
What are surveys?
Surveys are a flexible research method suitable for any phase of the design journey. They offer insights into both qualitative and quantitative aspects. However, it’s essential to note that they capture people’s feelings and thoughts, which might not always align with their actions.
Their popularity stems from their ability to collect ample data without incurring significant expenses. Numerous platforms, such as SurveyMonkey, Typeform, and Google Forms, provide a space for conducting these surveys.
But how do you ensure your survey is effective? I will be using the imaginary healthy delivery service Meal Masters’ survey example, to understand why they need to be crafted with care.
Why Surveys Matter in UX Design
Direct Feedback: Surveys offer unfiltered feedback from the target audience. This feedback is invaluable in understanding user needs, preferences, and pain points.
Data-Driven Decisions: Surveys provide quantifiable data, allowing designers to make decisions based on concrete evidence rather than assumptions.
Cost-Effective: Compared to other research methods, surveys can be a cost-effective way to gather large amounts of data in a relatively short time.
There are a couple of goals your survey has to achieve. Let’s start from the most basic one: How well did you understand your research goal?
Understanding the goal
Before making your survey, know what you want to find out from the participants. Your goals will guide your questions. With clear goals, you can also use other research methods alongside surveys for better results.
Analyzing the survey provided above, we can evaluate:
The survey is comprehensively designed to capture insights on potential customers’ dietary needs, preferences, spending habits, and experiences with meal delivery services, ensuring that all key aspects of the meal delivery market are addressed.
Avoiding bias: leading questions, sampling bias
Let’s analyze all 7 questions for potential bias. The survey questions are largely neutral, aiming to minimize bias in responses. In this case, it’s achieved by using the “Other” option (e.g. for those who don’t fit in the listed categories). Providing a broad range of payment options rather than a particular price point. Even though the majority of questions are multiple choice questions, the survey ends with an open-ended question, which allows respondents to provide any feedback they deem relevant. However, it’s essential to note that bias isn’t just about the questions themselves but also about the interpretation of the data. Also, while the questions might be unbiased, the way the survey is distributed or the audience it reaches can introduce other types of biases.
Asking leading questions is a clear manifestation of confirmation bias. Such questions guide respondents toward specific answers. The second question presents a presumption, seeking feedback on it, which often makes respondents inclined to concur with the presented view.
Leading questions can potentially distort your research results and lead you to inaccurate conclusions.
The second question structure biases the respondent towards choosing more frequent delivery options by both suggesting a norm in the question and skewing the response choices towards more frequent deliveries.
To avoid sampling bias spread your survey widely so everyone has a chance to answer. Use different methods to reach more people and always ask, “Are we missing someone? Who?”. Also, don’t rely too much on just a single study’s results.
Leave a place for open-ended questions for qualitative data
Open-ended questions are wide-ranging and let people answer in-depth, like asking the last question of this survey, “Please provide any additional comments or preferences you’d like us to know.” These questions let you hear directly from customers in their own words, not just pre-determined answers.
The answers are mostly descriptive, but sometimes you can spot common themes. You can use tools to find these themes or even use spreadsheets or any type of organizer and word clouds to see what words or ideas pop up often.
Too long to finish
Usually, if a survey is too long, people won’t finish it. It’s not just about how many questions there are, but also how long it feels. To get more people to complete your surveys: keep it straightforward, give people the option to skip the questions, and inform participants how long it will take for them to finish the survey.
Are there any trade-offs?
Like any research method, surveys come with their own set of trade-offs. Based on the survey provided above, here are some of the most common trade-offs associated with creating and conducting surveys:
Depth vs. Breadth: Surveys are excellent for gathering broad data from a large group of people. However, they might not provide the depth of insights that one-on-one interviews or focus groups can offer. While you can reach a wider audience with surveys, the responses might lack the nuance and depth that other qualitative methods provide.
Quantitative vs. Qualitative Data: Multiple-choice or Likert scale questions provide quantitative data that’s easy to analyze. However, they might not capture the full range of emotions or opinions. Open-ended questions can provide richer qualitative data, but they are more challenging to analyze, especially with a large sample size.
Response Bias: The way questions are phrased can influence how respondents answer. Leading questions or a limited range of response options can skew results. Additionally, those who choose to respond to a survey might have stronger feelings about the topic, leading to a non-representative sample.
Time and Cost: While surveys can be more cost-effective than other research methods, designing a good survey, distributing it, and analyzing the results can still be time-consuming. If a survey is too long, participants won’t finish it, yet shorter surveys might not capture all the necessary information.
Accuracy of Responses: Respondents might not always provide accurate answers. They might rush through the survey, misinterpret questions, or even provide answers they believe the surveyor wants to hear (social desirability bias).
Limited Context: Surveys don’t provide a lot of context. Unlike observational methods or interviews, you don’t get to see the environment the respondent is in, their body language, or their immediate reactions.
Over-reliance: Solely relying on surveys can be a pitfall. They are just one of the many tools in a researcher’s toolkit. It’s often beneficial to complement survey data with other research methods to get a holistic understanding of users.
To sum it up, surveys are great for collecting user feedback, but they have their limits. By understanding these limits, and pairing surveys with other research techniques, you can get deeper and more useful insights.