Anyone can create a good survey, right? You might think so. I definitely did.
I created my first “real” survey about six months ago. Titled “What’s New With The Brew?”, it asked about participants’ coffee habits and preferences. I created this survey for educational purposes: I was learning more about Sogolytics’s platform. While I was really only creating this survey as a training exercise, the lessons I learned when creating my first survey can probably be helpful for others, too. Creating this survey helped me realize what was important when designing a survey, and helped me identify some of the mistakes I made along the way. I didn’t realize many of these mistakes until I began reporting the results, so hopefully you’re reading this early in your survey creation process. (And yes, we’ve covered survey tips on our blog before, which you can read about here).
These four seemingly obvious mistakes followed me through the entire lifecycle of my survey.
It’s important to note that while you can change your survey after it’s live, I wouldn’t encourage it. Changing your survey while it is live can potentially skew your data. This is because some of your participants didn’t get the opportunity to see the new or old question(s) or answer(s), and so results aren’t necessarily valid or even representative. I realized most of my bigger mistakes after my first survey was live, so it was too late to correct them.
1. Lack of Direction
I knew I wanted to create a survey that dealt with coffee, but in all honesty I didn’t know what I was going to do with that information once I had it. My primary goal when I was first creating the survey was to learn about the creation process. Even with that, I didn’t consider the survey’s true purpose and direction. If you don’t know what you want out of the survey, it’s best to stay in the design stage and brainstorm the purpose of it before moving any further. It was easy for me to think of a random topic for a survey, but if I had defined the purpose of the survey it would’ve made the rest of the survey process that much easier. Lacking direction and purpose was the biggest mistake I made when creating my survey, and led to the next three mistakes I’ve listed below.
2. Poor Questions and Answers
Since I didn’t have a clear purpose for my survey, it was difficult to figure out which questions were appropriate or not. “How many questions should I have?” and “How can I make the questions flow well?” were questions that I constantly struggled with when I was creating my first survey. Also there were a few question and answer options I included that didn’t really belong because they were too broad. Even when knowing your purpose, selecting questions and wording them appropriately is still a challenge.
Here’s an example: In my coffee survey, I included McDonald’s as a possible answer for a variety of questions. Some of those questions involved food, and while I don’t mean to offend Starbucks or Peet’s, McDonald’s was clearly going to be selected for the food options. McDonald’s does have coffee, but that’s not what they are known for. After publishing my survey, I realized that those questions weren’t well written and, more importantly, they weren’t even relevant. If I could go back in time, I would’ve taken a look at this blog that addresses writing quality questions for your survey.
3. Incorrect Question Types
Selecting the correct question types is just as important as asking the correct questions, but this took me a while to figure out. Appropriate question types are more important than they initially appeared for me: I just thought it was fun to change up the question types. Unfortunately, that led to a less than stellar survey and, as a result, less than stellar reports. You need reports that are clear enough to allow you to understand the results and draw conclusions. If you don’t choose appropriate question types, your reports and data will be much more difficult to understand. In my first survey, I had a question at the beginning that asked the participants where they liked to get coffee. It was a radio button (single selection) and because of that it meant that if they had more than one answer — too bad! Now I know that a checkbox would’ve been more appropriate because they could have selected multiple answers if they wanted to. It also would’ve led to a better report for the responses to that question. If I could do it again, I definitely should have read this to help me when selecting question types for my survey. If you’re curious about the various question types Sogolytics has, you can check out this survey question type reference.
4. Audience Consideration
Whether it’s writing a blog or creating a survey, the audience is important. If you don’t know who you’re writing to, who do you expect to read it? This ties back to survey direction and purpose: you want to put yourself in the shoes of your participants. Is the survey too long? Are there too many questions? Are there too many questions on one page? Do the colors on the page look good? These are simple questions to ask yourself when creating and organizing your survey. While they are seem simpe and obvious, I somehow still forgot to ask those questions. “What’s New With The Brew?” should’ve been divided by the flow and focus of each set of questions instead of just split across four random pages. You want people to participate in your survey, so the more you put yourself in their shoes the better you’ll be able to understand what their experience will be. If you find your survey boring, how can you expect anyone to want to complete it?
All of these factors play into one another. From creation all the way to reporting, these four mistakes impacted the entire lifecycle of my survey. It’s important to remember that there will always be mistakes. The best way to not make those mistakes again is to learn from them and move on. Luckily for me, I learned these mistakes in an environment where I was encouraged to make them in order to learn. Unfortunately, many people make these mistakes where it really counts — and hurts. I wish you luck on your survey creation adventure!
Need help creating your first survey? Book a training call today!
