Which of these best conveys the message?
- It’s great to rate!
- Rate, don’t wait!
- Weight, don’t wait!
- Rate on!
As a writer, I’m okay with continuing to brainstorm sayings and slogans and reminders to encourage researcher to utilize rating questions. As an advisor and consultant to survey designers, however, I’d rather get the message through once and for all.
What’s so great about rating questions?
First: Not Every Question Needs to Be a Rating Question
As enthusiastic as I am about heaping high ratings on rating questions, it’s important to note: Not every question needs to be a rating question.
Consider the question above. Does this need to be a rating question?
In this case, there’s no relationship between any of the answer options. They can’t be arranged on a scale. Nobody’s going to expect to review results as an average. In this case, a Radio Button question is totally fine, because the most useful way to review results is by frequency, such as a bar graph that shows popularity of each answer option side by side.
Next: Answer Options on a Scale Should Be Weighted
When you’re asking about a participant’s level or degree of feeling — satisfaction, agreement, enthusiasm, belief — your answer options should be arranged in an orderly, scaled fashion, and they should be weighted. You choose the kind of rating question type that best meets your needs, and then ensure you’ve got weights included.
If you’re using a Rating Scale question, the weights are baked in. In many cases, this question type displays the actual weights as well as the labels for each option, or sometimes just for the anchor (end) options. This is a great choice of question when have short, easy-to-understand answer options that are simple and predictable. It’s also a good choice when you’re asking for an overall rating, like satisfaction with an entire event, before or after asking about specific key drivers. Want variety? Use the Smiley Rating Scale variation instead.
If you’re using a Rating Radio Button question, you’ll need to assign weights to each answer option when you create the question. This question type is especially useful when your answer options are longer phrases or even full sentences that represent the strength or degree along the scale. It’s also helpful when you need to use specific numbers for each answer option that might not make sense on a “line” Rating Scale. For example, imagine that a 6-point scale was used in the past, but your team has now decided to use 5-point scales on surveys moving forward. In this case, to make the data from the 5-point scale comparable to the past data, you’ll need to calculate an updated weight for each answer option. The awkwardness of these decimal details can be hidden behind the answer options, meaning that the weights will be collected but the participants will never see the numbers.
Yes, there are also more rating questions, like Symbol Rating Scale, Rating Drop Down, and the grid variations — Rating Radio Grid, Rating Drop Down Grid, Rating Scale Grid — but the same concepts apply across the board. Make sure your participants know what each point on the scale represents, either by labeling or explaining in the question itself (5 stars is good, but is 5 dollar signs good?).
Because: Rating Makes Stronger Reports
It’s nice to see a good frequency bar graph, but nothing beats the at-a-glance understanding provided by an average weighted score. Just want to know how many people chose a specific answer? Radio Button is fine. Want to know overall how satisfied readers are with the campaign slogan we’ve chosen, based on a 1-5 scale? Rating is required. Want more? Add segmentation to see how ratings vary by group.
Plus, why stop there? Switch between customized views — Net Intent and Percent Favorable — and you’ve got yourself some results that you can review quickly.
Heads Up: Check The Score Before You Launch
Simply entering answer options from “Very Dissatisfied” to “Very Satisfied” does not automatically generate a rating question. Instead, you need to choose the right question type and actually add in the weights. We’ve got a pretty smart platform, but it can’t always read your mind. Can you tell if this question is a Rating Radio Button or a Radio Button question?
The good news is, if you’ve added a Rating Radio Button, you can still add/update the numbers in the weight fields after you publish your survey. The bad news is, if you’ve added a Radio Button question instead, there’s nowhere to add the weights. At that point, it’s too late. Sure, you could export your data set and import it into a corrected survey, but you’re better off making time to double-check before you go live in the first place.
Ready, Set, Rate!
Now that you’ve got the idea that it’s great to weight, it’s no surprise that you can hardly wait to add weights! Not sure if you’ve got it figured out? Book a quick one-on-one call with a highly rated rating expert! We’re here to help, and it’s always better to start a weights work-out before your survey ends up live and out of shape.
