- \n
- “Testing”<\/strong> =\u00a0I clicked the link and it opened, so I guess it’s good to go.<\/em><\/li>\n
- Real Testing<\/strong> =\u00a0I totally understand the goals of this project, the purpose of each question, the possible answers, the requirements of the audience — and I’ve anticipated and addressed every possible point of participant confusion — and I’ve collected feedback from every project partner and stakeholder and we’re good to go.<\/em><\/li>\n<\/ul>\n
Unfortunately, in the run-up to a project launch, “testing” is much too common.<\/p>\n
<\/div>\nOf course, things happen. Even in our most perfect projects, requirements can change. Mistakes arise. “I thought it would work that way” meets “That’s not how it’s supposed to work at all.”<\/p>\n
<\/div>\nAlso, of course, nobody reaching out for help in these stressful situations wants to get a shrug and a “Better luck next time!” When is it too late to fix the problem — and is it always too late?<\/p>\n
<\/div>\n![\"fixing](\"https:\/\/www.sogolytics.com\/blog\/wp-content\/uploads\/2021\/10\/GettyImages-93119859-1024x683.jpg\")
<\/p>\n<\/div>\nAnswer 1: It’s always too late<\/strong><\/h2>\n
Imagine that you’re conducting a survey and you ask participants to choose their favorite color. Reviewing the results a few days later, you see that blue and green are at the top of the list — but you suddenly realize that you forgot to add red as an option. What to do?<\/p>\n
<\/div>\n- \n
- If you don’t add red, your results may not actually be representative of your participants’ true favorite colors.<\/li>\n
- If you do add red, it’s already “behind” in the ratings and can’t possibly be selected as many times as those colors that were always present. This means your report won’t fairly or accurately represent the selections participants might have chosen if all colors were present from the start.<\/li>\n<\/ul>\n
Those reviewing the results might even accuse the designers from intentionally excluding red from the original options — perhaps because of a bias against red or in favor of another color. Shocking!\u00a0While the question of favorite color is an overly simplified example, the illustration is clear.<\/p>\n
<\/div>\nWhether it’s adding or removing an answer option, adding or removing a question, or changing an existing question or answer, editing a live survey can result in complicated questions on the reporting end. A biased researcher might leave out any explanation, but the ethical research will be forced to include asterisks, footnotes, and narrative justification, at the very least. At worst, a researcher may have to completely toss out some early pre-edit responses as invalid, or scrap the entire project.<\/p>\n
<\/div>\nAnswer 2: It’s sometimes okay<\/strong><\/h2>\n
Imagine that you’re conducting a survey and you include a mandatory question about primary work location. Within an hour of launching, you’ve already heard from a dozen people telling you that they won’t be participating because you’ve left Herndon, Virginia, off of the list. What to do?<\/p>\n
<\/div>\n- \n
- If you don’t add Herndon, many people will choose not to participate because they’ve been left out. Worse, some may choose to select a completely different location, just to finish and be done with it. In the latter case, your data validity plummets.<\/li>\n
- If you do add Herndon, you’ll probably have to follow up with those initial dozen people and let them know that you’ve made the update and they’re re-invited to respond.<\/li>\n<\/ul>\n
In this case, it probably makes more sense to add the answer option than to leave it off. Here you’re asking for more of a demographic segmenting question rather than an opinion, so it’s less likely to be seen as an attempt to bias results. Still, there’s nothing quite so disengaging for a participant as realizing that the survey designer totally forgot about them.<\/p>\n
<\/div>\n![\"survey](\"https:\/\/www.sogolytics.com\/blog\/wp-content\/uploads\/2021\/10\/GettyImages-838089806-1024x683.jpg\")
<\/p>\n<\/div>\nAnswer 3: Check your tech options<\/strong><\/h2>\n
From a research perspective, editing a live survey is fraught with issues. You wouldn’t change protocols halfway through a series of focus groups or one-on-one interviews, and you shouldn’t mess with a validated research instrument halfway through a study.<\/p>\n
<\/div>\nTechnical tools, however, can empower researchers to make the right decision for each project. “Allowing” isn’t the same as “encouraging”, though, so keep in mind that your reports should reflect any changes that you made. Keep careful notes about how and when any changes were made so that any possible impact on results can be assessed in later analysis.<\/p>\n
<\/div>\nHow to edit a live survey: Basics<\/strong><\/h2>\n
First: Remember that most platforms won’t allow you to change everything you want, and also that your options will vary by package. When you open a live project that has already received responses, you’ll see a warning. You always read these kinds of messages, right?<\/p>\n
<\/div>\n![\"Edit](\"https:\/\/www.sogolytics.com\/blog\/wp-content\/uploads\/2021\/10\/Edit-Live-Project.png\")
<\/p>\n<\/div>\nEdit Text<\/strong>: If you just need to edit the text of a question or an answer, hover over the text and then click. If you click Edit in the menu bar that appears to the right, the system will think you’re trying to change the entire question, and you’ll have limited options.<\/p>\n
<\/div>\n![\"edit](\"https:\/\/www.sogolytics.com\/blog\/wp-content\/uploads\/2021\/10\/Edit-Question-Text-1024x105.png\")
<\/p>\n<\/div>\nUpdate Answers:\u00a0<\/strong>If you need to add or delete an answer option in a live survey, hover to display your options.<\/p>\n
<\/div>\n![\"add](\"https:\/\/www.sogolytics.com\/blog\/wp-content\/uploads\/2021\/10\/Add-answer.png\")
<\/p>\n<\/div>\nUpdate Questions<\/strong>: If you’re trying to add or delete a question in a live survey, you’ll see a warning. Heads up: Do you know what you’re doing and why?<\/p>\n
<\/div>\n![\"add](\"https:\/\/www.sogolytics.com\/blog\/wp-content\/uploads\/2021\/10\/Add-question-1024x296.png\")
<\/p>\n<\/div>\nHow to edit a live survey: Advanced<\/strong><\/h2>\n
Despite your best laid plans and it-seemed-like-enough-at-the-time testing, bigger problems can happen. At some point, you may not be able to salvage your current survey as it is and you may need to opt for an alternative. Either that, or you could just let it go on burning, right? Probably not. Even though you do need to prioritize the value of you results, you also need to prioritize the experience of your participants. If you’ve created a mess for them, the best possible thing you can do is gracefully clear the mess, excuse yourself, and offer an alternative as soon as possible.<\/p>\n
<\/div>\nThe two most common up-in-flames fails that we see are (1) issues with logic<\/a>, (2) issues with pre-population<\/a>, and (3) issues with anonymity<\/a>. Handled well, these are amazing features that have a hugely positive impact on both the participant experience and the quality of data collected. If they’re handled poorly — or forgotten about completely — that’s bad news all around.<\/p>\n
- Real Testing<\/strong> =\u00a0I totally understand the goals of this project, the purpose of each question, the possible answers, the requirements of the audience — and I’ve anticipated and addressed every possible point of participant confusion — and I’ve collected feedback from every project partner and stakeholder and we’re good to go.<\/em><\/li>\n<\/ul>\n