Attention check questions are survey items with one objectively correct answer, placed among the real questions to tell you whether a respondent is reading or clicking through. “Please select ‘Somewhat agree’ for this item” is the classic form. Anyone who picks anything else wasn’t reading, and now you know which rows to look at twice.
They’ve become more important as respondents have become cheaper. A paid panel pays per completed survey, which means the panelist who knocks out 40 surveys on a Tuesday earns ten times the one who does four, and nobody reads at that pace. Your own customers and employees aren’t immune. A 25-question pulse survey sent at 4pm on a Friday comes back with a surprising number of rows where the middle option is ticked all the way down.
The definition comes first, with the reason a 10% careless rate is enough to wreck a result. The six types follow, with a set of attention check questions examples you can lift straight into a questionnaire and the rules for writing and placing them.
Key takeaways
- An attention check has a right answer. That’s the whole difference between it and every other item on the survey, so if you can’t point to the one option that proves someone was reading, you’ve written a question rather than a check.
- Meade and Craig put careless responding at 10% to 12% in a student sample. Run that through a 400-response survey and you’re carrying about 40 rows of noise, which is plenty to flatten a six-point gap between two groups into nothing you could defend.
- One check per 10 to 15 minutes of survey is plenty. Three in a six-minute survey and your careful respondents start to feel tested, and people who feel tested answer differently.
- A failed check flags a row. Whether that row gets deleted is decided before fieldwork starts, in writing, and the same rule applies to every respondent, including the VP’s.
What are attention check questions?
Attention check questions are items inserted into a survey purely to measure whether the respondent is paying attention, with no interest in their opinion at all. They sit among the real questions, usually dressed to look like one, and each carries a single correct answer that anyone reading would give. Afterward you score them and decide how far to trust the rest of that person’s answers, which is a different use from any other item on the page.
That “correct answer” property is the whole definition. A satisfaction question has no wrong answer, however the respondent feels about your product. Put “To show you’re reading, choose ‘Never’ below” in front of someone and there’s one right answer; the respondent who picks “Sometimes” has just answered a question nobody asked. Methods papers call them instructed response items. Around the office you’ll hear trap questions, red herrings, validity items, whatever the last agency called them. The mechanics don’t change with the name.
Two cousins get mistaken for them. Survey screening questions decide whether someone belongs in the study at all, and a reverse-worded Likert item measures a real attitude while catching straight liners on the side. An attention check measures reading, and only reading. Say you run a 30-item employee survey and one item in the middle says “Select ‘Strongly disagree’ if you are reading this statement.” The 12 people who picked “Agree” weren’t lying about their manager. They were somewhere else.
Why attention check questions matter for survey data quality?
Attention check questions matter because careless responses look exactly like real ones in a spreadsheet, and there’s no filter that finds them after the fact. A respondent who clicked the middle option 30 times produced 30 valid-looking data points. Your averages absorb them, your crosstabs absorb them, and the only trace is a set of answers that don’t quite fit each other, which you’ll never spot across 600 rows by eye.
The size of the problem isn’t small. Using a student sample, Meade and Craig estimated that 10% to 12% of participants responded carelessly, and their paper is the one most methods courses cite on the topic. Picture what that does to a product survey with 400 responses. Around 40 of those rows are noise. Say the real gap in satisfaction between your two pricing tiers is six points; 40 rows of random clicking can pull it down to two, and the finding you were going to take to the pricing meeting is gone.
[Source: https://doi.org/10.1037/a0028085]
Careless rows also pull everything toward the middle, because the lazy click is the middle one. Your extremes get diluted, your correlations weaken, and anything you test for significance gets harder to detect than it should be. Berinsky, Margolis and Sances found that only about 60% to 70% of respondents passed their screener items in online political surveys, and the people who failed were also the ones who missed a flipped response scale later on. One inattentive habit predicts the next.
[Source: https://ajps.org/2015/03/16/methods-to-identify-inattentive-respondents/]
There’s a quieter benefit too. When you report a benchmark comparison, say against the Sogolytics Experience Index: Employee Edition Q3 2026 and its 1,000 full-time U.S. employees, you’re implicitly claiming your sample was answered with the same care. A documented check lets you say that out loud. Reviewers, clients and boards tend to ask.
Different types of attention check questions
The types of attention check questions in surveys fall into six families. Which one you want depends on how hidden the check needs to be and on who you’re trying to catch, because a speed-clicker and a bot fail different items.
- Instructed response items. The item tells the respondent which option to pick: “For quality purposes, please select ‘Disagree’ for this row.” These are the most common and the easiest to score. They catch people who aren’t reading the stem at all, and they’re nearly useless against someone who reads the first few words of every item and then clicks.
- Instructed manipulation checks. A longer passage asks the respondent to do something odd, like skipping the question below and typing a specific word in the comment box instead. They catch skimmers that the short form misses, because the instruction is buried in the middle of a paragraph. They also annoy attentive respondents more than any other type, and a few will fail on purpose out of irritation.
- Bogus or infrequency items. A statement that no honest, attentive person would endorse: “I have never used a computer” on a web survey, or “I am paid in gold bars.” No instruction is given, so the respondent can’t tell it’s a check, which is the point. The catch is finding a statement whose correct answer is obvious to everyone. By question 35 a surprising number of people will agree with almost anything.
- Logical consistency pairs. Two items that can’t both be true, several questions apart. Say question 4 asks how long someone has been a customer and question 19 asks whether they renewed their contract in 2023. Anyone who answers “less than a year” and “yes” has flagged their own row without knowing it. These are invisible to the respondent and useful for detecting bots, and they take real care to write so that an honest person can’t trip them by accident.
- Factual recall items. The block about a product ends, and one more item asks something the instructions already said: “Which feature did the previous section ask about?” You’re testing memory here as well as attention. A careful reader with a poor memory can fail one, so whatever you ask them to recall should be recent and hard to miss.
- Open-ended attention checks. They come in two forms, “In a few words, what was the last question about?” and “Please type the word ‘blue’ in the box.” The second is an instructed item in text form. The first is the one that catches copy-paste responders and bots, since a bot can fill a text box and can’t write a sentence about the question it just saw. Somebody has to read all 600 answers, which is the cost. In return you get the one check that’s close to impossible to fake.
| Type | Visible to respondent | Catches | Risk of false failures | Scoring effort |
|---|---|---|---|---|
| Instructed response item | Yes, if they read it | Non-readers, bots | Low | Automatic |
| Instructed manipulation check | Yes, when noticed | Skimmers | Medium; annoyance refusals | Automatic |
| Bogus or infrequency item | No | Random clickers, straightliners | Low to medium | Automatic |
| Logical consistency pair | No | Bots, random clickers | Medium; needs careful writing | Semi-automatic |
| Factual recall item | Sometimes | Skimmers | Medium; penalizes poor memory | Automatic |
| Open-ended check | Yes | Bots, copy-paste responders | Low | Manual |
Mix them. A 20-minute consumer panel survey might carry an instructed item near the start, a bogus item halfway through and a short open-ended check at the end. The bot fails all three. The tired human fails the last one. The person who skimmed the middle section fails the second, and that pattern tells you what kind of inattention you’re dealing with, which a single pass or fail flag never will.
Attention check questions examples
The attention check questions examples below are written to drop into a real survey, each with its correct answer noted and its type from the list above. Picture pasting the first one into a customer survey where every other item starts with “Our support team…”: it’ll stick out, and the skimmers will spot it. So put them in your own survey’s voice first. If you’re building from the Likert scale survey template, the first two slot into any grid as written.
Instructed response items:
- “This is an attention check. Please select ‘Somewhat satisfied’ for this item.” Correct answer: Somewhat satisfied. Works inside any satisfaction block.
- “To help us confirm you are reading each statement, choose ‘Never’ for this row.” Correct answer: Never. Fits a frequency grid such as a Likert scale survey.
- “Please leave this question blank and move to the next one.” Correct answer: no selection. Only use this where blanks are allowed, or the survey software will stop the respondent.
Bogus or infrequency items:
- “I have visited every country in the world.” Correct answer: Disagree or Strongly disagree.
- “I have never used the internet.” Correct answer: Disagree, on any web survey.
- “I read every word of the previous question twice.” Correct answer: Disagree. Attentive people are honest about this one, which makes an “Agree” worth a second look.
Logical consistency pairs:
- Early: “How long have you been a customer?” with the option “Less than 6 months.” Later: “How many annual renewals have you completed?” with the option “2 or more.” Both together flag the row.
- Early: “Which department do you work in?” Later: “Earlier you told us your department. Please select it again.” A mismatch flags the row.
Factual recall and open-ended checks:
- “The previous section asked about one of our products. Which product was it?” Correct answer: the product named in the block.
- “In your own words, what was the last question about?” Correct answer: any relevant sentence. A blank, a pasted URL or “good” fails.
- “Please type the word ‘orange’ in the box below.” Correct answer: orange, in any case.
Say you’ve got a 15-minute customer survey in four blocks. One instructed item in block two, one bogus item in block three and the “type the word” check at the end gives you three data points without making anyone feel tested. Three instructed items in a row, each announcing itself, gives you three chances to irritate the people whose answers you most want.
How to write effective attention check survey questions?
Decide what failure means
What a failure means gets decided before any item gets written. A failed check might remove the row, flag it for a sensitivity analysis, or end the survey for that respondent, and the wording of the check depends on which, as does the line in your consent text that says so.
Match the surrounding item format
The check gets written in the voice of the items around it. In an agree disagree survey built from short first-person statements, the check is a short first-person statement on the same scale, because a sudden shift in format is what tips people off.
Lock in one correct answer
One correct answer gets locked in, and a colleague who hasn’t seen the survey reads the item cold. If she can defend two options, the item goes back for another pass.
Place the instruction carefully
The instruction sits where a reader sees it and a skimmer doesn’t, which in an instructed item usually means the middle or end of the sentence, behind a plausible opening clause like “The product team asks that you…”.
Position the check mid-survey
The check lands after the respondent has settled in and before fatigue sets in. In most surveys that’s somewhere past question five and before the two-thirds mark, with a second check, if there is one, in the back third.
Test with five people
The survey gets tested with five people who don’t know which item is the check, at least two of them on a phone, since mobile survey design changes how much of an instruction gets read. If any of them fail it while reading carefully, the item is broken.
Set skip logic for failures
Skip logic gets set up for anyone the check should remove. Question display logic can route a failed response to a short closing page so the respondent isn’t asked another 20 questions whose answers you won’t use.
Report the pass rate
The pass rate goes into the methods note next to the response rate. At the next wave somebody will want to know whether 91% is better or worse than last time.
Writing attention check survey questions is mostly a matter of matching the check to its neighbors so closely that an inattentive respondent can’t tell it apart, while an attentive one can’t get it wrong. Most of the steps below get skipped when a check is bolted on the night before launch, and the check then fails in the field.
Picture a product manager with a 20-item feedback survey due Monday. Her first draft of the check reads “Select ‘Agree’ to confirm you’re reading,” which would be fine except every other item in the block opens with “The product…” and runs on the same five-point scale. So she rewrites it as “The product team asks that you select ‘Agree’ for this row,” and now it looks like its neighbors. Two testers read it cold and both pass. That’s the whole craft.
What makes a good attention check question?
A good attention check question is one that an attentive respondent passes without thinking about it and an inattentive one fails without noticing it. Every other quality follows from those two.
- One correct answer, obvious to anyone who read the item. “Select the option that best describes your mood” has five defensible answers, which makes it a survey question doing a check’s job badly.
- The same format as its neighbors, down to the scale and the length. Once a check stands out, the people who stopped reading everything else can pass it on sight, and you’ve learned nothing.
- Short enough to read in the time a real item takes. A 60-word instruction tests patience instead of attention, and the patient people are the ones you were trying to keep.
- No insults. “Prove you’re not a robot” and “Are you even reading this?” cost you goodwill from the 85% who are reading, and they’re the ones who’ll remember it at the next survey.
- It can’t be passed by guessing. A two-option check gives a random clicker a coin flip. A five-option scale cuts that to 20%, and a typed word cuts it to nothing.
- It survives translation. A survey going out in Spanish and Vietnamese as well as English needs an instruction that reads as plainly in all three, so idioms and anything clever are out.
- It records something useful on failure. A row flagged by a bogus item tells you what kind of carelessness you’re looking at, where a plain “failed” flag tells you nothing more.
A check that meets all seven looks dull, and dull is right. The best attention check questions are the ones no respondent remembers seeing.
What mistakes should you avoid when writing attention check questions?
- Giving the check its own font, a bold color or a page to itself. The people still reading pass it either way. The people who stopped reading three questions ago aren’t looking at fonts.
- Using a trick question. “Select the color that is not a color” trips the careful reader who overthinks it and waves the fast guesser through. That’s the opposite of what you wanted.
- Stacking three checks into the first five questions. By the third test in two minutes, respondents are answering to pass, and an answer given to pass tells you nothing.
- Making the check a required field with no correct option on screen. Say the survey stalls on question 11 for 300 people over a weekend and nobody finds out until Monday morning. The only place to test for that is a live survey maker with skip logic, since a Word document never stalls and so never shows you the problem.
- Treating one failure as proof of bad data. People misread. Somebody who missed one item in 40 and answered the other 39 consistently is worth keeping. Say so in the decision rule.
- Dropping failed rows without saying so. Berinsky and colleagues point out that attention tracks demographics, so quietly removing failers tilts the sample toward whoever reads most carefully. Report both versions, and let the reader see the difference.
- Writing a bogus item that some honest people could endorse. “I have never made a mistake at work” catches more than inattention.
- Copying a famous check word for word. A panel regular has met “Please select ‘Strongly agree’ to show you are paying attention” a few hundred times and passes it on reflex, so it says nothing about whether she read the rest.
- Relying on checks alone. A 40-item survey finished in 90 seconds tells you something. So does a grid with every row on the same column, and so does “asdf” in the open-ended questions. Together they’re more reliable than any single item.
- Forgetting that a check is still a survey item. The common questionnaire mistakes apply here too. A double negative trips honest readers. So does an undefined term, and two instructions packed into one double-barreled question trips nearly everyone.
When and how often should you use attention check questions?
Use attention check questions whenever the people answering have a reason to rush and nothing to lose by doing so, which describes most paid panels, most incentivized surveys and most long internal surveys. A three-question post-purchase pulse doesn’t need one, since the whole survey is shorter than the check would be. Small, identifiable groups need care too. A flagged senior manager in a 12-person leadership survey is a conversation you’d rather not have.
Frequency scales with length. Say a research manager runs an eight-minute customer survey and a 35-minute employee survey in the same quarter. The first gets one check and the second gets three, and she’s working to the same rule for both: one check per 10 to 15 minutes of survey time. A fourth check in the long survey would teach her almost nothing the third didn’t, and the irritation keeps climbing.
| Survey length | Attention checks | Placement |
|---|---|---|
| Under 5 minutes | 0 to 1 | Skip it, or one bogus item mid-survey |
| 5 to 10 minutes | 1 | After the first few items, before the midpoint |
| 10 to 20 minutes | 2 | One in the first third, one in the final third |
| 20 minutes or more | 2 to 3 | Spread evenly, one type each |
Some situations call for a check whatever the length:
- Paid panels and consumer survey panels, since the incentive pays for speed, and care is unpaid.
- Surveys with an open link anyone could forward, which is an invitation to bots and to curious outsiders with no stake in the answers.
- Studies with an experimental manipulation, because an effect measured on people who never read the stimulus is an effect of nothing.
- Any survey whose results will be compared wave to wave, so the attention rate itself can be tracked.
- Long employee surveys sent at the end of a quarter, when fatigue is predictable and a good survey response rate doesn’t guarantee a careful one.
Placement follows the same logic as frequency. The first few questions are too early, because everyone is attentive for the first minute. The last question is too late, since by then the damage is in the data. In a 24-item customer survey that runs about 12 minutes, that works out to an instructed item at question 6 and a bogus item at question 18. Neither one breaks the flow enough for anyone to remember it afterward, and between them they tell you most of what you need.
Conclusion
Attention check questions earn their place when they look like their neighbors, sit where attention drifts, and get scored by a rule you wrote down before anyone answered. Sogolytics’ survey design features give you the pieces: more than 20 question types so a check can match any block, question display logic and skip logic to route failed respondents to a closing page, and the option to randomize question order so panel regulars can’t learn where the check sits.
FAQs about survey attention check questions
Are attention checks questions mandatory in surveys?
No, and most short surveys never need one. They earn their place once respondents are being paid, once the survey runs past ten minutes, or once the results are headed for a board deck or a benchmark comparison, because at that point somebody will ask how you know people were reading.
What happens if a respondent fails an attention check?
Whatever you decided before launch, and deciding is most of the job. Most teams flag the row, then run the analysis twice, once with the flagged respondents in and once with them out, and only drop them if the two results disagree. Ending the survey on the spot is the other option; it spares the respondent 20 more questions and costs you partial data you might have wanted.
How many attention check questions should a survey include?
Work to one per 10 to 15 minutes of survey time. A 12-minute survey gets one or two and a 30-minute survey gets three at most, because somewhere past three your attentive respondents start to feel tested, and people who feel tested answer the items you care about differently.
Where should you place attention check questions in a survey?
Somewhere after the first few questions and before the final stretch, because nobody has drifted by question two and by question 40 the damage is done. With two checks, one goes in the first third and one in the last, each inside a block whose format it matches.
Can attention check questions bias survey results?
They can, in two ways. A check that reads as hostile puts attentive respondents on the defensive for the remaining 20 questions. Removing everyone who fails can tilt the sample toward the groups that read most carefully, which is why Berinsky and colleagues recommend reporting results with and without the flagged rows rather than quietly dropping them.


