Open-ended thematic analysis is the process of reading the free-text answers in your survey, tagging each one with a short code, and grouping those codes into themes that explain what people are trying to tell you. Say your last survey left you with 800 comments. Handled this way, they come out as five or six findings with a count and a couple of quotes each, and that’s something a manager will read.
Plenty of surveys close with “Is there anything else you’d like to share?” It’s where a frustrated customer finally types out what went wrong. Picture an analyst who exports those comments on a Friday afternoon, reads the first fifty and closes the spreadsheet, even though the reason behind every score above sits in that column. Thematic analysis of open-ended survey responses is how you get through all of them. It matters more now that AI tools sort comments in seconds and still need a person to check what they’ve sorted.
Braun and Clarke’s six phases come first, because everything after leans on them. After the codebook section comes the part everyone asks about: how much of the reading you can hand to AI.
Key takeaways
- A theme answers your research question. Take the comment pile from a pricing survey: “customers feel they pay for features they never use” is a theme someone can act on, and “pricing” is only the label on the folder.
- Under a few hundred comments, a spreadsheet and an afternoon or two will do. Past that, a text analysis tool saves days, and your job becomes checking its work.
- Cross every theme with your closed-ended scores. When the parking complaints all come from people who gave you a 1 or 2, you’ve found your first fix.
- Before the full set gets coded, two people should each code the same 50 comments. A disagreement over comment 37 costs ten minutes. Find it after 2,000 comments and you’re recoding for a week.
What is open-ended thematic analysis and why does it matter?
Open-ended thematic analysis takes a well-known qualitative method and points it at survey comments. Most are a sentence or a fragment (“parking again”), far shorter than anything you’d get from an interview, and a big survey can collect thousands. Each one also sits next to that respondent’s ratings and location. That’s the useful part, because you can see which group a theme comes from without asking a single extra question.
Two terms get mixed up here. A code is a label for what a single comment says, such as “wait time on phone” or “manager listens”. A theme is a broader pattern that ties several codes together and says something meaningful about your question, such as “people who call in feel ignored until they escalate”. If your final report reads like a list of nouns, you’ve stopped at codes, because a theme should carry a claim someone could act on.
People who answer an open question have put in more effort than people who tick a box, so their words deserve a careful read. Pew Research Center checked 92 open-ended questions from its American Trends Panel. On average, around 18% of respondents skipped them, against roughly 1% to 2% for closed-ended questions. Whoever did type something cared enough to bother. How much they give you depends a lot on how you worded your open-ended questions.
[Source: https://www.pewresearch.org/decoded/2021/10/14/why-do-some-open-ended-survey-questions-result-in-higher-item-nonresponse-rates-than-others/]
Picture a school district that sees its parent satisfaction score drop four points in a year. The ratings tell you something slipped, though they can’t say what. Read the comments and you might find that half of the unhappy parents mention the new pickup schedule, which is a problem a principal can fix by next semester. That’s the job of thematic analysis of open-ended survey responses. It ties a number to a cause. How much weight should text get next to your scores? That’s the old qualitative vs. quantitative research question, and it’s worth settling before you start.
Steps to conduct thematic analysis on open-ended survey data
Read before you code
Reading comes before coding. You’ll want to get through a big sample first, or every comment if there are fewer than 300. Keep a scratch list as you go. If “parking” turns up for the fourth time, it goes on the list, and so does the one odd comment about the gift shop.
Generate initial codes
Generate initial codes. Work through the comments and give each one a short label for what it says. It’s fine for a single comment to carry two or three codes when it covers more than one point.
Sort codes into themes
This is where codes get sorted into candidate themes. Say three codes keep turning up side by side: “slow refund”, “had to call twice” and “no update on ticket”. They probably belong under one theme about follow-through.
Review themes against the data
Every theme gets checked against the raw comments and the full data set. Some will overlap and get merged. Others try to cover too much and need splitting. A theme that rests on two or three comments usually doesn’t survive this step, and it shouldn’t.
Name and define each theme
Each theme gets a name and a definition of one or two sentences that says where its edges are. “Trust in billing” tells a reader more than “billing issues”, and it’s easier to remember in a meeting.
Write up the findings
The write-up reports each theme with how many respondents raised it and which segments raised it most. Two or three quotes show its range. Your stakeholders will usually read this in survey reports, so the theme summary goes before the quotes.
This guide leans on a 2006 paper by psychologists Virginia Braun and Victoria Clarke, published in Qualitative Research in Psychology, which set out six phases of thematic analysis. It’s the framework most people cite. They had interviews and focus groups in mind. Survey data gives you far more responses, and each one is shorter. It also arrives with a spreadsheet of respondent details next to every comment. The steps below adjust for that.
[Source: https://uwe-repository.worktribe.com/output/1043060]
In practice you’ll go round in circles a bit. A theme falls apart in Step 4, you realize the codes were too broad, and you’re back in Step 2. Say you’ve run a product feedback survey after a release, and your first pass produces a theme called “usability”. Go back to the comments and there are two stories in it. New users can’t find a setting. Long-time users are annoyed that a menu moved. Those deserve separate themes, since they lead to separate fixes. The closed-ended side of the same survey is a numbers job, and this walkthrough on how to analyze survey data covers it.
How to code open-ended responses and identify recurring themes?
Clean the responses
Cleanup is the boring part. Say you’ve got a 2,000-person customer survey; a few hundred rows will be blank or say “n/a” or “nothing”, and there’ll be exact duplicates. Delete those. Keep the respondent ID on every row, though. Without it you can’t link a comment back to its ratings and segment later. Good data management features in your survey tool save you most of that cleanup.
Build a codebook
Your first 50 to 100 responses become the codebook. Each code gets a name and a one-line definition, plus a rule for when it applies and an example. Without the rule, a new analyst hits “manager listens” and has to guess whether it’s recognition or communication. The rule answers that before anyone has to ask.
Check agreement between coders
Say two coders take the same sample and code it separately. On Monday they compare. Where they split on comment 23, somebody rewrites the definition until they wouldn’t split again, which beats arguing over the comment itself.
Code the full set
With the codebook settled, the full set gets coded. One comment can take more than one code. A “needs review” code catches the answers you can’t place yet, so nothing gets forced into the wrong bucket late on a Friday.
Count themes by respondent
Themes get counted by respondent. A man who mentions parking three times in one comment is one unhappy parker. Count mentions and he becomes three.
Choose representative quotes
Quotes get chosen once the counts are in. The good ones show a theme’s range, a mild version and a strong one. Anything that could identify the respondent gets stripped out before the quote goes in a deck.
Before you code a single comment, you decide where the labels come from. Inductive coding builds labels from the comments themselves. That’s the right call for a first survey on a topic, when nobody knows what’s coming. Deductive coding starts from a fixed list, usually last year’s categories. Results stay comparable from one wave to the next. In real projects the two get mixed. A team might start with last year’s twelve codes and add a thirteenth the week a new complaint shows up.
What follows is the unglamorous part.
A codebook doesn’t need to be elaborate. Here’s what a few rows might look like for a customer support survey:
| Code | Definition | Example response | Theme |
|---|---|---|---|
| Repeat contact | Respondent had to reach out more than once for the same issue | “Called three times before anyone fixed it” | Follow-through on support tickets |
| No status update | Respondent wasn’t told what was happening with their request | “Never heard back after I submitted the form” | Follow-through on support tickets |
| Agent knowledge | Comment about how well the agent understood the problem | “She knew exactly what to do” | Confidence in frontline staff |
| Transfer | Respondent was moved between people or departments | “Got passed to four different people” | Effort required to get help |
Two of those codes land on the same theme. That’s normal. It’s also the point where your analysis starts saying something a frequency count can’t. A spreadsheet can tell you that 140 people mentioned “transfer”. Add the people who called twice and the ones who heard nothing back. Together they describe a support process where the customer does the chasing.
How open-ended thematic analysis helps interpret survey responses?
Your scores say how much. The comments say why. Code the comments against respondent IDs and open-ended thematic analysis starts doing real work alongside the numbers:
- They explain a score. Say satisfaction dropped in March. The themes from that same wave show which part of the experience moved.
- They show who’s affected. Cross a theme with region or tenure and you’ll see whether a complaint is everywhere or sitting in one office.
- They catch problems you didn’t ask about. Nobody writes a closed question about the broken card reader at the north entrance. Somebody will mention it in a comment.
- They help you rank what to fix. Say 30% of your lowest scorers mention billing and 5% of your happiest do. Billing goes near the top of the list.
- They make trends readable. Keep the codebook stable and you can line up theme shares wave by wave. If “follow-through” was 12% of comments in spring and 20% in fall, someone should ask why.
Take an employee exit survey. The ratings might show departing staff scoring “career growth” lowest, which won’t surprise anyone in HR. The comments are more specific. Maybe people in two departments never saw internal openings posted, while everyone else did. That’s a process fix. You’d only catch it by reading the comments with the department column right beside them. When you rewrite the survey next year, knowing the difference between open vs. close-ended questions helps you decide which format carries which part of that story.
A theme nobody outside the analytics team sees won’t fix anything. When West Virginia Prevention Solutions moved off spreadsheets, county coordinators got direct access to their own data. With that kind of setup, a theme about one county gets read in that county.
Thematic analysis vs. content analysis for survey responses
People swap these two terms all the time, and in survey work they overlap a lot. Content analysis counts. It puts each comment into a predefined category so you can report how often each one shows up. Thematic analysis looks for meaning, building patterns from the text to explain what’s happening under the numbers. The table sets out where they differ.
| Aspect | Thematic analysis | Content analysis |
|---|---|---|
| Main question | What patterns of meaning run through the responses? | How often does each category appear? |
| Starting point | Usually inductive, built from the comments | Usually deductive, built from a set category list |
| Output | Named themes with definitions and supporting quotes | Category frequencies and percentages |
| Role of counts | Supports the themes but doesn’t define them | Central to the findings |
| Best fit | Exploratory surveys, new programs, “why” questions | Tracking surveys, large volumes, wave-to-wave comparison |
| Coder judgment | High, since themes are interpretive | Lower, since rules define each category |
| Typical risk | Themes drift toward the analyst’s assumptions | Counts miss context and nuance |
In practice, you’ll probably blend them. Suppose you run a quarterly customer survey. In the first wave, you’d use thematic analysis to find out what customers talk about and build a codebook from it. From the second wave on, you’d apply that codebook more like content analysis, counting each category so you can track shifts, while still reading for new themes that the old list doesn’t cover. That way, you get comparable numbers each quarter without losing sight of the comments that don’t fit any category yet.
Benefits and challenges of open-ended thematic analysis
The method asks for time and some discipline. What you get back:
- Any topic works, from patient experience to training feedback. Nobody on the team needs a statistics background.
- Real words end up in the report. A quote like “I waited 40 minutes and nobody told me why” tends to move a leadership team more than a bar chart does.
- Surprises show up. People write about what’s on their mind, so problems you never planned a question for appear anyway.
- Scores get context. Put next to closed-ended data, themes explain why scores moved and which segments moved them.
- It scales. The same method covers a 40-person class evaluation and a 20,000-response customer study, though the tools you’ll use change along the way.
The hard parts:
- Coding 1,000 comments by hand can eat a few working days, and the hours climb fast with volume.
- Two readers, two readings. Analysts can read the same comment differently, which is why the tested codebook and the second coder matter.
- Short answers. A comment that just says “the app” is hard to code until you look at that respondent’s other answers.
- Who writes. The people who leave comments aren’t always typical of everyone who took the survey. Check who’s behind a theme before you generalize.
- Vivid quotes. One angry paragraph can feel like a trend. It’s still one person.
A couple of rules set before you start handle most of this. One is a minimum count before anything gets called a theme. The other is a check on which segments each theme comes from. Say an employee survey comment reads “leadership doesn’t care”. If you can see that it came from a team that scored low on every leadership item, it carries weight. If it’s the only one of its kind among 900 comments, it belongs in a footnote.
How AI tools simplify thematic analysis of qualitative feedback?
Run the AI pass
The AI pass runs on the full set of responses and produces draft topics and sentiment.
Check each topic label
Someone on the team reads 50 to 100 responses from each topic and asks whether the label fits.
Merge, rename or split topics
Topics get merged, renamed or split until each one describes a single idea. A topic called “staff” that holds rude agents and slow agents is two ideas.
Fix missed sentiment calls
Fix sentiment calls the model missed, especially sarcasm and mixed comments.
Read the unplaced responses
Read the responses the tool couldn’t place, since new themes often hide there.
AI takes over the slowest part of thematic analysis of open-ended survey responses: the first read. A text analysis tool can make that pass for you. It groups responses into topics and tags sentiment, and you spend your time checking what it did. On a few thousand comments, that can turn a week of coding into an afternoon of review.
It’s at its best as a first draft that a person signs off on. A 2025 study by Leah von der Heyde and colleagues, accepted by the journal Survey Research Methods, tested large language models on more than 5,000 open-ended survey answers. A fine-tuned GPT-4o model matched the human codes closely, with agreement slightly above what two human coders reached with each other, while the off-the-shelf models scored much lower and struggled with catch-all categories such as “no reason” and “other”. The authors concluded that human validation is still needed, which matches what most survey teams find in practice.
[Source: https://arxiv.org/abs/2506.14634]
In Sogolytics, the Text Analysis report sorts open-ended responses into topics, tags each one as positive, neutral or negative, and shows a word cloud of frequent terms. You can search responses by keyword, filter by topic or sentiment and export the results, and where the model’s gotten a topic or sentiment wrong, you’re able to correct it by hand. If you’d like to read more on how tone gets scored, this explainer on what customer sentiment analysis is covers the basics, and word frequency has its own limits, as this piece on word cloud results shows.
To keep the speed without losing accuracy, most teams settle on a workflow like this:
Treat AI topics as codes, since that’s what they are, and do the theme work yourself. A model can tell you that 200 comments mention “billing”, but it takes someone who knows your business to see that 120 of them are about a confusing invoice layout introduced in March. This overview of AI in customer feedback analysis goes further into where it helps and where it doesn’t.
Conclusion
Open-ended thematic analysis pays off when every theme comes from a tested codebook and stays tied to the people behind it, with AI doing the first sort and you making the call. In Sogolytics, you’ll build the questions in the online survey maker and read the answers in the Text Analysis report and Verbatim Report, sorted by topic, sentiment and segment, with no spreadsheet on the side.
FAQs about open-ended thematic analysis
Can thematic analysis be used for questionnaires?
Yes. It’s one of the most common ways to handle the free-text boxes in a questionnaire. Picture a course evaluation that ends with “What could be improved in future sessions?”: you’d code each answer, group the codes into themes, then check which themes come from the students who rated the course lowest, and since those answers run shorter than interview transcripts, expect more of them and simpler codes.
How many themes should I have in thematic analysis?
Four to eight main themes is where most survey projects land, each with a few sub-themes underneath. If you’ve got 20, several of them are probably codes that haven’t been grouped yet. A good test is whether your stakeholders can still name them a week after the presentation.
How long does thematic analysis take on survey data?
A few hundred short comments take one or two days by hand, codebook included, and the write-up usually takes another day. Several thousand can take a week or more. That’s when many teams hand the first pass to a text analysis tool and spend their time reviewing it.
When should you use thematic analysis?
Reach for it when you need the why, like when a score moves and nobody in the room can explain it. New surveys and new programs are good candidates too. If you’re tracking the same categories wave after wave, a fixed coding frame closer to content analysis tends to be faster.
What are the main limitations of thematic analysis?
Subjectivity is the big one, since two analysts can read the same comment differently without a shared, tested codebook. It’s also slow at scale, and the people who bother to write comments may not represent everyone who took the survey. A second coder and a quick check of who’s behind each theme cover most of it.


