What AI can and can't do with customer feedback

An AI tool can read a lot of short comments quickly and suggest how they group together. It can't tell you what to do about them. That part stays with you.

Here's the plain version of both halves.

What the tool does well

Think of a stack of comment cards from a café. A tool can read the whole stack in one go and suggest a few piles: slow service, good coffee, noisy room. That's the first job.

The same idea has a long history in research. It's called thematic analysis, and it means finding patterns of meaning, called themes, in written answers. One description says it's about spotting and making sense of "patterns of meaning" in written data.¹

An AI tool is doing a quick version of that. It spots comments that say similar things in different words, even when they share no keywords. "Waited ages" and "took forever to get served" can land in the same pile.

Other jobs it can handle:

• Writing a short summary of each pile.
• Pulling out a typical comment from each pile.
• Labelling each comment as positive, negative or neutral.
• Tidying messy text, such as spelling slips or comments in more than one language.

Treat translations with care. Check them like anything else the tool produces.

Where it goes wrong

It sounds sure of itself even when it's wrong. Researchers call a confident false answer a hallucination. By one definition, it's any AI reply that states something untrue or misleading as though it were certain.²

For feedback, that could mean a theme nobody mentioned, or a quote that no customer wrote. The same source says chatbots built on large language models can slip "plausible-sounding" falsehoods into what they write.²

A quick example. You paste in a hundred comments about a gym. The summary says "members want longer opening hours". Maybe three people said it. Maybe none did, and the tool guessed from a comment about parking at night. You only know by checking, and the checking page shows how.

Positive, negative, neutral

Many tools label each comment with a mood. That's called sentiment analysis. A basic version sorts text into positive, negative or neutral.³

It's weaker than it looks. One source says computer systems struggle with negations, exaggerations, jokes and sarcasm. People usually handle those easily.³

So "Oh great, another wait" may come out as positive. A person reading it wouldn't make that mistake.

Counting is not the same as understanding

Good analysis is more than counting words. One description says thematic analysis goes beyond simply counting words or phrases.¹

A comment like "the staff were lovely but I'd never come back" has two ideas in it. A word count sees "lovely" and moves on. A person sees the problem.

Counting how many comments fall into each theme is its own job, with its own traps. The counting page covers it.

A sensible split of the work

Let the tool take the first pass. The rest is yours:

1. Read some of the comments under every theme.
2. Rename themes in words your staff would use.
3. Check any number the tool gives you.
4. Choose what, if anything, should change in the business.
5. Keep the original comments in a file the tool never edits.

The last step matters more than people expect. Your raw comments are the evidence. A summary is only a reading of them, and a machine's reading can be wrong.

Do your comments include names or phone numbers? Read the privacy page before you paste anything in.