Analyzing customer reviews and other open-ended feedback to hone strategy

Star ratings are an easy way to measure customer sentiment. But if that’s where you stop, you’re missing out on some rich data. Chances are that you’re sitting on an untapped source of data – the text of customer reviews and other open-ended feedback.

Why is it untapped? It’s complex, and it can be hard to know where to start, and hard to know what to do with all that new information you’ve just untapped.

Why is it worth the effort? It can give you insights that your number data cannot, and you can learn more about what your numbers mean. Even better, it can help you identify the things that will let you address multiple issues at once, because you’ll have more granular “why” data.

I cover two frameworks – sentiment analysis and thematic analysis – to get you started on tapping into this rich text data and figuring out what to do about it. If you’re into jargon, what I’m talking about is performing qualitative analysis on unstructured data.

Preliminaries

To begin, I’ll quickly define some terms.

In talking about this text data as qualitative data, what I’m highlighting is that this is not number data, which is called quantitative data.

In talking about this text data as being unstructured, I mean that it doesn’t have a predefined structure that flows easily into our metrics and analyses. People can write and say whatever they want, however they want, without it being nicely organized, or structured, into tidy boxes or database fields that they’re just filling in. This text is everywhere: in the open-ended text that goes along with the star rating, social media posts, text messages, email exchanges, transcribed customer support calls, focus group data, and other open-ended feedback 

To analyze and use this data, then, what we’re trying to do is figure out what people are writing about or talking about, and how that fits in with the tidy, predefined boxes, or classifications for the stuff we care about, and our metrics, analyses, and other factors in our decision-making.

Both of the frameworks I’ll cover can be applied with something as low-tech as highlighters on paper, and as hi-tech as artificial intelligence.

Sentiment Analysis

Sentiment analysis is a way of assigning an emotion or opinion-based classification to a chunk of text.

At its most basic, you’re assigning along a scale of positive, neutral, or negative, kind of like emojis. Let’s look at an example of a fictitious review of a spa experience. 

[November 2020] “This was a wonderful experience! It was relaxing and peaceful – the music, the understated décor, the herbal teas, and the massages. We took advantage of the spa’s holiday discounts for couples.Even so, the price was steep, but it’s nice as an annual treat when you’re in the mood for posh.”fictitious client of the fictitious spa

This review is what I call rich text. There’s a lot of information embedded in here. It’s rich in value, if only you can figure out how to tap into it.

However, applying basic sentiment analysis reduces all of this rich information, to a relatively uninformative positive. All of the reasons why, and all of the caveats as to why not, have been lost.

Still, basic sentiment analysis can be good for a quick instantaneous measure of the general opinion of your brand at a given point in time, like how people generally feel about the new ad or product you just released. Consistently having content circulating that promotes a positive attitude towards your brand can keep people in your sphere, and, hopefully, lead to buyers.

Fine Tune with Entities and Aspects

You can fine tune with a variation called entity-based sentiment analysis. An entity is a noun object, like a person, place, product, event, topic, or brand name. When applying this, you first scan for all of the entities you can identify, isolate what part of the text is referring to that thing, and then assign sentiments to those entities accordingly. This can help you do things like figure out who your customers think your competitors are, and how customers have felt about their experiences interacting with certain people at your company.

You can fine tune again with aspect-based sentiment analysis. An aspect is an attribute of an entity, like location, noise, cleanliness, battery life, and personality traits.

In the case of our fictitious review, we’ve got quite a few entities at play. Words like relaxing and peaceful are positive words, which are tied to the massage entity, so that would evaluate positively. The pricing aspect is a bit ambiguous due to the tension between steep and treat as descriptors, and how that’s decided would depend on your classification algorithm. What the spa does with this information will depend on why they think customers feel this way, something that isn’t provided by this type of analysis.

Thematic Analysis

Thematic analysis lets you dig into the “why” behind sentiments, and see what themes are emerging about your brand in the words of others. 

In addition to reviews of products and services, I also favor thematic analysis for analyzing company recommendations for zeroing in on the value your customers think you offer.

As with finely tuned sentiment analysis, you scan through for entities and aspects, but, instead of assigning along a predefined scale of sentiment, you’re working to identify the words, ideas, and themes that emerge from the text. When you find something of note, you highlight that chunk of text and assign a code to it. That code can be a word or phrase found directly in the text, or a word or phrase that you think best sums up what’s being talked about there.

So, you’re still assigning a label to a chunk of text, but thematic analysis codes are more bottom-up, coming from the text, instead of top-down, coming from the sentiment scale.

How does this impact how this new information is utilized?

Let’s go back to our example and see how this compares to sentiment analysis. In the review, I’ve highlighted the sentiments (relaxing, peaceful, steep, treat, and posh) in bold and yellow, the service offerings (massages) in underline and orange, and the pricing in italics and blue.

Instead of being limited to an ambiguous neutral sentiment regarding the pricing, you could use the words of your satisfied customers to market your massages as a treat, which a lot of spas already do. But how many of them market themselves as posh?

If this type of sentiment is repeated across multiple reviews, and you see a pattern emerging as something that draws in your happy customers, then you’ve isolated a differentiator for yourself. You haven’t had to brainstorm marketing copy – you’re speaking to customers in their own words. Even better, instead of worrying about adjusting your pricing model, you now have a characteristic that warrants higher prices – your poshness.

Let’s Compare Sentiment vs. Thematic Analysis

Compared to sentiment analysis, thematic analysis gives you greater nuance and granularity, and that nuance and granularity are better suited to taking action, because you’ll have a better handle on the “whys” that your customers are focusing on when forming their opinions. You can perform thematic analysis with automation and artificial intelligence, but it’s more likely that thematic analysis will be more manual. Even so, there’s some great software to help you out with large volumes of data for thematic analysis, particularly when you’re ready to synthesize. Humans might also be better able to understand the language than artificial intelligence algorithms, and better identify themes.

How can you use analyze your own customer reviews?

Depending on the volume of data you have, and the immediacy with which you need the results, you might need to invoke an automated solution for the type of analysis you’re choosing.

Will the automation get it wrong sometimes? Yes.

Will humans get it wrong sometimes? Yes.

Is it worth it anyway, even with some errors? Yes.

Some insight, with a little bit of error, is better than no insight at all.Remember, data is about clues, not facts. Whatever your data source, humans still need to do their big picture, synthesis, vision and strategy driven decision-making part, for what to do with that customer-related data.

I’ll leave you with two driving questions to keep in mind as you explore these frameworks and the data they produce:

• What nuance or insight can this give you that the quantitative data cannot?
• Are there things you can identify that will address multiple issues at once?