3 Ways to Maximize the Value of Open-Ended Responses from Customer Surveys

In my previous article on verbatim feedback, we explored some of the ways you can organize and systematically analyze open-ended feedback collected from your B2B customer surveys. Whether you leverage a text-analytics tool or read and code all free form responses manually, maximizing the value derived from this rich insight depends on what you do next. Here are some tips for showcasing the value of customer commentary.

Tip 1: Quantify the Verbatim Results

So long as you leveraged a codebook and utilized a consistent and defensible coding methodology, you can quantify the themes your customer’s highlight. While there are a few ways you can mathematically represent this information, there isn’t one method that’s necessarily the best.

Before getting into some specifics, it’s important to note that coding and quantifying commentary is not an exact science and is subject to interpretation. While there are codes you would apply to certain comments, a colleague may interpret the sentiment in a slightly different manner, so remember to seek input from others (see Tip 3 below).

There will also be instances where you’re unsure if a code applies or not. In such scenarios, I ask myself if I would be able to make a compelling case (usually to my client) that the code applies to the comment. If there is reasonable doubt, I won’t apply the code. With that in mind, here are some examples for quantifying verbatim coding results.

Example 1: Divide the count of theme/code mentions by the total number of comments received

If after coding 500 comments you find that approximately 200 mentioned the team’s responsiveness in a positive context, then roughly 40% of survey respondents who left an open-ended comment complimented their service team for responsiveness (200 theme mentions / 500 total comments = 40%).

Example 2: Divide the count of theme/code mentions by the total number of respondents who completed the survey whether they provided commentary or not

Applying the same scenario from the previous example, if 1,100 respondents completed the survey, then 18% of the participants mentioned the team’s responsiveness in a positive context (200 theme mentions / 1,100 total survey respondents = 18%).

Example 3: Compare the frequency of positive and negative mentions for a single theme (if applicable)

Although 200 individuals referred to the team’s responsiveness positively in their open-ended feedback, 150 respondents also spoke of negative experiences. In total, responsiveness was written about 350 times in a positive and/or negative context. As such, positive and negative mentions occurred at around 57% and 43% of the time, respectively. (200 positive mentions / 350 total theme mentions = 57% and 150 negative mentions / 350 total theme mentions = 43%).

Tip 2: Trend Verbatim Coding Results

Trending quantitative feedback survey over survey and measuring change is one of the reasons scale survey questions are so valuable. They allow you to measure the impact of your closed-loop activities, assuming survey design and data collection best practices were followed. Data professionals like myself often thrive in these environments because we’re trained to adhere to mathematical boundaries and logic. But as mentioned earlier, coding and then quantifying open-ended customer feedback doesn’t exactly follow a set of rules, so how can we trend this information?

The short answer: it’s difficult. We have the benefit of coding experience going back years and across many industries here at Satrix Solutions, so we’re used to ways to trend qualitative feedback. Tracking shifts and emerging themes within customer commentary is immensely valuable, so it’s definitely worth the effort.

For example, to understand why Net Promoter Score dropped from one survey wave to another, reviewing the customer comments from the open-ended question following “likely to recommend” is a great place to start. It’s important to ensure a consistent coding methodology and to apply your ‘counts’ in a similar manner. Keep a keen eye out for issues just bubbling up to the surface, as those may be customer frustrations you can get in front of or evidence that some of your improvements are taking hold. Also be sure to analyze verbatims in relative terms, as changes to your respondent population between survey A and survey B (for example) will impact your counts.

Tip 3: Collaborate on Verbatim Coding

Collaborating with colleagues’ when interpreting free form text can shed light on topics you might be overlooking. This likely applies to any work environment but it’s especially useful when attempting to understand the big picture behind customer commentary. We communicate and collaborate with our clients before and during our analysis work, which helps shape our verbatim codebook and how we interpret certain keywords and phrases. For me, this is like establishing scientific consensus. Challenging your own assumptions and inviting a diverse set of perspectives takes work, but it ultimately strengthens your findings.

Other Ways of Leveraging Open-Ended Customer Feedback

For just about all our codebooks, we like to track the number of instances a person is positively mentioned and share the results with our clients. Whether we showcase these quotes during a presentation or deliver them in a spreadsheet (some of our partners will share them with the entire company to exemplify service excellence!).

For companies with engineering or product teams, calling out roadmap requests, commentary on specific features/functionality, reactions to recent software updates, etc., is valuable insight. Going further, we can attribute these findings to certain customer segments which may help our client take a more targeted approach during their close-loop activities.

No matter the shape it takes, listening and acting on customer feedback is part of the genetic makeup of exceptional organizations. NPS, Customer Satisfaction (CSAT), and Customer Effort Score (CES) are great tools but gaining a holistic understanding of open-ended survey feedback increases the effectiveness of these metrics.

As this 2-part blog series shows, there is a lot we can do with free form customer input. Just remember that if your customers took the time to write down their thoughts, it’s important to them. Now it is on you to read it.

Related Posts

 

By |2020-10-05T15:06:06-07:00October 5th, 2020|Categories: Customer Experience|