What concerns Detractors? What make Promoters promoters? In this case study, we use Gavagai Explorer to acquire insight into the answers of an open-ended follow-up question in a Net Promoter Score survey of the Swedish Telecom business with 2535 respondents.
Net Promoter Score (NPS) is a metric for measuring the loyalty of a company’s customers based on their feedback. NPS is widely used across all kinds of industries, for instance in the travel and hotel business, software services, and telecommunications. Typically, NPS surveys are conducted continuously, so as to assess the performance of an organization over time. Each survey may include the responses from thousands of customers.
The NPS score is gauged off of the question:
How likely is it that you would recommend our company/product/service to a friend or colleague?
The answer is given on a scale from 0 to 10, where 0 means that the respondent will definitely not recommend the service. Respondents are categorised as belonging to one of three groups, depending on their answer: Detractors (answer 0 – 6), Passives (answer 7-8), or Promoters (answer 9-10). The NPS score itself is computed based on the relation between the three groups, and can range from -100 to 100. Positive values are generally considered good, and a value of +50 is excellent. NPS scores vary wildly between industries.
Now, as widely used as it is, NPS is not without criticism. One of the key points made against the case of using NPS is that one question that only accepts numerical responses, does not provide actual insight into why the customers answered the way they did. To mitigate this, the NPS question is sometimes accompanied by an open-ended query, asking customers “why?”.
But how do you make sense of thousands of free-form textual answers? Do you read them all?
No. Of course not. To make the most out of your hard-earned data you want to avoid confirmation bias, concept drift, and human errors made due to fatigue. You will need to have many people read the same set of answers. This is time consuming, tedious, and very expensive, and thus not practicable in any realistic setting.
In cooperation with CustX, we used Gavagai Explorer, a tool for rapidly acquiring insight from large text sets, to look into a benchmark NPS dataset obtained from a study carried out by CustX, QuickSearch, and Nordic Bench concerning the telecom business in Sweden during the first quarter of 2016. Of the 2535 respondents that provided an answer to the NPS question, 1128 also answered the follow-up question:
Finally, tell your story about the telecom business. Anything that you have experienced.
By loading the spreadsheet with NPS data into Gavagai Explorer, and then filtering it based on Detractors on the one hand, and Promoters on the other, we obtained a view into what actually concerned the respondents in less than 15 minutes. We found the answers to our initial questions as per the following.
What concerns Detractors?
Detractors most prominently express thoughts on switching services, churn and voice their concern regarding Telia and Telenor, two major operators, as well as toward services within the segments subscriptions, (cell) phones and TV.
What make Promoters promoters?
Promoters are promoters because they are predominately happy with specific providers, and specific services, that is Telia, Telenor and Tele2, and Internet via fibre and Broadband, respectively.
The figure below shows the top 10 themes expressed in the responses made by Detractors (red bars), and Promoters (green bars). On the right hand side in the figure, the distributions of positivity, negativity, and skepticism, as expressed by the respondents in conjunction with each theme are displayed. Note that the white pie charts indicate that no attitude was detected for any of the answers for the theme Nothing to say.
Detractors and Promoters in Swedish Telecom NPS survey, Q1 2016
As can be seen in the figure, the top theme for Detractors and Promoters alike is that of (Un)happiness. The theme is deliberately modelled to contain terms expressing happiness (satisfaction) and unhappiness since they are, in one sense quite similar: they just are at opposite ends of the same scale. As such, we wanted to investigate their combined coverage of the answers to the open-ended query. The sentiment scores reported as pie charts next to the theme tell the story of a difference between Detractors and Promoters: although perhaps surprisingly positive in their expressions when it comes to their (un)happiness, the Detractors are actually much less so compared to the Promoters for the same theme. To further understand the differences for this particular theme, we use the Gavagai Explorer to drill down into a couple of verbatim examples (with English translations in brackets).
Example sentences from responses by Promoters:
- Jag är i det stora hela nöjd med mina leverantörer.
[I am, by and large, happy with my providers.]
- Är jättenöjd med familjeflex abonnemanget som Telenor erbjuder
[(I) am very happy with the “familjeflex” subscription that Telenor offers]
- Universaltelekom är jag inte nöjd med
[Universaltelekom, I am not happy with]
Example sentences from responses by Detractors:
- De är jag nöjd med och jag tycker tele 2 är en bra operatör som man kan lita på
[Them I am happy with and I think tele 2 is a good operator that you can trust]
- Nöjd med Telia och dess tekning
[Happy with Telia and its coverage]
- Inte nöjd med telia då samtal bryts ofta och när man ringer dem och frågar varför så får man bara svaret att det inte är något fel på deras tjänst!
[Not at all happy with telia because calls are often disconnected and when you call them and ask why, you only get the answer that there is nothing wrong with their service!]
The same methodology can be applied to all themes shared between Detractors, and Promoters, that is, to first look into the theme itself, then use the sentiment scoring to further understand the differences between Detractors and Promoters, and finally assess qualitative differences by reading the verbatim sentences underlying the analysis.
Some general notes on the graphs in the figure. The attitudes expressed by Promoters are more positive than those of Detractors. The responses from Detractors are more evenly distributed across themes: the first theme ((un)happiness) is touched on by twice as many respondents than the second theme (Switching services, churn). While the Promoters graph shows that the first theme (again, (un)happiness) is almost three times as common than the second theme (Telia) among the respondents.
Do you work with answers to open-ended survey questions, NPS follow-up investigations, output from Customer Experience touchpoints, or other areas in which you need to gain rapid insight into large amounts of text? If so, we invite you to try out Gavagai Explorer.
About Gavagai Explorer
Gavagai Explorer enables individual analysts to single-handedly process amounts of data that would otherwise require tens or hundreds of analysts doing manual analysis.
Gavagai Explorer turns qualitative and unstructured text into quantitative measures by automatically identifying and ranking common themes, detecting associative expressions significant for each theme, as well as by scoring themes against multiple dimensions of sentiment, such as positivity, negativity, skepticism, and desire.
Gavagai Explorer is the Text Intelligence tool of choice for analysts who want to gain rapid insights into large text collections, such as answers to open-ended survey questions, NPS follow-up investigations, output from Customer Experience touchpoints, product reviews, and social media discussions around brands. The Explorer relies on the vast language knowledge continuously learned by Gavagai’s Semantic Memories, currently available in 46 languages. Gavagai Explorer is available at https://explorer.gavagai.se
