Prefer a single question to long and tedious surveys and use text analysis to understand the answers.
A survey from LastPass
I am a long-time user of LastPass and I introduced it in the company I co-founded, when we needed to manage our passwords.
And I am also in the SaaS business and appreciate the need for customer feedback which is why I wanted to help out by answering this survey I received from LastPass.
I do have a particular interest in surveys by virtue of the products my company develops (unstructured text analysis) and, finally, since Qualtrics is another option in the text-analysis market, it is always fun to see how they are used by their customers (LastPass used Qualtrics for this survey).
My LastPass survey experience was a failure
So I started the survey and at first it seemed promising: an NPS type question with the quantitative scale of 0 to 10 and an option to explain my rating.
So I put in a 6, after some consideration, and then wrote my 5 or 6 sentence answer covering my main points of pain in using LastPass in my company. I would say my answers are valuable from a customer experience analysis perspective. But I am only one person so they would have to be aggregated with many others’ of course, to get the full picture.
But when I clicked the arrow to submit my response, the survey brought me to another question, and at the top left corner there was an indicator of how long I had gotten in the survey: 0%, so I closed the window and never finished.
I don’t have time to answer lots of questions and I think this method of surveying – by carefully crafting a survey with all possible questions that might be valuable for the company and then asking your customers to painstakingly navigate it – is bad: you won’t get the responses you want. A lot of respondents will get survey fatigue and quit prematurely (like me). This is bad for you and annoying for the respondent who perhaps spent some effort on the first question, which was presumably lost when they quit the survey. And those who make it all the way to the end may not be the ones you actually care most about.
Try to keep the survey short
Instead you should keep the survey short. Just the one NPS question with its open-ended text answer to accompany the rating. And then rely on text analysis of the unstructured text to give you the insights you need. You will be able to find both overarching and very granular topics and by correlating with your meta-data, like the number of members of the respondent’s team, the time since they signed up, the amount of money they are spending each month and so on, you will be able to draw your conclusions, essentially answers to the question “what are the drivers I have to care about?”.
And you will get many more responses – a lot more – and that’s great because it strengthens the reliability of your findings; the more responses you get the better. But won’t the text answers be a problem when analyzing the survey results? And won’t that problem be aggravated by increased numbers of respondents? No, not if you use a proper tool to perform the analysis.
This is not a revolutionary change to your survey or to the survey methodology in general. We still use both quantitative and qualitative analysis but we get more responses and a richer and sometimes unexpected flora of insights from the text responses, which are also more plentiful.
Better reliability, less drop-off caused by survey fatigue, and more and better insights; the advantages of using a much simpler survey format are obvious. Maybe LastPass could try it.
P.S. I tried responding to the email with the survey invitation (it had the subject line “We want to hear from you!” after all) but unfortunately I received back a “this inbox is unmonitored” message.
