A popular type of question to ask in a survey is the ranking question, e.g.
Please rank each of the following items in order of importance
From the respondent’s perspective, this question involves more effort to answer than others because of the need to compare each item in the list against every other item in the list, and also requires more time to fill-in as the list gets longer.
In many cases though you don’t need to ask for every item in the list to be ranked. Asking for the top 3 (or even 2) can be enough to arrive at the same result.
The data table below is from real survey data on a question that ranked 8 items (although only showing the top 3 positions here):
| Item |
% Ranked
1st |
% Ranked
2nd |
% Ranked
3rd |
| Red |
26 |
19 |
14 |
| Blue |
28 |
20 |
15 |
| Pink |
3 |
13 |
17 |
| White |
20 |
17 |
13 |
| Black |
8 |
10 |
10 |
| Brown |
3 |
5 |
9 |
| Yellow |
7 |
10 |
12 |
| Green |
6 |
7 |
9 |
With this survey data the ranking is identical whether we got the respondent to rank all 8 items or just their top 3 (and if we only used the top 2 the results are the same except for positions 4 and 6 – btw the results above are blue first, red, white, then a big drop to pink, yellow, black, green and brown last).
The point though is that your ranking question data isn’t going to be the only factor in any business decision (other survey data should contribute for a start). Are you looking to identify the most important 5 or 6 factors (from a list of 26), or categorise the items into “high importance”, “medium” and “low”? When designing the survey – bear in mind how the data will be analysed – if you don’t need an ordered list of 26 priorities then don’t ask your customers to work out theirs!
Dan Wardle
Categories: Customer Surveys / Loyalty · Survey Design Tips
Tagged: ranking question, Survey Design Tips
I’ve been meaning to share this post on Top Right Corner’s blog that highlights two common mistakes when asking an open-ended verbatim type question after the Net Promoter Score question (e.g. would you recommend [company name] to your friends and family).
Asking the Right Question to get to the Key Drivers of NPS or Customer Satisfaction gives good practical advice how to avoid two possible effects that can turn up in customers’ verbatim comments.
Always remember that the wording of a question can have a big impact on how it is answered.
Dan Wardle
Categories: Net Promoter · Survey Design Tips
Tagged: Net Promoter, Survey Design Tips
Surveylab is looking for a graduate / recent graduate to join our small and friendly team as a junior web-developer. This is a technical role supporting the production of clients’ surveys from database setup to programming and Q.A.
The position is full time, based in south-west London (our office is in Epsom but we’re scouting for new offices to move our technical team into).
Read more on our website: web-developer vacancy
Categories: Surveylab
Tagged: graduate jobs, web developer jobs
29, September 2009 · 1 Comment
A post on The Messaging Times blog makes a case for at least testing whether sending emails at the weekend is more effective than mailing during the week.
The success of weekend mailings is obviously going to differ between organisations and their audiences, and before sending an email about a customer satisfaction survey on a Saturday my first thought is “does the organisation offer support or provide customer contact at the weekend?”, but there are some compelling arguments to trial it.
Looking at just one customer satisfaction survey that we mailed in September – with the initial mailing being sent on a Tuesday and a reminder being sent the following Monday, the number of click-thru’s follows a very typical pattern:
| Day |
No. of click-thru’s |
Tue 8
(email invitations sent) |
517 |
| Wed 9 |
258 |
| Thu 10 |
70 |
| Fri 11 |
38 |
| Sat 12 |
20 |
| Sun 13 |
15 |
Mon 14
(follow up emails to non-respondents) |
204 |
| Tue 15 |
83 |
| Wed 16 |
25 |
| Thu 17 |
25 |
| Fri 18 |
8 |
| Sat 19 |
9 |
| Sun 20 |
5 |
| Mon 21 |
9 |
| Tue 22 |
7 |
| Wed 23 |
3 |
| Thu 24 |
5 |
| Fri 25 |
3 |
| Sat 26 |
6 |
| Sun 27 |
1 |
For consumer customer satisfaction surveys we find that it generally doesn’t make a lot of difference when we mail Monday to Friday – after two-three weeks we typically collect the same number of responses overall. In the results above, the first weekend’s results don’t indicate a massive drop just because it’s a Saturday, and later weekends look like mini-spikes.
Mailing on a Saturday could just delay the first day’s big click-thru rate until the Monday. Or, perhaps mailing on a Saturday generates a significant response rate boost because many more people actually see the email and feel they have more time to respond..?
If we can test this out we’ll post our findings here. And if you have any experience we’d love to hear.
Dan Wardle
The Messaging Times post that inspired this one: Weekend Email Campaigns
Categories: Customer Surveys / Loyalty · Online Surveys · The email side of surveys
Tagged: email marketing, email survey, survey response rates
Jeffrey Henning has written a very good article in Research magazine – do phone surveys have a future?
The television didn’t make radio obsolete, even if it did push dramas, soap operas and serials on to the TV set.
The article is based on the US market, but all his points apply to the UK too.
Dan Wardle
Categories: "Research 2.0" · Online Surveys
Tagged: market research, telephone survey
Great Little Trading Co recently asked for feedback via their regular newsletter and avoided a common pitfall when embedding survey links in a newsletter by giving the link/news item top billing. Applies when then newsletter has more than one news item in an issue.
The subject line was 500 pennies for your thoughts (it works) and because it’s the newsletter the email has the benefit of being recognised in the inbox by customers/subscribers too.
The text is straight to the point:
- Help us (be Great)
- How? Take our survey
- What’s in it for me? £5 off
- Click here to start
I would expect GLTC got a good response rate from this email, and Marketing were probably still happy because the rest of the newsletter was still able to promote their offers.
Dan Wardle
More examples of emails used for customer surveys (including Lastminute.com and Borders) are here.
Categories: The email side of surveys
Tagged: customer survey emails, email survey, online survey
I went to Bluewater shopping centre this week where I visited the lego store (if it sounds like this was my first visit then you are very mistaken!) and on the receipt was a web link for a survey. Needless to say I logged in.
I am a big fan of lego (what’s not to like?) and shopping on a Thursday evening in early September I suppose you’re less likely to suffer bad luck with queues and what not. I answered all the questions with scores of 10 or 9 and the survey wasn’t anything to write home about, but today I received an email from Lego saying Thank you for your LEGO store feedback.
Ok, mine was a positive experience, and the email is no doubt automated (proof is in the mail-merge error in the first paragraph). However, the email still generated a positive feeling about the experience and the brand. I’m curious if I’d had a bad experience what they would have said…
Occasionally, we email a thank you message to our clients’ customers after a survey has closed (some clients also use our “customer at risk” alerts from our customer surveys to assist their customer service delivery) but more often we simply say thank you at the end of the survey itself (and trust that the client will act on the feedback).
Lego’s example has made me pause to think though – how do you say thank you to survey respondents?
Dan Wardle
Categories: Customer Surveys / Loyalty · The email side of surveys
Tagged: customer survey, lego
The role of Workplace Surveys
Business Week recently published an article about Employee Engagement – “It’s Not the Economy, Stupid” by Krisztina Holly from the University of Southern California and Jim Clifton from Gallup.
The article’s theme was that rather than worrying about increased unemployment in a gloomy economy, management should be concentrating on maximising the motivation of those still in jobs. It quoted recent Gallup research that indicated that less than 30% of the corporate workforce was truly engaged and as many as 20% were “actively disengaged”. It also made the point that strong employee engagement leads to increased customer engagement which in turn leads to higher revenues, profits and job opportunities for others.
The article went on to say that because management may be too concerned about economy issues they could be overlooking some of their basic managerial responsibilities. In particular they may be neglecting to ensure that their employees fully know what is expected of them and not allowing them to fully use their talents in their work roles. It quoted the US electronics retail chain, Best Buy (now linked with the Carphone Warehouse in the UK) that has linked improved employee engagement to increased profits. It reckons that
“for every one-tenth-of-a-point increase in employee engagement each Best Buy store increased profits by $100K per year” !
Although attention to employee engagement should be basic management common sense, in today’s complex and often diverse corporate environment it is important that senior management establish the tools to measure employee engagement and quickly identify issues undermining performance. That is why a regular (at least bi-annual) workplace survey needs to be conducted to keep abreast of employee opinion and engagement.
One of the concerns with fast moving and geographically spread corporate environments is the range of management skills that can exist within an organisation. Even though senior management may have all the right skills and abilities, and the correct management processes in place, it only needs one or two rogue managers lower down the chain to undermine all that good work.
By formally touching base with employees through a well designed workplace survey, management can measure engagement levels and ensure that the voice of the employee is properly heard, analysed and acted upon.
John Kemp
Categories: Employee Surveys
Tagged: Best Buy, employee engagement
Last week we appeared to be working on email more than anything else, which has sort of inspired me to look for good examples of customer survey emails. I’ll share what I find – the good, bad and ugly here on measuring-satisfaction.com.
What makes a good customer survey email?
The short answer is one that works (like any email marketing), although having been through my gmail inbox looking for examples it was telling how many emails asking for feedback about the customer experience were literally “knocked out” – brands can put a lot of love into the regular marketing newsletters but when it comes to the dirty word(s) customer care, the graphic designer always seems to be at lunch.
Not that it’s all about looks – the goal should be to maximise the survey’s response rate (and perhaps help service recovery depending on how the survey is positioned).
More to follow but meanwhile here are three examples from lastminute.com, ticketmaster.com (scroll down – third row from bottom of page) and Borders’ books.
Please feel free to share your own favourites!
Dan Wardle
View more examples:
Categories: The email side of surveys
Tagged: customer survey emails, Online Surveys, The email side of surveys
One other article I bookmarked this week. How to make data charts look good – Ten Chart Design Principles (screenshot below). Excellent…
Our own charts are customised for different clients. This one below is a fairly common example (chart from our ficticious company demo).
You might also like Smashing Magazine’s Data Visualization: modern approaches article (lots of high quality examples of data presented in anything but a pie chart).
Dan Wardle
Categories: Survey Design Tips
Tagged: data visualisation, pie charts, Survey Design Tips
