(we are still here, working on something quite exciting, in between the office move, new babies, new hires, and our clients of course…)

We’re hiring too! If you are a recent graduate/about to graduate or looking to start a career in IT check out our Systems Support Analyst vacancy.

When I picked up my new car from our local dealer a couple of days ago I struggled to suppress a smile when the salesman handed me a questionnaire for me to complete in front of him (which also needed signing), and then showed me another questionnaire (from the manufacturer) that I should receive in a few weeks, explaining that he is assessed on customers’ responses to this particular question circled here, yadder yadder yadder…

Photo by Greenkozi, on Flickr

When I picked up my last car (different make and dealer) I was offered an incentive of £15 of fuel if I brought the questionnaire into the garage to go through the feedback with them (“so they could have a chance to resolve any issues”).

The irony is that both salesmen/garages were fine – I was more than satisfied with the experience, the only thing they could have improved was the price (and I really wanted to go back to the first garage to buy our next car but they just don’t offer a model that fits our bigger family requirements, sorry).

Asking a customer to fill in a survey in front of your staff is wrong. And if your dealers/store managers/employees are compelled to “improve” their score by helping customers to fill in the survey you have a problem.

It’s easy to see why this is a bad approach to fielding customer satisfaction surveys. When I was about 10 or 11 we had our windows at home replaced with uPVC windows and I remember my mum complaining about the mess the installer made. When the job was finished she was asked to fill in a very short questionnaire and I watched the biro hover over the less than satisfied option before she swiftly ticked top-box, top-box, top-box for each question and handed it back. “Why didn’t you say anything?” I asked my mum. I don’t recall but I don’t think she answered.

The mess probably wasn’t all that bad and didn’t take much to put right (hoover and 5 minutes I should think). Was it a one-off or a recurring issue? The installer’s manager would never know what the customers really thought.

The issue for employees feeling pressured to improve their score (or ‘gaming’) is more difficult to resolve, and my response if this is the case is …

• perhaps the targets that trigger bonuses are simply too high (initially),
• scores fluctuate too wildly and/or not enough responses to provide reliable scores (if an employee’s score is based on 6 or 7 customer contacts then gaming just 2 results contributes to 33% of the score), or
• staff don’t have confidence in the process and/or feel penalised because of factors outside of their control (a delivery delay or the terms of the warranty are corporate issues – not down to the individual).

Finally, what purpose is the survey for? The example questionnaire I saw at the garage on Friday asked more of “did the salesman explain gap insurance?” and “did the salesman offer you our finance?” than “did we mess up?” and “how can we improve?” – which is more about conformance to procedures than customer satisfaction. Don’t mix the two.

Dan Wardle

If you are new to Net Promoter do read carphone warehouse puts emphasis on customer care (example of it in action) and Net Promoter Telecom industry scores for more information.

Whilst Satmetrix sells detailed reports you can download an excellent (and free!) summary document from their website ( you will need to register) that will tell you, for example, that in North America banks score between -9% and 81% with  an average score of 20%, supermarkets range from 21% to 69% with an average of 44.4% and online shopping from 12% to 71% with an average of 42%. There are some outstanding stars that account for the higher figures. For example, USAA a specialist financial services organisation servicing the military scored 81% for banking, Amazon scored 71% and Apple 78%.

Deborah Eastman, Satmetrix’s CMO summed up the results of their study by saying

“Best-in-class companies are those that put in the processes to continuously listen to, learn from, and take immediate action on whatever their customers tell them about performance”

– a statement we entirely agree with.

At Surveylab we have been using Net Promoter as a performance measure for over four years. Our clients often find it scary to start with particularly when the NPS^© score is initially considerably lower than the traditional ‘percentage’ satisfaction score. But once the initial panic of moving to a lower result is overcome, the benefits of this tighter measure are soon understood and used to drive improvement.

One client of ours started with an NPS of 56.5% in 2006 and is now regularly scoring in the 70%’s (an average of 73.8% over the last 12 months). Used to measure a number of external contractors, regular reporting, including Ranking and ‘Customer-at-Risk’ reports are used to provide monthly feedback including a range of diagnostic scores and verbatim comments. The customer at risk report, in particular, flags all detractors (those scoring six or less out of ten) triggering detailed investigation into the reasons for low scores.

The standard Net Promoter approach is to ask only one question – Based on your experience how likely are you to recommend ABC Co? Whilst we believe it is important to have one clear measure to use as an index, we find linking this index to a tight range of diagnostic questions and at least one qualitative response (verbatim comment) helps us to provide sufficient information to identify issues and drive improvement. We also believe this methodology works as well for workplace / employee surveys as it does for customer loyalty and satisfaction studies.

For further information and to download a selection of articles and white papers, please visit Surveylab’s resources page or give us a call to discuss our approach to customer and employee loyalty measurement.

John Kemp

In our line of work we process lots of data, most of it on behalf of our clients. For example, we receive sample files that contain lists of customers or employees to whom we will email survey invites and often the sample files contain other demographic data that is merged with the survey responses to be included in the analysis and reporting.

And of course we collect lots of survey answers, which contain all kinds of information. No matter how interesting or not the data appears, the data are valuable assets.

We have an Information Security Policy that is reviewed and updated (at least) every year, and also carry out an Information Security Risk Assessment. To be honest, I don’t find the audit the most exciting task but it is very important. While our company is small and our team of 6 aren’t stupid it is easy to assume “that goes without saying”.

When did you last consider information security?
Hopefully, not the last time there was a high profile news story involving a laptop, train and lost luggage! In an ideal world you think about security without realising – you use a password to access a system because it contains private information, or the IT Department won’t just email you a file of the last 2,000 customers’ emails without the authorisation from someone who can authorise such requests.

Some security issues aren’t that obvious though. For example, the eighth principle of the UK’s Data Protection Act says that “personal data should not be transferred to countries outside the European Economic Area without adequate protection”. A few other countries are deemed ok, e.g. Canada, but others are not, e.g. USA.

(the Information Commissioner’s Office has published various easy to read guidance, including International Transfers – legal guidance (opens in PDF), although remember that the Data Protection Act refers to the processing of “personal data” – click here for an overview of the Act)

Information Security isn’t all about computers and firewalls – it’s about processes too. For example, when sample files are passed to suppliers are they encrypted/password protected? Or, how do new users get added to the distribution list for tracking reports?

Don’t go overboard, but don’t take the security for granted either.

Dan Wardle

1. I expect economic recovery in my country to advance strongly in the first half of 2010
2. I expect economic recovery in my country to advance strongly in the second half of 2010
3. I expect economic recovery in my country not to advance strongly until early 2011

This is a question I saw recently which demonstrates the importance of asking questions correctly and checking/testing your survey thoroughly before launch (and also why compulsory questions should be avoided).

How do you answer this question if you don’t expect economic recovery to advance strongly in the next twelve months, or expect economic recovery to just stay slow until 2012, or beyond?

We regularly tell prospective clients that we can turn-around the survey programming in 5, perhaps 3 days, but most delays occur at the questionnaire draft and review stages. It is important to get the content right – otherwise your results are useless.

If you’re designing a survey Annie at LoveStats has sage advice:

Even if you much prefer DIY, I 100% believe that everyone needs a middle man. Whether that middle-man is a reputable survey company or another person in a completely different department, you absolutely must have fresh eyes.

Dan Wardle

Do you have any of the following at home?

• Nintendo Wii … Yes / No
• Sony Playstation 3 … Yes / No
• XBox 360 … Yes / No
• PC computer for playing games … Yes / No

Change the question ever so slightly:

Which of the following do you have at home?
Please click all that apply

[ ] Nintendo Wii
[ ] Sony Playstation 3
[ ] XBox 360
[ ] PC computer for playing games
[ ] None of the above

It is important to add a new option “none of the above” (or similarly worded option) in case the respondent needs to answer “no” to all the options offered.

By converting your Yes/No grid to a list of checkboxes the respondent can more quickly read down the list and tick off the relevant answers (compared to read each answer, scan across the screen to where the Yes/No button is, and repeat). It looks less daunting, and with longer lists the action of answering doesn’t feel like a chore.

When it comes to analysis, this result should also be more readily usable. Your online reporting will show a summary table with all the answers together, e.g.

Nintendo Wii – 50%
Sony Playstation 3 – 28%
XBox 360 – 38%
PC computer for playing games – 25%
None of the above – 11%

(these are all made up numbers)

However, if you had asked Yes/No questions the results would be independent of each other, so one result for Nintendo Wii would show:

Yes – 50%
No – 50%

And separately, the result for Sony Playstation 3:

Yes – 28%
No – 72%

Dan Wardle

Subject: Only 30 mins till lunchtime!

I definitely never actually opted-in to this mailing list but it’s a very different approach to the usual “Customer Satisfaction Survey” headline. Although the survey was more market research than for customer feedback, it might work well for employee surveys.

This message does assume that the email will land in inboxes before lunch. If you are firing out a few thousand (or less) – just because the emails have “gone” don’t mean they have all been delivered. Email does not guarantee time-critical communication.

Subject: Can you help us out this Friday?

This was the subject for PostcodeAnywhere’s annual survey. Again it’s different from the usual affair and appealing to our charitable feeling on the day.

Does the message mean that they don’t want to hear your feedback on Monday though? Possibly – this is a common problem for choosing a date and how to communicate the survey’s closing date.

Bonus points to the creator for making the version without images look so clean (left hand screenshot).

One other message leapt out in my inbox.

Thank you for taking part in our survey and prize draw

I’m not a customer of Zen Software yet, but receiving a thank you by email  is rare. I am more likely to buy the software (I probably would have anyway, but this has only helped strengthen my view).

Dan Wardle

The first is Panel Crazy – an observation about the number of errors present in surveys. Sheila Wilson wrote

Clients know that with the technology we now have, it is possible to script a project and complete the fieldwork online in a matter of hours. But perhaps they don’t know about the amount of thought, effort and creativity that is required for decent questionnaire design and in particular thorough quality control.

If you are planning a new customer feedback survey for the new year – give yourself plenty of time to

1. Write the first draft
2. To review and make changes when you have seen the online survey for the first time
3. Test (Q.A.) – make sure all paths through the survey have been tested!

The second blog post was Optimise rather than maximise response rates – which reminds readers that sending reminder emails and then even more reminder emails could have other effects on customers. Most interestingly Jeffrey Henning shared this nugget:

Pew Research Center found that a response rate of 25% did not materially change the quality of survey estimates when compared to the same study with a 50% response rate.

As I’ve said before you don’t need massive numbers of responses to a survey.

Dan Wardle

Is Surveylab missing anything I wondered? So I downloaded the PDF and was pleased to see the answer was no (according to this particular list of insights), but then there was one tip that I strongly disagree with.

Adding a prize draw to your survey does not necessarily mean a higher response rate. An opportunity for the customer to give feedback to improve the service/product/experience, or an employee being able to make their voice heard anonymously will be many respondents’ motivation, and sharing results and findings is enough incentive.

If you do use a prize draw, watch out for “professional” survey takers or users clicking through from competitions websites (especially if your survey is promoted on a public website) because these respondents could skew results wildly. Tip: find out if you can track the referring URL in your survey software or get your webmaster to check the website logs to monitor how respondents arrive at your survey, and also monitor the daily click-thru rate to the survey – a significant and unexplained spike in responses might mean your survey link has been posted on a forum.

Incentives can work, but their impact might not be in the intended way. If you are planning a customer or employee survey in the new year, our default suggestion would be “no incentive”. If the survey length is getting out of hand such that you have decided to offer a prize draw to win an iPod then rethink the prize and more importantly review the questionnaire – it’s probably too long.

Have a great 2010!

Dan Wardle