A popular type of question to ask in a survey is the ranking question, e.g.
Please rank each of the following items in order of importance
From the respondent’s perspective, this question involves more effort to answer than others because of the need to compare each item in the list against every other item in the list, and also requires more time to fill-in as the list gets longer.
In many cases though you don’t need to ask for every item in the list to be ranked. Asking for the top 3 (or even 2) can be enough to arrive at the same result.
The data table below is from real survey data on a question that ranked 8 items (although only showing the top 3 positions here):
| Item | % Ranked 1st |
% Ranked 2nd |
% Ranked 3rd |
|---|---|---|---|
| Red | 26 | 19 | 14 |
| Blue | 28 | 20 | 15 |
| Pink | 3 | 13 | 17 |
| White | 20 | 17 | 13 |
| Black | 8 | 10 | 10 |
| Brown | 3 | 5 | 9 |
| Yellow | 7 | 10 | 12 |
| Green | 6 | 7 | 9 |
With this survey data the ranking is identical whether we got the respondent to rank all 8 items or just their top 3 (and if we only used the top 2 the results are the same except for positions 4 and 6 – btw the results above are blue first, red, white, then a big drop to pink, yellow, black, green and brown last).
The point though is that your ranking question data isn’t going to be the only factor in any business decision (other survey data should contribute for a start). Are you looking to identify the most important 5 or 6 factors (from a list of 26), or categorise the items into “high importance”, “medium” and “low”? When designing the survey – bear in mind how the data will be analysed – if you don’t need an ordered list of 26 priorities then don’t ask your customers to work out theirs!
Dan Wardle
Dan
John Kemp