|To what extent do you agree or disagree with the following?|
“Boxers are more comfortable than briefs”
|Strongly Agree||…||[insert favorite response scale length/format]||…||Strongly Disagree|
Time after time I see debates about whether response scales should have an even or odd number of response options (meaning, should there be a midpoint). And alongside this, we see the parallel debate about what that midpoint should be “neutral”, “N/A”, or “don’t know”.
N/A and Don’t Know are not the same as Neutral
Now, I would sincerely hope the survey designers would give me, as a woman, a N/A option because, heck, how would I know? And even for men, well, some of my kilt-wearing male ancestors from Scotland could have also been in the breezy “N/A” category too.
My wanting an N/A response option is completely different from, say, a man choosing the neutral point on the scale (meaning, boxers and briefs are about as comfortable as each other). The difference is, a man who has tried both boxers and briefs is making an informed judgement based on experience, whereas even a neutral point is completely meaningless for me. I just don’t know!
Not applicable or don’t know is NOT the same as a neutral position and should not go in the middle of the scale!
And, in this case (for boxers vs. briefs), a neutral opinion is reasonable, so should be an option.
Is there ever a case on an agree/disagree scale where a neutral position would make no sense and should not be included? Some argue that this is the case, so please chime in on the post page under Comments and tell us when you think this is.
Not all scales are based on two opposing endpoints
The other point worth noting is that not every survey item is based on a positive/negative, opposing views, or two-extremes concept.
Some scales used in evaluation reflect a more developmental frame (like, “where would you rate your skill level on X”) where a neutral point would make no sense.
Are X-point scales “simply the best”?
I am always amazed how some people seem to have chosen 4, 5, 6, 7, or however many points as superior for all purposes and taking that as some ‘religious’ position. Sure, there’s been some empirical research done on these, and that’s worth paying attention to, but it’s in the context of particular types of scales. And not much of it has been done specifically in the context of evaluation work.
There are many different forms of survey items for which different response formats make sense – longer scales, shorter scales, midpoint, no midpoint, N/A option vs. none, and so on.
Scale design is about conscious decisions with the respondent’s perspective in mind
For every quantitative survey item, we need to **consciously** consider (a) whether there is any such thing as a neutral position AND (b) whether it’s possible that the entire question is just not applicable to some respondents (or, that they just wouldn’t know enough to be able to rate it – like asking my octogenarian parents which is better, an Android smartphone or an iPhone).
Let’s not drive respondents nuts by forcing them to answer in ways that fundamentally make no sense for what they have to say.
Or force them to make something up when they actually have nothing to say on the topic.
- Breaking out of the Likert scale trap
- Building causation into survey items about outcomes
- 2 more ideas for evaluation survey items – asking about comparative value
Comments? Please visit the post page to add your views!