Not sure this is the right place, but here goes.
After listening to the most recent episode (#407), I jumped on to do the listener survey and at a glance I could tell there were going to be problems.
So, question by question, what did the SGU listener Info survey get wrong.
1) What is your age?
Okay, this one is fine. Off to a good start. It is however a free text field and actually let me put 'spaghetti' as my age.
2) Sex:
Using the word 'sex' suggests the question is limiting itself to the biological sex of the respondent.
Only two options? What about Intersex? Or 'other'? Or those that do not like to be identified by their gonads?
In these enlighten days the moment this question is put out there the lack of other related questions is curious. No questions about Gender Identity? Sexual Orientation? Gender Expressions?
3) Education level:
The list of options here is very US centric. Down here in Australia we don't have 'middle' school. But, okay, we can figure it out I guess. But... wait a second, there are a bunch of tertiary education levels missing, where are all the pre-bachelor certifications? And why isn't this list in order. And, oh my, college, how cute. I once meet a guy that actually had 'GOLF101' on his transcript.
4) Employment status:
This one is mostly okay, just two oddities. The phrase 'employed for wages' suggests there is going to be another options for 'employed' on the list, maybe something like 'employed for board' or 'employed for food'. But when there is no other variate then the specification of wages seems odd. The other problem is, can we use something other than 'out of work', maybe something that doesn't scream "your a failure if you don't have a job", even just the dull 'unemployed' is less judgemental.
5) Household income:
So... you want this in my local currency right? You did put a dollar sign, so that limits it to about 20 currencies. Leaving US centricity aside, this also assumes a nuclear family in one house. If I rent do I count my flatmates? Should I include both my boyfriends income, even if they don't know about each other? What if me and the wife are separated, do I count her income? What if I just don't want to tell you?
6) Marital Status:
'Never married' makes it sound like a social failure, what's wrong with the word 'single'? I'm onto my third wife, so I'm widowed, divorced and married, but you don't have that as an option. If I'm in a civil union in a country/state that does not recognise gay marriage, which option should I pick?
7) Number of children in house?
What is it with houses that you find so fascinating? I only have the kids on the fourth weekend of the month, so they ain't here right now. Does anyone even do nuclear family any more?
Do you own or rent your home?
This is the first question to actually have a binary choice in the wording of the question itself, so obviously this is the first question to give 'other' as a valid answer.
9) Of the 400+ shows we have produced what percentage have you listened to?
Two words: percentile groups. If the options need to be broken down into 4 groups then the options should read '26%-50%', not just an outright '50%'. By giving the option of 'other' here and having 4 precise percentages listed, someone who has listened to 95% should put that in 'other' right?
Survey design is fiendishly complicated and best practise are always shifting, and I understand that your probably only gathering this data for some sort of marketing reasons.
I know I've been a bit silly on some of these, the age for example. But others such as sex and marriage are hot button issues these days, and should not be taken lightly.
Why is this even important? Because an off-putting question on a survey will usually result in the survey not being submitted, and that just skews the data.
All the best,
David.
PS: I hope whoever wrote the survey doesn't take all this too personally.
