Types of questions and response format
Survey questions can be loosely divided into two categories: structured and unstructured. Structured formats help the respondent respond quickly and easily while giving the researcher the ability to summarize responses efficiently, but they also constrain the respondent and limit the researcher's ability understand what the respondent really means. Additionally, you have to consider whether or not all alternatives are covered, the list length is reasonable and whether or not more than one item should be selected. Unstructured questions can be more difficult to write and time consuming to answer/interpret, but can provide the researcher with more information and understanding.
Examples of structured questions:
- Are you over the age of 18? Yes or No
- Rank the following in order of preference from best to worst... (1 being the worst and 5 being the best)
- Have you ever smoked marijuana?, If yes, about how many times have you smoked marijuana?
Example of unstructured questions:
- Please add any other comments.
- What do you see as the disadvantages of eliminating welfare?
Determining the question content, scope and purpose
For each question in your survey, ask the following questions:
- Is the question necessary/useful?
- Ex: Do you need the exact level of income or is an estimate ok?
- Are several questions needed?
- Ex: Consider breaking the following question down into two different questions: What do you think of the new benefits and hours?
- Do respondents have the needed information?
- Ex: What do you think of the TV show Game of Thrones? If your respondent has never seen Game of Thrones, they cannot be expected to answer the question. Consider asking if they have seen the TV show first.
- Does the question need to be more specific?
- Is the question sufficiently general?
- Is the question biased or loaded?
- Ex: "What do you see as the benefits of a tax cut" will give you biased survey results.
- Will the respondent answer truthfully?
- Ex: Some people don't like to disclose how much money they give to charity.
- Can the question be misunderstood?
- Ex: What do you mean when you ask a question about social media?
- What assumptions does the question make?
- Consider providing definitions when asking someone to put themselves into different categories.
- Is the time frame specified?
- Avoid using will, could, might, or may if you're not specifying a specific time frame.
- How personal is the wording?
- Consider avoiding questions that ask about how someone feels or whether or not they are personally satisfied.
- Is the wording too direct?
- Consider whether or not your question will elicit negative emotions based on personal experiences or if the question will lead the discussion towards discussing difficult issues directly.
Ethical Considerations
- Can you reach a sample of your intended population with the survey method you've selected?
- Ex: If you're conducting a phone survey, you're depending on everyone having a phone and a listed number.
- Is the population literate?
- Ex: Does your questionnaire contain difficult or technical vocabulary?
- Are there language issues?
- Ex: Can you produce multiple versions of your questionnaire?
- Will the population cooperate?
- Ex: Individuals conducting illegal activities may not want to divulge information.
- Are there geographic restrictions?
- Ex: Is your population of interest dispersed over too broad a geographic range for you to interview?
- Can all members of the population be sampled?
- Ex: Do you have a complete list of the population?
- Can you define the ideal respondent? Will screening questions be needed?
- Ex: Do you want to only interview the head of household?
- What types of questions will be asked and how complex will they be?
- Ex: Are you going to ask personal questions? Should your questions be broken into smaller parts?
- Can social desirability be avoided?
- Ex: Have you written your questions in a way that makes your respondents feel like they need to give you a specific answer?
- Can interviewer bias be avoided?
- Ex: Have you trained your interviewers on how to say the questions and how not to ask leading questions?