Demographic questions are often necessary to include in survey research. Here we visit the age-old question–where in the survey should they go?
Recently a collaborator designing a survey for an academic research study asked if it is best to put demographic questions at the beginning of a survey or at the end. Scott Crawford, SoundRocket’s survey methodologist, has been asked this many times.
“I lean towards placing demographics near the start,” says Crawford, “with a few considerations.”
Read on to learn more about Crawford’s advice for this familiar survey design question.
In the beginning, they were at the end.
Historically, demographics were collected by face-to-face interviewers observations, without any question asked. With the introduction of remote (telephone) and self-administered survey modes, survey researchers often placed demographics near the end of the survey. This is partly because asking about personal demographic information, like age, employment status, or income level, was considered sensitive. You don’t want to turn a respondent off at the outset. Researchers thought, if you hold the sensitive questions to the end, a trust has been built, and they are more likely to answer the questions and finish the survey.
And for consistency, or simply out of habit, many researchers have continued to place them at the end. And sometimes, research supports this traditional path.
However, times have changed. Asking about demographics in most situations is not considered as sensitive anymore. Survey participants expect to be asked demographic questions and they expect to be labeled and categorized by items such as age, gender, race/ethnicity, education level, job, or any number of other demographic categories. As recently seen in a National Science Foundation survey effort, not asking certain demographic questions in this day and age may be seen as a serious flaw.
“This is in part why I don’t feel they need to be buried at the end anymore”, says Crawford.
Tips and tradeoffs.
In recent years, we have learned how demographics correlate with item nonresponse, survey break-off, and key analytic goals. For example, in a 2022 study of over 3000 Americans, women were more likely to break off during a survey, whereas men were more likely to give poor quality answers while completing the survey. Placing the demographic questions closer to the front of the survey ensures that a researcher has at least demographic data on the respondents, so if someone breaks off, they can better evaluate the factors which may have impacted break off while querying the quality of the survey.
Most demographic questions are fairly simple factual questions to answer. Identifying your age or the state you live in is an easier task than responding to questions about your experiences or opinions. When demographic questions are placed at the beginning they can ease a participant into the survey and give them an immediate sense of accomplishment, setting your survey up for success. This helps avoid a miscalculation of the overall potential burden involved in participating.
Many researchers believe that reminding people of their demographic category may influence how they answer the questions to follow. For some topics this could lead to order bias. For example, asking about gender before asking about proficiency in math or science could lead women to respond more poorly about their own math or science aptitude, unknowingly reminded (and influenced) by a common stereotype. On the other hand, for other topics, placing demographics at the beginning of the survey may lead to more authentic responses. People tend to be more honest about themselves and in their answers if they believe they have shared aspects of their identity.
It doesn’t have to be all or none.
There’s no right or wrong approach here and there’s no rule saying demographic questions have to be presented all together. Crawford recommends starting with the demographic questions up front and then on a case-by-case basis, if there is reason to move an item towards the end, that can be done as the rest of the questionnaire is taking shape. You can move any questions that may be considered sensitive or could trigger a negative emotion to the end. You want the early part of the survey to feel smooth to participants so they want to continue and they should enter the substantive portion with an open frame of mind. It’s important to think about the topic of your survey and the specific questions, especially as they relate to the potential biases of your participant group (i.e., your sample). If the substantive responses to the survey may be biased by reminding the participant of their demographic category, they should be moved to the back.
“Sometimes, the best approach is a little bit of both,” says Crawford. “Put the more sensitive items at the end, and the less sensitive ones at the beginning.”