In the recent conference proceedings of the 2016 Computer Human Interaction (CHI) Conference in Human Factors in Computing Systems, Geoff Kaufman and Mary Flanagan present an eye opening piece of research on web survey methodology and the different reaction our brain has to content presented on digital vs. non-digital platforms.
Their work raises excellent questions about how we interact with content that is presented electronically vs. on paper – and raised awareness around the fact that even if we do everything we can to make the stimulus comparable visually, when dealing with paper vs. electronic, the context of the platform may be going beyond simple design.
Specifically, using the Construal-level Theory of Psychological Distance, the authors found evidence to suggest that respondents on a digital device will process information with a lower-level construal than those on paper. This means that those using digital will be more focused on the trees over the forest.
In thinking about this, I can’t help but wonder how this research may inform us in the field of survey methodology.
I WONDER…
If digital devices nudge survey responders to a more concrete & detailed way of looking at things, what does this mean for measures where we are asking respondents to quantify their behavior over some fixed period of time? Will respondents be more susceptible to recent events when estimating a long fixed term period if they respond to a survey via a web-based survey than if they respond via a paper survey?
AND I WONDER…
When we are asking about risky behaviors, does a low-level construal result in less risk taking about the reporting of such behaviors? Do we see lower reports of risky behaviors in web surveys vs. on paper as a result?
AND LASTLY, I WONDER…
How much of what we call “mode effect” in survey methodology can be explained by this difference – especially when an effort have been put forth to create comparable survey instruments between web-based surveys and paper surveys?
Like usual with these “wonders”… I don’t know the answers (yet). But this paper certainly did give me something to think about this week!
What’s a Research Wonder? Read this to find out…