Social Science Research: Are We Truly Innovating?

by | Oct 12, 2015 | Social Science Business

… or are social scientists just building a better rope?

When I think of innovation in social science research, I think of innovation in our “sister” field of land surveying.  (A field I think of too often as I correct confused extended relatives about the type of surveys that I’m involved in.)

In 2600 B.C. Egypt, “rope stretchers” were some of the first land surveyors.  They stretched rope to measure land distances.  But rope had its flaws – it was often weak, stretched, expensive, and in large amounts could be very heavy.  Small innovations took place to bring lighter, stronger, less stretchy, and less costly rope.  Eventually rope became metal and/or chain.

But through those innovations, surveyors were still “rope stretchers”.  Until the game was changed.   A disruptive innovation hit the land surveyor market – the laser.  With one piece of equipment, lasers provided the land surveyor with the ability to measure any distance between two points.

As we have moved from in-person interviews, to phone surveys, to mail surveys, to web surveys, to mobile surveys, we have made great strides and we have enhanced the quality of our science.  However, has social science research not simply been doing to surveys what the land surveyor was doing to rope for all those years?  We’re still just asking people questions.

We can (and sometimes do) capture a massive amount of paradata during surveys – are we using it?

Mobile devices are not just small web browsers that allow us to put a web survey in front of a respondent wherever they are.  Why are we ignoring most of the advanced capabilities that mobile devices bring us?

Consider these data collections:

  1. Should we really be asking respondent how much alcohol they drank today as a way to measure their blood alcohol content (BAC)?  Or should we start measuring BAC directly via bluetooth connected breath devices or skin alcohol measuring ankle bracelets?
  2. Do we ask respondents about their heart health, or should we capture their heart rate, activity levels, blood oxygen levels, and other health data through their devices and connected wrist bands?
  3. We know that respondents’ moods can impact how they interpret and respond to questions.  We also know that we can detect people’s mood  by looking at key indicators in their face.  Can social science researchers capture images of respondents using on-board mobile cameras to learn about mood, and thus assist us in analyzing their responses to the questions we ask?
  4. A significant number of answers to human behavior may reside in our genetic code.  Why are we not using that data in our analyses (or even our skip logic in surveys)?

Surveys are vital, but they reflect an artificial way to measure humans.  They are filled with sources of error.  Normal human behavior stalls and normal thinking stops while one is participating in a survey.  Simply updating the mode in which surveys are delivered will not change this much.

Maybe it is time for social science research to rethink the rope and look for the laser of our field.

Currently, the FDA only regulates true direct-to-consumer (DTC) genetic tests, which have no health care provider involved either before or after testing. Consumer-initiated, physician-mediated genetic tests are considered lab developed tests (LDTs), which currently do not require FDA oversight. 

 

Our Study Design

Our study was designed to simulate the experience of an everyday person who is considering doing a health-related genetic test. For this reason, we only reviewed website contents presented to a consumer before ordering a test. By limiting our data collection to pre-test content, instead of digging around or contacting the companies to fill in missing data points, gaps in public-facing information that consumers use to make ‘informed’ decisions were revealed.  

Also, while a genetic counselor supervised the project, a research assistant (RA) conducted most of the website investigations. The RA was familiar enough with genetics and genetic testing to understand and identify the information presented on the websites, but has not had the clinical exposure that might create bias from knowing how specific tests work “behind-the-scenes”. 

 

To Sum Up

We set out to understand the landscape of health-related consumer genomics testing from the public perspective. By limiting our research (by design) to public-facing pre-test website content, we could not complete our data collection as set out in the protocol. However, this uncovered an important observation: consumer genomics websites are highly variable in content, readability and ease of use. 

This begs the question, if we can’t find basic test information on a consumer genomics website, how does a consumer have enough information to make an informed choice about testing? 

Stay tuned for Part 2 in this series, where we will dig into our study findings and reveal our most interesting observations.  

 

 

As experts in FDA user comprehension studies for consumer genomics companies seeking 510(k) clearance, we are interested in how everyday people access and understand health content that is meant for them. If you need help optimizing your consumer-directed health communications, we’ve got the in-house expertise and experience to meet your needs. Let’s chat

About the Author

Scott D. Crawford

Scott D. Crawford is the Founder and Chief Vision Officer at SoundRocket. He is also often found practicing being a husband, father, entrepreneur, forever-learner, survey methodologist, science writer & advocate, and podcast lover. While he doesn’t believe in reincarnation, he’s certain he was a Great Dane (of the canine type) in a previous life.