Survey Science: Our Story, Part 1

by | Jul 14, 2015 | Business Leadership, Events, News

Survey Sciences Group, LLC: From Rubber Bands to Plastic Bags 

Riding from house to house on my scooter, aiming to “porch” every edition of the San Jose Mercury News that I could (especially for the big tippers in the small ranch on the corner), the 16-year-old version of me loved the freedom that came with running my own business.  How did this lead to an interest in survey science, you ask?

While I suspect I would have made different choices in today’s more eco-friendly atmosphere, I learned that if I used plastic bags (previously reserved only for those rare California rainy days), I could fold and prepare the papers about 20% faster than when I used rubber bands.  The added cost of the supplies of bags was small in comparison with the added papers I could deliver in that saved time. I can still remember the confusion I felt when my route manager suggested that I shouldn’t do this because the News had always used rubber bands.

I had no idea at the time, but this situation led me to conduct my very first experiment in survey science – a two-question insert into the next bill I delivered to each of my customers:

Q1.  How do you prefer to receive your News:

  • With a rubber band (that sometimes breaks and doesn’t protect the front page from driveway scratches)

  • In a plastic bag so that the paper is better protected, whether or not it is raining (and just in case my neighbor’s sprinkler sprays it)

Q2.  If you prefer to receive rubber bands, please tell me why you prefer rubber bands?

Before I had a single course or on-the-job experience to teach me the difference between biased and unbiased questions, I saw the questionnaire as a highly manipulatable tool. I could generate the survey data that I knew I wanted.

Upon receiving the results, I was not surprised to learn that my customers agreed with my preference for the plastic bag – and they were more than willing to call the News to specifically request that I use the plastic bag daily.  The News could not ignore their customers’ request!

I anticipated this, but I did have one holdout who strongly preferred the rubber band.  This was why I had asked Q2, as I wanted to understand his motivations. It turned out he had a use for them, and felt the rubber bands that were used by the News were far superior to those that he could get at the local stationary store.  After a quick discussion, we arrived at a solution – when he needed more rubber bands, I would purchase a bag of them from the News and deliver it to his door with his paper (enclosed in a plastic bag). I offered this service for free, but I was always given a $20 tip whenever I delivered his bag. My first foray into survey science!

My love for business, for thinking differently, and for understanding the quality of social science survey research grew from this early experience.  While many things changed, the core values have remained.

Over the course of the next few days, I will be posting a series I am calling “Our Story” to share the story of SoundRocket – where we came from, what we have done over the past 10 years, and where we are going.  I hope you enjoy the ride!

Currently, the FDA only regulates true direct-to-consumer (DTC) genetic tests, which have no health care provider involved either before or after testing. Consumer-initiated, physician-mediated genetic tests are considered lab developed tests (LDTs), which currently do not require FDA oversight. 


Our Study Design

Our study was designed to simulate the experience of an everyday person who is considering doing a health-related genetic test. For this reason, we only reviewed website contents presented to a consumer before ordering a test. By limiting our data collection to pre-test content, instead of digging around or contacting the companies to fill in missing data points, gaps in public-facing information that consumers use to make ‘informed’ decisions were revealed.  

Also, while a genetic counselor supervised the project, a research assistant (RA) conducted most of the website investigations. The RA was familiar enough with genetics and genetic testing to understand and identify the information presented on the websites, but has not had the clinical exposure that might create bias from knowing how specific tests work “behind-the-scenes”. 


To Sum Up

We set out to understand the landscape of health-related consumer genomics testing from the public perspective. By limiting our research (by design) to public-facing pre-test website content, we could not complete our data collection as set out in the protocol. However, this uncovered an important observation: consumer genomics websites are highly variable in content, readability and ease of use. 

This begs the question, if we can’t find basic test information on a consumer genomics website, how does a consumer have enough information to make an informed choice about testing? 

Stay tuned for Part 2 in this series, where we will dig into our study findings and reveal our most interesting observations.  



As experts in FDA user comprehension studies for consumer genomics companies seeking 510(k) clearance, we are interested in how everyday people access and understand health content that is meant for them. If you need help optimizing your consumer-directed health communications, we’ve got the in-house expertise and experience to meet your needs. Let’s chat

About the Author

Scott D. Crawford

Scott D. Crawford is the Founder and Chief Vision Officer at SoundRocket. He is also often found practicing being a husband, father, entrepreneur, forever-learner, survey methodologist, science writer & advocate, and podcast lover. While he doesn’t believe in reincarnation, he’s certain he was a Great Dane (of the canine type) in a previous life.