Jim Harbaugh – A Solution to Sagging Response Rates?

by | Sep 2, 2015 | Fun, Survey Methodology

Bear with me — but I have an idea.  I may have stumbled upon the solution to a troubling trend in our science.  Response rates are dropping.  What can we do?

We used to say that it was critical to get high response rates in our studies.  We spoke of our fight against our nemesis — “Non-Response Error”.

But then it became clear that we were losing that fight, so we suggested that maybe it wasn’t so critical after all.  The term itself — “Non-Response Error” — was shortened to “Nonresponse”.  Then we developed the code “AAPOR RR#2”.

Anyone else out there feeling the deja-vu?

University of Michigan football fans may remember when our nemesis “The Ohio State University” turned into “Ohio State University” with our refusal to include the “The”.  Then, more recently, their name was dropped to simply “Ohio”.  For football fans the comparable plummeting figures (to our response rates) were known as the win-loss % (or maybe we can call it UM#1?), home game attendance (or UM#2), and of course the dreaded killer of major football programs, the “Appalachian State Rate” (or UM#3).

Any good survey researcher knows that trends do reverse themselves.  Numbers that go down do sometimes go back up.  How can we turn our response rates around?

In football, we may have stumbled upon a solution!  Without a single game played, one man has immediately made the UM#3 rate become irrelevant; has caused UM#2 to blast back to 100%; and if you were to survey Michigan fans today, you would come to believe that we just finished the Collegiate Summer Football League undefeated!  (For those who are football challenged, there is no such thing.)

All this was accomplished by speaking one name — Harbaugh.

Survey researchers — we must take note.  Jim Harbaugh can bring our response rates back up!  Let’s use Harbaugh to recruit those respondents (he’s good at recruiting)!

He will raise excitement by hiding the starting question in our surveys until the moment respondents open the questionnaire so that question competition is fierce.

By looking at what he is saying, it is clear that Harbaugh is already thinking about us.  When respondents are sick and cannot complete our surveys, we’ll have him step in:

data-animation-override>
“I don’t take vacations. I don’t get sick. I don’t observe major holidays. I’m a jackhammer.”

— Harbaugh on “Best Day to Launch Survey” panel

Survey respondents – You can be a jackhammer too!

When we need our respondents to know that we have their back, and that we will be there for them when they need:

data-animation-override>
“Yeah, it’s like coming out of the mother’s womb. You’re in a nice, warm, cozy environment — safe. And now you are out into the chaos and bright lights. It’s a happening. It’s all those things rolled into one.”

— Harbaugh on responding to the National Survey of Family Growth

Harbaugh’s enthusiasm for increasing response rates is not necessarily where his contribution needs to end.  He has solutions that can be applied to survey research and questionnaire design, as well.  His event calendar ideas have some merit…

data-animation-override>
“It’s like Thanksgiving. It’s like New Year’s Day. It’s like a family reunion. And having it all rolled into one. Most people think of January 1st as the start of a new year. To people who espouse to Catholicism and Christianity, they might correlate that with the birth of Christ. Us in football, the start of spring practice and the first day of summer training camp are what you look at as the New Year with fireworks going off, it’s your birthday. It’s being born back into football, it’s a happening.”

— Harbaugh on building a responsive design with an event history calendar to help respondents report on their activities

I realize that some detractors may suggest that Harbaugh hasn’t proven anything yet.  That he has yet to even win (or play for that matter) a game.

And to those detractors I say, “You are right!”

But hold it right there — John U. Bacon, the man who wrote Bo Schembechler’s memior, has put it in his new book’s title: Endzone: The Rise, Fall, and Return of Michigan Football.   Mr. Bacon believes in the man sufficiently to declare Michigan football has returned.  We can be so confident as well!

So let me close with this — Harbaugh for American Association for Public Opinion (AAPOR) President!  And then let me make it so by proposing a new book — a sequel to A Meeting Place (which if you do not know is a history of AAPOR).

It will be called A Recruiting Place: Harbaugh, the Error Eraser.

Go Blue!

Currently, the FDA only regulates true direct-to-consumer (DTC) genetic tests, which have no health care provider involved either before or after testing. Consumer-initiated, physician-mediated genetic tests are considered lab developed tests (LDTs), which currently do not require FDA oversight. 

 

Our Study Design

Our study was designed to simulate the experience of an everyday person who is considering doing a health-related genetic test. For this reason, we only reviewed website contents presented to a consumer before ordering a test. By limiting our data collection to pre-test content, instead of digging around or contacting the companies to fill in missing data points, gaps in public-facing information that consumers use to make ‘informed’ decisions were revealed.  

Also, while a genetic counselor supervised the project, a research assistant (RA) conducted most of the website investigations. The RA was familiar enough with genetics and genetic testing to understand and identify the information presented on the websites, but has not had the clinical exposure that might create bias from knowing how specific tests work “behind-the-scenes”. 

 

To Sum Up

We set out to understand the landscape of health-related consumer genomics testing from the public perspective. By limiting our research (by design) to public-facing pre-test website content, we could not complete our data collection as set out in the protocol. However, this uncovered an important observation: consumer genomics websites are highly variable in content, readability and ease of use. 

This begs the question, if we can’t find basic test information on a consumer genomics website, how does a consumer have enough information to make an informed choice about testing? 

Stay tuned for Part 2 in this series, where we will dig into our study findings and reveal our most interesting observations.  

 

 

As experts in FDA user comprehension studies for consumer genomics companies seeking 510(k) clearance, we are interested in how everyday people access and understand health content that is meant for them. If you need help optimizing your consumer-directed health communications, we’ve got the in-house expertise and experience to meet your needs. Let’s chat

About the Author

Scott D. Crawford

Scott D. Crawford is the Founder and Chief Vision Officer at SoundRocket. He is also often found practicing being a husband, father, entrepreneur, forever-learner, survey methodologist, science writer & advocate, and podcast lover. While he doesn’t believe in reincarnation, he’s certain he was a Great Dane (of the canine type) in a previous life.