When the pandemic began forcing companies to close their physical doors in March 2020, SoundRocket’s Founder and Chief Vision Officer Scott D. Crawford packed up a few belongings and told employees, “Let’s see how this goes. I’m sure we’ll return in a couple of months.” But they didn’t. Three months later, with the pandemic unrelenting and work-from-home working, SoundRocket became a 100% remote company.
“It turns out the office was a security blanket.
…Our customers were already remote to us, and we were accustomed to having a portion of the company traveling. So we were better prepared for this than we thought,” says Crawford.
During this time, SoundRocket was in the middle of an FDA user comprehension study for Ancestry.com, who was submitting a 510K to obtain FDA authorization to market their genetic health risk (GHR) test directly to consumers in the US. The pandemic challenged regulatory precedents and guidelines, as COVID-19 safeguards meant user comprehension studies could no longer be held in-person. Pre-COVID, the FDA strongly encouraged in-person data collection for these studies, and it wasn’t certain what their position would be going forward.
Pioneering live, online user comprehension studies
SoundRocket successfully navigated this journey, ultimately convincing the FDA that their newly developed Live Televideo Interview protocol met all regulatory requirements for approval.
Fast forward to today, and Crawford looks back on the steps SoundRocket took get there:
1. Use technology, but keep the human experience.
For user comprehension studies in support of DTC GHR tests the FDA prefers in-person, one-on-one meetings. In-person data collection can ensure research subjects are engaged with the study process. The experience had to feel like it was in-person and still meet guidelines. But how? And how quickly?
2. Make swift decisions and fail fast (if necessary).
With the regulatory clock ticking for Ancestry.com to maintain their place in line with the application for this new test—pandemic or no pandemic—SoundRocket had to move quickly. Having tried self-administered surveys in the past for these studies, they knew that people don’t pay close attention to study tasks if left entirely on their own. “The FDA requires a 90% comprehension rate for genetic health risk tests,” says Crawford. “With fully self-administered studies, the same test that would score a 90% in person would struggle to reach 70% in a self-administered design. We needed to find another option—and fast.”
Crawford is an avid podcast listener and remembered an episode of The Pitch (a “Shark Tank” like podcast where startup companies pitch their ideas to investors) where a new company called User Interviews was looking for venture funding. User Interviews’ solution was a platform of already-vetted study participants who were engaged and interested in participating in research via remote video connections. With the COVID-19 pandemic in full swing, Crawford remembered User Interviews and quickly connected with their remote live interview panel of research participants. Within days, SoundRocket had an enterprise level account established with them to provide over 400 study participants in the coming months.
3. Hold true to your quality standards. Work differently to get there.
Previously, SoundRocket had set aside the idea of online panels for this high quality regulatory user comprehension work. To survive in the new environment, it would be necessary to find a way to maintain that quality without relying on face-to-face interviewing. This meant engaging with study participants to ensure their focus while also ensuring a diversity that matches the U.S. population. User Interviews and SoundRocket collaborated on a recruitment to meet specific age, race/ethnicity, education, gender, and other related factors—ultimately producing a study population that was reflective of the diversity in the U.S.
With a captive and diverse population, a study protocol was born that included live, online user comprehension studies hosted on SoundRocket’s Zoom account. This provided a study atmosphere where the research team engaged directly with study participants during the interview as if they were seated in the same room. Participants remained as engaged as they had when these interviews were held in-person.
4. Be flexible & look to science to support your pivot.
Having seen all previous user comprehension studies conducted in-person, the FDA was rightly concerned about efforts to conduct these studies online. Decades of research has been conducted on different modes of survey research—comparing self-administered to interviewer-administered, telephone interviews to live televideo interviews, and using a variety of technologies to support data collection. It did not take long to develop a protocol that was supported by the survey methodology literature. Live televideo interviews could be, in fact, conducted with the high level of quality expected by a federal agency. The COVID-19 pandemic served as the catalyst to allow all parties involved to have the openness to consider a new approach—one that ultimately resulted in a final FDA authorization (K192947.)
5. Open your eyes & keep them open!
Serendipitously, the collaboration with User Interviews proved useful to SoundRocket in another way—User Interviews had always been a virtual company. This gave SoundRocket a partner who could also help shine the light on a path to becoming a fully remote company. When asked, User Interviews staff were always glad to share their experiences and lessons learned in this area.