The long survey that worked

      6 Comments on The long survey that worked

phone-958066_1920
I am a big fan of public transportation. It gives me the chance to people-watch, nap, read, listen to podcasts, and generally avoid being stuck in traffic. Recently, I got a bit of a market research lesson on my bus ride to work.

He spent how long on that survey?

I sat on the packed bus, looking around to note the number of people on their phones. I happened to glance at the phone screen of the person next to me and noticed he was taking the “I side with” 2016 political quiz. I thought, “Ah, yes, I remember taking that last election and seeing my friends’ answers on Facebook. I didn’t realize they had updated it. I wonder how he’s answering.”

Then, as we continued our journey, I realized that the quiz was actually fairly long – or at least was taking him a long time.

Now, as a market researcher, we hear time and time again how surveys need to be shorter, packing a punch, only asking the most pertinent questions. When it comes to mobile, this is an even more ubiquitous refrain.

But here’s the thing: we were on that bus for about 30 minutes. 25 of those minutes were spent by this individual taking one survey.

What I noticed

I couldn’t help but notice a few things about this experience. (Please note: I did NOT read his answers as he was taking the quiz. I was far enough away to see some general things.)

  • The survey was mobile-friendly. It was easy to read, with only single-choice options being asked, and when ratings were presented, it wasn’t on sliding scales, but simple scales with buttons clearly marked for each option on the scale.
  • He was captivated by the survey. I noticed him scrolling up and down, reviewing answers given previously, seeming to read through every possible option before answering. He was definitely an engaged respondent.
  • The survey has a single purpose: to tell you which of the many political parties in the United States you might agree with most.
  • To accomplish this purpose, the survey is split into very clearly defined sections. Each question within a section seemed to offer three options, one of which would expand to show more options. When I later went to the quiz to see it myself, I was intrigued by the fact the answers were such that you were presented with either of the polar opposite opinions, then given “Other stances” as the option to see more nuanced variations.
  • He never seemed to get tired of the survey.
  • The results looked fairly extensive, using charts to plot where one’s stances on groups of social issues seemed to be.

My takeaways

  • If your survey is about something that the respondent is passionate about, they are likely to be more willing to spend more time on the topic.
  • This survey was truly device-agnostic. It rendered differently on mobile than it did on my laptop, and both were well-laid out.
  • It’s possible to keep everything on a single page, and still keep someone interested in the survey.
  • The results were easy to review.

My own personal opinion is that this quiz/survey is popular in part because it tells us something about ourselves, and can even help us learn about political parties we may not have been aware of previously. But it did make me consider why people would never think of spending that sort of time on most market research studies. The answer is fairly easy: the topics typically aren’t nearly as compelling. Frankly, they don’t always have to be, either. But occasionally, when done right, and when the topic IS compelling, you can get people to spend more than 5 minutes taking a survey on any device.

6 thoughts on “The long survey that worked

  1. Annie Pettit, CRO Peanut Labs

    Love it. Researchers need to do this more often. Bringing people into research labs would help but the live real experience is the best there is. Observational research like this has been part of psychology/sociology/anthropology since the beginning and market researchers need to do it more often.

    Reply
    1. Zontziry (Z) Johnson Post author

      We don’t do it nearly enough. We kind of throw stuff out there that’s been done before and hope it still works as well as it did the last time it was tried, which is somewhat ironic, come to think of it…

      Reply
  2. Dan Kvistbo

    Enjoyable read Zontziry. While I agree to your points, I think that we shouldn’t underestimate the difference between giving and taking, between the purpose of “telling you” and asking you to “tell me” (and offer nothing substantial in return for your time). It’s the difference between push and pull – and it’s enormous. That, I think, is the most important lesson – and one that the market research industry has yet to truly understand, embrace and act upon.

    Reply
    1. Zontziry (Z) Johnson Post author

      Great point, Dan! We are definitely still trying to figure out how to break away from the traditional method of asking and hoping the respondent will just feel like answering and providing something in return – more than just points to redeem for something later, as done on so many panels, too. What do you think we could do to help that shift happen in market research?

      Reply
      1. Dan Kvistbo

        I think there are a number of alternatives that are better than (or could be great supplements to) the traditional point systems, e.g. better communicating the purpose of individual studies, sharing tidbits of insights with the people who participate, for instance by letting people know how their responses (even if only to just a few of the given survey questions) compare to other people – perhaps restricted to their individual segment. This would 1. make it more personalised and relevant and 2. circumvent client concerns in terms of risking overall results being shared publicly. There are many things we could do better and the tech is there. The challenge is that it takes an effort and adds a bit of complexity to the individual study. In other words, time and costs, that clients aren’t always thrilled to pick up…

        Response rates have been dropping for ages and there are of course, many other contributing factors than the value proposition we offer respondents. Yet overall, it seems to me that the industry at large, has its eyes too focused on the next quarterly financial report – than the long term prospects of how we can motivate people to continue to take part in research and engage them properly when they do.

        A huge and important topic! And if the industry doesn’t up its game, particularly the panel vendors – who are currently squeezing the last few completes out of worn-out panels, someone or something else will take their place.

        Anyway, thanks for responding to my comment above Zontziry. I was delighted to come across your blog and look forward to future thoughts and perspectives on what’s down the road! 🙂

        Reply

Leave a Reply to Dan Kvistbo Cancel reply

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.