Tuesday, April 03, 2012

New Google Consumer Surveys: revolutionizing academic data collection?

Surveys are a key data collection tool in several academic research areas. As opposed to experiments or field studies that yield observational data, surveys can give access to attitudes, reaching "inside the head" of people rather than observing their behavior.

Technological advances in survey tool development now offer "poor academics" sufficiently powerful online survey tools, such as surveymonkey.com and Google forms. Yet, obtaining access to a large pool of potential respondents from a particular population remains a challenge. Another challenge is getting fast responses -- how do you reach people quickly and get many of them to respond quickly?

We may now have a solution that is affordable for academic research: A few days ago Google announced a new service called "Google Consumer Surveys". Similar to Ad Sense, where Google places ads on websites of publishers (and pays the publishers a commission), with Consumer Surveys, Google places a single-question survey (=poll) on websites of publishers. The publishers require website users to complete the poll to get access to premium content.

Google Consumer Surplus: How it works (from their website)

The good:

  • Very affordable: the charge for each response is $0.10 (=only $100 for the magic number of 1,000 responses). Or, for an audience targeted by demographics or some trait, it is $.50 per response (more here).
  • Fast: Google will likely post the polls on pages with high traffic.
  • Google presents the results with attractive charts
  • Getting IRB permission may be easier, given the stringent policies that Google mandates
The bad:
  • You can only post one question at a time. For a longer survey, breaking it up into single questions means that not the same person is answering all the questions. Also, each additional question increases the cost exponentially.
  • Google does not supply the poll creator with the raw data. You only get aggregated data. You can choose the aggregation (inferred age, gender, urban density, geography, or income). This is likely to be a huge "bad" for researchers who need access to the raw data for more advanced analyses than those provided by Google. 
  • Currently Google only offers this service for websites in the US. To collect information from users visiting non-US website we will all have to continue holding our breath.
A curious anecdote: I filled in the support contact form to ask a few extra questions. I received speedy and helpful answers (within 24 hours), but they all landed in my Google Spam folder!

2 comments:

Brett said...

Hey Galit,

You can export the data to CSV:

http://support.google.com/consumersurveys/bin/answer.py?hl=en&answer=2449700

There's a row in the CSV for every answer you receive. This makes it easy to build a pivot table or use your favorite stats tool.

Let us know what else you need!

-Brett

Galit Shmueli said...

Hi Brett,
Initially I saw this option and was also hopeful ("You can choose to export these results in CSV or image format"), so I contacted support and got the definitive answer that this is not the raw data. You can export the aggregated results that you see in the chart to CSV. Here's the response from the Google Consumer Survey support team:

We don't offer the export of individual responses from the surveys results. Google Consumer Surveys aggregates and analyzes responses from users and presents results in an simple online interface. You can choose to export these results in CSV.