Trustworthy Online Controlled Experiments (A Practical Guide to AB Testing) by Ron Kohavi & Diane Tang & Ya Xu

Author:Ron Kohavi & Diane Tang & Ya Xu
Language: eng
Format: azw3
Publisher: Cambridge University Press
Published: 2020-04-02T03:00:00+00:00

Surveys

To run a survey, you recruit a population to answer a series of questions (Marsden and Wright 2010). The number of questions can vary, as can the type of questions. You can have multiple-choice answers, or open-ended questions where users give a free-form response. These can be done in-person, over the phone, or online directly on your app or site or via other methods of reaching and targeting users (such as Google Surveys (Google 2018)). You can also run surveys from within products, potentially pairing them with controlled experiments. For example, the Windows operating system prompts users with one or two short questions about the operating system and about other Microsoft products; Google has a method to ask a quick question tied to a user’s in-product experience and satisfaction (Mueller and Sedley 2014).

While surveys may seem simple, they are actually quite challenging to design and analyze (Marsden and Wright 2010, Groves et al. 2009):

Questions must be carefully worded, as they may be misinterpreted or unintentionally prime the respondents to give certain answers, or uncalibrated answers. The order of questions may change how respondents answer. And if you want to get data over time, you need to be careful about changes to the survey, as the changes may invalidate comparisons over time.

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.