A/B Testing: The Most Powerful Way to Turn Clicks Into Customers by Dan Siroker & Pete Koomen

Author:Dan Siroker & Pete Koomen [Siroker, Dan]
Language: eng
Format: mobi
Publisher: Wiley
Published: 2013-08-06T14:00:00+00:00

Evangelize and Galvanize: Lizzie Allen at IGN

It's one thing to get people excited about A/B testing; it's another thing entirely to encourage people to make it an integral part of their daily practice. This is what it takes, though: making the testing culture at your organization contagious.

This might seem like a daunting task. But if an entry-level data analyst can single-handedly turn a 15-year-old, 300-person company into a shining example of A/B testing prowess, you can do it, too.

When Lizzie Allen joined the gaming news site IGN as a data analyst in 2010, the company had never heard of A/B testing. Allen was astounded that such a prominent content website did not use this approach to test its assumptions, especially when making editorial decisions. So she took on the challenge of introducing the company to A/B testing, and helping to establish a culture where decisions would be rooted in data. “When you tweak any sort of process in a large organization, you have to throw your weight around. When you're entry-level, you don't have a lot of weight to throw around. I had to use some untraditional and somewhat ballsy methods in order to promote A/B testing,” Allen says.

In month one, she introduced the company to testing through training sessions. She worked with folks on the editorial team to educate employees throughout IGN about the practice. These sessions specifically centered on the value of testing two different headlines to see which one garnered more clicks. When the early buzz and excitement started to dwindle, Allen perceived that people saw A/B testing as an extra step in an already established, already functioning process. “There wasn't a foundation of data-minded people to support it, and I needed to cultivate that,” she says.

Her strategy, then, was to shock people—a lot of people—to prove why A/B testing was so vital to business. Allen gamified A/B testing by turning the site's tests into a competition. (IGN is a gaming site, after all.) The “A/B Master Cup” was born. Inspired by the website whichtestwon.com, Allen would once a week send out a test that IGN had run that week and ask people to choose which variation they thought had won. She used the company's internal chat tool (Yammer) to send out screenshots of the different variations. At the end of each month, she would crown the person who picked the most test winners correctly as the “A/B Master.”

She found that, overwhelmingly, everybody failed. In fact, some months went without a winner at all, because people guessed wrong so often. The contest started a lot of conversations, and began to instill a sense of humility, and also of intrigue. One week at a time, Allen built a groundswell of data-driven thinkers who were curious and eager to “figure out the puzzle that is the Internet user base for a video game content publishing site.”

When asked for advice for building a testing culture at a company, Allen puts it simply:

Be obnoxious. Question assumptions. Be that jerk in the

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.