Experimentation Works by Stefan H. Thomke

Author:Stefan H. Thomke [Thomke, Stefan H.]
Language: eng
Format: epub
Publisher: Harvard Business Review Press
Published: 2019-02-19T16:00:00+00:00

True experimentation organizations embrace a new leadership model (see the sidebar “Leadership and Large-Scale Experimentation”). Instead of viewing leaders primarily as decision makers, the model encompasses three important responsibilities. First, a senior executive’s job is to set a grand challenge that can be broken into testable hypotheses and key performance metrics (e.g., “Best customer experience in the industry”). Second, they need to put in place systems, resources, organizational designs, and standards (e.g., tools, program management, skills training) that allow for large-scale, trustworthy experimentation. And third, executives need to be a role model for all employees. That means living by the same rules as everyone else: having their own ideas subjected to tests and demanding that experiments, not just feature or product releases, are integrated into business roadmaps. Leaders also have to savor surprises and pay close attention to the seven cultural attributes described in this chapter. The watershed moment arrives when results from experiments become part of the regular management meetings and workflows, just like running the numbers.

Leadership and Large-Scale Experimentation

An Interview with Mark Okerstrom, CEO, Expedia Group

Stefan Thomke (ST): How important is the scientific method at the Expedia Group?

Mark Okerstrom (MO): The scientific method is absolutely critical to how the Expedia Group operates and competes. At any one time, we’re running hundreds, if not thousands, of concurrent experiments, involving millions of visitors. When experiments work, we roll them out globally. Between 2016 and 2018, the Expedia brand alone ran thousands of product tests annually; and this is just in product/user experience, and only on one brand. Large-scale experimentation happens across all of our other brands. Beyond innovation in user experience, experiments are running in areas including human resources, sales, and traditional television advertising. That’s approximately seven thousand product leaders, engineers, and data scientists actively involved in experimentation. And in some form or another, the majority of our over twenty-five thousand full-time employees use the scientific method as well. We’ve also involved many of our external partners in testing hypotheses with us around better ways to work together.

ST: What is the role of the CEO in building a culture for large-scale experimentation?

MO: As CEO, it’s not my job to say what product decisions are right or wrong. Essential to my role is to be an organizational system architect of sorts. I need to ensure all the essential elements are in place to create the environment where innovation and experimentation can thrive. I need to ensure that the right culture, incentives, resources, business processes, and organizational design are in place. A common platform, along with standardized tools and metrics across brands and divisions, is also very helpful. People then need to be trained in the scientific method, to know how to formulate a good hypothesis, to understand what a minimum viable product is, and to find efficient and low-cost ways to build and test them. They need to accept that failure (and oftentimes, a lot of it) is a requirement for success—and that’s hard for a lot of people.

Getting to this

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.