Contributor: Yoram (Jerry) Wind, PhD, The Lauder Professor Emeritus of Marketing; Academic Director, The Wharton Fellows Program; Director, SEI Center for Advanced Studies in Management, The Wharton School, University of Pennsylvania
Nano Tools for Leaders® are fast, effective leadership tools that you can learn and start using in less than 15 minutes — with the potential to significantly impact your success as a leader and the engagement and productivity of the people you lead.
The Goal:
Make innovation more cost-efficient and less risky through continuous, adaptive experimentation.
Nano Tool:
Let’s say your team comes up with ideas for two new projects. You choose project A. Revenue the first year surpasses expectations and the project is declared a success. But what if project B would have generated even greater revenue, as well as driving interest to other existing company offerings? In the “choose one” model, you have to base your decision on instinct — with no way of comparing the potential of both projects.
Alternatively, the most innovative and agile companies today are using adaptive experimentation (AE) to conduct continuous real-world experiments that provide rapid insights and improve business decisions. Not confined to innovation projects, it can also be applied to decisions about budget allocation, advertising campaigns, pricing, and more. Experimenting with variations can tell you what effect each action is having, and what to do next. By pursuing several approaches, you can learn which alternatives will add the most value, and which ones are likely to waste resources. Using this “fail fast, fail cheap” experimental approach can help you identify the decisions that could lead to the best outcomes, yielding better results with each step.
Because measuring success or failure is embedded in the process, and because failure yields valuable information, permission to fail is an integral element. AE also creates a compelling competitive advantage: continuous experimentation means your competitors can’t discern your master experimental design.
How Companies Use It:
- The audacious experiments and achievements of Google’s famous X Labs include self-driving cars, balloons that broadcast wi-fi around the world, and contact lenses that can measure glucose levels. Google uses “rapid eval” to test multiple ideas — rejecting most of them — and embraces a culture in which failure isn’t the goal, but is in many respects the means.
- In 2014, Facebook launched the Airlock platform for testing and refining new features on its mobile interface. The mobile A/B testing infrastructure lets them expose users to two versions of every new app, which are the same in all aspects except for a specific test variable. Airlock lets them compare metric data from each version and various tests, and then decide which version to ship or how to iterate further.
- During the summer of 2014 Uber was in competition with Lyft for the New York market. Uber ran pricing experiments in real time to understand how price-sensitive the demand for rides was. The experiment helped them understand when and how to adjust prices to ensure optimal outcomes for customers and drivers.
Action Steps:
Adaptive Experimentation is a continuous loop, rather than a linear process. The learning that takes place in Step 5 lets you refine your approach and decide on your next objectives as you circle back to Step 1 — which means you improve your outcomes over time.
- Determine your objectives: what achievement(s) is worth the time, effort, and resources needed to run an adaptive experiment?
- Create a culture of innovation: involve your organization’s architectural processes and structures, including technology, reward systems, and incentives. To get buy-in from everyone involved and encourage innovation, assure them that performance areas will not be penalized, and compensate as if the old strategy was being used.which to implement them. Strategies should be innovative, involving multiple approaches that are significantly different and that employ radically different methods for resolving a challenge or taking advantage of an opportunity.
- Design the experiment: develop many full execution ideas, including a selection of different markets in which to implement them. Strategies should be innovative, involving multiple approaches that are significantly different and that employ radically different methods for resolving a challenge or taking advantage of an opportunity.
- Implement in a controlled and regulated manner: put an ongoing measurement system in place to evaluate your efforts as well as to monitor markets and control for external effects.
- Analyze your results: use lessons about what worked and what didn’t to develop your next series of experiments.
Whether used to develop new products and services, target marketing programs, or create better customer experiences, AE can help you encourage breakthrough ideas and create an innovation culture.
Additional Resources:
- Beyond Advertising: Creating Value Through all Customer Touchpoints, Yoram (Jerry) Wind and Catharine Findiesen Hayes (Wiley, 2016). Offers a concrete set of principles, including Adaptive Experimentation, based on the insights of 200 of the world’s most forward-thinking executives, innovators, and academics.
- The Power of Impossible Thinking: Transform the Business of Your Life and the Life of Your Business, Yoram Wind, Colin Crook, with Robert E. Gunther (Wharton School Publishing, 2006). Provides practical strategies for developing new ways of seeing, changing to a new mental model, swapping among a portfolio of models, and improving models through constant experimentation.
- Yoram (Jerry) Wind, The Lauder Professor Emeritus and Professor of Marketing, is academic director of, and teaches Adaptive Experimentation in, Wharton’s Global CEO Program: A Transformational Journey and the Wharton Fellows: Master Classes and Networking for Senior Executives. He also teaches in Digital Marketing Strategies for the Digital Economy
About Nano Tools:
Nano Tools for Leaders® was conceived and developed by Deb Giffen, MCC, director of Custom Programs at Wharton Executive Education. Nano Tools for Leaders® is a collaboration between joint sponsors Wharton Executive Education and Wharton’s Center for Leadership and Change Management. This collaboration is led by Professors Michael Useem and John Paul MacDuffie.