What are your 3 best tips to get management interested in CRO?
Firstly, showcase the monetary benefits. Highlight how even minor changes have the potential to yield significant profits. Secondly, put emphasis on risk management. Discuss how CRO/experimentation can help avoid expensive mistakes. Lastly, underscore the learning value. Show them how testing provides insights into customer behavior that can't be obtained elsewhere.
What is most important for a CEO to understand regarding experimentation that, in your experience, is under-communicated?
CEOs must grasp that experimentation is a means of guiding strategic decisions, not a tactic for quick wins. The value of experimentation comes from long-term learnings and development, not short-term upticks in KPIs.
What is the most neglected/underestimated thing within CRO?
The value of 'losing' tests. Negative results are viewed as failures when, in fact, they provide immensely valuable insights. Failure is a crucial part of the learning process.
What is your favorite way of celebrating winners/learnings from experiments with the organization/team?
By sharing the stories. Every experiment, successful or otherwise, is a narrative about your customers, your assumptions, and your growth as a company. Celebrate the process, not just the results.
What is the most important trait to be a conversion optimizer?
Curiosity and a critical scientific attitude. To excel in CRO, one must be deeply curious about human behavior, technology, business, and the many ways these areas intersect.
How do you communicate the value of running an AB test program?
By illustrating its capacity to generate incremental revenue, mitigate risk, and provide learnings that drive strategic decisions.
How do you communicate the value of having "losing" tests?
I emphasize that they prevent us from implementing ineffective or even harmful changes, saving us money and time in the long run, plus the positive benefit that can be gained from learning about how customers behave.
How do you measure the success of the AB test program?
Success isn't just about winning tests; it's about generating insights that lead to business improvements. So it's not just about conclusive tests but about the richness of the learnings derived.
Could you give us an insight into your equivalent of Insight-Action-Result?
Observe - Test - Learn. Observe customer behavior and the performance of the site, Test hypotheses derived from observations, Learn from the results and apply these learnings moving forward.
How much qualitative research do you do when creating hypotheses for a/b-testing?
It's an essential part of the process. It provides depth and context to quantitative data, helping us understand the 'why' behind user behavior.
What are the most common misconceptions about CRO you have encountered?
That CRO is just about increasing conversion rates, and that CRO is a quick-fix solution for underperforming websites. Also, that CRO generates revenue in and of itself. It doesn’t; it just helps you make better decisions about things which generate revenue.
What do you recommend to companies with low traffic?
Focus on gathering qualitative insights and experiment with big changes that can provide noticeable results, even with low traffic.
What do you think is the future of CRO?
The rise of AI and machine learning will undoubtedly influence CRO, but the human element - understanding customer needs, emotions, and motivations - will remain central to the process.
