Interview with Natasha Senior on “Experimenting Beyond Conversion”

Natasha, what is your talk about?

I’ll be sharing our journey at Sky, where we’ve shifted our experimentation focus from traditional conversion metrics to operational ones. Like reducing contact center calls, average handling time, and repeat calls.

Instead of just trying to increase revenue, we’re using experimentation to drive cost savings and make bigger business decisions.

What’s the key message you want to get across?

The key message is that experimentation shouldn’t be owned by one person or just focus on conversion. It should be a company-wide initiative, and the experimentation backlog should be the product roadmap.

What’s important for your business might not always be conversion. Sometimes, it’s about reducing costs or improving operational efficiency.

Why is this important?

Too often, CRO is seen as just “conversion rate optimization,” but A/B testing can help you make the right decisions for any business metric whether that’s increasing app usage, reducing churn, or getting more people into stores.

It’s about using data to make the best decisions for your business, not just chasing conversions.

What are some common pitfalls?

A big one is misunderstanding the underlying math—like not accounting for visits when measuring percentage drops.

Another is the challenge of educating developers about the importance of A/B testing. It’s crucial to continuously educate and build trust across teams, especially as developers and team members change.

What should people do after your talk?

Step back and ask: is conversion really the right metric for your business? Look at the bigger picture—maybe operational metrics or loyalty are more important.

Experimentation can be more successful when it’s tied to what matters most at the top of your organization.

What’s your vision for the future of CRO?

I hope we move away from the term “CRO” and embrace “experimentation” as a core, product-focused way of working.

Experimentation should be a necessity for making business decisions, not just a “nice-to-have.” It’s something the whole company should be involved in—not just one person or team.

Don’t miss Natasha’s session at CJAM 2025 on October 23rd!

Seats are limited and going fast, secure your ticket today to be part of the event.