Interview with Jorden Lentze from Booking.com

Hi Jorden, what's your journey into experimentation and what drives your work today?

My journey into experimentation started back in 2010 when I was working at Moneyou, a subsidiary of ABN Amro. I was responsible for various aspects of the business including the website, and I was constantly faced with decisions about website changes. That's when I discovered A/B testing and began running experiments out of curiosity.

Now at Booking, experimentation is used extensively for safe releases, product validation, and innovation. What drives me today is how often we're surprised by being wrong - it keeps things interesting and reminds us why testing is so important.

What can we expect from your CJAM 2025 session?

I'm really looking forward to the Stockholm event. I'll be sharing four examples of Booking's experimentation journey over the past 20 years. Despite Booking's current scale, we started small and made plenty of mistakes along the way, so these will be practical learnings that companies at various stages of experimentation can relate to.

What's the key message you want to get across?

The main message is about the importance of taking a broad approach to building a successful experimentation program. It's not just about the tooling - it's about the people too. This comprehensive approach ultimately leads to better and faster decision-making.

Why is this so important?

By making quicker decisions, companies can learn faster than their competitors, and that contributes to overall success. I'll also be sharing examples of mistakes we've made and experiments we've run specifically on apps to illustrate these points.

What skills do you think are most important for someone who wants to succeed in experimentation?

Storytelling is absolutely the most valuable skill set for those working in experimentation. It's essential for so many aspects of the job - convincing stakeholders, presenting experiment results in an engaging way, helping internal customers understand data, and creating clear project documentation. You need to be able to translate data into impactful narratives about customer experiences.

If you could give one piece of advice to teams just starting out with CRO, what would it be?

Start with the biggest problem your company is facing. That's where you'll get the most attention and support. At Booking, our initial experiments were actually driven by the need to manage expectations from hotels regarding the impact of our releases.

What are the common pitfalls? What should teams do differently?

The biggest pitfall is expecting immediate conversion optimization results. I'd suggest giving yourself a year to learn and establish proper processes. Start with easy, quick wins to prove the concept, but balance that need for early impact with long-term process development through effective stakeholder management from the very beginning.

Also, while it's important to address fundamental customer problems, allocate some bandwidth for trying out novel ideas - things that customers might not explicitly request but could be valuable.

How do you work to get the whole organization onboard with the experimentation mindset?

From my experience at the Dutch bank, visibility is key. We did monthly updates of experiment learnings in open spaces where everyone could see them. But the crucial part is understanding your different stakeholders - IT, design, branding - and tailoring the narrative of experimentation to each of them.

I have to be honest though - while everyone wants an experimentation culture, company culture is deeply rooted and changes gradually. Rather than trying to force a change, find approaches that fit your existing culture. Even at a data-driven company like Booking, maintaining an experimentation culture requires ongoing effort, similar to maintaining a marriage.

What's your vision for the future of CRO?

I see AI significantly easing the initial stages of experimentation, particularly in research, data analysis, and accessing past experiment data. AI could help us search through vast amounts of historical data, making it more accessible and useful.

I'm not concerned about job security for experimentation specialists though. I foresee continued importance for collaboration between specialists, design, and engineering in smaller, agile teams. Experimenters are actually well-suited to navigate the changes brought by AI, given our familiarity with testing and failure.

Despite all the AI hype, the real complexity in development lies in understanding what to build, and AI isn't highly proficient in that area yet. Human expertise will remain crucial.

I've also noticed the terminology evolving - "product experimentation" is gaining traction alongside or in place of "conversion rate optimization." I see CRO as part of a broader toolkit that includes experimentation and A/B testing, with different terms sometimes representing different scales or focuses of optimization efforts.

Don’t miss Jorden's session at CJAM 2025 on October 23rd!

Seats are limited and going fast, secure your ticket today to be part of the event.