4 ways to transform product development with experimentation

Last week, product leaders and developers gathered to discuss the future of product development alongside Spotify and Mentimeter. Reflecting on this movement, Conversionista’s Andreas Gref (AG) highlights that what we are seeing right now is not necessarily a sudden breaking point, but a rather steady evolution that has been happening for some time:

“It is a fundamental shift in how we think about the relationship between code and user insights, and how we organize our teams to ensure we aren't just shipping – but actually succeeding”.

From guessing to knowing

The goal is to stop guessing and start knowing. By intertwining development, design, and analysis, companies can move away from risky "Big Bang" releases toward controlled experiments. Here are four fundamental shifts AG identifies as defining modern, experiment-driven product organizations.

The approach

1. Teams move independently

Modern product development requires breaking down silos between developers, designers, and analysts. Instead of a central IT department, companies create small, cross-functional teams that own a specific feature, like search. These teams are equipped with all the competencies they need to act and make decisions without friction.

These teams are equipped with all the competencies they need to act – from developers and designers to analysts who understand user behavior. This setup allows them to make decisions and test improvements without the friction of cross-departmental bureaucracy.

Ready to break down the silos and empower your teams? Let’s explore how you can shift from a traditional IT-order model to a modern, autonomous setup. Contact us for an assessment.

2. Moving in the same direction

While teams are autonomous, they must remain aligned. A shared framework for goals and metrics ensures everyone moves in sync. At Spotify, almost all code goes through a controlled rollout or experiment, allowing teams to monitor their own KPIs while ensuring local changes don't cause global losses. If a new feature for Mentimeter, such as a new subscription plan or feature, is launched, the team can see the effect early and decide whether to proceed or pivot.

3. Learn fast by testing everything

Experimentation isn't just for big ideas; it’s for risk mitigation on everything from bug fixes to new features. This constant testing reveals exactly what impacts the user experience and what backfires.

As AG points out, when you test systematically, you often find that many ideas shouldn't be rolled out at all. Spotify has found that a high percentage of features they intended to ship are actually pulled back after testing because they didn't meet the requirements or created new problems. This prevents the product from becoming a "junkyard" of features that add complexity without value.

4. Ship with confidence

To move fast without breaking things, teams use technology like feature flags. This allows anyone in the organization to:

• Turn features on or off instantly if problems arise.
• Test different variants on real users in a controlled environment.
• Base decisions on actual behavior — like music played or presentations started, rather than opinions.

How to start tomorrow

Shifting an entire organization is a long journey, but AG suggests starting small:

• Find an ally: It is hard to drive change alone. Find a colleague with a different skillset who shares your curiosity.
• Understand the friction: If you are in research or product, learn the developers' day-to-day problems so you can propose solutions that actually help their workflow.
• Audit your data: Check if it’s currently easy to see the impact of a release or if it requires heavy manual analysis.

Ready to dive deeper?

Explore how Spotify Confidence can help your team ship with certainty. Reach out and connect with our experts at hej@conversionista.se