The importance of experimentation

Interviewer:  
George Lavric, Partner Manager, Continental Europe at Optimizely 

Interviewees: 
Carsten K. Schou Hjaeresen, VP of Marketing Technology at Valtech
Mark Bisbjerg, Director, Commercial Insights at Valtech 

Let’s kick off with the basics; why is experimenting so important? 

Carsten: 
We know that the Pandemic accelerated everything when it comes to digital…. For some industries, there has been more innovation since 2020 than over the last 100 years combined. That level of disruption can be overwhelming, especially for more traditional companies. Experimentation - and the resulting data - helps companies to understand and validate where they should make the best investments.  

Mark:  
For many companies, particularly those who are less used to working with data, their decisions can be driven by the influence of a few senior people or even one key person. We use the acronym ‘Hippo’ to describe that phenomenon, or ‘The Highest Paid Person’s Opinion’. Often, that person can be wrong, so taking their lead isn’t always the right approach to take. Experimentation takes the subjectivity away and uses a more objective approach to decision making. 

What departments need to take the lead in experimentation? How do you structure the teams?

Carsten: 
In a dream world that direction comes from the C-suite. But, often the more senior you are, the further away you are from the detail. So, the C-suite shouldn’t define what experiments need to be run, but they can define the overarching strategy, the KPIs that need to be influenced, and the metrics they are measured by. Where they need to step away is when you start to explore how to achieve these things. 

Mark: 
Almost all digital departments and markets have a need for experimentation. When it comes to team structure, we try to make experimentation programs global or organization-wide and we have three models for how we usually organize our teams:  

Centralized: Where you have a central experimentation team with specialists that support each of the product teams or markets. 

Decentralised: Where we integrate a specialist in the market or product team. In that way they get closer to the markets or the products and can work with them on a day-to-day basis. 

Centre of excellence: A combination of both, dependent on the client.  

How do you make sure that the C-suite prioritizes experimentation and incorporates that into their digital strategy? How do you make sure that you get executive buy-in?

Mark:
The most important approach is to communicate value early on. We often have a North Star KPI that guides us; that could be leads for B2B clients or sales per visitor for ecommerce clients. Once we know that, we can start measuring the experimentation efforts and communicate that to the C-suite. It's really important to communicate the value that a successful test can have on the business over a long period because then they can see the real Return on Investment. 

How do you make sure to run the right experiments and to get learnings from it? 

Mark: 
Once you’ve aligned on the business goals, it’s time to dive into the data. We call this pain point analysis, it’s a very explorative process, leading to different questions and new tests. 

Most of these experiments show a result that you can’t implement. But these are just as important as the ones that are successful in achieving an uplift. They are validating that what you have is better than your hypothesis. Building your next test based on the results of your previous test is an important step in constantly keeping the wheels going. 

Carsten: 
You have to be brave enough to think big. That means looking at the data and working out if you’re doing the right things… the right kinds of experiments. When I work with clients, it’s often the case that they already run experimentation. For me it’s about questioning whether those are the right experiments to be doing and how we can look at the bigger picture.  

Google is sunsetting Optimize at the end of September 2023. What does that mean for customers and why should they be looking for a platform like Optimizely now? 

Mark: 
The sunset of Google Optimize is going to force clients to really reconsider their current experimentation efforts. I hope that it will trigger them to re-prioritize experimentation and ramp it up within their business. 

We can definitely support them on that journey. Building the processes and providing the acceleration they need to scale. We have great experience, straight forward playbooks and golden tests that we can bring to the table.  

Carsten:  
We can also define the relevant strategies: Where do you want to take your brand and your products? With more pace and new players coming to the market, it becomes more and more important to be agile.  

For us Google optimize is a strong product to get started with, but not necessarily one to scale on. It’s just not good enough for the disruptors of tomorrow, for the digital native brands that are coming. We actually run Optimizely Experimentation ourselves because we see how much value it can bring.

If I’m a marketer and am used to working with Google optimize - how can I educate myself about the other platforms out there that can take me to that next level.  

Mark: 
I have a very fast answer for that: Optimizely experimentation can make all your dreams come true. If you select any other tool you're going to be working with limitations, and you need to know what those limitations are.  

Thank you both!

Want to learn more about experimentation? Read all about how we have partnered with Optimizely to support our enterprise customers in a quick and easy transition to Optimizely Experimentation.