Published by Sue Johns-Chapman
Published Date 08.09.2026 
Being shortlisted for CRO Agency of the Year is a great moment for our team at Online Dialogue. Not simply because it recognises the experiments we run, but because it reflects what we believe CRO should ultimately achieve: helping organisations make better decisions.
Most organisations do not lack ideas, data or activity. What they often lack is clarity. What deserves attention? Which assumptions are actually true? What do customers really need? And which choices are likely to have meaningful impact?
Experimentation helps turn assumptions into evidence. But to make that valuable at scale, organisations also need to become better at understanding, prioritising and learning. That is where we believe CRO becomes more than optimisation.
A multidisciplinary way of solving the right problem
Online Dialogue is an experimentation and CRO agency based in the Netherlands. For 17 years, we have been helping organisations improve how they understand customers, validate assumptions and make better digital decisions. Our team consists of experimentation consultants, behavioural psychologists, UX designers and data analysts who work together on client programmes.
That combination matters, but the disciplines themselves are not the point. Their value comes from how they work together.
Data helps us understand what is happening. Behavioural psychology helps explain why it may be happening. UX turns those insights into possible solutions. Experimentation allows us to validate whether the assumptions behind those solutions actually hold.
A useful experiment therefore rarely starts with a button colour. It starts with a relevant business or customer problem, a clear assumption and a decision that is worth improving.
From running tests to building a learning capability
We often work embedded within client teams for longer periods of time. The ambition is not to become indispensable. Quite the opposite: we want organisations to build the knowledge, processes and confidence to make strong evidence-based decisions themselves.
For us, that means helping teams not only run experiments, but also improve how they frame problems, prioritise opportunities, interpret evidence and retain learnings.
A successful experiment is not only one that produces a conversion uplift. An experiment can also be valuable because it disproves an assumption, prevents investment in the wrong solution or gives a team a better understanding of customer behaviour.
That learning becomes even more valuable when it changes the next decision.
From a CRO pilot to 145 experiments at NS
A good example is our work with Dutch Railways (NS).
What started as a relatively small CRO pilot developed into an organisation-wide experimentation programme. Over 14 months, we trained and coached 13 NS specialists across several clusters. The number of experiments grew from 21 in 2022 to 145 in 2024.
The number itself is not the most important result. More important is what sits underneath it: experimentation became part of daily workflows, knowledge spread across teams and ownership increasingly moved to NS itself. The programme is now being transferred to its internal Centre of Excellence.
That is what scaling experimentation looks like to us: not simply doing more tests, but creating the conditions in which better validation and learning can happen across an organisation.
Building experimentation into decision-making
We have taken a similar approach with DPG Media and Vattenfall.
Together with DPG Media, we helped develop a mature experimentation programme within the subscription domain. Through coaching, training, playbooks and tooling, validation became more structurally embedded in decision-making. Instead of experimentation being something that happened at the end of a process, it increasingly became part of how ideas were assessed before launch.
At Vattenfall, the challenge was to develop Customer eXperience Optimization (CXO) from an initiative into a broader organisational capability. We helped define the process, train colleagues and support adoption through ambassadors and internal initiatives. Over time, CXO developed into a widely adopted, data-driven way of working that continues to scale through internal ownership.
The harder part of CRO is not the test
One of the recurring challenges in our field is the expectation that experimentation should produce quick wins. Of course commercial impact matters. But when CRO is reduced to a stream of isolated optimisations, its potential stays limited.
The real challenge is rarely whether an organisation can run an A/B test. It is whether teams can decide what is worth testing in the first place, whether they are willing to challenge assumptions and whether learnings actually influence future decisions.
That requires more than tooling. It requires capacity, stakeholder alignment, good problem framing and a learning mindset.
The organisations where experimentation creates the most value are not necessarily those with the most traffic or the largest teams. They are the organisations that are willing to make room for evidence, reflection and learning, even when the answer is not an immediate conversion uplift.
That is why our work increasingly focuses on capability building as well as execution. We train and coach teams, improve hypothesis quality, help structure experimentation processes and bring different disciplines and stakeholders together around evidence.
We also use new technology where it genuinely improves that process, including AI-supported workflows for capturing insights and developing stronger hypotheses, and advanced quantitative methods such as cluster analysis to build more evidence-based customer understanding and prioritisation.
Recognition on a European level
“Being shortlisted for CRO Agency of the Year on a European level means a lot to us. It is recognition that the way we approach experimentation – combining behavioural science, data, UX and long-term capability building – resonates beyond our own market.
For a relatively small, independent agency from the Netherlands, being recognised alongside leading agencies from across Europe feels significant. It reflects not only the work of our own team, but also the trust of the organisations we work with and the client teams that have committed to making experimentation part of how they work and make decisions.
After 17 years of working in experimentation and CRO, it is especially rewarding to see that approach recognised internationally.” – team Online Dialogue
And perhaps most importantly, it gives us extra motivation to keep pushing the field forward: towards experimentation that is not just about short-term uplift, but about helping organisations understand customers better, make stronger decisions and keep learning.
Future of experimentation according to Online Dialogue
For us, the future of CRO is not simply about optimising more. It is about helping organisations become structurally better at understanding customers, choosing what matters, validating assumptions and learning from the outcome.
We will keep combining behavioural science, data and experimentation to help organisations make those decisions with more confidence.
And for now, we are very happy to see that approach recognised on the CRO Agency of the Year shortlist.
Website: https://onlinedialogue.nl
LinkedIn: https://www.linkedin.com/company/onlinedialogue
