Embracing ambiguity in experimentation

We prefer options that are known to us.

The ambiguity effect is a cognitive bias that describes how we tend to avoid options that we consider to miss information. It can cause companies to remain committed to failing systems, rather than opting for improvement.

When we practice experimentation we have to embrace ambiguity. It is hard because decision-making is influenced by how much information we have. We would like not to avoid options for which we feel we don’t have sufficient information. If we did, we may never discover something that can either accelerate our learning or bring us to a halt.

So how can we get better at identifying what we don’t know about an option, as well as recognizing the potential benefits of choosing the more ambiguous option?

Thinking time

We need to learn to override our initial impulse to avoid ambiguous options and situations. We can achieve this by putting the time into making decisions.

The less ambiguous option may seem more desirable. However, if we reframe the situation, it may reveal that the less ambiguous option is not as superior as it seemed. Furthermore, we can use second order thinking to help us think beyond what we know. Second order thinking requires analyzing the potential impact of our decision into the future.

When addressing the more ambiguous option, we need to consider what can go wrong and what can go right. When faced with ambiguity, we default to the worst case scenario, forgetting that maybe the outcome could be better.

Ask the right questions

We tend to focus a lot on answers. Answers are solutions to problems.

Instead, during experimentation we want to prioritize questions. By asking questions we signal that we want to learn. We should not be afraid of admitting we do not know the answer.

By asking questions you’re actually doing something very important. You are assessing the understanding of your peers. Everyone has a different set of skills and experiences. Our interpretation of results will differ.

During experimentation, we want to manage the rise of decision fatigue. Questions help us manage the quality of our conversations by remaining active listeners.

Have a strong opinion

You can’t build products that people want without a strong opinion. Before you formulate a strong opinion, you first need to understand how the customer sees your product. An end-to-end approach alongside design thinking can surface a new perspective. A “product” is not a website or an app.

A strong opinion provides the right constraints to focus on innovation. You can’t research your way to innovation despite having done research. You have to understand the customers’ experiences without expecting your customers to tell you the solution. Customers are your inspirator and you are the creator.

Structured thinking

By its very nature, experimentation can feel chaotic. The chaos can manifest day-to-day at a team level and at an organisational level in the way we make decisions. The reason why our thinking is in chaos is that we have no structured thinking system in place.

We were taught at school what to think, we were never taught how to structure our thinking. Thinking leads to actions, leads to results. When we have a structure for thinking in place we can take on different perspectives to help us answer effectively and efficiently “what’s next?”.

On a past product team, we were given the problem of optimising a key market vertical. After some initial research, we decided to build again from scratch. To structure our early thinking, we borrowed from systems thinking and design thinking to formulate a strong opinion. Friday’s quickly became our most cherished group thinking time. Without this early approach, we may not have surpassed business objectives and hit the desired go-live time frame.

How are you embracing ambiguity in experimentation?