Summary.    Successful businesses focus on three core principles of their customer experience: They put the digital experience in the business context. They recognize that customers are not created equal. And they make zero-based decisions. Continuous improvement is fine,...

An online casual gaming company was testing a new piece of functionality. It did a standard A/B test where half of the players saw variant A and half saw variant B. The test concluded that variant A improved a critical element of the site’s conversion rate with a high degree of confidence, and the business moved all of their players to the winning variant. Unfortunately, revenue immediately started to decline. Only when the business looked further did it make a critical discovery. What is typical in casual gaming apps is that often only 1% of the visitor base spends any money. So, while variant A performed better overall, variant B performed much better for visitors who actually spent money.

This example highlights two of the most common challenges that businesses face when attempting to improve their digital engagement with customers. One is assuming that all customers are the same, and the other is focusing on a siloed digital outcome rather than a business outcome. It was true that variant A improved conversion rate but, in this case, conversion rate was not aligned to profit. As the saying goes, “The operation was successful, but the patient died.”

Another common challenge is around how to prioritize and focus customer experiences. As businesses increasingly invest in their digital engagement with consumers, there is a never-ending backlog of potential enhancements enabled by new ideas, improved device capabilities, and the relentless progress of AI — all which continuously increase consumers’ expectations. Standing still is never an option, but knowing where to focus and what will make the greatest difference to your customers’ experience is not easy.

As consumers, we all recognize the importance of a seamless customer experience. The purpose of this article is to highlight the typical things that go wrong when developing customer digital experiences, and to propose an approach that will help capture the huge opportunities of getting it right.

Successful businesses focus on three core principles:

• Put the digital experience in the business context. While it might be alluring to focus on conversion rate and to frame it as the objective of the digital experience, that is rarely the route to success. It is critical to understand how the digital experience drives profit and to focus on solving a business problem.
• Recognize that customers are not created equal. Leverage your customers’ digital experience and understand what is important for different customer segments, particularly your high value and high potential value customers. Then personalize the experience when appropriate.
• Make zero-based decisions. Continuous improvement is fine, but it’s important to be willing to think from first principles, and to re-examine and justify what you are doing and what you could do differently. This helps ensure that the digital roadmap is prioritized and focused on the highest impact actions.

Put The Digital Experience in a Business Context

Digital experiences can be thought of as a funnel — customers enter a site or app with an objective in mind, such as booking a taxi, ordering a pizza, or buying shoes. Some succeed and some fail. Most businesses embarking on managing this journey start by measuring the different stages of the funnel. Overall conversion rate (the percentage of customers who achieve their objective) is often seen as the ultimate metric of success.

However, any digital experience is (almost always) a window into the overall business proposition. The digital experience is tightly interdependent with all aspects of a business and cannot be assessed or optimized in isolation — just look at the spikes in conversion rate during any sale or promotion. Clearly, individual elements of a digital experience can be improved — faster speed, bug fixing, better navigation — and these types of enhancements are rarely bad, but often miss the bigger opportunity.

A digital fashion retailer was focused on improving its “add to basket” rate by testing lots of different product page formats. But when they looked at product conversion rates over time and across different products, they realized that the conversion rate for individual products was dominated by the product’s quality, price, and availability. Great products, in-stock and priced competitively, had a conversion rate 10 times higher than the average product.

Their focus immediately shifted from page formats to delivering a better experience of availability, by nudging customers to enter their size and then directing them to the products available in their size. The retailer also started to augment the measurement of “add to basket” and conversion rates with measurement of the customer experience. Measuring and optimizing customer viewed availability — are the products the customer is viewing in stock in their size — turned out to be much more important than fiddling with different button colors.

Another example further illustrates the challenge of finding the real opportunities. A food delivery company was trying to optimize how to show expected delivery times in their app to drive customer satisfaction. However, when they spoke to unhappy customers, they realized the failure point was quite nuanced — the problem was less about the messaging and more about when the expected delivery time varied frequently between point of order and delivery. The expected arrival time was being updated every few minutes and was thought of as “helpful” but actually drove significant dissatisfaction. Moreover, the delivery company was only storing the order time and delivery time, making the issue challenging to diagnose. The company recognized that the root cause of customer dissatisfaction was nothing to do with the digital journey but was driven by the algorithm for predicted delivery time (which they subsequently improved).

These examples highlight the need to unpack the “digital” element from the interaction with the rest of the business. An observed failure in a digital journey is often nothing to do with the digital journey but is a symptom of the overall business proposition. This pitfall is all too common when digital has become a distinct division, disconnected from the rest of the business. Digital experiences do not respect organizational silos.

Customers Are Not Created Equal

A hotel chain identified high bounce rates on search traffic from Google as an issue — bounce rate is the percentage of visitors who enter and leave a site immediately, and only ever see one page. Their initial digitally focused solution was to conduct A/B testing on different landing page layouts. The impact on the overall bounce rate, while positive, was relatively small. Further analysis of the data revealed an interesting insight: the bounce rate was driven by the customer’s device and the specificity of their search. For example, a customer searching for “hotels near Geneva” on a desktop preferred a large map and default radius to be able to see all the hotels in the Geneva area. But a customer searching for “hotels near Geneva airport” on a mobile device preferred a much smaller default radius around the airport. The landing page design was not the issue. The real business opportunity was to personalize the default map radius based on the customer’s search characteristics which had a dramatic impact on both bounce rate and overall business performance.

Personalizing your customer experience extends to understanding what is important for your high value, and high potential value, customers. This same fashion retailer started analyzing its product availability for different customer types, and discovered that its highest value customers had the worst availability experience. Further digging revealed that the higher value customers were typically searching for either smaller or larger clothing sizes. However, the retailer had been making decisions about size ratios based on old-fashioned rules of thumb, and had not adjusted to the new online reality which changed both the data available and also the customer mix. They subsequently changed their approach to optimizing size ratios.

As we saw in the opening example, A/B testing is seen as the gold standard but is fraught with complexity and misinterpretation. A critical challenge is failing to look beyond the headline results. Only by analyzing outcomes at an atomic level — by devices, browsers, customers, or marketing sources — can one really understand the true efficacy of a test, and whether the best approach is to select the A vs. B, or to personalize (as with the hotel example above). Oversimplifying will at best miss opportunities and at worst lead to significant waste. Nothing about testing is easy.

Make Zero-Based Decisions

It is typical to have teams focused on continuously improving the digital experience. It is less common to have a zero-based mindset with teams willing to look at a problem afresh. But businesses that regularly challenge their priorities and actions from first principles make better decisions.

A retailer developed a product recommendation system that offered cross-sells and upsells at various points across its site. The algorithm that powered it was intended to optimize “add to basket” and conversion rates. When the new algorithm went live, they were delighted with the results. It performed much better than the previous recommender.

However, they wanted to challenge the face-value assumption of success. Further analysis revealed insights that all was not as it seemed. Looking across all products and inventory, only about 12% of the inventory value was ever getting recommended. It turned out that the recommendation algorithm was resolutely focused on recommending bestsellers, which often would have sold anyway, and was much less prone to recommend overstocks, new products, or products that were harder to sell. The metric that had been selected to optimize was misaligned with the business objectives of sell through, gross margin, and inventory efficiency.

Put It All Together

Digitally led businesses are overwhelmed by opportunities. Identifying the top 100 ideas is easy, identifying the top 10 is hard. Being intentional about where to focus is a significant management challenge and at its core it is an analytical challenge. What to measure to drive the right diagnosis? Which digital objective to optimize? How to understand the interconnectedness of the digital experience to the rest of the business? And as the examples above have highlighted, there’s rarely a shortcut to doing the analysis.