Dynatrace Opportunity Insights uses AI prediction and real-user data to optimize business outcomes

Performance and error optimization is complicated, and the approach to driving improvement has long relied on static recommendations disconnected from a site’s end users and their experiences. While traditional methods can help you understand when your users face poor performance or errors, understanding if and how much these problems impact your business remains largely guesswork. The new Opportunity Insights app (available to subscribers of the Dynatrace Business Insights service) takes an industry-first approach to shift this paradigm: using predictive AI on top of your application’s historical real user data, we identify the primary performance/error metric impacting user experience on each page and quantify the predicted business impact of recommended optimizations. With this information, business and application owners are empowered to make decisions based on their impact on the bottom line.

Performance, errors, and user experience

Dynatrace has long understood the importance of performance as a foundational component of user experience and the impact that page speed and any friction introduced by errors have on user behavior. When a page is slow to load, users are quick to abandon the site; research by Google, Vodafone, Akamai, and others has repeatedly shown that even small (~100 ms) degradations in page performance can increase abandonment, lost revenue, and lead to persistent changes in user behavior. Additionally, JavaScript or HTTP errors that impact a page’s functionality or overall experience can be even more immediate and lasting, leaving users with the impression that a site’s problems reflect the brand, leading to a longer-lasting negative impression.

Despite this broad, conceptual understanding of the importance and impact of poor performance and errors on end users, it’s difficult for site owners to fully understand and quantify the experience of their unique site on its users. Until now, the best approach was to take generic, static recommendations based on the experience of large internet brands and their user bases, apply them to a site, and hope to see improvement in the business. Sometimes, these recommendations are so far from current performance or cost so much to implement that they seem irrelevant or unattainable. The addition of more and more metrics over time has only made this increasingly complex. As a result, it’s challenging to get business and resources focused on performance and error optimization without supporting data that shows how those optimizations will impact your organization’s financial outcomes.

Site owners and businesses are left asking where they should invest and what exactly they should optimize to balance the best experience for their users with the least amount of effort that will lead to improvements in the business.

Performance and errors in context

What if you could answer the question, “What is the smallest change I can make to this page to maximally reduce user-exit rates and improve conversion rates?”

With Opportunity Insights, Dynatrace uses predictive AI and your site’s real user data to identify opportunities where improvements in performance and errors can make the most significant difference in retaining users and sessions while reducing exit rates and converting more customers.

To provide precise, contextual recommendations for performance and error improvement on a given page, Opportunity Insights does two key things:

• First, it looks at user behavior and performance/error metrics across millions of sessions and identifies which candidate metrics show the most significant impact on user experience and should be considered a metric to improve. Performance metrics to improve can be Visually Complete, Speed Index, or other timing metrics associated with the page load cycle.
• Opportunity Insights then predicts the business impact of improvement in those metrics and suggests a target for the most significant return on effort,” expressed as an increase in retained sessions, conversion percentage, or revenue per month.

From an underlying list of hundreds of opportunities, Opportunity Insights surfaces the Top 3 opportunities that will provide you with the most significant return on effort.

How we do it

Opportunity Insights’ predictive AI is powered by a patent-pending structural causal machine learning model (connecting the cause and effect of performance, errors, and behavior) that is trained on each application’s unique real user session data across millions of sessions to analyze errors, performance metrics, and behavior patterns. It looks at these metrics across key pages to pinpoint where your users are most impacted and which metric directly impacts each page.

With this deep understanding of the intersection of performance, errors, and real user behavior, Opportunity Insights simulates over 650 million possible scenarios of performance improvement per application to surface precise recommendations for optimization that will yield the most significant increase in retained sessions for the least amount of effort. Armed with this prediction, it’s easy to understand the potential payoff of improvements and better prioritize your development efforts.

In evaluating our model’s accuracy in understanding performance/errors and behavior, we use historical real user data and a standard 80-20 train-test split to train our model. It predicts user behavior based on performance/error experience. We’ve tested it across various applications and industries, and it accurately predicts user behavior about 90% of the time. When the suggested metric is optimized, the user-exit rate drops, and the number of additional retained sessions increases as predicted.

We also tested our business impact prediction in real-world scenarios. We used customer applications where our model identified improvement opportunities. After implementing the suggested optimizations, we found that ~20% more users and sessions were retained than predicted, showing that our model’s predictions are conservative.

Early user feedback

Preliminary user feedback validates the ability of the app to help application teams, and owners quickly and easily understand where to optimize and the business impact of optimizations. Here are a few customer highlights:

• The director of e-commerce at a leading budget airline stated, “This is exactly what we need [in our decision-making], connecting application performance and business outcomes.”
• The IT Manager of a leading US insurance company stated, “This aligns with the direction we are taking the company, which is to make decisions based on business impact.”
• A platform monitoring manager from a leading payment company stated, “Honestly, this should be driving where they (the teams) are doing bug fixes and performance improvements.”

Get started

Opportunity Insights is available with the Dynatrace Business Insights service. It was released to a subset of Business Insights customers as a preview in April 2024 and is now generally available with Dynatrace version 1.296.

Download the whitepaper to learn more about the technology underlying Opportunity Insights and how you can quickly understand where to strategically invest development resources so they have the greatest impact on your business.

If you’re already a Business Insights customer and want to learn more, please contact your Insights Team. If you’re not a Business Insights customer but are interested in Opportunity Insights and the value Dynatrace Business Insights can bring your customers and your business, please contact your Dynatrace Account Executive or Customer Success Manager.