Extend business observability: Extract business events from online databases (Part 1)

Business leaders benefit from in-the-moment business insights. They frequently articulate the need for real-time visibility into business data to support agile business decisions. But existing business intelligence (BI) tools often lack the broad context, ease of data access, and real-time insights needed to understand and improve customer experience and complex business processes.

The key challenges include:

• Business data is often difficult to access, resulting in fragile data pipelines.
• Data is not delivered in real time; it’s often delayed by weeks or longer.
• Business data often lacks IT context, which prevents effective BizOps collaboration.

Dynatrace business events address these systemic problems, delivering real-time business observability to business and IT teams with the precision and context required to support data-driven decisions and improve business outcomes.

Dynatrace business events provide precise, real-time business metrics that support fine-grained business decisions and auditable business reporting. They offer lossless access to hard-to-reach business data embedded in in-flight application payloads, ensuring that valuable information is not missed. Additionally, Dynatrace business events enable organizations to explore and analyze large, long-term data sets without pre-indexing, which allows for flexible and comprehensive data analysis.

Extend business observability to data at rest

In our past blog post about business agility, we looked at a retail sales use case example to investigate potential causes of underperforming store locations. We also looked at a pizza chain example, connecting each customer order to the fulfillment process milestones that followed, including the handoff to the delivery agent.

In both examples, we used Dynatrace OneAgent® deep payload inspection to capture business data in motion. There are also many cases where business data—transactional, inventory, or financial—is at rest or in use, stored in a database. For comprehensive business observability, you need access to this data in real time. This can be accomplished using Dynatrace extensions. Dynatrace extensions can easily query data from various databases and store the results in Grail™, the Dynatrace data lakehouse. Once the data is in Grail, it can be transformed, queried, reported to dashboards, and more.

Business data is more than metrics

Dynatrace Extensions enable the expansion of Dynatrace monitoring to encompass any technology that provides an interface. For instance, the SQL datasource facilitates universal database queries across commonly used databases, subsequently transmitting the results to Dynatrace in the form of metrics or logs.

However, in the real world, business-related data isn’t limited to metrics. Business data should be viewed through a different lens, storing it separately while preserving the unique characteristics that enable business observability:

• Certain business data, such as product names, customer details, sentiments, order dates, payment methods, and more, are not simple metrics. Instead, they can consist of various data types: strings, integers, float, timestamps, and combinations of values.
• Such business observability can’t reside in traditional databases or data warehouses and thus needs to be in a data lakehouse that can unify and contextually analyze observability, security, and business.
• Metrics lack the contextual information to automatically trigger actions such as targeted outreach to impacted customers or automations to remediate process anomalies. Business events, however, capture specific occurrences or actions, allowing organizations to understand triggers, respond promptly, and foster collaboration among teams for improved customer experiences and business outcomes.

To get past the basic metric limitations, we created a custom extension to extract business data from existing databases and store it in Grail. Here’s a peek at the approach:

Business observability

Business observability refers to gaining insights into a business’s operation, performance, and behavior in real time. It involves collecting and analyzing data from various sources within an organization, such as IT systems, applications, customer interactions, and business processes, to gain a comprehensive view of how the business is functioning. An effective business observability solution should make it easy to ingest business data from any source, including databases.

Similar to the concept of observability in IT systems and applications, business observability focuses on capturing data at different layers of the business and making it easily accessible and understandable for analysis and decision-making. It goes beyond traditional business intelligence by providing real-time, granular, and contextual data that enables organizations to identify patterns, trends, anomalies, and correlations across different business dimensions.

Illustrating the value of business observability

Business observability helps you understand and evaluate the performance and effectiveness of systems in achieving their intended business goals. While observing individual requests is essential for performance engineering purposes, taking a business lens perspective provides deeper insights into the actual value delivered by the underlying system.

For example, consider an e-commerce website aiming to maximize sales. By implementing business observability, you can analyze conversion rates, sales patterns, and order fulfillment times. This enables you to identify bottlenecks, optimize user experiences, and make data-driven decisions to improve sales performance.

Similarly, in the case of a ride-sharing app, business observability allows you to monitor metrics like ride acceptance rates, driver and rider satisfaction, and average wait times. By analyzing these business-oriented indicators, you can optimize an app’s algorithms, allocate resources effectively, and enhance the overall experience for both riders and drivers.

For an insurance provider, business observability provides insights into key metrics such as policy sign-ups, claim processing times, and customer satisfaction levels. By closely monitoring these business-focused metrics, you can identify areas for improvement, streamline processes, and deliver better service to your customers.

Business observability not only ensures that systems perform well technically, it also ensures that systems are aligned with their intended business objectives. By gaining visibility into the business value delivered by these systems, you can make informed decisions, optimize performance, and ultimately achieve your business goals more effectively.