Privacy spotlight: Retain data in Grail with 1-day precision to better meet your compliance requirements

The constantly evolving challenges of privacy-conscious data management require more than traditional data retention settings, which are limited to weeks or months. With Dynatrace Grail™ you can configure data retention periods with day-level granularity for up to 10 years, ensuring your ability to meet legal requirements while maximizing the value you get from Dynatrace.

Streamline privacy requirements with flexible retention periods

Data retention is a critical aspect of data handling, and it’s not just about privacy compliance—it’s about having the flexibility to optimize data storage times in Grail for your Dynatrace use cases. Most observability solutions rely on fixed, default retention periods, or provide only limited configurability with pre-defined retention periods to choose from (number of days, weeks, or months), or require you to set up rules to archive and extract logs.

Holding onto personal data beyond the minimum required timeframe can introduce privacy risks, inefficiency, and excessive storage costs. Implementing clear data deletion policies is also closely linked with the legal requirement of personal data minimization. The core privacy requirement to consider when defining retention periods is to keep data no longer than required for a specific purpose. While this might seem straightforward, the challenge lies in determining precisely what this means in diverse scenarios. Deciding on the correct duration for retaining data is context-specific; it depends on the nature of the data, its intended use, legal requirements, and your evolving business needs.

For example, log data, which can include personal data, should have a shorter retention period when used to troubleshoot and debug performance issues in your testing environment (compared to log data used for audit logs) because the data might not be needed after a few days. Fixed or predetermined timeframes lack the adaptability needed for diverse data management scenarios. Archiving options fail to provide a dynamic, user-friendly solution for tailoring retention periods.

Customize retention periods with buckets in Grail

You can define retention periods in custom buckets for all the data you store in Grail. Grail buckets function like folders in a file system. Each bucket should contain only those records that should be handled together as a set. You can learn more about custom bucket retention periods in our recent blog post, which explains how to enhance data management and includes best practices for setting up buckets with security context in Grail. Besides improving query speed, reducing related costs, and enabling organizations to target a specific business unit or production stage, custom buckets enable you to segment your data for specific use cases.

Segmentation of use cases establishes the perfect landscape for tailoring retention periods to the purposes for which data is collected. You might have specific data retention needs based on your industry and use cases. For example, industries such as finance and healthcare have specific regulations that dictate audit log retention periods ranging from months to several years.

With Dynatrace, you have full control over Grail retention periods—you choose how long to store each portion of data in line with your requirements. You can select the desired retention period for your data in bucket configuration settings, with the available retention periods ranging from 1 day to 10 years, plus an additional week (3,657 days). A hard deletion process is initiated when there’s no legal or business reason to continue retaining data in alignment with your configuration settings. The system reviews the timestamp of all data segments to see if any have reached the end of their specified retention period. Data that has exceeded its retention period is flagged for deletion, and the data is permanently removed from Grail, making it unrecoverable.

The Storage Management app makes it simple to set data retention periods

While the Grail bucket management API is a powerful approach you can apply to your data management policies in an enterprise setting at scale, a simpler method is also available. The Storage Management app on the Dynatrace® platform lets you manage all custom buckets for Grail in a visual user interface. This makes the data retention rules easy to grasp and implement even for non-technical users and auditors.

The Storage Management app provides an intuitive overview of Grail buckets for different types of data you store with Dynatrace (for example, logs and business events. Other data types will be available soon). For each type you can inspect existing buckets along with their defined retention periods and current status. The app also provides a quick way to extend or shorten a retention period for existing buckets, rename buckets, add new buckets, or delete buckets to completely wipe the data they contain.

What’s next?

• Sign up for a Dynatrace free trial account to start using Dynatrace today.
• Start managing your Grail buckets with the Storage Management app, to be released with Dynatrace version 1.281.
• Explore our documentation, which explains data retention periods and how to manage custom Grail buckets.
• Learn more about our commitment to providing you with control and transparency over your customers’ personal data in the Dynatrace Trust Center.