Kubernetes logging made easy: Comprehensive Kubernetes visibility with Dynatrace

The Dynatrace Log Module for Kubernetes is a fully automated, full-service approach to Kubernetes log management and analytics. Dynatrace is your one-stop shop and go-to solution for all Kubernetes needs.

Logging is integral to Kubernetes monitoring

In the ever-changing and evolving software development landscape, logs have always been – and continue to be – one of the most critical sources of insight. Log data is essential, whether you’re troubleshooting and conducting forensics into past problems, investigating potential security issues, or debugging in real time.

However, when working with Kubernetes, its distributed and ephemeral nature means that logs are scattered across multiple nodes and pods, making it difficult to ensure that all logs are preserved, easily accessible, and enriched with necessary context for future analytics.

A robust log collection solution must ensure:

• Central log management: Logs must be centralized, preserved, easily accessible, and connected with other signals from your Kubernetes environments to allow effective monitoring and troubleshooting.
• Context-aware and topology-rich logs: Logs must be enriched with metadata and contextual information, allowing powerful log analytics of all distributed traces, metrics, and events within the Kubernetes topology. This allows analysis of logs from out-of-memory containers or pods with application errors.
• Powerful access controls: The ability to define fine-grained access control for logs based on Kubernetes namespace or cluster level is essential. This ensures that users can only access the necessary logs, helping maintain the highest security, compliance, and operational efficiency.

At Dynatrace, we support whichever integration you use to collect logs from your systems, whether it’s OpenTelemetry, Fluent Bit, or other tools. Moreover, we’re continuously looking to improve our customers’ experience, which is why we’ve enhanced our fully automated, full-service log streaming solution with the Dynatrace Log module.

The updated solution allows you to:

• Stream and analyze logs from your Kubernetes environments without running OneAgent on each Kubernetes node.
• Flexibly choose the level of observability you need. For example, you can start small with only Kubernetes platform monitoring and log analytics and grow into comprehensive observability maturity with distributed tracing, security analytics, real-user monitoring, and more. This flexibility extends to privileges, ensuring you can grant the least possible number of privileges in Kubernetes to get the needed data.
• Get insights into logs from short-lived containers and pods such as InitContainers or Jobs.
• Easily onboard log analytics within the Kubernetes app and control log ingest and management centrally to ensure optimal experience. Rather than configuring log collection locally, as you might have done with open source log shippers you’ve used, you can now centrally define and manage which logs Dynatrace collects.
• Effortlessly get logs in context as the Dynatrace Log Module for Kubernetes is fully managed for you; Dynatrace handles all upkeep, configuration, and lifecycle management.

Let’s look at how the new capabilities integrate with the Dynatrace platform.

One platform, complete log value

Not having to lift a finger to stream your logs is awesome, but that’s not the only value the Dynatrace Log Module provides.

Central log management

You have the ability to fully manage the lifecycle and configuration of your Kubernetes log collection within the Dynatrace platform. Gone are the days when you needed to manage the configuration of a log shipper across all your clusters. Dynatrace allows you to do this on one platform. You can centrally manage and control data collection, data masking, data dropping, data transformation, and data retention vs distributed configurations at the source. Additionally, the ability to dynamically define retention periods and compliance settings gives you much more flexibility when working and allows you to get the data you need to resolve issues easily. This ensures the smooth and reliable operation of applications running on Kubernetes.

Troubleshooting and remediation

Logs are critical to understanding and maintaining system health and performance. Dynatrace helps you quickly troubleshoot and easily remediate in a variety of ways, allowing you to:

• Inspect and understand logs from crashing application containers and other workloads in the Dynatrace Kubernetes app.
• Slice and dice log data with traces and Kubernetes topology in Notebooks with DQL.
• Drive evidence-based investigations for security issues and data forensics.
• Easily derive metrics and events from logs for dashboarding and prediction.

• Automate remediations with log data in the Workflows app.
• Dive into log data to explore surrounding logs and patterns using the Dynatrace Logs app.

Cost management and allocation

Dynatrace allows you to manage costs by controlling the amount of ingested logs, the retention period for which logs are kept for analysis, and query timeframes for your analytics. You can also manage ingest and retention by filtering log sources, managing buckets, and picking the right license model for your needs.

Additionally, you can leverage existing labels or annotations to enrich log data with cost-allocation metadata, making it easier to allocate expenses accurately. This allows for precise cross-charging, helping ensure that bills are charged to the correct departments.

Log enrichment

Dynatrace enriches every log line with additional Kubernetes metadata. In addition to cost-allocation use cases, you can enrich logs with security context and other common Kubernetes context metadata.

This allows you to define IAM rules to control access to log data so that only users with specific roles or permissions can query certain logs. This ensures that teams see only the logs they need, helping your organization maintain the highest levels of privacy and security. Further, if you also monitor your applications with Dynatrace, your log lines will be enriched with span and trace IDs for context-rich analytics of log, trace, and metrics data.

Get started

Seamlessly manage your logs with the new Dynatrace Log Module. Get started today to see how Dynatrace can help you with your observability and security needs.

_{Dynatrace and the Dynatrace logo are trademarks of the Dynatrace, Inc. group of companies. All other trademarks are the property of their respective owners.}