5 years of external pentesting: How the Dynatrace security posture has strengthened

Over each of the past five years, the Dynatrace security team has tasked external security firms with conducting penetration tests of our SaaS and Managed components. These tests are performed to verify the effectiveness of our security controls. These tests also provide insights to our customers in the form of independent, third-party reports about our security posture.

The entire product is in scope

Dating back to March 2016, the first-ever Dynatrace external penetration test was performed. With a test duration of 21 days, the focus was on the Dynatrace web application, ActiveGate, OneAgent, and our mobile app.

Fast forward to 2020. NCC Group, the biggest external penetration testing company world-wide was charged with testing all aspects and components of both Dynatrace SaaS and Managed deployments. With a total of 45 testing days, the external team was tasked with testing the following scope:

• Dynatrace web application
• Cluster Management Console
• SSO (SaaS and Managed)
• Public REST API
• OneAgent
• ActiveGate
• Synthetic Monitoring
• ActiveGate plugins
• OneAgent plugins
• Session Replay

We’ve raised the bar for security at Dynatrace

Just by looking at the declining number of security vulnerabilities found, it’s clear that the Dynatrace security posture has taken a massive leap forward.

During the first external penetration test in 2016, a high number of severe issues were found although the scope of these issues was rather small. Over time, the scope and the number of testing days increased while the number of severe vulnerabilities dropped to an all-time low of only two findings in 2020.

A major contributor to this increased level of security are the SDLC controls that we’ve implemented since our dedicated product team of certified security professionals was established in May 2017.

“For software this complex, the report is clearly an indication of an excellent result. It is quite clear that the Dynatrace team consists of competent developers that are able to write very well-tested and defensive code. Despite the complexity, the quality stands out for not leaving much room for errors.”

 – Cure53 (External Pentest Company) after testing the OneAgent component

Pentesting helps shape a better product

Penetration test findings are not only useful for informing us about missing security defenses. They also play a vital role in shaping our product, by making it more understandable and easier to configure. Over the past years, we’ve not only strengthened our security posture, we’ve also contributed significant product improvements, for example:

Improved password policy

An initial finding showed that the password policy for local users in Managed clusters wasn’t strong enough and could not be configured. Together with fixing this security concern, our internal team also introduced a new password policy settings view which enables you to adapt password complexity to your needs. This has been in place since Dynatrace Managed version 1.206.

Concurrent logins overview

It was reported that there was no overview of current sessions logged into Dynatrace Managed. A new view was introduced in the Cluster Management Console that not only provides a good overview, it also provides Dynatrace Managed admins with full control over their users. For details, see Configure and manage user sessions.

Dynatrace extensions 2.0

Early reporting showed that it was possible to inject code via Python plugins. Rather than simply addressing the limited scope of this vulnerability, our team moved multiple steps further: the new Dynatrace extensions capability will soon be fully configurable via a newly crafted API. Additionally, code signing will be introduced to fulfill your security needs. The first version of these enhancements are targeted for release at the end of Q1 2021.

The road to continuous security evaluation

Some may argue that external penetration testing only provides a statement about a product’s security posture at a given point in time. However, we’re not just remediating the findings of our penetration tests. In our continuous improvement cycle, penetration test findings play an increasingly prominent role in challenging us to address the question “How could we have found this vulnerability in an automated fashion?”

The Dynatrace product security team has therefore been working on creating a continuous testing pipeline that automatically scans for all previously found pentest vulnerabilities, among others. This internal testing environment covers a multitude of tasks that previously had to be performed manually. This gives our team more time to focus testing on all new features and components before they go to market.

The goal

External penetration tests will continue to deliver insights into our security posture from an external perspective. We will of course continue to strive to reduce the number of vulnerabilities found in external penetration test reports to zero.

If you’re interested in our latest penetration test results, please contact your Dynatrace account representative and ask for the latest report. We’ll be happy to share the results with you once you’ve signed an NDA.