Running a successful internal bug bounty program

With the topic of IT security receiving more and more attention each day in media coverage, Dynatrace is proud to announce that we’ve just completed the “first season” of our internal bug bounty program.

What is a bug bounty program?

According to HackerOne, the industry leader in external bug bounty programs, a bug bounty program is described as:

A program where ethical hackers are invited to report security vulnerabilities to organizations, in exchange for monetary rewards for useful submissions. Bug bounties are commonly seen as the most effective and inexpensive way to identify vulnerabilities in live systems and products.

So, what then is an “internal” bug bounty program? You’ve probably already guessed. In internal bug bounty programs, a company’s employees are encouraged to search for security vulnerabilities within the product portfolio in exchange for monetary reward.

Introducing Hack.DT

We considered the television character Elliot Alderson (Mr. Robot) to be the ideal role model for this sort of work. Combined with the catchy title “Hack.DT”, our internal bug bounty program began in January at our Linz office. The program launch was broadcast to all Dynatrace locations around the globe (see photo below).

The objective for each participant was simple: Find as many Dynatrace security bugs as possible and report them to the bug bounty supervisory team. Each submitted issue was evaluated for validity. Valid submissions received a reward calculated based on the Common Vulnerability Scoring System (CVSS).

Advantages of the program

External penetration tests are expensive and require that testing personnel be well acquainted with the product under evaluation. Having all internal R&D employees on board to search for security bugs within a codebase that they work with on a daily basis is however far easier and brings tremendous benefits. No time is spent beforehand understanding the environment.

While participating in the program, employees learn to think about security-related topics and begin to view source code from a different perspective. The opportunity to see the findings of colleagues can inspire others to work harder at unearthing vulnerabilities. Healthy competition is then created in which all participants work to receive the highest monetary rewards.

Our in-house security team also learned a great deal from our findings. Without the effort of performing a complicated gap analysis, we were quickly able to see where security improvements should and will be implemented.

Risks & problems of the program

Validating findings and determining appropriate rewards can honestly take a lot of time. The process of triaging can also be a little tedious as it eats up time you would otherwise spend on testing. Nevertheless, the process can offer huge benefits to any company.

Most every business deals with some sort of sensitive data. Such data needs to be protected against both internal and external actors. This means it’s important that vulnerabilities and sensitive data only be disclosed to a small number of employees. At Dynatace, we set up a secret endpoint where our colleagues had the ability to report their findings. This provided the necessary control over all sensitive information.

To determine appropriate monetary rewards, each security issue must be rated. This can create a fairness dilemma as some employees (i.e., those with a strong security background) can elaborate on the worst-case interpretation of risks better than other employees can.

Measures of success

Our bug bounty results provided our security team with a solid base from which to measure our success going forward. Automated controls have been established to prevent the reoccurrence of identified issues; this is mainly handled by via regression testing. We’ve also configured some static code analysis tools to detect certain security vulnerabilities should they re-appear in the future.

Based on what we’ve learned, we’ve reevaluated our security posture and are now targetting black spots in our IT landscape. Chief software architects and software development leads are invited to consult with the product security team throughout the planning of security-critical components.

Last but not least, internal security champions who receive advanced security training are being put in place. This will help evangelize security awareness across all Dynatrace software development teams, ultimately leading to a more secure product.

How many bugs did we find?

We know this is the question on many of your minds. While we can’t share the exact number of bugs we found with you, know that the number was nearly double what we anticipated. The entire reward pool was exhausted. We had to double the bug-bounty budget to accommodate the high number of findings and to show our appreciation for what was reported.

Stay tuned

If there’s one thing certain, it’s that Hack.DT will return for a second season, with a higher bug-bounty budget, more participants, and, of course, increased security benefits for Dynatrace customers.