Implementing Agile project management in an open-source project

How we found a project management style that works for our open-source project Keptn

I have been part of the open-source project Keptn since its inception in January 2019 and it’s exciting to see how the project — what we created originally as a demo for our Autonomous Cloud Lab at Dynatrace — has now become a real open-source product that is not only used by many companies worldwide, but is also a Cloud Native Computing Foundation (CNCF) Sandbox project.

What is Keptn?

Just to give you context: Keptn can be described as a control-plane for DevOps automation of cloud-native applications. Thanks to a declarative approach, it enables you to automate your application delivery and operations, without the need of bulky, obscure, and inflexible code. (You can read more about Keptn here.)

Keptn was created as an open-source project straight from the beginning. When I started, I was working directly on the code and taking care of releases. Now, I have moved on to a Technical Product Manager role, which means that I drive the Keptn roadmap, take care of the use cases, organize the team and the working processes. We have achieved a lot in the past years, both technologically and community-wise, but one of the challenges we faced was finding a way to organize the software development process.

After trying a couple of different techniques, we have now settled with an Agile framework that fulfills our needs and ensures that we deliver good software regularly.

Since one of the key aspects of open source is that everything needs to be shared with the community, I thought it would be a good idea to document how we do things here.

And if you are also part of an open-source project, please share in the comments how you organize your work and if you have any feedback on our way of doing things!

What’s the difference between open-source and “normal” projects?

Based on my experience at Keptn, I learned that there are three specific aspects that can be challenging when managing an open-source project. These are:

• Results, code, processes need to be shared with the community
• Changing team of developers
• Difficult to rely on deadlines and estimates

Let’s look further into these points.

Sharing with the community

Coming from a closed-source project, it was challenging to get used to sharing everything about Keptn publicly. Companies are infamous for being very secretive and this is especially true for the software business, as great code and algorithms give them the edge on the market.

In open source, the public aspect is not limited to the code. We also have open meetings, discussing future releases, new features, and the Keptn roadmap publicly.

But we see this as a strength more than a drawback. Being open with your project means that you also open yourself to feedback from contributors that are outside of your company, meaning that you get new ideas that you might not have got by staying in your bubble. Conversely, we contribute to the evolution of software development with our own software code, that is available for anybody to use.

Changing team of developers & difficult to rely on deadlines

A large part of the development of Keptn is done by our amazing contributors, who decide to join the project and take on tasks to make this product successful. It’s fantastic to have volunteers who want to contribute to Keptn, but at the same time we face the challenge of a fluctuation of people working on the software. Whilst some have been working with us for a long time, many only implement one task before moving on to another project.

This also brings us to the next point: it’s hard to rely on only contributors because you cannot be sure that things will be delivered by a specific deadline. Since, and rightly so, Keptn is not their priority or full-time job.

In a software development company, you would usually have a team of developers working on a project, on which you can rely to achieve deadlines. And the team members would most likely stay the same for long periods of time. This helps give stability to the project.

In open source, it is not always possible to have this stability.

How we face these challenges at Keptn

When we started the project at first, both internal developers and contributors would work together with a shared Kanban board. Based on milestones, I pre-selected which items should be done first and what later, and things worked pretty well from the point of view of development.

However, as I was responsible for managing releases, it made it difficult for me to make a “release cut”. When you work in a Kanban mode, it isn’t always possible to predict what will be done for which date. So, it was difficult for me to predict when a milestone-based release could be finished.

Since contributors were working on very important features too, we often had the problem that some tasks were unfinished for long periods of time. And since it’s voluntary work, nobody can impose a deadline on contributors.

So, after some deliberation, we decided to split our developers into two groups:

1. A core team

2. And a contributor “team”

This split allows us to compensate the fluctuations that are typical of an open-source project, with the stability of a “corporate” software development project. Although the two teams work in different ways, they are not walled off from each other. The core team members and the contributors collaborate continuously, and contributors are encouraged to participate in all discussions. The main difference is the way work is organized: core team works in a sprint cadence, a modified type of Scrum, whilst contributors work in a Kanban way.

Let’s look into this in more detail.

The Core Team’s process

For organizational purposes, our core team is also split into two parts: we currently have 3 people working in the Core (GoLang) and 3 people working on the Bridge, the end-user interface (Angular).

The core team works in 2-week sprints and has releases every 4 weeks.

We start the sprint with a Sprint Planning on Friday afternoon and finish the sprint on the second Thursday with a Sprint Review. In the Sprint Planning, the team plans what their availability is for the next two weeks and how many stories they can accomplish. At the Sprint Review, the team shows what has been accomplished in the past sprint (which is an open event that anybody can attend).

The core team has its own sprint board (see this one for Sprint 219 as an example) and a new board is created for each sprint.

Since we have a strict schedule of sprints and releases, it’s much easier now to cut new releases. And it helps the team stay more focused as well, because they know exactly what needs to be done for which date.

Every Tuesday we have a Backlog Refinement meeting, where the Product Owner presents new user stories to the development team. The team can ask for clarifications to optimize the user stories for a more efficient implementation. You can view the current refinement board here: Keptn Refinement Board.

This meeting is crucial to ensure that only user stories that are ready are accepted into the sprint. Developers then can concentrate their efforts on the implementation, without losing time asking for clarifications.

The contributors’ process

For contributors, we created a Kanban Working Items board, which makes it easy to know what tasks you can take and what is already being worked on. Even here, user stories are refined by the Product Owner before they are moved to this board.

If anybody wants to contribute, they just need to leave a note on an issue saying they want to work on it, and then it gets assigned to them by us. If there are any further questions or discussions, contributors can contact us on our Slack channel or write directly in the issue’s comments.

Having a different board for contributors enables us to make a split between items that are higher priority and need to be finished by a certain date, and items that have no strict deadline. If something is finished in time for a release, it will be merged together with the new release candidate.

We are open to anybody who wants to contribute — the more, the merrier! There are no requirements, besides being a team player and presenting your work at the Community and Developer Meeting, which takes place every second Thursday on alternate weeks to the Sprint Review meeting.

Agile project management in Open Source

Splitting up the work between two teams, one working in sprints and one working by priority, allowed us to become more efficient and able to plan releases in advance. It seems to be the best way for us to organize our work at the moment — but this does not mean that we won’t update and adapt in the future! And it’s important to note that the teams don’t work in silos. We value open communication and collaboration at Keptn. Plus, there is no difference between a core team member and a contributor, just the way the work is organized.

And as mentioned before, we would love to know how others organize their work in open-source projects! So, leave a comment if you’d like to share how you do things.

If you are interested in joining the Keptn community, check out our website, our GitHub, and our Slack channel. We are always looking for motivated contributors who love working in great teams!

Implementing Agile project management in an open-source project was originally published in Dynatrace Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.