Upbound is a fully managed control plane platform that scales up Crossplane, one of the most developer-loved Cloud-Native Computing Foundation (CNCF) projects, to manage tens of thousands of cloud resources via Kubernetes. Upbound is used by several companies, including Deutsche Bahn, Gameloft, Millenium bcp, and many more. They also happen to be the creator and maintainer of Crossplane.


Upbound’s Director of Engineering, Jean du Plessis, quickly understood that the cost of an engineer per minute is far higher than a CI minute. Therefore, self-hosting runners and paying the hidden operations costs of managing, supporting, and patching security fixes, while tempting, was never in the cards for them. They were never going to be experts at managing CI runners—nor should they be! So, they initially chose the well-trodden path of GitHub-hosted runners to start their CI journey. And for some time, that worked.

The Problem

Kubernetes pushing GitHub to the edge.

While 8-core GitHub-hosted runners initially served Upbound’s needs, limitations soon emerged. Given the nature of their product, Upbound’s engineering team runs a large Kubernetes-based test suite consisting of many interconnected services, databases, and other external dependencies. Although the GitHub-hosted runners were managed and saved their engineering team from self-hosting CI infrastructure, they were now losing valuable time waiting as their sluggish CI jobs struggled to run the Kubernetes-based test suite. Sometimes it would take up to 1 hour to finish, drastically increasing their feedback loop and delaying how quickly engineers could merge code.

Prior to Blacksmith, waiting for our CI was like waiting for paint to dry.

— Jean du Plessis, Director of Engineering, Upbound.

In general, running Kubernetes-based test suites is extraordinarily resource-intensive, and Upbound was definitely feeling the pain of running them in CI. There needed to be a better way, and Jean was determined to find one!

The Solution

Choosing Blacksmith over GitHub.

Upbound’s entire CI pipeline is built on GitHub Actions, so moving away from the GitHub Actions ecosystem was not a viable option. So the promise of Blacksmith being a drop-in replacement for GitHub-hosted runners was a very intriguing proposition for Jean. Shortly after, he spoke to his Tech Lead about Blacksmith and planned on monitoring the performance improvements of Blacksmith runners over a week-long observation period.

But before the observation period began, Jean began to notice our other software features: our colocated caching service, which is a drop-in replacement for GitHub’s cache action. This would speed up Upbound’s cache speed from 100MB/s to over 400MB/s by colocating their cache artifacts with their Blacksmith runners. Jean also took notice of Blacksmith’s CI analytics dashboard, which would provide Upbound with a single view of their CI pipeline’s performance, failure rate, and costs. Suddenly, this observation period wasn’t just about evaluating infrastructure, but also the software tooling that ran on top of it. The promise of making the end-to-end CI experience far faster, far better. So these additional features became part of the observation period as well.

After the week-long observation period was over, the results came in, and Blacksmith had outperformed GitHub in both performance, cost—and most importantly to us—experience. And while Jean had also considered other managed solutions, frankly they “just didn’t work,” Jean shared with us. So shortly after, Upbound started migrating their entire CI to Blacksmith!

Switching to Blacksmith was quite seamless.

— Jean du Plessis, Director of Engineering, Upbound.

The Result

An engineer’s time is worth its weight in gold.

Today, Upbound runs all their CI jobs on Blacksmith and enjoys the best of both worlds: zero CI operating costs and far less time waiting on CI jobs to complete than before. But rather than just taking our word for it, take a look at the raw numbers for yourself. Jean was nice enough to help out and share the before and after stats he collected on some of their CI jobs during the observation period to quantify Blacksmith’s impact on Upbound’s CI pipeline. Most notable is that two of their more time-consuming jobs—unit tests and integration tests—were completed in far less than half the time it previously took.

The amount we pay an engineer per minute is far higher than any CI minute. Blacksmith helps us save where it really counts.

— Jean du Plessis, Director of Engineering, Upbound.

Previously, CI was something to be avoided at Upbound. Now, they actually feel so comfortable with their CI that they are running more CI than ever before. So much so that they decided to bump up their runners to a larger 16-core machine since, given Blacksmith runners cost 50% less than GitHub-hosted runners, it was cost-neutral for them, making the additional performance boost essentially free.

If you’re currently dealing with slow CI, stop right now, cancel all running jobs and migrate in under 5 minutes to Blacksmith with our Migration Wizard! You won’t regret it.