Buildkite or Jenkins: Choosing the right tool for you

So you've been running Jenkins for a while, and it started great—a simple setup with elegant pipeline definitions and a thriving ecosystem of plugins at your fingertips—but things slowly changed:

• You ran Jenkins as a monolith, but performance suffered, plugins clashed, and you introduced a single point of failure.
• You ran Jenkins as disconnected controllers, but governance and compliance went out the window as teams did different things and got siloed.
• You filled every gap with a plugin from the community, but they weren't maintained and introduced security holes.
• You embraced the power of a full programming language for pipeline definitions, but new team members struggled to reason about the behavior and introduced conflicting conventions.

CI/CD tools are critical in software development, so choosing the right one is important. When they go down, so does your developer productivity. You need a tool that's flexible enough to fit your needs without creating a huge maintenance burden.

In this article, we'll explain how Buildkite improves on many of Jenkins' great features by striking the right balance between flexibility, convenience, and control.

The legacy of Jenkins

It's hard to overstate Jenkins' role in launching the first generation of CI/CD tools. Jenkins' unopinionated architecture, node management features, and rich plugin ecosystem have made it the default build and deployment tool for many companies.

But Jenkins has been around for a while now. And that age shows in its outdated UI and numerous security issues (both in its core code and plugin ecosystem).

Uber ran into these limitations building its Go monorepo in Jenkins. Jenkins' limited UI made it impossible to parse log output and debug build issues. Plus, the team spent more time than it wanted solving issues with the speed, scalability, and ongoing maintenance of its Jenkins instance. Things got so bad that Uber couldn't confidently change its build pipelines without causing downstream disruptions. To solve this, they migrated to Buildkite.

Watch Uber Engineering's conference talk "Building Fast, Reliable and Scalable CI at Uber with Buildkite" to see how they quickly and confidently deploy changes at scale with Buildkite.

Buildkite’s approach

While Jenkins is a general automation engine with plugins to add additional features, Buildkite Pipelines is a product specifically aimed at CI/CD. You can think of Buildkite Pipelines like Jenkins with the Pipeline suite of plugins. To simplify it, we'll refer to Jenkins Pipeline as just Jenkins and Buildkite Pipelines as Buildkite.

At a high level, Buildkite follows a similar architecture to Jenkins:

• A central control panel that coordinates work and displays results.
  • Jenkins: A controller shown in the web UI.
  • Buildkite: The Buildkite dashboard.
• A program that executes the work it receives from the control panel.
  • Jenkins: A combination of nodes, executors, and agents.
  • Buildkite: Agents.

However, while you're responsible for scaling and operating both components in Jenkins, Buildkite manages the control panel as a SaaS offering (the Buildkite dashboard). This reduces the operational burden on your team, as Buildkite takes care of platform maintenance, updates, and availability. The Buildkite dashboard also handles monitoring tools like logs, user access, and notifications.

The program that executes work is called an agent in Buildkite. An agent is a small, reliable, and cross-platform build runner that connects your infrastructure to Buildkite. It polls Buildkite for work, runs jobs, and reports results. You can install agents on local machines, cloud servers, or other remote machines. The agent code is open-source, and you can view it on GitHub. You can also use hosted agents, which lets Buildkite handle the infrastructure while you work on your pipelines.

The following diagram shows the split in Buildkite between the SaaS platform and the agents running on your infrastructure.

Buildkite provides a web interface, handles integrations with third-party tools, and offers APIs and webhooks. By design, sensitive data, such as source code and secrets, are only seen by the agent executing your pipeline. Since you can run agents on your own infrastructure, cloud servers, remote machines, or use Buildkite’s hosted agents, you have full control over your sensitive data. This decoupling also provides flexibility and the ability to scale the build agents independently while Buildkite manages the coordination, scheduling, and web interface.

In Jenkins, you manage concurrency by having multiple executors within a single node. In Buildkite, you run multiple agents on a single machine or across multiple machines, which can autoscale according to your needs—or use hosted agents for a managed, autoscaling experience.

Security

Security is crucial in CI/CD, protecting sensitive information, system integrity, and compliance with industry standards. Jenkins and Buildkite have different approaches to security, which impact how you manage your CI/CD pipeline's security.

Securing a Jenkins instance requires:

• Careful configuration.
• Plugin management.
• Regular updates to address security vulnerabilities.

You must consider vulnerabilities in both the base code and plugins. Additionally, since Jenkins is a self-hosted solution, you are responsible for securing the underlying infrastructure, network, and storage. Some updates require you to take Jenkins offline to perform them, leaving your team without access to CI/CD during that period.

Buildkite's hybrid architecture, which combines a centralized SaaS platform with open-source build agents, provides a unique approach to security. Buildkite takes care of the security of the SaaS platform, including user authentication, pipeline management, and the web interface. Build agents, which can run on your infrastructure or on Buildkite, allow you to maintain control over the environment, security, and resources. This separation reduces the operational burden and, if self-hosting agents, allows you to focus on securing the environments where your code is built and tested.

Buildkite does not have or need access to your source code (except when using hosted agents). Only the build agents need access to clone your repositories. Hosting agents on your infrastructure gives you all the benefits of a SaaS platform without many of the common security concerns.

Buildkite also doesn't store your secrets. Instead, Buildkite integrates with best-in-class tools like AWS Secrets Manager and Hashicorp Vault to use directly in your pipelines.

Both Jenkins and Buildkite support multiple authentication providers and offer granular access control. However, Buildkite's SaaS platform provides a more centralized and streamlined approach to user management, making it easier to enforce security policies and manage user access across your organization.

Pipeline configuration

Like Jenkins, Buildkite lets you create pipeline definitions in the web interface or a file checked into the repository. Most people use the latter to include their pipeline definitions next to the code, managed in source control. The equivalent of a Jenkinsfile is a pipeline.yml.

Rather than the Groovy-based syntax in Jenkins, Buildkite uses a YAML-based syntax. The YAML definitions are simpler, more human-readable, and easier to understand. And you can even generate pipeline definitions at runtime with the power and flexibility of dynamic pipelines.

In Jenkins, the core description of work is a job. Jobs contain stages with steps and can trigger other jobs. You use a job to upload a Jenkinsfile from a repository. Installing the Pipeline plugin lets you describe a workflow of jobs as a pipeline.

Buildkite uses similar terms in different ways. Pipelines are the core description of work. Pipelines contain different types of steps for different tasks:

• Command step: Runs one or more shell commands on one or more agents.
• Wait step: Pauses a build until all previous jobs have completed.
• Block step: Pauses a build until unblocked.
• Input step: Collects information from a user.
• Trigger step: Creates a build on another pipeline.
• Group step: Displays a group of sub-steps as one parent step.

Triggering a pipeline creates a build, and any command steps are dispatched as jobs to run on agents. A common practice is to define a pipeline with a single step that uploads the pipeline.yml file in the code repository. The pipeline.yml contains the full pipeline definition and can be generated dynamically.

A simple pipeline looks like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
steps:
  - label: "Build"
    command: build.sh
    key: build

  - label: "Test"
    command: run-tests.sh
    key: test
    depends_on: build

  - label: "Deploy"
    command: deploy.sh
    depends_on: test

There are steps to build, test, and deploy. Each step depends on the previous completing and runs a script with the full behavior. You can build on this with more advanced features like dynamically defining the pipeline at runtime, lifecycle hooks, and passing artifacts between steps.

Plugin system

Plugins are an essential part of both Jenkins and Buildkite. They help you extend the products to further customize your CI/CD workflows.

Rather than a web-based plugin management system like Jenkins, you manage Buildkite plugins directly in pipeline definitions. That means teams can manage plugins on a pipeline level rather than a monolith approach.

For example, the following pipeline runs test.sh inside an app service container using Docker Compose:

1
2
3
4
5
steps:
  - command: test.sh
    plugins:
      - docker-compose#v4.14.0:
          run: app

The command uses the official Docker Compose plugin and is equivalent to running:

1
docker-compose run app test.sh

This makes Buildkite plugins more decentralized and allows for easier version control.

Jenkins plugins are typically developed in Java and are closely integrated with the Jenkins core, which may lead to compatibility issues when updating Jenkins or its plugins. Buildkite plugins are written in Bash and loosely coupled with Buildkite, making them more maintainable and less prone to compatibility issues.

Conclusion

Buildkite is a CI/CD tool that balances power and flexibility. Its hybrid architecture gives you security and control where you need it while reducing the burden of managing a control panel. You can also use hosted agents for a fully managed experience. The pipeline configuration and plugins are simple to read and can support complex workflows.

Migrating from Jenkins to Buildkite provides a more flexible, scalable, and secure build infrastructure for your applications.

Ready to try Buildkite and see what we're talking about? Sign up, check out the documentation, or schedule a demo.