Build Matrix for Multi-step Pipelines

We've added Build Matrix so you can create multiple jobs from a single, multi-variant step definition. 🧮

Build Matrices can also include multiple dimensions, different combinations with the adjustments key, and matrix jobs can be grouped together to clean up build pages. ⚡️

Read more about it in the blog, documentation, or on Twitter.

Libby

Group steps with the `group` config option

We've added a new group pipeline step type so you can group steps together, set dependencies between them, reduce YAML repetition, and clean up your build pages 🏗✨

You can read more about it in the blog post, documentation, or on Twitter.

Libby

Datadog CI Visibility Integration

For teams using Datadog, we've recently made it easier to send information about your Buildkite pipelines to Datadog’s Continuous Integration Visibility. This is a simple integration that any organization using both Datadog and Buildkite can enable to get insights into their pipeline’s performance over time. 📈

For more details on the integration check out the documentation 📚

Libby

Elastic CI Stack for EC2 Mac

For teams that run Buildkite Pipelines and build Xcode based software projects for macOS, iOS, iPadOS, tvOS, and watchOS apps, you can now run your Buildkite Builds on AWS EC2 Mac instances using a CloudFormation template. 🎉

This experimental template creates an Auto Scaling group, launch template, and host resource group to launch a pool of EC2 Mac instances that run the Buildkite Agent.

📣 Big shoutout to Buildkite customer Oliver Koo for his early input into this feature 🙏

📚 For details on how to prepare and deploy this template to your AWS Account, checkout the Auto Scaling EC2 Mac documentation, or jump straight into Elastic CI Stack for EC2 Mac.

Libby

Test Analytics private beta

Identify, track, and fix problematic tests with ✨Test Analytics✨, now in private beta.

test-analytics_cover.png

🔍 Identify flaky tests: See which tests are most disruptive with automatic flaky test identification and analysis over time.

👀 Monitor speed & reliability: Get alerts when slow tests are introduced.

📈 Deeply analyze performance: First-class framework integrations provide in-depth performance analysis, helping you find external dependency bottlenecks and more.

🏎 View real-time results: Watch your test suite run in real-time and get immediate performance insights.

Learn more and join our waitlist at https://buildkite.com/test-analytics.

Michelle

Job Priority Attribute for Pipeline Steps

For teams running a lot of jobs who want more control over what order their jobs run in, we've introduced the Job Priority attribute to specify its priority within its queue. 🙋🏻‍♀️

It's a new attribute defined on a step in the pipeline configuration that changes how the dispatcher assigns work to agents. The higher the value you set for priority, the sooner a job will be prioritised. 🔝

Our recently released Eager Concurrency feature can also be used in conjunction with Job Priority.

Libby

SCIM deprovisioning support for Okta and Azure AD

Enterprise teams using Okta or Azure AD for SSO with their Buildkite organization can now optionally enable SCIM to automatically deprovision users ⚡️

image.png

Okta and Azure AD SSO with SAML is still available to all users.

A step-by-step guide to enabling SCIM support is available in our Single Sign-On with Okta and Azure AD documentation 📚

Harriet

Eager Concurrency

Traditionally, concurrency groups enforce strict ordering of the jobs within them. Concurrent jobs will be limited to the number set for the group, and will only start executing in the order they were created. 📝

However, sometimes you only need the limit, and it's fine for eligible jobs to run in whatever order their dependencies allow. 🏃🏻‍♀️

Which is why we've added a concurrency_method step attribute you can set to 'eager'. This will allow any job, up to the concurrency limit, to start as soon as it's eligible, regardless of creation order. 🔀

As an example:

1
2
3
4
5
steps:
  - command: echo "Using a limited resource, only 10 at a time, but we don't care about order"
    concurrency_group: saucelabs
    concurrency: 10
    concurrency_method: eager

For more information, check out our guide to Controlling Concurrency.

Libby

One-shot agents with the acquire-job flag

For teams running their own compute scheduler or those needing single-use agents, we've added the --acquire-job flag to the Buildkite Agent 🧰

With this flag, agents are mapped 1:1 with jobs. This makes them perfect for folks using a system like Kubernetes, Nomad, or ECS, or anyone who wants a clean-room environment.

acquire-job.gif

To get started, pass the --acquire-job flag with the job ID to the buildkite agent start command. When you start an agent with this flag it will run the job then exit rather than polling for more work.

You can use acquire-job with agents v3.17.0 and above, anywhere that the agent can run 🏃🏻✨

For details on how to use the flag, see the agent cli documentation on Running a single job.

Harriet

HMAC signed webhooks

For teams wanting to add an extra layer of security to their webhooks, we have just added the option to include HMAC signatures with your Webhook Notification services 🔐

You'll find the HMAC signature option under the Token section when editing or creating Webhook Notifications.

image.png

For more information on verifying webhook signatures and defending against replay attacks, see our Webhook API documentation.

Harriet

Conditional webhooks

For complete control over when to trigger builds, use conditional expressions to filter incoming webhooks 📥

image.png

Build filtering with conditionals is available with any repository provider. You can add a conditional on your Pipeline Settings page or via the REST API 🔀

For more information on supported conditionals, check out our guide to Using Conditionals.

Harriet

PagerDuty Change Events Integration

Send Change Events to your PagerDuty services whenever a build completes 🚨

Filter Change Events notifications with conditionals to make sure you're sending only the information your team needs

image.png

The Buildkite integration can be installed from the PagerDuty Service Directory 📚

Find the step-by-step guide to sending PagerDuty change events in our PagerDuty integration guide.

Harriet

Announcing the Buildkite Newsletter

We've had a lot to talk about at Buildkite lately! We relaunched our blog, we've been updating and revamping our documentation, and have a ton of upcoming projects, customer stories, and tutorials we can't wait to share.

If that sounds like a lot to keep track of, we've got you covered! We're launching a 📰 newsletter 📰 to provide curated highlights, so you can stay in the know whatever your schedule ✨

Sign up here and you'll get a handy digest of our favorite features, updates, and blog posts delivered directly to your inbox 🗞📬🚀

For the latest news as it happens, this changelog will remain your one-stop source for Buildkite updates. You can also check out the blog and our docs directly to stay up to date on all things Buildkite 💚

Nick

Keep your secrets safer

Nobody likes accidentally leaking secrets in a build log. The Buildkite agent can now redact secrets present in your environment variables before they are uploaded to your logs 👷🏻

redacted-log.png

Turn it on for your agents, and add your custom variable patterns.

Thanks to Buildkite customer David Baggerman for doing the initial work on this feature 💚

Sam

New in the Buildkite Agent: Remove a build annotation

With the release of v3.28.1 of the Buildkite Agent, we've introduced a new command to remove existing annotations from a build 📝🚫

buildkite-agent annotation remove removes an annotation entirely, where previously you could only blank their content by passing an empty string to buildkite-agent annotate.

This allows for some creative uses like showing bits of context while builds, tests, deployments or lints run, and removing them from view when they're no longer relevant.

We're looking forward to hearing about what you make with the extra annotation abilities 😄🎉

Jessica

Datadog tracing for CI builds

For teams needing detailed observability in their CI/CD pipelines, we've recently released support for configuring Datadog as a tracing backend on the Buildkite agent to help monitor build and job metrics 🕵🏻

Huge thanks to our open source collaborators 💚

image.png

To get started, install the Datadog Agent on your Buildkite Agent machine(s) and configure the following environment variable: BUILDKITE_TRACING_BACKEND="datadog"

For more details, check out the agent configuration docs.

Chloe

Official Buildkite Terraform provider :terraform:

Folks using Terraform to manage their infrastructure, great news! We've published an official Buildkite provider to the Terraform Registry :terraform:

image.png

For details of the configuration syntax and supported resources, check out the official docs on the Terraform Registry.

A big thanks to Buildkite customer Jarryd Tilbrook who developed the initial version of the provider and was happy for us to pick it up 💚

James

Introducing early access to Buildkite’s Managed macOS CI

Update, June 2021: We are no longer offering Buildkite-hosted macOS agents. For more information on spinning up your own Buildkite agents on macOS, see this guide from Stark and Wayne or check out the docs.

image.png

We are excited to announce an exclusive program for our macOS agents managed service, providing the first scalable CI platform for Mac and iOS development 🚀

To learn more and to resister your interest, check out our introduction to Hosted Mac Agents and our blog post about the announcement 📚✨

Keith

Copy environment variables to clipboard

Those occasional times when you need to grab a copy of all the environment variables on a job, wouldn't it be great if there was a button for that? 🤔

Screen Shot 2020-10-29 at 4.36.26 pm.png

Now there is! We've added a button to the job Environment tab to copy all of your environment variables and their values straight into your clipboard 🚀

Shevaun

Archive Pipelines 🗃

If you manage a lot of pipelines you will be pleased to hear that we've added the ability to archive pipelines that are no longer in use. Archived pipelines have all their historical data preserved, but won't run new builds, and are not shown on the Pipelines page by default. You'll find a list of archived pipelines in the Team Selector:

View archived pipelines.png

Archive pipelines from the Settings page, under Pipeline Management, or using the GraphQL API.

Archive Pipelines.png

Happy Archiving!

Juanito

Start turning complexity into an advantage

Create an account to get started with a 30-day free trial. No credit card required.

Buildkite Pipelines

Platform

  1. Pipelines
  2. Pipeline templates
  3. Public pipelines
  4. Test Engine
  5. Package Registries
  6. Mobile Delivery Cloud
  7. Pricing

Hosting options

  1. Self-hosted agents
  2. Mac hosted agents
  3. Linux hosted agents

Resources

  1. Docs
  2. Blog
  3. Changelog
  4. Webinars
  5. Plugins
  6. Case studies
  7. Events

Company

  1. About
  2. Careers
  3. Press
  4. Brand assets
  5. Contact

Solutions

  1. Replace Jenkins
  2. Workflows for AI/ML
  3. Testing at Scale

Support

  1. System status
  2. Forum