Buildkite Test Engine speeds up builds with real-time flaky test management and intelligent test splitting. Drive accountability and get more out of your existing CI compute with performance insights and analytics.
Isolate low quality tests automatically to keep your business unblocked.
Eliminate wasteful retries and keep merge queues moving by automatically categorizing and quarantining problematic tests, including flaky test detection.
Map tests to teams and automatically assign ownership when troublesome test cases are detected.
“ Being able to deliver software to production quickly and consistently is a critical capability for Persona Identities. Flaky tests were significantly impacting our engineering productivity; Buildkite's Test Engine cut flakey test disruptions by 50% in only four weeks, which translated into substantial productivity gains.”
- Ian Chesal
- Head of Infrastructure
See which tests are slowing your builds sooner, so you can solve your biggest problems faster.
Identify and optimize slow or poor performing tests using timing data analysis from all test runs.
Group tests into parallel jobs based on timing data—using the built-in intelligent sorting or your own rules for distributing tests.
Get test insights from any CI/CD tool: Buildkite Pipelines, GitHub Actions, Jenkins, CircleCI, and more.
Gain critical test insights with a unified view.
Understand what’s running and what needs attention across your test suites, all from one place. Unit tests, integration tests, end-to-end tests—see analysis of your regression testing health in a single view.
Automate reports on the status of unreliable or troublesome tests across teams on a configurable cadence.
Automatic tracing helps you see why tests are slow or behaving differently between executions.
Got a question that’s not on our list? Want a demo? Just want to chat? Get in touch.
Flaky tests are automated tests that produce inconsistent or unreliable results, despite being run on the same code and environment. They cause frustration, decrease confidence in testing, and waste time while you investigate whether the failure is due to a genuine bug.
By default, Test Engine detects flaky tests by surfacing when the same test is run multiple times on the same commit SHA with different results. The tests might run multiple times within a single build or across different builds. Either way, they are detected as flaky if they report both passed and failed results.
Test Engine also offers several other flaky test detection configurations based on those used by top software teams.
Yes! Use our supported collectors for RSpec, Jest, Cypress, pytest, Swift, and more—or easily implement your own collector. Or, use Test Engine with JUnit XML-formatted test results (the industry standard for test result metadata).
Buildkite Test Engine uniquely identifies each test by its combination of test suite, scope, and name. These “managed tests” are used to power Test Engine's analytics, as well as to provide a stable basis for billing purposes.
We use a 90th percentile (P90) billing method for managed tests. This approach encourages continuous improvement of your testing practices while maintaining predictable costs.
To accomplish this, Buildkite measures usage by calculating the number of managed tests that have executed (run) at least once each day, and at month's end, disregard the top 10%. By disregarding the highest usage days, you can freely add new tests, experiment with different testing strategies, or temporarily increase test coverage without worrying about billing spikes.
Yes! All new organizations get a 30-day free trial of the Pro plan to try the key features. See Pricing for all the details on the plans.
Buildkite provides an SLA of 99.95% uptime and a status page to track any incidents.
Create an account to get started with a 30-day free trial. No credit card required.