---
component: ROOT
version: "2026.1"
slug: ROOT/guides/flaky-test-detection-guide
canonical_url: "https://docs.gradle.com/develocity/2026.1/guides/flaky-test-detection-guide/"
title: "Develocity Flaky Test Detection Guide"
description: "Overview of flaky test detection in Develocity."
keywords: []
status: current
---
# Develocity Flaky Test Detection Guide
Flaky, or non-deterministic, tests are a serious and prevalent problem in modern software development. An unreliable test suite with flaky tests wastes developers' time by triggering unnecessary test failure investigations that aren’t the result of their code changes, and delaying the integration of their code.
Often tests which report flaky results aren’t themselves unreliable, but caused by flawed production code or test infrastructure. Thus, it’s important to periodically identify and fix the most severe flaky tests.
Develocity provides [Test Failure Analytics](https://gradle.com/develocity/product/failure-analytics/) which gives you tools for quicker root cause analysis.
## How Flaky Test Detection Works
Develocity identifies flaky tests in a single build and across multiple builds. For a single build, a test outcome is marked as `FLAKY` if it fails and succeeds within the execution of a single Gradle task, Maven goal, Bazel target, or sbt task. When this occurs, flaky tests analysis becomes available in Build Scans and in the Develocity Tests Dashboard.
Flaky Tests Overview Shown in the Dashboard
This typically requires _retrying_ failed tests, which is an industry-standard way to identify flaky tests. When a test is retried, Build Scans capture the test output from all failed executions and the first successful execution for comparison. Develocity also detects flaky tests if no retry mechanism is set up or if the flaky test didn’t succeed within the configured retry limit. Tests that were executed and had different outcomes for the same task/goal inputs will be marked as cross-build flaky.
Example of Cross-Build Flakiness
## Flaky Test Detection Setup
Common test execution frameworks such as JUnit provide mechanisms for retrying tests, typically requiring extra code to annotate tests that are known to be flaky.
However, enabling test retry via your build doesn’t require source code changes and applies to your entire test suite. Importantly, this allows you to analyze _newly-introduced_ flaky tests in the Develocity Tests Dashboard.
One other important aspect to consider is whether to fail the build when flaky tests are encountered. Historically, retry mechanisms have allowed builds to succeed. When enabling test retry through Gradle or sbt, it’s possible to enable flaky test detection _without_ silencing flaky failures. This comes at the cost of continuing developer disruptions, however; and should be considered carefully.
### Gradle
> [!NOTE]
> The configuration examples in this section assume you are using Develocity Gradle plugin 3.17 or later. For older versions, refer to the [(Legacy) Gradle Enterprise Gradle Plugin User Manual](https://docs.gradle.com/develocity/legacy/gradle-plugin/legacy/#test_retry).
The Develocity Gradle plugin version 3.12 or above integrates the test retry functionality offered by the Test Retry Gradle plugin.
*
Kotlin
*
Groovy
**build.gradle.kts:**
```
tasks.withType().configureEach {
develocity.testRetry {
if (System.getenv().containsKey("CI")) {
maxRetries.set(3)
}
failOnPassedAfterRetry.set(true)
}
}
```
**build.gradle:**
```
tasks.named('test', Test) {
develocity.testRetry {
if (System.getenv().containsKey("CI")) {
maxRetries = 3
}
failOnPassedAfterRetry = true
}
}
```
See [test retry functionality in Develocity documentation](https://docs.gradle.com/develocity/gradle/4.4/gradle-plugin/#test_retry) to learn about all the useful features and configuration options.
When using an older version of the Develocity Gradle plugin, you can use the test retry functionality of the test retry plugin version 1.1.4 or above.
*
Kotlin
*
Groovy
**build.gradle.kts:**
```
plugins {
id("org.gradle.test-retry") version "1.6.4" (1)
}
tasks.withType().configureEach {
retry {
if (System.getenv().containsKey("CI")) {
maxRetries.set(3)
}
failOnPassedAfterRetry.set(true)
}
}
```
**build.gradle:**
```
plugins {
id('org.gradle.test-retry') version '1.6.4' (1)
}
tasks.named('test', Test) {
retry {
if (System.getenv().containsKey("CI")) {
maxRetries = 3
}
failOnPassedAfterRetry = true
}
}
```
See the [Test Retry Gradle plugin documentation](https://github.com/gradle/test-retry-gradle-plugin) and [introductory blog post](https://blog.gradle.org/gradle-flaky-test-retry-plugin) to learn about all the useful features and configuration options.
#### Compatibility
The test retry functionality provided by the Develocity Gradle plugin is available from version 3.12 onwards, and requires Gradle 5.0 or newer.
The test retry functionality is currently supported by the following frameworks (other versions are likely to work as well, but haven’t been explicitly tested):
* JUnit4: 4.13.2
* JUnit5: 5.9.2
* Spock: 2.3-groovy-3.0
* TestNG: 7.5
### Maven
The Maven Surefire and Failsafe plugins provide configuration properties which cause the test runner to retry each failing test a configured number of times.
Configuring these properties in your project causes failing tests to be rerun immediately after they fail. If a test passes and then fails, Develocity will record a `FLAKY` outcome for the test.
**pom.xml:**
```
2
2
```
> [!NOTE]
> Test retry works the same way when using [Test Distribution with the Develocity Maven Extension](https://docs.gradle.com/develocity/test-distribution/3.7/#maven_configuration).
See the [maven-surefire-plugin documentation](https://maven.apache.org/surefire/maven-surefire-plugin/test-mojo.html#rerunFailingTestsCount) and [maven-failsafe-plugin documentation](https://maven.apache.org/surefire/maven-failsafe-plugin/integration-test-mojo.html#rerunFailingTestsCount) for configuration details.
#### Compatibility
Surefire/Failsafe’s `rerunFailingTestsCount` supports the following test frameworks:
* JUnit 4.x
* JUnit 5.x (requires at least Surefire/Failsafe 3.0.0-M4)
### Bazel
Bazel provides a common [`flaky` attribute to test rules](https://docs.bazel.build/versions/main/be/common-definitions.html#common-attributes-tests) which causes Bazel to rerun failing tests up to three times, equivalent to specifying `--flaky_test_attempts=3` for test runs.
If any subsequent test execution passes after a failure, the test is marked as `FLAKY`, and the test target may succeed.
**BUILD:**
```
java_test(
name = "foo",
flaky = True
)
```
See the [Bazel user manual](https://bazel.build/docs/user-manual#flaky-test-attempts) for more information.
### sbt
The Develocity sbt plugin version 1.0 or above supports test retry.
**build.sbt:**
```
ThisBuild / develocityConfiguration ~= { previous =>
previous
.withTestRetry(
previous.testRetry
.withMaxRetries(if (sys.env.contains("CI")) 3 else 0)
.withFlakyTestPolicy(FlakyTestPolicy.Fail)
)
}
```
See the section [Using Test Retry](https://docs.gradle.com/develocity/sbt/1.4/sbt-plugin/#using_test_retry) in the Develocity sbt plugin user manual to learn about all the useful features and configuration options.
### npm
The Develocity npm agent supports detecting flaky tests for [a selection of test runners](https://docs.gradle.com/develocity/npm/4.0/npm-agent/#viewing_test_results), based on their native test retry mechanism. When enabled, failing tests will be re-executed until they pass or retry limit is reached. In case a test failure is followed by success a `FLAKY` test status will be assigned and visible in the Build Scan.
Refer to the [Jest](https://jestjs.io/docs/jest-object#jestretrytimesnumretries-options) and [Mocha](https://mochajs.org/declaring/retrying-tests/) documentation to learn about enabling test retries.
## Resources for Flaky Test Analysis
With flaky test detection enabled, you will be able to identify the most severe flaky tests and their trends using [Develocity Test Failure Analytics](https://gradle.com/develocity/product/failure-analytics/).
Here are some resources which show you how to best leverage these tools:
* [Preventing Flaky Tests from Ruining your Test Suite](https://gradle.com/blog/prevent-flaky-tests/) — More on flaky test detection, common sources of flakiness, flaky test management strategies
* [Flaky Test Management with Develocity](https://gradle.com/blog/flaky-test-management-gradle-enterprise-2020.2) — A real-world example debugging a flaky test in Gradle using Develocity
* ["Flaky Test Days" primer](https://gradle.com/blog/do-you-regularly-schedule-flaky-test-days/) — How Gradle engineering uses "Flaky Test Days" to manage flaky tests
* [Develocity API samples](https://github.com/gradle/gradle-enterprise-api-samples/) — Example how to automate flaky test management using Develocity API
* * *
If you have any questions or need any assistance contact the Develocity support team or your customer success representative.