Develocity Maven Extension User Manual

The easiest way to configure a build environment to authenticate with Develocity is to use the following goal:

When executed, it opens your web browser and asks to confirm provisioning of a new access key. You will be asked to sign in to Develocity in your browser first if you are not already signed in.

When confirmed, a new access key will be generated and stored in the .develocity/keys.properties file within the Maven user home directory (~/.m2 by default).

Any existing access key for the same server will be replaced in the file, but will not be revoked at the server for use elsewhere. To revoke old access keys, sign in to Develocity and access “My settings” via the user menu at the top right of the page.

Access keys can also be configured manually for an environment, when automated provisioning is not suitable.

Develocity access keys are long-lived, creating risks if they are leaked. To avoid this, users can use short-lived access tokens to authenticate with Develocity. Access tokens can be used wherever an access key would be used. Access tokens are only valid for the Develocity instance that created them.

To create an access token:

The requested permissions and project ids can be specified as comma-seperated lists or repeated parameters. For example, ?projectIds=a,b&projectIds=c is valid and will request projects a, b, and c.

If the user creating the token does not have one of the requested permissions or projects, Develocity will respond with a 403 Forbidden error. If an access token is used to authenticate the creation request, its permissions and projects will be used for this check instead of the user’s. The request will also error if the requested lifetime would cause the new access token to expire after the one used to authenticate the request. Together, this means you cannot create an access token with more access or a later expiration than the credentials used to authenticate the request.

Here is an example using CURL to create an access token:

The resulting token would have the following permissions:

And it would have access to these projects:

The token would only be usable for one hour.

When you publish Build Scans to a Develocity instance, you must configure the location of the Develocity server.

The precise URL you need depends on the hostname that your Develocity instance has been configured with. If in doubt, be sure to ask whomever manages that instance.

You may encounter a warning about an untrusted certificate when connecting to Develocity over HTTPS. The ideal solution is for someone to add a valid SSL certificate to the Develocity instance, but we recognise that you may not be able to do that. In this case, set the allowUntrustedServer option to true:

In order to publish to scans.gradle.com, you need to accept the terms of use.

Be sure to check the terms of use at the URL shown in the above fragment.

Once you have accepted the terms of use, you can start publishing Build Scans to scans.gradle.com.

The Gradle Jenkins plugin prominently displays links to the Build Scan for any Gradle or Maven build that produce Build Scans, which makes it convenient to access the Build Scans. See the following screenshot for an example:

Besides that, the plugin allows you to instrument all CI jobs and to publish a Build Scan without modifying the underlying projects. This eases the adoption of Build Scans within the company and allows configuring the Develocity Gradle plugin or Develocity Maven extension in a central place. For more information please read the documentation here.

The TeamCity Build Scan plugin prominently displays links to the Build Scan for any Gradle or Maven build that produce Build Scans, which makes it convenient to access the Build Scans. See the following screenshot for an example:

Besides that, the plugin allows you to instrument all CI jobs and to publish a Build Scan without modifying the underlying projects. This eases the adoption of Build Scans within the company and allows configuring the Develocity Gradle plugin or Develocity Maven extension in a central place. For more information please read the documentation here.

The Develocity Bamboo plugin prominently displays links to the Build Scan for any Gradle or Maven build that produce Build Scans, which makes it convenient to access the Build Scans. See the following screenshot for an example:

Besides that, the plugin allows you to instrument all CI jobs and to publish a Build Scan without modifying the underlying projects. This eases the adoption of Build Scans within the company and allows configuring the Develocity Gradle plugin or Develocity Maven extension in a central place. For more information please read the documentation here.

The Develocity GitLab templates allow you to instrument CI jobs and to publish a Build Scan without modifying the project’s build scripts. This eases the adoption of Build Scans within the company and allows configuring the Develocity Gradle plugin or Develocity Maven extension at the CI level. For more information please read the documentation here.

This is the default. There are many advantages to publishing Build Scans regularly, such as being able to track the behavior and performance of a build over time. It makes no sense relying on ad-hoc publishing of scans in such a situation as it’s easy to forget on the command line. Should you decide to explicitly enforce this default option, you can do this as follows:

This approach means that you get a Build Scan for every successful and failed build that runs, including from your continuous integration infrastructure and your developers.

If you want to deactivate Build Scans for a particular build, you can pass the -Dscan=false system property to Maven.

We imagine that when you first start experimenting with Build Scans, you won’t want to publish them all the time until you become familiar with the implications. Even then, you may have good reason not to go all-in and automate the process. That’s where one-off Build Scans come in.

If you only want to publish Build Scans when explicitly requested, use the following option:

You can also publish a Build Scan for the most recently run build by invoking the build-scan-publish-previous goal.

Many of you will want a bit more control over exactly when Build Scans are published without resorting to using -Dscan each time. You may want to publish Build Scans only under the following conditions:

Such scenarios are covered by configuring a custom predicate.

Capturing file fingerprints increases the amount of data transmitted to the Build Scan server at the end of the build. If the network connection to the Build Scan server is poor, it may increase the time required to transmit. Additionally, it may also increase the data storage requirements for the Build Scan server. In such cases, capturing file fingerprints can be disabled.

However, if you are using Develocity and utilising its Build Cache to accelerate your builds, it is strongly recommended to keep it enabled as identifying which files have changed between builds with build comparison is extremely effective for diagnosing unexpected Build Cache misses.

If you are using Develocity for Predictive Test Selection enabling capture of file fingerprints is a prerequisite.

File fingerprint capture can be enabled/disabled programmatically, via a system property, or via the develocity.xml configuration file.

You may want to skip capturing build or test logging for security/privacy reasons (i.e., some logging may leak sensitive data), or performance/storage reasons (i.e., some goals/tests may produce a lot of logging that are irrelevant for your usage of Build Scans).

Logging capture can be enabled/disabled programmatically, via a system property, or via the develocity.xml configuration file.

Tags are typically used to indicate the type or class of a build, or a key characteristic. They are prominent in the user interface and quickly inform a user about the nature of a build. A build can have zero or more tags.

Note that the order in which you declare the tags doesn’t affect the Build Scan view. They are displayed in alphabetical order, with any all-caps labels displayed before the rest.

You can see the effect of a custom tag in figure 2.

Builds rarely live in isolation. Where does the project source live? Is there online documentation for the project? Where can you find the project’s issue tracker? If these exist and have a URL, you can add them to the Build Scan.

Links can also be added in your project POM.

The <name> is simply a string identifier that you choose and that means something to you.

You can see the effect of a custom link in figure 2, which shows how a label for a given VCS (here GitHub) becomes a hyperlink that anyone viewing the Build Scan can follow.

Some information just isn’t useful without context. What does "1G" mean? You might guess that it represents 1 gigabyte, but of what? It’s only when you attach the label "Max heap size for build" that it makes sense. The same applies to git commit IDs, for example, which could be interpreted as some other checksum without a suitable label.

Custom values are designed for these cases that require context. They’re standard key-value pairs, in which the key is a string label of your choosing and the values are also strings, often evaluated from the build environment.

As with tags, you can filter Build Scans by custom values in Develocity.

What if you want to execute some code based on data that is only available late in the build? For example, you might want to label a build as "built-from-clean" if the clean goal was run. But you don’t know if that’s the case until the goal execution plan is ready.

The Maven extension provides a buildFinished() hook that you can use for these situations. It defers attaching custom data until the build has finished running. As an example, imagine you want to report how much disk space was taken up by the output directory. The build doesn’t know this until it’s finished, so the solution is to calculate the disk space and attach it to a custom value in the buildFinished() hook:

The buildFinished() action has access to a BuildResult instance that you can use to determine whether the build failed or not, like so:

You might want to perform a custom operation when a Build Scan is published, like notifying an internal tool of your company. To do this , you can use the BuildScanApi#buildScanPublished() method:

Some data that you may wish to add to your Build Scan can be expensive to capture. For example, capturing the Git commit ID may require executing the git command as an external process, which is expensive. To do this without slowing your build down, you can use the BuildScanApi#background() method:

This method takes a function that will be executed on a separate thread, which allows Maven to continue without waiting for the expensive work to complete.

All background work will be completed before finishing the build and publishing the Build Scan.

Any errors that are thrown by the background action will be logged and captured in the Build Scan.

See the BuildScanApi#background() API reference for more information.

You may want to ensure that a given operation is only executed once for the whole execution of a multi-project Maven build.

The Maven extension provides an executeOnce() hook that you can use for these situations. It must be provided with an identifier, and can call any service provided by the API. The identifier is used to guarantee that the provided action will be executed at most once.

Build Scans capture certain identifying information such as the operating system username, hostname, network addresses, and CPU-intensive process names. You may choose to obfuscate this data so that it is not decipherable in Build Scans when viewed. To do this, you can use the BuildScanApi#obfuscation() method.

With Maven extension v1.6.3+, you can register obfuscation values on the develocity.xml. These will always be applied, even in case of very early build failures, that would prevent programmatic configuration via the BuildScan API. If the BuildScan API gets eventually called, obfuscation functions registered with it will have precedence over the values defined via XML.

The following examples show registering obfuscation functions for the different identifying data.

See the BuildScanApi#obfuscation() API reference for more information.

Every goal, test suite and test case has an outcome. For a goal, the outcome is either SUCCESS, indicating that the goal was successful, FLAKY, which means that at least one test suite was flaky, or FAILED, if a test suite has failed.

For test suites, the outcome can be SUCCESS, meaning that all executed test cases were successful, FAILED, indicating that at least one test case failed, FLAKY, meaning that at least one test case was successful only after retry, or SKIPPED, meaning that the test suite was not executed.

For test cases, the outcome can be SKIPPED, meaning that the test case was not executed, SUCCESS, meaning that the test case was successful, FAILED indicating that the test case was failed, or FLAKY, meaning that the test case was successful only after retry.

On a goal, the total time corresponds to the total amount of wall clock time the goal took to execute.

On a test suite, the total time corresponds to the total amount of wall clock time the test suite took to execute.

On a test case, the total time corresponds to the total amount of wall clock time the test case took to execute. Clicking on a test goal will open the details for this test goal, which will display the results of the test suites which ran as part of this test goal.

When using background Build Scan uploading (default behaviour since Maven extension version 1.5, see this section for configuration options) upload failures are not visible in the build logging due to occurring in a background process after the build has finished. Instead, errors are logged to a file located at ~/.m2/.develocity/build-scan-data/upload-failure.log. If this additional information does not help to resolve the failure, please contact technical support and include the contents of this log file.

If the background upload process fails to start, a warning is shown in the build console and uploading is performed in the build process. If this occurs, please contact technical support with the log files located at ~/.m2/.develocity/build-scan-data/<<extension-version>>/pending-uploads/*.log.

Build Scans attempt to determine the host name of the machine. An issue affecting macOS can cause a delay when doing this in some environments.

If you see a warning during your build that resolving the local host name is slow, you can workaround the problem by adding a host name mapping to your /etc/hosts file.

Add these lines to your /etc/hosts file, substituting your computer name for 'mbpro' in the below snippet:

Several Maven plugins fork a new Maven build during the execution of a build. When the project is configured to publish a Build Scan, the forked Maven build will publish one as well. Depending on the use case, this behavior is desirable or not. The Develocity Maven extension does not detect if it’s executed within a forked Maven build to avoid publishing a Build Scan. In some circumstances and specific setups it can lead to errors on the forked Maven build and we then advise you to disable the Build Scan functionality of the Develocity Maven extension for such builds. This does not affect the use of the Build Cache.

The following sections show how to disable publishing a Build Scan for various Maven plugins that spawn forked Maven builds.

The extension uses a local Build Cache to store build outputs in the local filesystem. It prevents network round-trips by storing both outputs that local builds created and outputs that were downloaded from the remote Build Cache.

Develocity provides a Build Cache node that is built into the server. Additionally, remote Build Cache nodes can be spun up and connected to the server. By default, the built-in Build Cache node of the Develocity server is used.

The following snippet shows you how to ignore any file called META-INF/build.properties on any runtime classpath in the given project. You can share this setting across many projects by putting it in the pluginManagement section of your parent POM. You can use ANT-style patterns like META-INF/**/*.sql as well.

Since properties files are a common file format to store data generated by the build, the normalization DSL provides special support for ignoring specific entries in properties files. Again, ANT-style patterns can be used to match relevant properties files. In the following example any files named build.properties in com/example or any subdirectory are matched and the value of the property build.timestamp is ignored:

A common location to store properties files is the META-INF directory. Thus, the normalization DSL provides a shortcut for normalizing files matching META-INF/**/*.properties

The <metaInf> configuration element offers even more convenience configuration settings for normalizing the contents of the META-INF directory. For example, you may only want to ignore one or more attributes in MANIFEST files, e.g. Implementation-Version instead of ignoring the whole file. This can be done as follows:

If you want to ignore MANIFEST files completely there’s the <ignoreManifest> configuration as a shorthand for that. It’s the equivalent of adding META-INF/MANIFEST.MF as an ignored file.

In situations where you have several changing files in the META-INF directory, you might want to ignore the contents of that directory completely. The <ignoreCompletely> configuration is a shorthand for adding META-INF/** to ignored files.

The normalization provides a way to ignore certain system properties by their key

The predefined inputs and outputs of any supported goal can be augmented by declaring them in the <pluginManagement> section of your pom.xml file. A common use case for declaring additional input files are test cases that read from a location that is not automatically tracked. For example, you might have Cucumber specification files located in the src/test/specs directory. Any change to the files in that directory may change the result of running the tests. So changing specification files should result in rerunning the tests and not loading the results from the cache. Without additional configuration the Build Cache is unaware of these additional inputs and will therefore load the result from the Build Cache even if specification files change.

You can disable caching on a fine-grained level in the <pluginManagement> section of your pom.xml file. The following will disable caching for all executions of the failsafe plugin in the given project:

You can also disable caching for a specific execution. Other executions of that plugin will then still remain cacheable. The following will disable caching only for the systems-integration-test execution of the failsafe plugin. Other tests will remain cacheable.

By default, outputs are stored in the Build Cache if it is enabled and the build includes the clean lifecycle phase. Disabling storing outputs of a specific goal can be achieved by setting the storeEnabled option to false on the outputs configuration of the goal’s plugin or execution.

An example use case is when local development has the side effect of corrupting outputs of a goal (e.g. when the IDE and Maven use the same compilation output directory). Without disabling storing outputs to the Build Cache, this can lead to issues with corrupt cache entries being loaded which make downstream goals fail (to resolve such a situation you can use the -DrerunGoals command line argument). To avoid this situation, the above sample configuration for the compile goal disables storing its outputs in a local-only enabled Maven profile. On CI, outputs will still be stored in the Build Cache so that local developer builds are able to reuse cache entries from prior CI builds.

The following example shows how to configure the local profile to disable storing outputs generated by the compile goal. To check if the build is executed locally we check that the environment variable CI is not set.

You can conditionally skip a goal execution by configuring the skipIfTrue option with one or several Mojo properties of type boolean. If the value of any configured property is true, then the goal execution will be skipped.

An example use case is a goal that supports skipping of its execution via setting a specific boolean property. Even if a goal execution is skipped via such a property, the Build Cache will still attempt to retrieve outputs from a previous execution. In order to avoid unnecessary computation and network overhead, the name of the specific boolean property can be declared via skipIfTrue.

All the supported goals are already configured to skip the Build Cache if the goal is skipped.

The extension allows you to make any goal cacheable, beyond the ones that are supported out of the box. Take great care to define all of the goal’s inputs and outputs before doing so, to avoid false cache hits and follow-up errors.

When making goals cacheable, you don’t need to repeat the values of all their inputs and outputs. You can simply provide the name of each property and the extension will look up the value in the goal’s configuration. The extension will make sure that you have handled all configuration parameters of the goal in this way. If some parameter is irrelevant for the purposes of caching, e.g. because it only affects console output, you can tell the extension to ignore it.

For input properties, the extension supports all primitives, Strings, Enums and Collections, Arrays and Maps of those. Any other types need to be broken down using the nestedProperties (for a single complex type) or iteratedProperties (for a Collection of complex types or a Map of complex values) configuration.

The extension provides several loggers to make analyzing problems with build caching easier. To show the effective Build Cache configuration, use the develocity.goal.cache logger:

The develocity.goal.cache logger will also print the result of determining the cacheability of the executed goals:

Sometimes you might encounter a situation where a goal execution is not avoided by using the Build Cache although you would expect it to be. For example, if you run the same build twice without any changes, the outputs of all supported goals should be retrieved from the local Build Cache (if it is enabled). If this is not the case, this almost always is caused by unstable inputs, e.g. a timestamp being added to a file by some build logic. In order to identify which inputs change between builds the Maven build comparison feature can be used. Simply run the same Maven build twice and once the Build Scans have been published, they can be compared in Develocity. In this example, the build was configured to write a timestamp to the build.properties file. When comparing the two builds this shows up nicely in the comparison.

Once the changing input is identified, the build can be changed to be reproducable or normalization can be used to ignore the changing input.

Some widely used Maven plugins are a common cause of cache misses because they produce changing build results. This chapter shows you how to solve them.

When there is a pom.xml configuration for the Develocity Maven extension on a parent pom.xml, such as runtime classpath normalization or declaring inputs and outputs for given plugins, and another configuration on a child pom.xml, configurations may merge incorrectly, resulting in a misconfiguration of the Develocity Maven extension.To see the effective pom.xml that is used during the execution of the build on any given project mvn help:effective-pom can be executed, and the effective pom.xml is printed to the console. If a check of the Develocity Maven extension configuration shows that it’s invalid or not properly merged, then a different merge strategy can be applied as mentioned in this blog post.

In this example, we have a parent project and a child project which both have specific Develocity configurations for the compiler and Surefire mojos. The following excerpt is from the parent pom.xml:

This excerpt is from the child pom.xml where we want to keep the configuration of the compiler mojo defined on the parent pom.xml and add another property as input to the Surefire mojo:

Due to the merging strategies available for Maven POM files there are several pitfalls:

With the above setup the merged child POM produced by retrieving the effective POM via executing mvn help:effective-pom resulted in the following merged child POM:

In rare circumstances the Build Cache might be filled with an invalid entry, e.g. when another process deletes the outputs of a goal while the cache entry is being created. In this case you can use the -DrerunGoals command line argument to rerun the goals and overwrite the faulty cache entry.

The develocity.xml configuration file supports two types of expressions that are evaluated when reading the configuration file:

For both expression types, the following objects can be referenced:

In addition, the following objects and functions can be referenced in SpEL expressions:

The following example shows how to use both expression types to configure local and CI builds with a single develocity.xml file. It uses the JENKINS_URL environment variable (which is present in builds on Jenkins) to determine whether the build is running locally or on CI. Based on that, it enables the local Build Cache and background Build Scan upload only for local builds but enables writing to the remote Build Cache only for CI builds. Moreover, it determines the URL of the remote Build Cache based on the fictional REGION environment variable. Lastly, it uses Maven-style expressions to configure the remote Build Cache credentials based on custom environment variables that are typically injected by the CI server.

Unless there are annotation processors on the classpath, the extension uses compile avoidance so your sources are only recompiled if the signatures of the classes on the compile classpath have changed.