Build Cache

Build Caches store outputs from Gradle tasks and Maven goal executions and reuse them for later builds, rather than building the outputs again.

Develocity provides a Build Cache service as part of each installation, and allows optionally connecting one or more remote nodes to use as discrete caches or replicas for reducing network latency.

For information on how to use the Develocity Build Cache, consult the Getting Started guides for Gradle or Maven.

Configuration

The Develocity provided Build Cache service is configured with the following defaults:

All cache access is disabled
Target storage size of 50 GiB (how much data to retain in the cache)
Maximum artifact size of 100 MiB (the largest cache artifact to accept)

To change this configuration, click Build Cache in the Administration dashboard, accessible from the upper right of the application.

Build Cache Administration Dashboard

Build Cache Storage

Develocity stores Build Cache artifacts in object storage, either in Develocity-provided embedded object storage, or in external, user-managed object storage.

See the Kubernetes Helm Chart Configuration Guide or Standalone Helm Chart Configuration Guide for object storage configuration details.

Embedded Object Storage

Embedded object storage is used by default to store Build Cache artifacts. While the Build Cache service integrated into Develocity server scales horizontally, the embedded object storage component that it uses out-of-the-box doesn’t. It’s strongly recommended to connect Develocity to an object store that’s highly scalable, such as Amazon S3, Google Cloud Platform or Microsoft Azure. Cloud-based object storage services are fault-tolerant and reduce Develocity operational overhead.

External Object Storage

Your Develocity installation should be hosted as close as possible to the external object storage service. Host your Develocity instance on AWS if you’re using Amazon S3, GCP if you’re using Google Cloud Storage, or Azure if you’re using Azure Blob Storage, etc. Within the same cloud provider’s services, it should also be hosted within the same geographic region.

Space Management

The amount of space used by the Build Cache service is controlled by the target storage size setting. The optimum target storage size value depends on how many cache entries are stored and the size of the cache entries stored. Develocity installations with an increased Build Cache maximum artifact size are likely to require more space to maximize cache effectiveness.

As new projects are created or existing projects grow, increasing the target storage size could become necessary to maintain cache hit rates.

Embedded object storage has a fixed capacity. If used, ensure that the Build Cache target storage size is less than the configured embedded object storage capacity. See the Kubernetes Helm Chart Configuration Guide or Standalone Helm Chart Configuration Guide for embedded object storage configuration details.

Build Cache Storage and Usage Observability

The Usage tab of the Build Cache administration page visualizes indicators that provide insight into cache usage patterns and whether the cache size is undersized, affecting build performance due to a too-short retention period.

Please note that these metrics apply to the logical Build Cache that’s served by the Develocity server and Develocity Edges. They don’t provide insight into Develocity Build Cache nodes.

Least Recently Used Cache Entry Age

For least recently used cache entries, visualizes how long since the entry was last used.

Specifically, two metrics are charted:

The time since last access of the least recently used entry (among all entries) stored in the cache ("Oldest").
The median time since last access of entries stored in the cache ("Median").

It’s optimal for the longest time since the least recently used entry was last used to be equal to the typical lifetime of feature branches in your organization’s source control.

A Graph Showing the Oldest and Median Time Since Last Access of Cache Entries

Cache Misses for Recently Evicted Entries

The rate of cache misses for recently evicted entries.

If entries are frequently attempted to be used soon after eviction, i.e., the trend is above zero more often than not, it’s a sign that the cache is churning.

A Graph Showing the Rate of Cache Misses for Recently Evicted Entries

Storage Size

The current size of the cache and the storage size required to achieve a seven-day retention of cache entries.

Develocity is conservative in recommending increases to the storage size. A recommendation is emitted only if there is indication that an increase is necessary over a longer period of time, rather than at an instant.

Note that Develocity can’t recommend a storage size decrease in case Build Cache storage is over-provisioned.

A Graph Showing Cache Size and Storage Needed for a Seven-Day Retention Period

Incoming Data Storage Rate

The rate at which data is incoming to the cache.

A Graph Showing the Rate at Which Data Is Incoming to the Cache

Access Control

By default, access to the Build Cache is disabled. There are two access control mechanisms that can be used. Which one you should use depends on the version of the Develocity Gradle plugin or Develocity Maven extension you will be using.

Version 3.11+ of the Develocity Gradle plugin and version 1.15+ of the Develocity Maven extension can authenticate to the Build Cache with Develocity access keys. Permissions for these users are managed via the same access control mechanism used for other functions. Users with the “Build Cache read” permission can read data from any connected Build Cache, while users with the “Build Cache write” permission can read and write data from any connected Build Cache.

How to configure your builds to supply credentials is described in the Develocity Gradle Plugin User Manual for Gradle, and in the Develocity Maven Extension User Manual for Maven.

Anonymous access control can also be configured using the same anonymous access control mechanism used for other functions.

Remote Cache Nodes

Remote cache nodes can be installed separately from Develocity. They can be used to separate cache artifacts, distribute load, or improve build performance by having a better network link between the build and the node.

By connecting remote cache nodes to a Develocity instance, you are able to configure them centrally from Develocity, and have them replicate entries from other nodes in the network.

The Develocity Edge User Manual covers how to deploy Develocity services like Build Cache closer to where builds are run.

Connecting With Develocity

To connect a remote node to your Develocity installation, you first need to create a record for the node with Develocity by following either of these options:

Manual Registration
Unattended Registration

Manual Registration

Visit /cache-admin of your Develocity installation, and select Nodes from the left menu. In the Remote nodes Create new node section, enter the name for your node and click Create new node.

Remote Build Cache Node Administration Dashboard

The node will now be listed in the Existing nodes section. Each node is assigned a key and a secret. To view it, click the View button in the Secret column for the newly created node.

Example of Viewing a Remote Node Secret

The secret is only viewable for 5 minutes after node creation. If the node secret is lost, use the regenerate function to issue a new secret, which will then be viewable for 5 minutes.

As a last step you need to update the node configuration with the assigned key and secret.

Unattended Registration

In case nodes are required to be managed by provisioning software (e.g., Terraform) Develocity offers an unattended option to register the node via the Develocity API. Consult the access control section of the API manual on how to connect to the Build Cache section of the Develocity API.

Once you have access to the Develocity API, you can register a node via the Develocity API.

The following example uses develocity.example.com as the installation host, 7asejatf24zun43yshqufp7qi4ovcefxpykbwzqbzilcpwzb52ja as the access key of the user which has the Cache admin role, and cURL as the HTTP client.

First, a new node (referred to as node-1) is registered in Develocity using the Create/Update endpoint by specifying a unique descriptive value for the name of the node, e.g.:

curl -X PUT https://develocity.example.com/api/build-cache/nodes/node-1 \
  -d '{"enabled":true}' \
  -H "Authorization: Bearer 7asejatf24zun43yshqufp7qi4ovcefxpykbwzqbzilcpwzb52ja" \
  -H "Content-Type: application/json"

If applicable, a replication configuration (in this example a preemptive replication from parent-node-1 is configured) can be specified at creation time:

curl -X PUT https://develocity.example.com/api/build-cache/nodes/node-1 \
  -d '{"enabled":true,"replication":{"source":"parent-node-1","preemptive":true}}' \
  -H "Authorization: Bearer 7asejatf24zun43yshqufp7qi4ovcefxpykbwzqbzilcpwzb52ja" \
  -H "Content-Type: application/json"

Second, a request to the secret endpoint is made using the name of the node used in the previous step. This generates a key-secret pair that the node requires for connecting to Develocity:

curl -X POST https://develocity.example.com/api/build-cache/nodes/node-1/secret \
  -H "Authorization: Bearer 7asejatf24zun43yshqufp7qi4ovcefxpykbwzqbzilcpwzb52ja"

Output

{
  "key":"fqysvmfd3u5dcgis5reunrxjce",
  "secret":"cpgovxdo6f6phkdyxdrfzyy43q5hflatc2umzgguxv6adap2qyyi"
}

As a last step you need to update the node configuration with the returned key and secret. Consult the reference documentation of the Develocity API for further functionality with regard to node management.

Update Node Configuration

The node must also be configured with details of the server to connect to, including the assigned key and secret. Consult the Develocity Registration section of the Build Cache Node User Manual for information on how to do this.