Develocity Artifact Cache CLI User Manual

Limited Availability

The Develocity Artifact and Setup Cache is in Limited Availability.

Please contact the Develocity support team to get started with Develocity Artifact and Setup Cache.

Introduction

Develocity Artifact and Setup Cache enables caching of build dependencies and build inputs required by Gradle, Maven and npm builds. This is especially useful in ephemeral CI environments where build agents are frequently created and destroyed, leading to the loss of local caches between builds.

Artifact Cache is designed to eliminate the bottlenecks caused by downloading dependencies from the WAN (Wide Area Network). It acts as an intermediary between your builds and external repositories (such as Maven Central and npm) or internal artifact managers (like Artifactory).

Setup Cache targets the initialization phase of the build, which is often a hidden tax on developer productivity. In a typical "cold" CI run, build tools like Gradle must perform compute-intensive tasks before they even begin executing the build. This includes recompiling build scripts, parsing configurations, and calculating file hashes.

Develocity Artifact and Setup Cache is delivered as a unified service through Develocity Edges which serve as persistent cache for build artifacts that Artifact Cache CLI can later use to restore CI agent state.

Using Artifact Cache CLI

The Artifact Cache CLI is a command-line tool that stores and restores build artifacts (build dependencies and other build inputs) to the unified Develocity Artifact and Setup Cache service provided by Develocity Edge nodes.

Artifact and Setup Cache Overview

Key Components

Artifact Cache CLI

An application that runs on the CI agent. Handles capture and retrieval of build dependencies and other build inputs to and from the Develocity Artifact and Setup Cache service.

Edge

Comprises one or more Edge nodes deployed near CI infrastructure. Each Edge node manages available storage through eviction, stores and retrieves build artifacts to Artifact Cache CLI clients when requested.

Develocity

Manages access control and enables Edge discovery for Artifact Cache CLI clients.

Image

A metadata file that lists all build artifacts available from the Develocity Artifact and Setup Cache for a specific CI job. Tracked separately for each CI job, branch, and environment.

Develocity Artifact and Setup Cache Operation Flow

Edge Discovery

The Artifact Cache CLI uses Edge discovery to connect to the optimal Edge:

  1. Artifact Cache CLI reads the access key of a CI agent Develocity user from the environment variable DEVELOCITY_ACCESS_KEY.

  2. Access key identifies a Develocity user.

  3. The CI agent Develocity user has the Edge location configured.

  4. Artifact Cache CLI connects exclusively to that Edge for the Restore and Store phases.

 The "Preferred location" for the CI agent’s user must be set to one that doesn’t have Develocity server in it. A user’s "Preferred location" can be changed in Develocity by logging in as that user and navigating to the "My Settings - Location" page in the right-hand menu.

Restore Phase

  1. CI job starts - Fresh ephemeral CI agent is created.

  2. Checkout code - Repository is cloned.

  3. Artifact Cache CLI queries Edge - Requests the image for this job/branch.

  4. Edge returns image - Lists all known artifacts for this job.

  5. Restore artifacts - Artifact Cache CLI downloads only missing artifacts.

Build Phase

  1. Build runs - The build tool uses the restored artifacts when it runs.

Store Phase

  1. Build completes - The build tool process exits.

  2. Artifact Cache CLI scans directories - Identifies new or changed artifacts.

  3. Store new artifacts - Upload to Edge node.

  4. Update image - Records the up-to-date artifact list.

  5. Image stored to Edge - Updated image is ready for use by the next CI job.

Requirements

For details on the build infrastructure and other requirements to use Develocity Artifact and Setup Cache, please refer to the System Requirements section.

Getting Started

Develocity Artifact and Setup Cache integrates with any CI platform that supports running shell commands.

See these guides for specific platforms that can help you get started:

Artifact Cache CLI Commands

Refer to the Artifact Cache CLI Commands Reference for detailed information on available commands and their usage.

Observability

Build Scan Reporting

By applying version 2.1.0 or above of the Common Custom User Data Gradle plugin or Common Custom User Data Maven extension, Artifact and Setup Cache configuration and restore phase metrics will be automatically added to every Build Scan as Custom Values.

Example Build Scan with Artifact and Setup Cache reporting
Example Build Scan with Artifact and Setup Cache reporting
 The "Artifact Cache: Restore wall-clock duration" value is the total duration of the restore phase from both the Artifact and Setup Cache.

Develocity Artifact and Setup Cache Service Monitoring

The Develocity Artifact and Setup Cache service exposed by Edge nodes can be monitored by following the Monitoring section of the Develocity Edge User Manual.