This document refers to installing Gradle Enterprise on its legacy platform. For installing on its current platform, please see the following documents:

This manual covers the installation and operation of Gradle Enterprise. It is useful for administrators and maintainers of Gradle Enterprise installations, and build engineers that maintain or develop Gradle or Apache Maven™ builds that leverage Gradle Enterprise.

If you’re interested on deploying Gradle Enterprise into your own Kubernetes cluster, jump into these specific instructions.

Installation requirements

This section outlines the installation resource requirements and configuration aspects that need to be resolved during installation.

Supported operating systems

Server versions of the following Linux distributions are supported:

  • Debian 7.7+

  • Ubuntu 14.04.5 / 16.04 / 18.04 / 20.04

  • Red Hat Enterprise Linux 7.4 - 7.9 / 8.4

  • CentOS 7.4 - 7.9 / 8.4

  • Amazon Linux 2014.03 / 2014.09 / 2015.03 / 2015.09 / 2016.03 / 2016.09 / 2017.03 / 2017.09 / 2018.03 / 2.0

  • Oracle Linux 7.4 - 7.9 / 8.4

Docker

Gradle Enterprise requires Docker to be installed. Versions between 1.10.3 and 20.10.7 are supported, but we recommend version 20.10.7. For detailed requirements and installation guides see the docker installation docs.

On November 20, 2020, rate limits for anonymous and free authenticated use of Docker Hub went into effect. As the Replicated Docker images are hosted in Docker Hub you will need to provide your credentials. See the instructions in the Standard Installation section.

Virtualization

Gradle Enterprise requires a modern Linux operating system with Docker support. For trial and evaluation purposes, you can use a virtual machine to perform the installation on a non-Linux system. Several desktop virtual machine providers, such as VirtualBox, exist for this purpose.

Some considerations when using virtualization in conjunction with Gradle Enterprise are:

  • Any of the above listed Linux distributions will still work, even in a virtualized environment.

  • You will need to forward the necessary ports to the host system in order to configure Gradle Enterprise and access the application.

    Newer versions of MacOS may have built-in services listening on these ports, in particular, port 8800. If you encounter errors binding to these ports you may need to forward them to a different port on your host.
  • When using a virtual machine provider such as VirtualBox, the IP of your virtualized environment will differ depending on how you have configured networking for the virtual machine. For example, with VirtualBox, if you have set it up with the default networking mode (NAT), then the IP address of the Gradle Enterprise installation will be the same as that of your host system.

    When running in NAT networking mode, a VirtualBox machine will not be able to communicate with a service running on the host system. This has the implication that a virtualized Gradle Enterprise installation will not be able to communicate with a remote build cache node running on the same host as the VirtualBox instance. For this, you will need to investigate a different network mode (such as Host-only), or you will need to install the build cache node on a different host within your organization.
  • If you remap the ports 80 or 443 of your virtual machine, you will need to ensure that the Gradle Enterprise ports are reconfigured to match on the Settings page of the admin console. Failing to do this will cause some areas of the Gradle Enterprise UI to go to an incorrect address, and the server will generate an incorrect URL for build scans.

  • Gradle Enterprise can consume a considerable amount of disk space depending on how actively it is used. Keep this in mind when provisioning a virtual disk. Also, depending on the virtualization method used, disk performance may not be consistent with native disk access so Gradle Enterprise performance is likely to be affected.

Using desktop virtualization software such as VirtualBox or Vagrant is not recommended for production deployments.

CPU & memory requirements

  • Quad-core 2GHz or better CPU

  • 16 GB free RAM

Storage requirements

Embedded database

By default, Gradle Enterprise stores its data in a database that is run as part of the application itself, with data being stored in a directory mounted on its host machine. It is also possible to store most data in a user-managed database - see the below for details.

Capacity

Gradle Enterprise stores the majority of its data in its “installation directory”, which defaults to /opt/gradle. The recommended minimum initial capacity to provide for this directory is 250 GB. It is recommended to create a specific volume for the installation directory to avoid having another consume the space required for Gradle Enterprise, and to ensure at least 10% of the volume’s space is free at all times.

The amount of space used and rate of growth is dependent on your usage of Gradle Enterprise. See the managing disk space section for how to avoid running out of disk space for the installation directory.

The following are additional disk capacity requirements:

Location

/tmp

1 GB

/var/lib/docker

30 GB

/var/lib/replicated

30 GB

Backups

It is recommended to create backups of your Gradle Enterprise installation for disaster recovery purposes, by using Gradle Enterprise’s built-in backup scheduling. By default, backups are stored in the “backups” directory of the installation directory. To simplify disk space management, it is recommended to either mount a separate volume at this location or change this setting to a separate volume. The backup location can be specified at installation time, and changed later.

Performance

Disk performance has a significant impact on Gradle Enterprise. For production workloads, storage volumes should exhibit SSD-class disk performance of at least 3000 input/output operations per second (IOPS). Most desktop-class, non-SSD disk drives do not provide this level of performance.

You can run the following benchmark to estimate your disk performance:

docker run --rm -v /opt/gradle/fs-test:/var/lib/postgresql/data postgres pg_test_fsync

The majority of reported results should indicate greater than 3000 ops/sec.

Amazon Web Services

If you are using Amazon Web Services (AWS) and Elastic Block Storage (EBS) to host your Gradle Enterprise instance you should ensure that you provision 3000 or more IOPS for the data volume. Keep in mind that general purpose (or gp2) volumes are limited to 3 IOPS per gigabyte, meaning that 200 GB volume will only provide a maximum of 600 IOPS. If you are provisioning a volume smaller than 2TB you should consider using a provisioned IOPS (or io1) volume.

The recommendations above are based on average workloads. For projects with more complex builds or teams that produce a large number of build scans, Gradle Enterprise may require higher I/O performance than suggested.

User-managed database

Gradle Enterprise can be configured to store data in a database that is hosted and managed separately to the rest of the application. This can help with performance in some circumstances by allowing database resources to be scaled separately to Gradle Enterprise. It can also simplify backups and volume management when using a database provider that provides tooling for these out of the box.

When using a user-managed database, some Gradle Enterprise administrative features such as automated backups and build scan disk space management features that respond to low disk space scenarios are not available. Additionally, some installation local disk storage is still required for log files and Build Cache artifacts.
Compatibility

Gradle Enterprise needs to connect to a PostgreSQL version 12 compatible database.

Performance and capacity considerations

For similar workloads, Gradle Enterprise will need similar storage capacity and IOPS throughput as for the embedded database mentioned above. Please see your database provider’s options for provisioning and ensure that they meet these requirements.

When Gradle Enterprise and the database are not deployed in a private network near each other, consideration should also be given to ensuring that the database is hosted as close to the Gradle Enterprise host as possible. In a cloud provider this typically means deploying within the same availability zone or equivalent. This will ensure the lowest latency and highest throughput. Gradle Enterprise will not perform adequately under load if latency to its database is too high.

Backups

When using a user-managed database, backups must be handled by the database provider. Gradle Enterprise built-in backups will not operate in this mode.

Build Scan storage

By default, Gradle Enterprise stores Build Scan data in the configured PostgreSQL database. This has the advantage of being simple to deploy and operate. It has the disadvantage of being expensive and difficult to scale for larger installations. Utilizing an additional S3-compatible store is a compelling deployment option for larger installations that produce large volumes of data.

Please see below for more details.

Networking configuration

The following TCP ports on the host system are used by the application:

Port Network Interface Required Description

80

Public

When SSL is disabled

HTTP port used to access the application

443

Public

When SSL is enabled (default)

HTTPS port used to access the application

8800

Public

Always

System admin console

9870-9881

Public

Always

Replicated Operator

5432

docker0

Always

PostgreSQL database

22003

docker0

Always

Internal metrics collection

8080-8086

docker0

Always

Build Scans and Build Cache servers

Only ports 443 (or 80 if not using HTTPS) and 8800 are required to be accessible from outside the host. All the other ports are used for inter-process communication on the host itself over the specified network interface.

Internet access

In addition to the ports listed above the host system will also need access to the internet for retrieving application images, syncing license information, etc. Depending on your firewall or proxy configuration you may need to whitelist individual websites. Please refer to this document for detailed information on all external websites necessary for installation and proper system function.

If internet access is not possible you can still install Gradle Enterprise via an airgapped installation.

Application hostname

When installing Gradle Enterprise, you will be asked to provide a hostname. This should be the hostname that users of the installation use to access it and therefore should resolve within your network.

This value can be changed later.

If you are unsure what name to assign, or you do not yet have a hostname assigned, you can use the public IP address of the server, or localhost if you only intend to access the installation from the same host for the time being.

The public IP address is the IP address you would use to SSH on to the host system.

If you are using a proxy or external SSL termination server, you should enter the hostname of this service as it is this name that people will use to access Gradle Enterprise.

HTTPS SSL Certificate

It is strongly recommended that production installations of Gradle Enterprise are configured to use HTTPS with a trusted certificate, or to use an external SSL terminating server to secure your data whilst in transit.

If you do not have a certificate to use you may either:

  1. Allow Gradle Enterprise to generate an untrusted, self-signed, certificate

  2. Disable HTTPS and use plain HTTP

Using an untrusted certificate requires extra build script configuration for both Build Scans and the Build Cache, in addition to requiring users to accept the untrusted connection in their browser each time they access the application. However, your data will be encrypted whilst in transit on the network.

Using plain HTTP means that no extra build or browser configuration is required, but your data will not be encrypted while in transit on the network.

Untrusted SSL certificates

By default, Gradle Enterprise uses the default trust settings of the Java runtime that it ships with when connecting to other systems using SSL. If your organization uses certificates that are not signed by a trusted certificate authority, you must perform additional configuration for this to work. This may be the case if you use self-signed certificates or an internal certificate authority.

Additional certificates to trust can be configured by during installation/configuration by enabling “Trust additional certificates for SSL communication” and uploading all required certificates as a single PEM file.

If using self-signed certificates, the certificate of each service must be uploaded. If using an internal certificate authority, its certificate must be uploaded.

HTTP and non-anonymous access

When Gradle Enterprise is configured to use HTTP instead of HTTPS, it must be accessed via a private IP address to be able to sign in. This also applies when using external HTTPS termination. This is a security restriction that reduces the risk of secrets being transmitted without encryption being stolen.

A private IP address is any within ranges 10.0.0.0/8, 172.16.0.0/12 or 192.168.0.0/16.

Installation flavors

Gradle Enterprise supports multiple installation mechanisms.

Standard installation is recommended as it is simple and requires few steps. It requires the host system to have internet access for installation and upgrade. During operation, Gradle Enterprise does not require a connection to the internet.

Airgapped installation can be performed on a host without an internet connection. Installation materials are downloaded from a system with internet access and then manually transferred to the host system. Airgapped installations are not available for trial licenses.

Unattended installation

Gradle Enterprise can be installed without user intervention, by specifying configuration options in files before starting the installer. This allows scripted installations or installation by configuration management tools. For assistance in performing an unattended installation, please submit a support ticket at support.gradle.com.

Installation

Gradle Enterprise leverages Replicated for installation, configuration and software updates. Replicated orchestrates downloading and installing the Gradle Enterprise application on the host system as well as checking for periodic updates.

Replicated, as well as the Gradle Enterprise application both run on the host in the form of Docker containers. Management of these containers is done transparently by Replicated, however regular Docker commands can be used to inspect the status of running containers.

While the regular docker commands can be used to manage Gradle Enterprise application containers, it is highly recommended that maintenance actions such as starting and stopping the application be done via the Replicated admin console.

Standard installation

  1. During installation you will be asked for any required proxy settings. You should have this information on hand before proceeding with the next step. Later you may also be prompted to supply the host’s public IP address if the installer was unable to automatically detect it. If this happens, the IP address should be left blank.

    If you intend to run the Gradle Enterprise application with SSL enabled (which is recommended) you should also have the public certificate and private key available prior to beginning installation.

  2. Create a file called /etc/replicated.conf and set the following properties to persist your Docker credentials for future upgrades.

    {
        "DockerHubUsername": "your-username",
        "DockerHubPassword": "your-password"
    }
  3. Install Replicated on the target host by issuing the following command on the host system. Supplying the password inline is not recommended to avoid unintentionally leaking the values in the shell history or the process table. You can simulate supplying a password inline by adding your password to a file, your_password.txt in the example, and piping it into STDIN.

    cat your_password.txt | docker login --username "your-username" --password-stdin && \
    curl -sSL https://get.replicated.com/docker/gradleenterprise/stable | sudo bash
    As part of the installation procedure, Replicated will create a user replicated (if one doesn’t exist already), and add it to the docker group of users.
  4. Open a web browser and navigate to https://«hostname»:8800 to access the Replicated admin console.

  5. Enter a valid Hostname used by the Replicated admin console. This name should be routable on your network and resolve to the IP address of the host system. It will be used when generating self-signed certificates for the Replicated admin console.

  6. For SSL used by the Replicated admin console you can either use self-signed certificates or provide your own.

    1. To use self-signed certificates simply select Use Self-Signed Cert.

    2. To provide your own certificates provide files for Private Key and Certificate and select Upload & Continue.

    install ssl 2017.7
    These settings can always be changed later via the Console Settings screen, accessible under the install gear icon icon in the upper-right corner of the Replicated admin console.
  7. Select Choose License and navigate to the location of the license file which was sent to you via email.

    install license 2017.7
  8. Choose a method for securing the Replicated admin console. Either Anonymous or Password, and select Continue.

    install password 2018.3
    Should you choose to change these settings later, you can do so by visiting https://«hostname»:8800/create-password.
  9. The system will perform some pre-flight checks to test the host system for compatibility. Although the software can be installed without passing these checks it is highly recommended that any deficiencies be addressed. Proper system functionality cannot be guaranteed if the host system does not meet the required criteria.

    install preflight check 2022.1

    Select Continue.

  10. The Replicated admin console is now installed, and the Gradle Enterprise application will begin starting in the background. At any time you can select Dashboard from the navigation menu to see the current status of the application.

  11. Gradle Enterprise is installed with default settings for how it is accessed via the network and where it stores its data on the host system. At any time you can select Settings from the navigation menu to change this configuration.

    replicated settings 2022.1

    The same network address settings specified for the Replicated admin console are used as the network address for Gradle Enterprise by default, except for the port. The port used for Gradle Enterprise defaults to the default port for the specified protocol (i.e. 80 for HTTP and 443 for HTTPS). The hostname, port(s), protocol and SSL certificate can be changed from what is used to access the Replicated admin console.

    By default application data will be stored in /opt/gradle, which is referred to as the “installation directory”. This location can be changed, along with optionally separate locations for backups and log files.

Further configuration aspects are configured via the Gradle Enterprise application once Gradle Enterprise has started. Please see Post-installation setup.

Airgapped installation

Replicated already performs Docker cleanup management such as deleting dangling images or removing stopped containers. In an airgapped environment, there might be a race condition while executing docker system prune and the restart of containers, leading to a deletion of a required Docker image. Therefore, it’s not recommended to perform this kind of manual cleanup.
  1. Along with your provided license file you will be given a link and password to access the Gradle Enterprise airgap bundle download page.

  2. Select Get Download Link and download the airgap bundle.

    If using wget to download the airgap bundle you will need to use the --trust-server-names option or rename the downloaded file to .airgap. You may also need to surround the URL with quotes (").
  3. Transfer the airgap bundle to the host system.

  4. Download the Replicated admin console installer.

  5. Transfer the installer bundle to the host system.

  6. Run the Replicated admin console installer.

    tar xzvf replicated.tar.gz
    cat ./install.sh | sudo bash -s airgap
  7. Open a web browser and navigate to https://«hostname»:8800 to access the Replicated admin console.

  8. Enter a valid Hostname used by the Replicated admin console. This name should be routable on your network and resolve to the IP address of the host system. It will be used when generating self-signed certificates for the Replicated admin console.

  9. For SSL used by the Replicated admin console you can either use self-signed certificates or provide your own.

    1. To use self-signed certificates simply select Use Self-Signed Cert.

    2. To provide your own certificates provide files for Private Key and Certificate and select Upload & Continue.

    install ssl airgap 2017.7
    These settings can always be changed later via the Console Settings screen, accessible under the install gear icon icon in the upper-right corner of the Replicated admin console.
  10. At the following screen, select Choose license and select the license file which was sent to you via email.

    install license airgap 2017.7
  11. Then on the next screen choose Airgapped and click Continue

    install choose installation airgap 2017.7
  12. Enter the location of the .airgap file you downloaded and transferred earlier.

    install airgap package 2017.7
  13. Select Continue.

  14. After the airgap packages have been extracted, choose a method for securing the Replicated admin console. Either Anonymous or Password, and select Continue.

    install password 2018.3
    Should you choose to change these settings later, you can do so by visiting https://«hostname»:8800/create-password.
  15. The system will perform some pre-flight checks to test the host system for compatibility. Although the software can be installed without passing these checks it is highly recommended that any deficiencies be addressed. Proper system functionality cannot be guaranteed if the host system does not meet the required criteria.

    install preflight check 2022.1

    Select Continue.

  16. The Replicated admin console is now installed, and the Gradle Enterprise application will begin starting in the background. At any time you can select Dashboard from the navigation menu to see the current status of the application.

  17. Gradle Enterprise is installed with default settings for how it is accessed via the network and where it stores its data on the host system. At any time you can select Settings from the navigation menu to change this configuration.

    replicated settings 2020.4

    The same network address settings specified for the Replicated admin console are used as the network address for Gradle Enterprise by default, except for the port. The port used for Gradle Enterprise defaults to the default port for the specified protocol (i.e. 80 for HTTP and 443 for HTTPS). The hostname, port(s), protocol and SSL certificate can be changed from what is used to access the Replicated admin console.

    By default application data will be stored in /opt/gradle, which is referred to as the “installation directory”. This location can be changed, along with optionally separate locations for backups and log files.

Further configuration aspects are configured via the Gradle Enterprise application once Gradle Enterprise has started. Please see Post-installation setup.

Database setup and configuration

By default, Gradle Enterprise will use an internal database that stores its data inside the installation directory on the host. In this configuration, there is nothing else to set up regarding the database.

database embedded

When Gradle Enterprise is configured to store data in a user-managed database, it must be provided with connection settings for that database. This includes standard connection settings like host, port and database name, and also credentials.

The database must have already been created prior to configuring Gradle Enterprise with connection details.

database external connection 2022.1

There are two options for credentials. If provided with credentials for a database superuser (such as the postgres user that is common on PostgreSQL database instances), Gradle Enterprise can perform all further database setup, and can do so for all subsequent Gradle Enterprise upgrades. This is the recommended configuration.

database external provide superuser

Note that in some installations, and often in cloud-based databases, the typical credentials provided by the database provider are not a true superuser, but have many of the same abilities. For example, the supplied postgres account in Amazon RDS Postgres databases is not a true Postgres superuser but has the rds_superuser role. Such accounts are fine to configure Gradle Enterprise to use.

In some instances it may be be preferable to not supply Gradle Enterprise with database superuser credentials. It is possible to instead run a database setup script on the database, which will set up less privileged accounts for the application to use, and some privileged functions needed for the application to run. The credentials for the accounts must then be set by the user and provided to Gradle Enterprise in the Settings screen.

database external no superuser

Please contact Gradle support for assistance in setting up a database without providing superuser credentials.

Post-installation setup

Basic installation settings for Gradle Enterprise are configured via the Replicated admin console (i.e. the application used to perform the installation thus far). This includes how Gradle Enterprise is accessed via the network, and where it stores its data on the host system. Other configuration aspects are configured via the Gradle Enterprise application. Only users with the administration permission can do this, and they can do so by using the user menu in the top right of the application and choosing “Administration”. Until user accounts have been created, or in the case that no user accounts will be used, the system user can be used for this purpose.

Please read each of the following sections for information on recommended configuration.

System user password

The “system user” is an ever-present local user account that can be used to bootstrap the configuration of an installation, or for emergency local access in case of a failure with an external authentication provider once configured.

It is strongly recommended to immediately change the system user password from its default after installation.

To change the system user password, visit Gradle Enterprise in a web browser via the hostname that was specified during installation. Use the “Sign in” button at the top right of the page to sign in as the system user by using the username system and password default. You will then be required to specify a new password for the system user account. The new password should be recorded and kept secret as it can be used to access Gradle Enterprise as an administrator.

After setting the system user password, you will be redirected to the administration section.

It is recommended that the system user account not be used regularly. Instead, real administrator user accounts should be created by configuring access control.

Access control

Gradle Enterprise allows locally managed user accounts and permissions, and externally managed via an LDAP service or SAML 2.0 identity provider.

Initial setup can be performed by the system user. Subsequently, any user with the administration permission can configure access control by using the user menu in the top right of the application to access “Administration” and then “Access control”.

The following permissions are available to grant access to the described function:

Permission Description Default role name

Build scan view

Allows viewing of build scans and associated build data and use of Predictive Test Selection

ge-scans-viewing

Build scan publish

Allows publishing of build scans

ge-scans-publishing

Test Distribution

Allows use of Test Distribution

ge-distributed-testing

Build data export

Allows exporting build data via the API

ge-data-export

Build Cache read

Allows reading of Build Cache data

ge-cache-read

Build Cache read and write

Allows reading and writing of Build Cache data

ge-cache-write

Build Cache admin

Allows administration of Build Cache functionality

ge-cache-administration

Admin

Allows general administration of Gradle Enterprise (e.g. access control)

ge-administration

For locally managed accounts, permissions must be assigned for each user. A default set for all new users can be specified.

For externally managed user accounts (LDAP or SAML 2.0), permissions can be managed locally or by group/role membership defined by the provider. When using provider-defined membership, each permission can be mapped to one or more roles. By default, each permission maps to a unique role with the “default role name” specified in the table above. This can be changed for each permission.

Anonymous access

By default, Gradle Enterprise allows anonymous viewing and creation of build scans. This makes it easier to get started by reducing build configuration, but may not be suitable for your environment. Anonymous access to the built-in Build Cache node is not enabled by default.

Permissions for anonymous users can be changed by going to “Administration” via the top right hand user menu, then “Access control” → “Anonymous access”.

Authenticated build access

Builds can authenticate with Gradle Enterprise by supplying an “access key”.

Please consult the Gradle Enterprise Gradle Plugin User Manual or Gradle Enterprise Maven Extension User Manual for guidance on how to configure builds to authenticate with Gradle Enterprise.

Local users

Locally managed user accounts can be created to allow users to access Gradle Enterprise. They are not mutually exclusive with externally managed user accounts and both can co-exist provided the usernames and emails are unique.

Setup
  1. From the Administration page, navigate to “Access control” → “Users”

  2. Click Add.

  3. Enter details for the user and set an initial password.

  4. Assign the required roles for the user.

  5. Click Save.

SAML 2.0

A SAML 2.0 identity provider can be configured to allow users to access Gradle Enterprise using their organization credentials. User accounts for users authenticating with the SAML provider will be created on first login.

A user cannot login via a SAML provider if a locally defined account exists for the same username or email.

Sign out from Gradle Enterprise will not log users out of the SAML identity provider.

Setup
  1. From the Administration page, navigate to “Access control” → “Identity provider”

  2. Check ”Enable external identity provider”

  3. Choose “SAML 2.0” from Identity provider type options

  4. Enter a name for the identity provider

  5. Create a SAML application at your identity provider using the displayed “Service provider SSO URL” and “Service provider entity ID”.

  6. Download the metadata for the SAML application from your identity provider, and select this file for the “Identity provider metadata file” field.

  7. Configure signing/encryption options

  8. Configure attribute mapping (described below)

  9. Configure role management (described below)

  10. Click Save.

If any signing or encryption is being used, use “Download service provider config” to obtain a configuration file that will need to provided your identity provider.

Attribute mappings

User’s “first name”, “last name” and “email” attributes can be obtained from the identity provider or prompted for on first login.

To obtain an attribute from the identity provider, select “Manage in identity provider” for the attribute and specify the name of the SAML attribute that will provide the value. Attribute changes made at the SAML identity provider will only take affect after either a user initiated logout, administrator force logout, or session expiry.

Locally managed attributes can be updated for a user by an administrator.

Roles

User role membership can be defined by the identity provider or managed locally.

To use identity provider specified role membership, select “Defined by identity provider” in the “Role membership” section. The name of the SAML attribute that defines the roles for a user must be specified, along with the values to map to Gradle Enterprise access roles.

When using “Defined by Gradle Enterprise” as the “Role membership” option, the default roles for every user can be specified. Users will be assigned the roles when they first sign in. Changing the default roles will not change role membership of any existing users. Administrators can change role membership for individual users after they have signed in for the first time.

LDAP

An LDAP identity provider can be configured to allow users to access Gradle Enterprise using their organization credentials. User accounts for users authenticating with the LDAP identity provider will be created on first login.

A user cannot login via an LDAP provider if a locally defined account exists for the same username or email.

Setup
  1. From the Administration page, navigate to “Access control” → “Identity provider”

  2. Check ”Enable external identity provider”

  3. Choose “LDAP” from Identity provider type options

  4. Complete the connection details for your LDAP provider.

  5. Configure user attributes (described below)

  6. Configure role management (described below)

  7. Click Save.

Attribute mappings

There are five required fields in Gradle Enterprise: “username”, ”first name”, “last name”, “email“, and “UUID”. These fields are required to be mapped to fields in your LDAP identity provider.

Commonly, the “UUID” attribute is “entryUUID”, however for LDAP providers where this is missing, another sensible unique persistent ID should be used. For Active Directory, this is usually called “objectGUID”.
Roles

User role membership can be defined by the identity provider or managed locally.

To use identity provider specified role membership, select “Defined by identity provider” in the “Role membership” section. Details on where to find roles and how they are defined must be provided.

For most LDAP providers it is common for the role object class to be “groupOfNames”, however for Active Directory this is usually “group”.

When using “Defined by Gradle Enterprise” as the “Role membership” option, the default roles for every user can be specified. Users will be assigned the roles when they first sign in. Changing the default roles will not change role membership of any existing users. Administrators can change role membership for individual users after they have signed in for the first time.

Recursive group membership is supported, via an opt-in option.

Build Scan storage

Build scans can be stored in the Gradle Enterprise database, or in in Amazon S3. By default, build scans are stored in the database that Gradle Enterprise is configured to connect to.

Disk space management

In most installations, storage space usage is dominated by the storage of Build Scan data. The amount of space used is dependent on how many scans are being published and how much information is being recorded for each build.

When Build Scan data is stored in the database, compression and deduplication techniques are used. This means that data growth non-linear; storing data for twice as many builds does not mean that twice the space will be required.

When Build Scan data is stored in S3, each scan is stored as a compressed, self-contained object. This makes estimation of required storage easier: twice as many scans results in roughly twice as much storage being used.

To control the amount of storage space used, Gradle Enterprise can be configured to remove build scans based on their age. To avoid running out of disk space, you can configure automatic deletion of old build scans when the amount of free space drops below a specified percentage, as well as automatic rejection of incoming data when free space drops below a specified percentage. Additionally, the system can be configured to send warning emails when free space drops below a specified percentage.

These settings can be changed by going to “Administration” via the top right-hand user menu, then “Build Scans”.

A configuration that maintains a predictable build scan retention period is:

  1. Specify a maximum build scan age

  2. Send a warning email when there is less than 10% of space free

  3. Reject incoming data when there is less than 5% of space free

When storing scans in the database, an alternative configuration that stores as much build scan history as space permits is:

  1. Do not specify a maximum build scan age

  2. Automatically delete old build scans when there is less than 15% of space free

  3. Send a warning email when there is less than 10% of space free

  4. Reject incoming data when there is less than 5% of space free

When enabling automatic deletion of old build scans when disk space is low, be mindful that the result of another process filling the volume that Gradle Enterprise is using will be that all build scan data will be deleted.

If backups are created on the same volume, make sure to leave enough room for them in your thresholds. For example, if the total space that your backups take up is 40% of the disk, the above recommended settings would be 55%, 50% and 45%. Store backups on a separate volume to simplify space management.
Configuring disk space management percentage thresholds is currently only supported when using the embedded database. Please consult your database provider for monitoring and alerts on database disk space when using a user-managed database.
When storing Build Scans in S3, the space used will not be taken into account in the above calculations. This means that if your installation is running low on database space, the likely cause will not be Build Scan data, so automatic deletion of build scans wil not recover much space. It is not recommended to set an auto-deletion threshold when storing Build Scan data in S3.

Amazon S3 Build Scan storage

It is possible to store build scans in Amazon S3. While this is a more complex setup, there are a number of benefits:

  • High traffic installations typically see an improvement in build scan processing and cleanup performance.

  • S3 is a highly scalable and fault tolerant service, typically more so than an individual database.

  • It is cheaper to store more scans.

  • With the vast majority of Build Scan data in S3, the main Gradle Enterprise database typically requires much less storage, and the installation (for embedded databases) or the user-managed database can be provisioned with fewer resources.

  • Database backup management is simpler when the database is much smaller.

There are some downsides:

  • Installation is more complex.

  • More memory must be allocated to the Gradle Enterprise application.

  • Cloning a Gradle Enterprise instance requires more steps, since a database backup is not the only thing required to restore data.

  • S3 storage is only supported for AWS installations.

Due to latency and traffic costs, it is only recommended to use S3 storage if your Gradle Enterprise instance is hosted in AWS.

There are many S3-compatible object stores such as MinIO or Google Cloud Storage. If you wish to use a service other than Amazon S3, please contact Gradle Enterprise technical support.
S3 prerequisites

Storing build scans in S3 requires the following:

The latter option uses credentials provided by AWS dynamically to access S3.

S3 configuration

Once the above bucket, policy and necessary credential setup is done, we can configure Gradle Enterprise.

Go to the "Administration" → "Build Scans" page, then scroll down and select "Enable S3 build scan storage".

s3 scan storage 2022.3
  • Enter the AWS region and bucket name.

  • Optionally adjust where in the bucket Build Scans will be stored.

  • If using a static access key / secret key pair, enter these.

  • If using instance profile credentials, select "Obtain from environment".

  • The connection to S3 can be tested using the "Test S3 Connection" button. It is strongly recommended to test the connection to S3 on the admin page to troubleshoot connection issues before making permanent configuration changes.

  • Once successful, also select "Incoming Scan Storage" → "Store incoming build scans in S3".

  • Click "Save".

  • You will be prompted to restart to allow the configuration changes to be applied. Do not do this yet.

  • Go to the "Administration" → "Advanced".

  • Increase the "App" → "Heap memory" setting by 2048 MiB.

  • Click "Save".

  • You will be prompted to restart to allow the configuration changes to be applied. Do so.

Once Gradle Enterprise has restarted, all new Build Scans will be stored in S3.

Backup and disaster recovery

Embedded database

Gradle Enterprise provides system backup and restore capabilities to facilitate disaster recovery. Backups can be triggered manually or scheduled to be done automatically on a periodic basis, and require no system downtime.

The backup process produces a single compressed archive which can be used to restore all user data to a state at the time of the backup. It is highly recommended that this archive additionally be copied to an off-host location, in the case of a complete loss of the host system or volume used to store backup archives.

In addition to disaster recovery, backup archives can be used to migrate your Gradle Enterprise data to a new host system. This can be useful in scenarios where a trial instance is promoted to a production one or in the event that you require upgrading your Gradle Enterprise instance hardware.

Creating backups

The backup schedule and settings can be configured by going to “Administration” via the top right hand user menu, then “Backups”.

An administrator can be notified of the backup process outcome, which is particularly important in the case of failures. This requires administrator email details and SMTP server settings to be configured for the Gradle Enterprise instance.

In addition to configuring automatic scheduled backups, you may trigger a manual backup at any time by running the following command on the host system.

replicated admin backup
By default, backups are stored in the “backups” directory of the installation directory. To simplify disk space management, it is recommended to either mount a separate volume at this location or change this setting to a separate volume. This setting can be changed in the Replicated admin console. It cannot be changed in the Gradle Enterprise Administration section.
Restoring from a backup

Once a backup archive is created you may use it to restore a Gradle Enterprise instance to the backup state at any time. Backup archives contain only a copy of the application database and not configuration settings entered in the Replicated admin console. If restoring to a completely new host you will first need to perform a fresh installation and restore any configuration options manually.

Executing the restore_snapshot.sh script requires 7zip to be installed on the host system. This can typically be done by installing the p7zip package using your Linux distribution’s package manager.
  1. If restoring to a new host, perform a fresh installation as described in the Installation section above.

  2. Copy the previously created backup archive to a location on the host system.

  3. Stop the server by visiting the Dashboard page of the Replicated admin console and selecting Stop Now.

  4. Execute the following command on the host system as root.

    /opt/gradle/tools/restore_snapshot.sh \
        -y \ # assume that the answer to any question which would be asked is yes. Useful for unattended executions"
        /opt/gradle/data/backups/backup.7z # location of backup archive
  5. When the script has completed, return to the Replicated admin console Dashboard and select Start Now.

User-managed database

When data is stored in a user-managed database, backups and recovery is the responsibility of the user or database provider.

Creating backups

Please see your database provider for details on how to make backups and schedule them regularly.

Restoring from a backup

To restore a backup, perform the following steps:

  1. If restoring to a new host, perform a fresh installation as described in the Installation section above.

  2. Stop the server by visiting the Dashboard page of the Replicated admin console and selecting Stop Now.

  3. Follow your database provider’s instructions on restoring a backup. This may be running a tool like pg_restore or restoring a database snapshot, depending on your database provider and backup type.

  4. If your restored database is in a new location (for example, some cloud providers will restore a snapshot by creating a new database instance), update the connection details on the Settings page of the Replicated admin console.

  5. If you are not providing superuser credentials to Gradle Enterprise, run the current database setup script used during Database setup and configuration on the database and ensure that working application and migrator passwords are set on the Settings page.

  6. Save the settings, return to the Replicated admin console Dashboard and select Start Now.

Email notifications

Email server settings can be configured by going to “Administration” via the top right hand user menu, then “Email server”.

With an email server specified, Gradle Enterprise can send email notifications on completion of backups or when disk space is low. See the Disk space management and Creating backups sections for more details.

Upgrading

Database

For major version upgrades (e.g. 2021.2.4 to 2021.3 or later), if data is stored in a user-managed database and superuser credentials are not supplied, the database setup script must be run prior to the upgrade. Contact Gradle support to get the correct script for the major version to which the system is being upgraded.

Standard installation

Before upgrading, be sure to check the Upgrade notes section for any special considerations when upgrading from older versions of Gradle Enterprise.

  1. Open a web browser and navigate to https://«hostname»:8800 to access the Replicated admin console.

  2. Checks for updates are done periodically. When an update is available you will see something similar to the screen below. You can explicitly check for updates at any time by selecting Check Now.

    update dashboard 2019.2
  3. Select View Update.

  4. On the Release History screen select Install Update to install the latest released version.

    Updating requires the application to be temporarily shutdown. This means the application will not be accessible and information for builds run during this downtime period will not be captured.
    update install 2017.7
  5. The updated application images will be downloaded and installation will begin immediately once the download has completed.

  6. Select Dashboard from the navigation menu. The status Started will be shown when installation is complete and the application is up and running.

    update started 2017.7

After upgrading, Docker may keep older Gradle Enterprise images on disk. These unused images can consume a significant amount of disk space. You can clear up space taken by unused Docker images at any time by running the following command:

docker images --filter dangling=true -q | xargs docker rmi

Airgap installation

  1. Check in the release notes whether Replicated has been upgraded as part of the release. If so, follow steps 4-6 in the airgapped section above to upgrade to the required Replicated version.

  2. Confirm the update path used to search for new airgap bundles. The Replicated admin console will look for updates in this folder. You can configure this in the Airgap Settings settings section of the Console Settings page which is accessible via the install gear icon in the navigation menu.

    update airgap 2017.7
  3. Follow steps 1-3 in the airgapped section above to get the latest airgap bundle. You will want to place the .airgap file in the location confirmed in the step above.

  4. You can now upgrade the system using the instructions in the upgrading section.

Uninstalling

To uninstall Gradle Enterprise you first need stop the application. Navigate to the dashboard at https://«hostname»:8800, and click “Stop Now”.

Then follow the steps to uninstall Replicated, which can be found here.

You can then remove the replicated user that was created on install, by running the command:

sudo userdel -r replicated

This will remove the user, along with their home directory.

Troubleshooting

Issues with the Replicated admin console

If you experience difficulty accessing the Replicated admin console then you may need to restart the Replicated services on the host system. To restart Replicated, issue the following commands:

Debian/Ubuntu
sudo service replicated restart
sudo service replicated-ui restart
sudo service replicated-operator restart
CentOS 7/RHEL 7/Fedora
sudo systemctl restart replicated replicated-ui replicated-operator

Resource usage

Standard Linux process utilities such as ps and top can be used to inspect the resource usage of the application.

Both Replicated and the application run in Docker containers. The following is sample output from running the command ps -aufx which displays the application’s processes.

process list

The Replicated admin console (https://«hostname»:8800/dashboard) also displays CPU and memory usage of the application’s Docker containers.

dashboard resource usage 2017.7
Both the Memory Usage and CPU Usage charts have a window of 15 minutes and are representative of the most recent usage data.

Alternatively, docker stats can be used to get more detailed resource usage information about the individual Docker containers.

docker stats

Support

If you are experiencing issues with Gradle Enterprise or related components please submit a support ticket at support.gradle.com, including details of the issue and an attached Support Bundle.

Support Bundles assist our engineers in troubleshooting your issue by providing the following information about your Gradle Enterprise installation:

  • Host Information

    • Operating System

    • CPU

    • Memory

    • Disk Usage

  • Logs

    • Gradle Enterprise

    • Replicated

  • Docker Configuration

  • Gradle Enterprise Database Statistics

To generate a support bundle:

  1. Open a web browser and navigate to https://«hostname»:8800 to access the Replicated admin console.

    update dashboard 2019.2
  2. Select Support from the navigation menu.

    support dashboard 2019.2
  3. Click the Download Support Bundle button.

    Generation of the Support Bundle may take several minutes to complete. Once the Support Bundle has been generated it will be automatically downloaded.

If the Replicated admin console is unreachable, log on to the box hosting Gradle Enterprise and run:

replicated support-bundle $(replicated apps | tail -n1 | awk '{print $1}')

If you have questions about a particular build scan, please submit a support ticket at support.gradle.com, and attach a dump of the build scan.

To generate one, replace /s/ with /scan-dump/ in the build scan url, e.g. https://your-ge-server/scan-dump/vbdei7xhyojq2

Getting started

Build scans

A build scan is a shareable record of a build that provides insights into what happened and why.

For information on how to create build scans, please consult the links below.

Compatibility between versions of Gradle Enterprise and the Gradle Build Scan plugin or the Gradle Enterprise Maven extension can be found here.

Build Cache

Build caching dramatically decreases build times for both local development and continuous integration builds. Build Caches store outputs from Gradle tasks and Maven goal executions and serve them to later builds to reuse, instead of the outputs being built again.

Gradle Enterprise provides a built-in Build Cache node as part of each installation, and allows optionally connecting one or more remote nodes to use as discrete caches or replicas for reducing network latency.

For information on how to use the Gradle Enterprise Build Cache, please consult the Getting Started guides for Gradle or Maven.

Build Cache administration is available at /cache-admin of your Gradle Enterprise installation.

Configuration

The built-in Build Cache is configured with the following defaults:

  • All cache access is disabled.

  • Target cache size of 10 GiB (how much data to retain in the cache).

  • Free space buffer size of 1 GiB (how much free disk space to ensure is available at all times, reducing cache size if necessary).

  • Maximum artifact size of 100 MiB (the largest cache artifact to accept).

  • No limit on cache entry age.

To change this configuration, click the Nodes item in the left menu to access the node listing, then click View built-in node details.

controller built in details 2022.1

If you are using an installation method that gives the built-in build cache node an exclusive data volume (like Ship), the "Target cache size" section may instead look like this:

controller exclusive volume mode 2022.1
Access control

By default, access to the built-in cache node is disabled. There are two access control mechanisms that can be used. Which one you should use depends on the version of the Gradle Enterprise Gradle plugin or Gradle Enterprise Maven extension you will be using.

Version 3.11+ of the Gradle Enterprise Gradle plugin and version 1.15+ of the Gradle Enterprise Maven extension can authenticate to the build cache with Gradle Enterprise access keys. Permissions for these users are managed via the same access control mechanism used for other functions. Users with the “Build cache read” permission can read data from any connected build cache, while users with the “Build cache write” permission can read and write data from any connected build cache.

To control access for earlier versions, separate username and password credentials can be specified in the “Build Cache” section of Gradle Enterprise, in the details section for each build cache node. This section of Gradle Enterprise is only accessible by users with the "Build cache admin" permission.

How to configure your builds to supply credentials is described in the Gradle Enterprise Gradle Plugin User Manual for Gradle, and in the Gradle Enterprise Maven Extension User Manual for Maven.

Anonymous access control can also be configured using the same anonymous access control mechanism used for other functions, or by configuring node-specific anonymous permissions in the details section for each build cache node. When both are configured, the most permissive setting is used.

When the anonymous access level is set to Read and Write, writing to the build cache does not require credentials which may allow anyone to write malicious cache entries.

Remote nodes

Remote build cache nodes are installed separately from Gradle Enterprise. They can be used to separate cache artifacts, distribute load, or improve build performance by having a better network link between the build and the node.

By connecting remote nodes to a Gradle Enterprise instance, you are able to configure them centrally from Gradle Enterprise, and have them replicate entries from other cache nodes.

Installation and operation of remote build cache nodes is documented in the dedicated Build Cache Node User Manual.

Connecting with Gradle Enterprise

To connect a remote node to your Gradle Enterprise installation, you first need to create a record for the node with Gradle Enterprise by following either of these options:

Manual registration

Visit /cache-admin of your Gradle Enterprise installation, and select Nodes from the left menu. In the Remote nodes > Create new node section, enter the name for your node and click Create new node.

controller after new node added 2021.2

The node will now be listed in the Existing nodes section. Each node is assigned a key and a secret. To view it, click in the View button on the Secret column for the newly created node.

controller secret 2021.2
The secret is only viewable for 5 minutes after node creation. If the node secret is lost, use the regenerate function to issue a new secret which will then be viewable for 5 minutes.

As a last step you need to update the node configuration with the assigned key and secret.

Unattended registration

In case nodes are required to be managed by provisioning software (e.g., Terraform) Gradle Enterprise offers an unattended option to register the node via the Gradle Enterprise API. Please consult the access control section of the API manual on how to connect to the Build Cache section of the Gradle Enterprise API.

Once you have access to the Gradle Enterprise API, we can start with the registration of a node via the Gradle Enterprise API.

The following example uses ge.mycompany.com as the installation host, 7asejatf24zun43yshqufp7qi4ovcefxpykbwzqbzilcpwzb52ja as the access key of the user which has the Cache admin role, and cURL as the HTTP client.

First, a new node (referred to as node-1) is registered in Gradle Enterprise using the Create/Update endpoint by specifying a unique descriptive value for the name of the node, e.g.:

$ curl -X PUT https://ge.mycompany.com/api/build-cache/nodes/node-1 \
  -d '{"enabled":true}' \
  -H "Authorization: Bearer 7asejatf24zun43yshqufp7qi4ovcefxpykbwzqbzilcpwzb52ja" \
  -H "Content-Type: application/json"

If applicable, a replication configuration (in this example a preemptive replication from parent-node-1 is configured) can be specified at creation time:

$ curl -X PUT https://ge.mycompany.com/api/build-cache/nodes/node-1 \
  -d '{"enabled":true,"replication":{"source":"parent-node-1","preemptive":true}}' \
  -H "Authorization: Bearer 7asejatf24zun43yshqufp7qi4ovcefxpykbwzqbzilcpwzb52ja" \
  -H "Content-Type: application/json"

Second, a request to the secret endpoint is made using the name of the node used in the previous step. This generates a key-secret pair that the node requires for connecting to Gradle Enterprise:

$ curl -X POST https://ge.mycompany.com/api/build-cache/nodes/node-1/secret \
  -H "Authorization: Bearer 7asejatf24zun43yshqufp7qi4ovcefxpykbwzqbzilcpwzb52ja"
{
  "key":"fqysvmfd3u5dcgis5reunrxjce",
  "secret":"cpgovxdo6f6phkdyxdrfzyy43q5hflatc2umzgguxv6adap2qyyi"
}

As a last step you need to update the node configuration with the returned key and secret. Please consult the reference documentation of the Gradle Enterprise API for further functionality with regard to node management.

Update node configuration

The node must also be configured with details of the server to connect with, including the assigned key and secret. Please consult the Gradle Enterprise Registration section of the Build Cache Node User Manual for information on how to do this.

Replication

The bandwidth and latency of the network link between a build and a build cache significantly affects build performance. Replication allows users to use a cache node that they have a better network link to, while reusing artifacts from another cache node that is further away. This is particularly effective for geographically distributed teams.

The replication settings for each node can be configured via the node’s configuration page in Gradle Enterprise. They can not be configured for remote nodes via the remote node’s user interface or configuration file.

A typical arrangement is to have continuous integration builds push to a cache node on a local network, and have other nodes used by developers in different locations, ideally on their local network, use it as their replication source.

replication network

Replication is one-way (i.e. replication sources do not copy artifacts from nodes using them as a source). Furthermore, a node that is acting as a replication source cannot itself use a replication source.

Replication is not supported with remote nodes earlier than 3.0. This setting will not be shown for such nodes.

By default, cache entries are replicated on demand. Nodes will copy a cache entry from their replication source when that particular cache entry is requested for the first time. This avoids copying entries between nodes that will never be used, but forces builds to wait for the transfer on first use.

Preemptive replication enables better build cache performance, at the expense of more network transfer between nodes. When preemptive replication is enabled, nodes will copy cache entries from their replication source as soon as they are added. The downside of this approach is that entries may be copied that are never used. If network bandwidth usage between nodes is not a concern, this is the best configuration.

Only cache entries that are added to the replication source after the node has connected are copied preemptively. Any already existing entries will be copied on demand.

Preemptive replication is only supported for nodes of version 4.0 or later. If the node or its replication source is older than this, replication will be on demand.

Health monitoring

The Health page provides an overview of the operational status of all enabled build cache nodes.

controller health 2021.3

Each node may have a status of OK, WARN or ERROR. If all nodes are reported as OK, you can be assured that your build cache network is operating well.

WARN indicates that there is a non-critical issue, or that there is a problem that is expected to resolve itself soon. If the issue does not resolve itself, or for immediately critical issues, the ERROR status is used.

A monitoring tool friendly version of this page is available via the Monitoring page link towards the bottom of the page. This endpoint returns the statuses in plain text. If there are any ERROR statuses, the HTTP response status code for this page will be 503. Otherwise, it will be 200.

Test Distribution

Gradle Enterprise Test Distribution takes your existing test suites and distributes them across multiple compute resources to execute them faster. It requires connecting one or more test execution “agents” to Gradle Enterprise. Builds that have enabled Test Distribution connect to Gradle Enterprise, which then distributes the tests to be executed on connected agents.

Connecting builds

Please consult the Gradle Enterprise Test Distribution User Manual for information on build configuration.

Gradle Enterprise users must be granted the “Test Distribution” access control role to be able to use Test Distribution.

Builds and agents connect to the Gradle Enterprise server using a WebSocket connection. Thus, you need to ensure WebSocket support is enabled on every load balancer or proxy that is used in between the build/agent and the Gradle Enterprise server.

The “Jobs” tab provides an overview of the currently active jobs, where each row represents a running test task or goal in a Gradle or Maven build, respectively.

testdistribution jobs 2021.2

The first part of the label indicates the root project name of the build, while the second part identifies the particular task or goal. The count of assigned/desired agents shows how many agents are currently assigned to/desired by the job. More agents may be assigned as they become available. The count of compatible agents shows how many currently connected agents have capabilities that satisfy the requirements of the job.

API keys

API keys are used by agents to authorize their connection to Gradle Enterprise, and for authorizing connections to Test Distribution APIs provided by Gradle Enterprise.

API keys can be generated on the “Configuration” tab. A single API key may be used my multiple agents. Multiple can be created to facilitate a rolling update if required. Revoking an API key prevents it from being used for future registrations and API access. Connected agents will be forcibly disconnected one hour after the API key they used to connect is revoked.

testdistribution configuration 2021.2

Agents

Utilizing Test Distribution requires connecting additional compute resources, called agents, to Gradle Enterprise. The Test Distribution agent is available as a Docker image or standalone JAR and is easy to deploy and operate via Kubernetes or any modern compute platform. Please consult the Gradle Enterprise Test Distribution Agent User Manual for more on deploying and operating agents.

The “Agents” tab shows all currently connected agents.

testdistribution agents 2021.2
Auto scaling agents with pools

An agent pool is a group of agents that are to be scaled up and down based on demand. Each pool has a generated unique ID that is specified by an agent when connecting to indicate its pool, and can be used by compute platform auto scalers to determine the desired number of agents at any time for that pool.

Please refer to the Gradle Enterprise Test Distribution Agent User Manual for step-by-step instructions for auto scaling with Kubernetes.
testdistribution agent pool 2022.1
Minimum and maximum size

The minimum and maximum size of an agent pool is used to determine the desired number of agents at a given moment.

Ultimately, the actual number of agents is determined by the compute platform, which may impose incompatible limits. Where possible, ensure that the limits imposed by the compute platform are either identical to or compatible with those configured in Gradle Enterprise.

To temporarily disable a pool, set both minimum and maximum to zero.

Capabilities

Agents in a pool are expected to provide a certain set of capabilities. While an agent may provide additional capabilities, it will be rejected when connecting without at least the mandatory capabilities configured on its agent pool.

While computing the number of desired agents for a pool its capabilities are matched against the requirements of the current jobs. A pool is only utilized if it satisfies all of a job’s requirements. Therefore, you should at least add a jdk=…​ capability to each agent pool since such a requirement is added implicitly to each job. Moreover, each agent has an implicit os=…​ capability which should be added to the agent pool for the sake of consistency and in case it’s an explicit requirement of any job.

Already connected agents that don’t provide all mandatory capabilities will stay connected for up to 4 hours after adding capabilities to an agent pool. During that time they will be shown with a warning indicator on the “Agents” tab. In order to add capabilities to an existing pool, it is recommended to first add the capability to all connected agents and then edit the agent pool.

Priority

Pools with higher priority are utilized before pools with lower priority. If tests to be executed can be executed by agents from different pools, agents of the pool with the highest priority are requested. When scaling to meet demand, higher priority pools are scaled to their configured maximum before scaling lower priority pools.

You can change the priority of agent pools by dragging them on the “Configuration” tab. Pools that are higher on the list are prioritized over those below.

Deletion

Agent pools can be deleted at will when they are no longer needed. It is recommended to stop all agents in the agent pool before deleting it.

Already connected agents will stay connected for up to 4 hours after deleting an agent pool. During that time they will be shown with a warning indicator on the “Agents” tab.

Querying pool status

Agent auto scaling relies on auto scaling machinery of a compute platform regularly querying Gradle Enterprise for a pool’s desired size and starting/stopping agents to match the desired size.

The https://«hostname»/distribution/agent-pools/«pool-id» endpoint conveys a pool’s desired size at the moment, along with other information about the pool, as a JSON document.

Accessing the endpoint requires an API key via the X-API-KEY HTTP header. It can also be accessed by administrators in a browser when signed in to Gradle Enterprise.

The following demonstrates using curl to access the endpoint:

curl -H "X-API-KEY: «api-key»" https://«hostname»/distribution/agent-pools/«pool-id»

Which will produce output of the following form:

{
  "id": "«pool-id»",
  "name": "Linux",
  "capabilities": [
    "jdk=11",
    "os=linux"
  ],
  "minimumAgents": 1,
  "maximumAgents": 100,
  "connectedAgents": 25,
  "idleAgents": 9,
  "desiredAgents": 16
}

Auto scalers need only consider the desiredAgents field and adjust the number of running agents to match this number.

Historical usage

The “Usage” tab provides a historical overview of workload in terms of number of active jobs, agent utilization, and agent pool metrics, for the last hour, 24 hours, or 7 days.

testdistribution usage 2021.2

By default, all jobs, agents, and agent pools are shown. Adding requirement/capability criteria shows only the jobs with at least the specified requirements, only the agents with at least the specified capabilities, and only the agent pools with at least the specified capabilities.

Predictive Test Selection

Gradle Enterprise Predictive Test Selection allows developers to get faster feedback by running only tests that are likely to provide useful feedback on a particular code change using a probabilistic machine learning model. It is available for Gradle and Apache Maven™ builds.

Builds that have enabled Predictive Test Selection connect to Gradle Enterprise to select relevant tests. Gradle Enterprise users must be granted the “Build scan view” access control role to be able to use Predictive Test Selection. For information on how to use Predictive Test Selection, please consult the Predictive Test Selection User Manual.

Appendix A: Upgrade notes

Appendix B: Amazon S3 Build Scan storage

Create an S3 bucket for Build Scan storage

You will need to know the AWS region that your Gradle Enterprise installation is running in.

If using the AWS web console:

  1. Go to S3 in the AWS console

  2. In the navigation pane, choose Buckets and then choose Create bucket.

  3. Enter the name you want for your bucket e.g. gradle-enterprise-build-scans. Make sure the bucket is in the same region as your Gradle Enterprise installation. Leave the other settings as their default values.

  4. Click Create bucket.

Alternatively, use the AWS command line utility:

aws s3 --region «your-gradle-enterprise-aws-region» create-bucket --bucket «your-bucket-name»

Create a policy allowing access to the S3 bucket

The following policy allows access to the S3 bucket. Granting the policy to a user or role that Gradle Enterprise will access it as is a separate step.

If using the AWS web console:

  1. Go to IAM in the AWS console

  2. In the navigation pane, choose Policies and then choose Create policy.

  3. Click on the JSON tab to define the policy using JSON directly.

  4. Enter the following policy JSON, replacing «your-bucket-name» with the name of your bucket:

    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Sid": "VisualEditor0",
                "Effect": "Allow",
                "Action": [
                    "s3:ListBucket"
                ],
                "Resource": [
                    "arn:aws:s3:::«your-bucket-name»"
                ]
            },
            {
                "Sid": "VisualEditor1",
                "Effect": "Allow",
                "Action": [
                    "s3:PutObject",
                    "s3:GetObject",
                    "s3:DeleteObject",
                    "s3:AbortMultipartUpload"
                ],
                "Resource": [
                    "arn:aws:s3:::«your-bucket-name»/*"
                ]
            }
        ]
    }
  5. Click Next: Tags.

  6. Add any tags that your organisation requires, and click Next: Review.

  7. Enter a name for this policy e.g. GradleEnterpriseBuildScanStorage, and optionally also a description

  8. Click Create policy.

Alternatively, use the AWS command line utility:

  1. First, save the above JSON policy to a file policy.json.

  2. Run the following command:

    aws iam create-policy \
      --policy-name "«your-policy-name»" \
      --policy-document file://policy.json

    It is possible to add tags and a description using the CLI, see the documentation.

  3. Note the policy ARN.

Create an AWS user to access S3 for Build Scan storage

If using the AWS web console:

  1. Go to IAM in the AWS console

  2. In the navigation pane, choose Users and then choose Add users.

  3. Enter a user name e.g. gradle-enterprise-build-scans.

  4. Select Access key - programmatic access for the AWS credential type.

  5. Click Next: Permissions.

  6. Select Attach existing policies directly.

  7. Select the IAM policy you created above, and no others.

  8. Click Next: Tags* and add any tags that your organisation requires.

  9. Click Next: Review, then Create user

  10. At this point, you will be shown the access key credentials for your new user, the Access key ID and the Secret access key. Store these somewhere secure. They will be needed to configure Gradle Enterprise for S3 Build Scan storage.

Alternatively, use the AWS command line utility:

aws iam create-user --user-name "«your-user-name»"
aws iam create-access-key --user-name "«your-user-name»"

It is possible to add tags using the CLI, see the documentation.

The output will contain the access key credentials for your new user in the AccessKeyId and SecretAccessKey fields. Store these somewhere secure. They will be needed to configure Gradle Enterprise for S3 Build Scan storage.

Create an AWS role for EC2 instance profile access to S3 for Build Scan storage

If you already have a role that your EC2 instances are launched with, skip these steps and instead attach the policy to the existing role directly.

If using the AWS web console:

  1. Go to IAM in the AWS console

  2. In the navigation pane, choose Roles and then choose Create role.

  3. For the Trusted Entity type, choose AWS service, then EC2 for the Use case.

  4. Click Next.

  5. Find and select the IAM policy you created above.

  6. If you are running Gradle Enterprise in Amazon EKS and wish to use node-associated roles, also add the built-in AmazonEKSWorkerNodePolicy and AmazonEC2ContainerRegistryReadOnly policies, and an applicable network policy. See here for full steps on finding these in the console.

  7. Click Next.

  8. Add a role name, e.g. gradle-enterprise-build-scans.

  9. Add any tags that your organisation requires.

  10. Click Create role.

Alternatively, use the AWS command line utility:

  1. Create a file ec2-role-trust-policy.json and save it with the following content:

    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Effect": "Allow",
          "Principal": { "Service": "ec2.amazonaws.com"},
          "Action": "sts:AssumeRole"
        }
      ]
    }
  2. Run the following commands, using the ARN of the policy created above:

    aws iam create-role \
      --role-name "«your-role-name»" \
      --assume-role-policy-document file://ec2-role-trust-policy.json
    
    aws iam attach-role-policy \
      --role-name "«your-role-name»" \
      --policy-arn "«your-policy-arn»"
    
    aws iam create-instance-profile \
      --instance-profile-name "«your-role-name»"
    
    aws iam add-role-to-instance-profile \
      --instance-profile-name "«your-role-name»"
      --role-name "«your-role-name»"

    It is possible to add tags using the CLI, see the documentation.

If you already have Gradle Enterprise installed on an EC2 instance, please follow these instructions to associate the instance with the new profile.

Appendix C: Gradle Enterprise Admin CLI JAR downloads

Appendix D: Gradle Enterprise config schema downloads