Gradle Enterprise Helm Kubernetes Installation Manual

Gradle Enterprise is distributed as a Helm chart.

Helm can manage components of a cluster, tracking what has been installed and upgrading it gracefully. The helm command is used to install Gradle Enterprise into the cluster and manages each installation / upgrade as a Helm release. See the Helm documentation for explanation of Helm concepts. This is referred to as a Helm-managed install in this guide, and is the recommended way to install Gradle Enterprise.

There are alternative installation options for more environments for which this is not suitable - see Advanced installation.

Gradle Enterprise can store data in either:

The two have different trade-offs. Please consider these prior to installation of Gradle Enterprise.

Gradle Enterprise has been tested as compatible with Kubernetes API versions 1.11.x to 1.24.x. Later versions may be compatible but have not been verified to work.

Gradle Enterprise does not use any platform specific features and is expected to work on all platforms, but not all are verified to work. Gradle Enterprise is verified to work on Minikube, K3s, EKS, and OpenShift. It has also been tested as compatible with GKE and AKS.

Gradle Enterprise requires Helm version 3.5.x (or later) to install. It is recommended to use the latest version available as this will have all known security vulnerabilities addressed. Helm releases have specific supported Kubernetes versions. Please check the Helm Version Support Policy to ensure compatibility with your Kubernetes version.

The user running Helm or applying its output should have permission to create Kubernetes resources in the designated namespace.

If your Kubernetes environment has fine-grained permissions such that your Kubernetes account may not be able to create certain types of resources, it is recommended to inspect the resources that Gradle Enterprise creates by running helm template and viewing the output.

Helm configuration can be provided in several ways:

Nested items in the values file have equivalents on the command line. So a values file with this content:

is equivalent to passing --set global.hostname=ge.example.com --set database.type=embedded on the command line.

It is also possible to put more complex data into a Helm values file, such as a whole file:

Unless otherwise indicated, most values are optional and have usable defaults.

Although the most commonly required configuration options are documented below, the example values file that can be found here and is included in the distribution has descriptions for all supported configuration options. After installing Helm and adding the Gradle Helm repository, this file can be viewed by running the following command:

The output of this command can be redirected to a file to use as the starting point for your configuration:

It’s also possible to see these values when installing from an airgap bundle. After downloading the bundle and transferring it to a host with Helm installed, extract and then run helm show values with the included chart:

In an airgap installation, Helm must be provided the hostname of the internal image registry. If that registry requires authentication, then the name of a Kubernetes secret that can be used as an image pull secret must also be provided.

You will have been provided with a Gradle Enterprise license file. This file can be used in any testing, staging or production deployments.

Your Gradle Enterprise license file must be provided to Helm. This can be done in a number of ways.

You can provide the file to helm using --set-file:

Alternatively, you can embed the file into your Helm values file:

It is also possible to specify the Gradle Enterprise license in its compact form, which is just the "data" part of the license.

This can be more convenient in some provisioning setups.

It is also possible to specify the license file as a Kubernetes secret. This can then be managed manually, or by tooling such as Sealed Secrets or AWS Secrets Manager.

To manually create a secret with the Gradle Enterprise license:

Then configure Helm with the name of the secret:

When specifying the license as a secret and pulling images from the default registry (i.e. you have not specified a custom registry by setting global.image.registry), you also need to create an image pull secret for the default registry and provide it to Helm. To create the image pull secret for the default registry, use the content of your license file as the password:

As described above, a hostname for the application must be supplied:

There are a number of ways to route web traffic to Gradle Enterprise from outside the Kubernetes cluster. These include:

Which you choose to use will depend on your organizational policies and available infrastructure.

Connecting to the application over HTTPS is strongly recommended. Gradle Enterprise can be configured to securely serve traffic over HTTPS based on user-provided certificates. If certificates are not supplied, self-signed certificates will be generated and used, though this is not recommended for production operation.

By default, Gradle Enterprise will use an embedded database that stores its data in a persistent volume provided by the cluster. In this configuration, there is nothing else to set up regarding the database.

When Gradle Enterprise is configured to store data in a user-managed database, it must be provided with connection settings and credentials for the database. These can be provided either to Helm as configuration or as Kubernetes ConfigMap and Secret resources. Configuring these in Helm is the simplest approach during installation. Providing these values via external Kubernetes resources allows performing update (e.g. changing credentials) without having to rerun helm, and allows integration with other Kubernetes tooling (e.g. for Secret provisioning).

Standard connection settings like host, port and database name must be provided. Optionally, additional JDBC parameters can be specified.

These can also be provided as a ConfigMap. The name of the ConfigMap must be provided, and the resource itself must be created prior to starting Gradle Enterprise.

The ConfigMap is then configured like this:

The port, dbname and jdbcParams properties are all optional. The default database name is gradle_enterprise.

The database must have already been created using CREATE DATABASE, the createdb command or an equivalent mechanism in a cloud database interface prior to configuring Gradle Enterprise with connection details.

There are two options for credentials. If provided with credentials for a database superuser (such as the postgres user that is common on PostgreSQL database instances), Gradle Enterprise can perform all further database setup, and can do so for all subsequent Gradle Enterprise upgrades. This is the recommended configuration.

Note that in some installations, and often in cloud-based databases, the typical credentials provided by the database provider are not a true superuser, but have many of the same abilities. For example, the supplied postgres account in Amazon RDS Postgres databases is not a true Postgres superuser but has the rds_superuser role. Such accounts are fine to configure Gradle Enterprise to use.

These credentials can also be supplied as a Kubernetes Secret:

The Secret should have the typical username and password keys, encoded using Base64:

In some instances it may be be preferable to not supply Gradle Enterprise with database superuser credentials. It is possible to instead run a database setup script on the database, which will set up less privileged accounts for the application to use, and some privileged functions needed for the application to run. The credentials for the accounts must then be set by the user and provided to Gradle Enterprise via Helm configuration or Secrets.

These credentials can also be supplied as Secret resources. There are two accounts that must be configured, requiring a Secret each.

In this example stringData is used (which does not require Base64 encoding) to show the correct usernames to use. The usernames are currently not configurable.

Please contact Gradle support for assistance in setting up a database without providing superuser credentials.

A number of Gradle Enterprise components can be scaled horizontally to provide greater performance and availability.

Gradle Enterprise’s scalable components are implemented as Kubernetes StatefulSet resources. It is also possible to alter their replica count directly using kubectl scale, allowing integration with other Kubernetes tooling.

It is only recommended to alter the replica count directly when connecting to a user-managed database, as available embedded database connections are configured based on the replica count. The user-managed database should have (160 ✖️ replica count) connections available.

Gradle Enterprise needs to know when it is being deploying to an OpenShift cluster.

Many aspects of Gradle Enterprise’s behaviour can be configured via the Admin user interface. These are described in the Administration Manual.

It is possible to pre-configure Gradle Enterprise with the same settings that can be set via the Admin UI. This is useful to create a working Gradle Enterprise instance in a completely automated way, with no need to configure anything via a user interface.

The settings take the form of a yaml file. This file can be hand-written or exported from Gradle Enterprise. For more details on creating a Gradle Enterprise configuration yaml file, see Unattended configuration in the Administration Manual.

Having prepared a Gradle Enterprise configuration file and optionally an encryption key, you can provide it to Helm in one of two ways.

Example key:

Example yaml file:

The file and encryption key can be provided inline in a Helm values file under the global.unattended property:

Or they can be supplied as helm arguments using --set-file:

If your cluster has internet connectivity, perform an online installation as described below.

Airgap installations require a Docker or compatible registry available on an internal network that is accessible from the cluster to which Gradle Enterprise images can be pushed.

Airgap installations require a specific entitlement on your license. Please contact Gradle if you need an Airgap-enabled license.

For Airgap installations:

We recommend you save all the files into a single transfer directory so that it is easy to transfer to other hosts. For example:

See the appendix for discussion of advanced installation options.

At this point, it should be possible to see the Helm release installed:

You can inspect the status of the Gradle Enterprise pods:

For airgap installations, it is recommended to remove the following files after installation:

Once Gradle Enterprise has been installed, files used during installation are not required at runtime and can be removed if desired. However, the following files may be useful to preserve, as they may aid in future upgrades or maintenance:

If you have configured more than one replica (see Horizontal scaling configuration), you must scale the replicas down to one before upgrading to avoid having mixed versions running at the same time.

Before applying the upgrade, run the following command to scale the application down to one replica:

Where «deployed-version» is the running version of Gradle Enterprise, not the version being upgraded to.

The upgrade to the new version can now be executed. If your upgrade processes uses --reuse-values, you will need to include a --set=global.scaling.replicas=N argument where N is the desired replica number. The following example performs a version upgrade and changes the replica number in one operation:

Storage classes cannot be changed once a persistent volume claim is in place. If you need to alter any configured Gradle Enterprise storage classes, please contact Gradle support for assistance.

If you are experiencing issues with Gradle Enterprise, please see the Administration Manual for information about how to submit a support ticket.

Starting with 2022.2, Gradle Enterprise instances installed with Helm validate their licenses with Gradle over the internet. Once running, Gradle Enterprise will periodically check for license updates. Before upgrading a Helm-based installation to Gradle Enterprise 2022.2 or later, ensure that harbor.gradle.com is reachable from the Gradle Enterprise host.

See Network connectivity requirements and Verifying network connectivity for more information.

Upon upgrading, a data reindexing process will be initiated in the background with Gradle Enterprise being usable for its duration. CPU usage will be increased and performance may be slightly degraded. For large installations storing many Build Scans, the reindexing process may take several hours. During this time, some builds may be omitted from cross-build data visualizations. Recent builds are prioritized, making their data available sooner.

Improvements to Predictive Test Selection in this release, that improve its effectiveness, require model retraining and data reindexing.

For installations that have enabled Predictive Test Selection or the Predictive Test Selection Simulator, this occurs automatically upon upgrade and may temporarily degrade performance. During this time, Predictive Test Selection will be unavailable with all tests being selected at build time, and simulation data will be unavailable. Most installations will complete this process within 24 hours.

Users that have not enabled Predictive Test Selection or the Predictive Test Selection Simulator are unaffected.

When asking Helm to install or process a chart, it is possible to pick a number of sources:

Which of these to use depends somewhat on network and policy requirements.

Many organizations require the ability to customise Kubernetes manifests prior to applying them to their cluster. Kustomize is one commonly used tool to do this. It is possible to alter the Helm-generated Kubernetes manifests prior to Helm applying them to the cluster, using Helm Post Rendering via the --post-renderer flag. In this way, manifests can be customised while still allowing Helm to manage what is applied to the cluster.

The post-renderer is given the combined manifests as input and must provide the altered manifests to standard output.

An example of a Helm post renderer for Kustomize is provided below:

An alternative to running helm install directly on your cluster is to have Helm generate Kubernetes manifests that can then be applied via kubectl or other Kubernetes tooling. In this mode, Helm does not keep track of previously installed versions - the resources in the cluster are not tracked and Helm cannot report on the currently installed version.

The most common reasons to do a user-managed installation are:

There are some other downsides:

An internal Helm chart repository can be used to mirror the Gradle Enterprise Helm charts.

Using helm pull to fetch the charts is the recommended approach since the Gradle Enterprise Helm repo index uses relative urls.

Starting with 2022.2, Gradle Enterprise instances installed with Helm validate their licenses with Gradle over the internet. Once running, Gradle Enterprise will periodically check for license updates. Before upgrading a Helm-based installation to Gradle Enterprise 2022.2 or later, ensure that https://harbor.gradle.com is reachable from your Kubernetes cluster.

See Network connectivity requirements and Verifying network connectivity for more information.