Version 2022.4

This manual covers the installation of Gradle Enterprise into an existing Kubernetes cluster, using the Helm Kubernetes package manager. It is useful for administrators of Gradle Enterprise installations.

This Gradle Enterprise installation method is suitable if your organisation has Kubernetes infrastructure or if you are familiar with Kubernetes and are comfortable operating a Kubernetes cluster. Alternatively, you can install the Gradle Enterprise standalone distribution.

Before installation

Installation Overview

Gradle Enterprise is distributed as a Helm chart.

Helm can manage components of a cluster, tracking what has been installed and upgrading it gracefully. The helm command is used to install Gradle Enterprise into the cluster and manages each installation / upgrade as a Helm release. See the Helm documentation for explanation of Helm concepts. This is referred to as a Helm-managed install in this guide, and is the recommended way to install Gradle Enterprise.

There are alternative installation options for more environments for which this is not suitable - see Advanced installation.

Database type

Gradle Enterprise can store data in either:

  • An embedded database that uses a local directory or volume to store its data, or

  • A user-managed database that is completely separate from Gradle Enterprise.

The two have different trade-offs. Please consider these prior to installation of Gradle Enterprise.

Embedded database

When using the embedded database, Gradle Enterprise will run a PostgreSQL database in a container and store data in a persistent volume in your cluster. In this mode Gradle Enterprise can run backups on a regular or cron-like schedule.

The embedded database has several advantages:

  • Simple setup: there is no other database to provision or configure.

  • The database will get upgraded in-place to the latest PostgreSQL version supported by Gradle Enterprise with the relevant Gradle Enterprise upgrade.

  • Gradle Enterprise features that need disk space usage information are supported.

  • For smaller installations it is often cheaper than using a separate database.

However, there are downsides to the embedded database, in particular for larger installations:

  • Disk I/O throughput can be a bottleneck to processing on a busy system. This can be difficult to tweak in many setups.

  • Backup management can get difficult past a certain database size. Specifically, backup generation can be slow and create load on the server, and managing backup files for a terabyte-sized database can be awkward.

  • There is no ability to keep a standby database available.

  • The database is a single point of failure. However, for single-node Gradle Enterprise installations any system outage will likely take the service down, regardless of the database.

User-managed database

A user-managed database can be any PostgreSQL 12, 13, or 14 compatible database, but the most common usage is a cloud-based database provider such as Amazon RDS or Aurora. These have a number of advantages, particularly in large installations:

  • Database systems need a different set of resources to be provisioned to run smoothly. For example, a high-traffic Gradle Enterprise instance processing a lot of Build Scans will need a lot of memory, but the database needs a lot of I/O throughput and CPU. Separating the database to a different system that can be scaled differently can allow more cost-effective resource allocation.

  • Most cloud-based database providers have tools that allow efficient snapshotting of databases, both manually and scheduled. This allows faster and more convenient backup and data management.

  • Many cloud database instances allow scaling of resources easily after an initial installation. This makes adding more storage or upgrading the instance type easier than changing the configuration of nodes that Gradle Enterprise is installed on.

  • It is possible to run standby databases for failover in the case of an outage. If self-hosting, this can be done using something like wal-e, and many cloud database providers have this functionality built in to the service.

  • Some cloud database providers have hot standby and failover functionality built in to the service.

There are some downsides to using a user-managed database:

  • Extra system(s) to provision and configure.

  • Administrator must ensure network connectivity to the cloud instance, and consider network latency when choosing where to set up a database.

  • Backups must be completely managed by the administrator or cloud database tool.

  • Disk space management and alerting must be completely managed by the administrator or cloud database tool. Gradle Enterprise features that allow certain actions at disk space percentage thresholds are not available.

  • There are extra security considerations such as credential cycling.

Build Scan storage

By default, Gradle Enterprise stores Build Scan data in the configured PostgreSQL database. This has the advantage of being simple to deploy and operate. It has the disadvantage of being expensive and difficult to scale for larger installations. Utilizing an additional S3-compatible store is a compelling deployment option for larger installations that produce large volumes of data.

Please see the Administration Manual for more details.

Installation requirements

This section outlines the installation resource requirements and configuration aspects that need to be resolved during installation.

Supported Kubernetes versions

Gradle Enterprise has been tested as compatible with Kubernetes API versions 1.11.x to 1.24.x. Later versions may be compatible but have not been verified to work.

Supported Kubernetes platforms

Gradle Enterprise does not use any platform specific features and is expected to work on all platforms, but not all are verified to work. Gradle Enterprise is verified to work on Minikube, K3s, EKS, and OpenShift. It has also been tested as compatible with GKE and AKS.

Helm requirements

Gradle Enterprise requires Helm version 3.5.x (or later) to install. It is recommended to use the latest version available as this will have all known security vulnerabilities addressed. Helm releases have specific supported Kubernetes versions. Please check the Helm Version Support Policy to ensure compatibility with your Kubernetes version.

Network connectivity requirements

Runtime network connectivity requirements

Gradle Enterprise can be run in clusters whose nodes can pull images from the internet, referred to as an online install in this guide, or in clusters without internet connectivity, referred to as an airgap install in this guide.

When starting, online installations of Gradle Enterprise will validate its license with Gradle over the internet. Once running, Gradle Enterprise will periodically check for license updates.

An online installation of Gradle Enterprise will not start if it cannot connect to both registry.gradle.com and harbor.gradle.com.

Airgap installations of Gradle Enterprise do not require internet connectivity when running (airgap installations do not validate the license on startup and do not periodically check for updates) and therefore do not require any special firewall rules. However, the airgap bundle used during installation must be downloaded from an internet-connected host. See Airgap installation for details.

Airgap installations require a Docker or compatible registry available on an internal network that is accessible from the cluster to which Gradle Enterprise images can be pushed.

Airgap installations require a specific entitlement on your license. If you need an airgap-enabled license, please contact your customer success representative.

Installation network connectivity requirements

Downloading the Gradle Enterprise Helm chart and container images requires internet access.

For online installations, Helm must be run on a host that has access to both the internet and the Kubernetes cluster.

For airgap installations:

  • The installation files must be downloaded on a host with access to the internet

  • The images must be uploaded to the internal container registry on a host with network access to the internal container registry (no internet access required)

  • Helm must be run on a host with network access to the Kubernetes cluster (no internet access required)

  • There must be some mechanism for transferring the downloaded installation files to where they will be used

Network connectivity verification

See Verifying network connectivity for instructions on verifying network access prior to installation.

Resource requirements

Node group specification

If you are planning to provision a dedicated cluster for your Gradle Enterprise installation, our recommended node group specification for that cluster is 3 nodes, each with roughly 4 CPU units and 16 GiB memory (e.g. AWS EC2’s m5.xlarge). If your workload is very low, you can use 3 smaller nodes, each with 2 CPU units and 8 GiB memory (e.g. AWS EC2’s m5.large).

Resource requests and limits

If you are planning to install Gradle Enterprise in an existing cluster, we recommend ensuring access to at least the following CPU units and memory: 4 CPU units, 16 GiB memory. The Gradle Enterprise Helm chart’s total resource requests and limits are: 6.85 CPU units, 13.53125 GiB memory; and 15.55 CPU units, 25.03125 GiB memory, respectively.

In practice, provisioning only minimal resources will be too little for most workloads. Instead, you should use our recommended values.

Kubernetes required permissions

The user running Helm or applying its output should have permission to create Kubernetes resources in the designated namespace.

If your Kubernetes environment has fine-grained permissions such that your Kubernetes account may not be able to create certain types of resources, it is recommended to inspect the resources that Gradle Enterprise creates by running helm template and viewing the output.

Storage requirements

Storage Class

Gradle Enterprise uses persistent volume claims for storing data, logs and backups. You will need to provide the name of the desired storage class to be used for provisioning persistent volumes. Different storage classes can be specified for the three different types of storage used.

Some pods are associated with multiple persistent volumes and for Kubernetes platforms with multiple availability zones, the pods and their persistent volumes must be located in the same zone. In this case it is recommended to use a storage class with a volumeBindingMode of WaitForFirstConsumer to ensure that all persistent volumes are provisioned in the same zone that the pod was scheduled in.

It is strongly recommended to use storage classes that allow persistent volume claim expansion if available. This makes expanding storage used as usage of Gradle Enterprise increases straightforward.

Capacity

The recommended minimum capacities for the persistent volumes are:

Description Size in GB

Build Scans

250*

Build Scans backups

250*

Build Cache

10

Test Distribution

10

Logs

2

* Will not be present when using a user-managed database - see below.

If you are producing many build scans a day or intend to retain build scans for long periods of time you may want to consider provisioning more storage. If your storage provisioner does not allow expanding volumes you should also consider preparing for future data growth.

Performance

Disk performance has a significant impact on Gradle Enterprise.

For production workloads, the data storage class should exhibit SSD-class disk performance of at least 3000 input/output operations per second (IOPS). The storage classes used for logs and backup volumes may be slower.

If you are experiencing performance problems, consider reviewing your Database type and Build Scan storage options. You can also contact Gradle support for advice.
Amazon Web Services

If you are using Amazon Web Services (AWS) and Elastic Block Storage (EBS) to host your Gradle Enterprise instance you should ensure that you provision 3,000 or more IOPS for the data volumes. Keep in mind that some general purpose volumes (e.g. gp2) are limited to 3 IOPS per gigabyte, meaning that a 200 GB volume will only provide a maximum of 600 IOPS. If you are provisioning a volume smaller than 1TB you should consider using gp3 or a provisioned IOPS volume.

The recommendations above are based on average workloads. For projects with more complex builds or teams that produce a large number of build scans, Gradle Enterprise may require higher I/O performance than suggested.

User-managed database

Gradle Enterprise can be configured to store data in a database that is hosted and managed separately to the rest of the application. This can help with performance in some circumstances by allowing database resources to be scaled separately to Gradle Enterprise. It can also simplify backups and volume management when using a database provider that provides tooling for these out of the box.

When using a user-managed database, the "Build Scans" and "Build Scans backups" capacity requirements will be fulfilled by the database instead of Gradle Enterprise. As such, it is recommended that your user-managed database has a capacity of at least 250 GB.

When using a user-managed database, some Gradle Enterprise administrative features such as automated backups and build scan disk space management features that respond to low disk space scenarios are not available.
Compatibility

Gradle Enterprise needs to connect to a PostgreSQL version 12, 13 or 14 compatible database.

Using PGBouncer with a user managed database is not supported. Gradle Enterprise uses and depends on PostgreSQL connection parameters that are not supported by PgBouncer.
Performance and capacity considerations

For similar workloads, Gradle Enterprise will need similar storage capacity and IOPS throughput as for the storage volumes mentioned above. Please see your database provider’s options for provisioning and ensure that they meet these requirements.

When Gradle Enterprise and the database are not deployed in a private network near each other, consideration should also be given to ensuring that the database is hosted as close to the Gradle Enterprise host as possible. In a cloud provider this typically means deploying within the same availability zone or equivalent. This will ensure the lowest latency and highest throughput. Gradle Enterprise will not perform adequately under load if latency to its database is too high.

Backups

When using a user-managed database, backups must be handled by the database provider. Gradle Enterprise built-in backups will not operate in this mode.

Networking configuration

Application hostname

When installing Gradle Enterprise, you will need to provide a hostname, such as ge.example.com. This should be the hostname that users of the installation use to access it and therefore should resolve within your network. This may depend on how you intend on directing external traffic into your cluster to access Gradle Enterprise. Please refer to this section for more information.

Proxy configuration

By default, Gradle Enterprise requires an internet connection to make several outbound HTTP requests (such as to validate its license) on startup. In case your organization requires all outbound HTTP traffic to go through an HTTP proxy, you must perform additional configuration for this to work.

HTTP proxy configuration can be specified at installation time via the unattended configuration mechanism, under the network section in the unattended configuration file.

For example, using the inline Helm values support to include the unattended config in a values file:

values.yaml
global:
    unattended:
      configuration:
        version: 5
        systemPassword: ...
        network:
         proxy:
           protocol: http (1)
           host: proxy.gradle.com (2)
           port: 8080 (3)
           excludedHosts: (4)
             - some.external
             - '*.internal'
           auth: (5)
             username: proxy_user
             password: "aes256:B0uVHRDhng+PraUI:2bOz71vKTexz0QH5:z7lO+1wOC/tA3izLAwV0BXMugg=="
1 The protocol used to connect to the proxy. Note that this is not the protocol used to connect to the destination/target addresses. Supported values are http and https, if no value is provided http will be used as the default protocol.
2 HTTP proxy host name.
3 HTTP proxy port, if no value is provided 80 will be used ad the default port.
4 A comma-delimited list that controls what hosts should not be proxied. The list can contain individual host names as well as domain patterns (e.g. '*.internal') which match all hosts for a particular domain. Any requests sent to these hosts will be sent directly rather than being sent through the HTTP proxy. Gradle Enterprise is connecting to very few external hosts, see the network connectivity section for more info.
5 A username and password used to authenticate with the HTTP Proxy.

Proxy configuration can also be configured after installation, see the Proxy configuration section of the administration guide. Proxy SSL certificates can be configured following HTTPS SSL certificate section.

HTTPS SSL certificate

It is strongly recommended that production installations of Gradle Enterprise are configured to use HTTPS with a trusted certificate.

Gradle Enterprise natively supports serving traffic over HTTPS when configured with a certificate and key. If you intend to use an ingress controller for directing external traffic to Gradle Enterprise, you may opt to terminate HTTPS there. It is also possible to terminate HTTPS connections in an external reverse proxy.

Untrusted SSL certificates

By default, Gradle Enterprise uses the default trust settings of the Java runtime that it ships with when connecting to other systems using SSL. If your organization uses certificates that are not signed by a trusted certificate authority, you must perform additional configuration for this to work. This may be the case if you use self-signed certificates or an internal certificate authority.

Additional trusted certificates can be specified at installation time via the unattended configuration mechanism, using the additionalTrust field in the unattended configuration file. The value of this field should be the X509 certificates to trust in PEM format, newline-seperated if there are more than one.

For example, using the inline Helm values support to include the unattended config in a values file:

values.yaml
global:
    unattended:
      configuration:
        version: 5
        systemPassword: ...
        network:
         additionalTrust: |
             -----BEGIN CERTIFICATE-----
             MIIDfzCCAmegAwIBAgIURqPslYGu7cHXs22q3RK6e5L87PwwDQYJKoZIhvcNAQEL
             ...
             s10yB5VjVBES6A22rYwYb8mImpQiVP/mr4ao5U5m+h50l3E=
             -----END CERTIFICATE-----
             -----BEGIN CERTIFICATE-----
             DSE3a3CCAmegAwIBAgIURqPslYGu7cHXs22q3RK6e5L87PwwDQYJKoZIhvcNAQEL
             ...
             s10yB5VjVBES6A22rYwYb8mImpQiVP/mr4ao5U5m+h50l3E=
             -----END CERTIFICATE-----

Additional trusted certificates can also be configured after installation, see the Untrusted SSL certificates section of the administration guide.

Helm configuration

Many aspects of Gradle Enterprise can be configured inside the application in the Administration screens. However, some things must be configured via Helm. These are typically things that are necessary to get Gradle Enterprise up and running and serving requests.

Prior to installing Gradle Enterprise, you will need to prepare a Helm values file and have your Gradle Enterprise license and SSL certificates (optional) available on the host where you will run Helm.

Providing configuration to Helm

Helm configuration can be provided in several ways:

  • Passing values directly to the helm command using --set or --set-file.

  • Creating a Helm values file and passing it to helm using --values.

  • Editing the default Helm values file in the chart prior to running helm.

Nested items in the values file have equivalents on the command line. So a values file with this content:

values.yaml
global:
  hostname: ge.example.com
database:
  type: embedded

is equivalent to passing --set global.hostname=ge.example.com --set database.type=embedded on the command line.

It is also possible to put more complex data into a Helm values file, such as a whole file:

values.yaml
global:
  license:
    file: |
      multi
      line
      content
      here

Unless otherwise indicated, most values are optional and have usable defaults.

Example values file

Although the most commonly required configuration options are documented below, the example values file that can be found here and is included in the distribution has descriptions for all supported configuration options. After installing Helm and adding the Gradle Helm repository, this file can be viewed by running the following command:

$ helm show values gradle/gradle-enterprise

The output of this command can be redirected to a file to use as the starting point for your configuration:

$ helm show values gradle/gradle-enterprise > values.yaml

It’s also possible to see these values when installing from an airgap bundle. After downloading the bundle and transferring it to a host with Helm installed, extract and then run helm show values with the included chart:

$ helm show values gradle-enterprise-2022.4.7.tgz (1)
1 The file argument is a Helm chart archive as created by helm pull or extracted from an airgap bundle, not the airgap bundle itself.

Airgap configuration

In an airgap installation, Helm must be provided the hostname of the internal image registry. If that registry requires authentication, then the name of a Kubernetes secret that can be used as an image pull secret must also be provided.

values.yaml
global:
  image:
    registry: registry.example.com/gradle-enterprise
    imagePullSecret: example-image-pull-secret-name

Gradle Enterprise license

You will have been provided with a Gradle Enterprise license file. This file can be used in any testing, staging or production deployments.

Your Gradle Enterprise license file must be provided to Helm. This can be done in a number of ways.

You can provide the file to helm using --set-file:

$ (helm command) --set-file global.license.file=path/to/gradle-enterprise.license

Alternatively, you can embed the file into your Helm values file:

values.yaml
global:
  license:
    file: |
      ====================== GRADLE ENTERPRISE LICENSE V1.0 ======================
      License file ID: YC3IWLASSXB5E
      Issued to: ACME Co.
      Issued at: 2022-02-11T05:11:38Z
      Expires at: 2023-02-11T00:00:00Z
      ============================================================================
      R0VMRgF4nBWOSZKCMAAAX+QUu3BUIJAIwUQiwsViEwMIDKOyvH701n3p6nJBYxoRHnAUnwHwKLjb
      YZMAbN8B5BMv3HaCdc9PbcG+Hkdz93XW+gsOY34jP+WC1iKC38SeCCKANn57HP7Bx/lVmFCLROHL
      Sv5hP4xlXO9kf40n35x4GgHhk5txTQQcMsmvG8Uz0W/hNN9eQlaMzsK8ZyKOSX0GsBN+dsp5Wq10
      /kVuf6u8LfUCPc72q79TWCvmFvNQD6RemcTxAbfILTNuaMsNgLjdCjYVK7padCLiolv2wc1P3pXF
      GoAExap9ZPKZTXpeaAaPeFF2qANCH/eXVW7+8kVxz0VH4PPnPjXc1yWzmHNTBu2WbpfDaSJil/g8
      lqB6zfaOvfSKQGkVMNthtqjJdSCmne3gWXpFJpGAe7dKkrD9K/vpoUZNUWuV7dFbFpqkNhJn+XAQ
      r2cjAak5TluR14aCsX6Fr4JgfSPyWNYdOB7DXrlnNT7d3p+/cFoqGVOvqjfDrnIkVBPLK5PCufU8
      VQzz/fagwC6dMtq0MEw5bphm5+LQeFPg/AOoua9P
      ====================== END GRADLE ENTERPRISE LICENSE =======================

It is also possible to specify the Gradle Enterprise license in its compact form, which is just the "data" part of the license.

values.yaml
global:
  license:
    file: R0VMRgF4nBWOSZKCMAAAX+QUu3BUIJAIwUQiwsViEwMIDKOyvH701n3p6nJBYxoRHnAUnwHwKLjb...

This can be more convenient in some provisioning setups.

It is also possible to specify the license file as a Kubernetes secret. This can then be managed manually, or by tooling such as Sealed Secrets or AWS Secrets Manager.

To manually create a secret with the Gradle Enterprise license:

$ kubectl create secret generic my-example-license-secret --from-file=license=path/to/gradle-enterprise.license

Then configure Helm with the name of the secret:

values.yaml
global:
  license:
    secretName: my-example-license-secret

When specifying the license as a secret, an image pull secret must also be specified. If performing an airgap installation, an image pull secret must be supplied anyway if the internal registry requires authentication. If performing an online installation, the license file also needs to be created as an image pull secret:

$ kubectl create secret docker-registry my-example-license-image-pull-secret \
    --docker-username=ignored \
    "--docker-password=$(cat ./enterprise/internal-installs/gradle-enterprise.license)" \
    --docker-server=registry.gradle.com
values.yaml
global:
  image:
    imagePullSecret: my-example-license-image-pull-secret

Hostname

As described above, a hostname for the application must be supplied:

values.yaml
global:
  hostname: ge.example.com

Exposing Gradle Enterprise outside your cluster

There are a number of ways to route web traffic to Gradle Enterprise from outside the Kubernetes cluster. These include:

Which you choose to use will depend on your organisational policies and available infrastructure.

Supplied Ingress

Gradle Enterprise can create a Kubernetes Ingress resource for you that is managed as part of your Helm release. The Ingress will be bound to the hostname that is configured.

values.yaml
ingress:
  enabled: true
Your Ingress-Controller implementation has to be compatible with GRPC to support some Gradle Enterprise features (e.g. Bazel Build Scans). For more details about this, please consult your Ingress-Controller implementation.

User-managed routing

For compatibility and configuration reasons, we strongly recommend to use the supplied Ingress definition. If the provided one is not suitable, you will need to provide your own configuration aligned with Gradle Enterprise supplied Ingress.

Gradle Enterprise creates a service named gradle-proxy exposing ports 80 and 443 (when HTTPS is enabled, see below) for accessing the application. This is the service to which alternative routing infrastructure such as load balancers will need to route traffic.

You may need to configure your ingress differently to support Bazel using GRPC. If you need help on this subject, please contact Gradle Enterprise support for assistance in setting up this.

HTTPS

Connecting to the application over HTTPS is strongly recommended. Gradle Enterprise can be configured to securely serve traffic over HTTPS based on user-provided certificates. If certificates are not supplied, self-signed certificates will be generated and used, though this is not recommended for production operation.

HTTPS terminated at the built-in Ingress

When using the Gradle Enterprise supplied Ingress, HTTPS is enabled by default. It can be disabled to serve traffic over HTTP only:

values.yaml
ingress:
  enabled: true
  ssl:
    enabled: false

SSL certificates can be provided inline in the Helm values file:

values.yaml
ingress:
  enabled: true
  ssl:
    cert: |
      -----BEGIN CERTIFICATE-----
      MIIDKjCCAhKgAwIBAgIRAPNTIHf6/oUuzMKm3ffGNOgwDQYJKoZIhvcNAQELBQAw
      HDEaMBgGA1UEAxMRYXV0by1nZW5lcmF0ZWQtY2EwHhcNMjExMTMwMTU1NDU5WhcN
      ...
      Cn/3yUirFVTslrSYKAemLw8btLO6FDF9dc/lq1o7tKsYVuhEcjqnTah7puJjEN9h
      z+P5RmRxU/kaaFB+Vuw1pRezbaAtZNorVgXnBwrdseY4zLGyhAcGcR9v+VtCiQ==
      -----END CERTIFICATE-----
    key: |
      -----BEGIN RSA PRIVATE KEY-----
      MIIEpQIBAAKCAQEA4qV8JlqDMi7y85Ykq8dn7uIsi609D6KuFtlc+UvNYjatz0+u
      QzIr3iw//qf7sM8nx8fhGwuWvUWeCE6zbgKjuxDH82J9NQ0ctf70n0qVTeyW1CKR
      ...
      XlOfXr/xvkXA66PROgvVxfwpN/GNrLXFi1HvMg7MVZJUZQpNzpAzw5JTk2MbawOl
      G7tI0qQ6F20e5R4tPpEDKCFZykyvgGMhfLzsvVlrgaVW8QbVK4YWNtQ=
      -----END RSA PRIVATE KEY-----

Or supplied as helm arguments using --set-file:

$ (helm command) \
    --set-file ingress.ssl.cert=path/to/ssl.crt \
    --set-file ingress.ssl.key=path/to/ssl.key

Or the certificate can refer to a Kubernetes TLS Secret already available in the cluster:

values.yaml
ingress:
  enabled: true
  ssl:
    secretName: example-ssl-certificate-secret-name

HTTPS terminated at the built-in proxy service

If you want to expose Gradle Enterprise without Ingress Controller, HTTPS needs to be enabled at the gradle-proxy service.

values.yaml
proxy:
  ssl:
    enabled: true

SSL certificates can be provided inline in the Helm values file:

values.yaml
proxy:
  ssl:
    cert: |
      -----BEGIN CERTIFICATE-----
      MIIDKjCCAhKgAwIBAgIRAPNTIHf6/oUuzMKm3ffGNOgwDQYJKoZIhvcNAQELBQAw
      HDEaMBgGA1UEAxMRYXV0by1nZW5lcmF0ZWQtY2EwHhcNMjExMTMwMTU1NDU5WhcN
      ...
      Cn/3yUirFVTslrSYKAemLw8btLO6FDF9dc/lq1o7tKsYVuhEcjqnTah7puJjEN9h
      z+P5RmRxU/kaaFB+Vuw1pRezbaAtZNorVgXnBwrdseY4zLGyhAcGcR9v+VtCiQ==
      -----END CERTIFICATE-----
    key: |
      -----BEGIN RSA PRIVATE KEY-----
      MIIEpQIBAAKCAQEA4qV8JlqDMi7y85Ykq8dn7uIsi609D6KuFtlc+UvNYjatz0+u
      QzIr3iw//qf7sM8nx8fhGwuWvUWeCE6zbgKjuxDH82J9NQ0ctf70n0qVTeyW1CKR
      ...
      XlOfXr/xvkXA66PROgvVxfwpN/GNrLXFi1HvMg7MVZJUZQpNzpAzw5JTk2MbawOl
      G7tI0qQ6F20e5R4tPpEDKCFZykyvgGMhfLzsvVlrgaVW8QbVK4YWNtQ=
      -----END RSA PRIVATE KEY-----

Or supplied as helm arguments using --set-file:

$ (helm command) \
    --set-file proxy.ssl.cert=path/to/ssl.crt \
    --set-file proxy.ssl.key=path/to/ssl.key

Or the certificate can refer to a Kubernetes TLS Secret already available in the cluster:

values.yaml
proxy:
  enabled: true
  ssl:
    secretName: example-ssl-certificate-secret-name

Then, you need to create the following service in the namespace where Gradle Enterprise is installed:

gradle-enterprise.external-loadbalancer.yaml
apiVersion: v1
kind: Service
metadata:
  name: gradle-enterprise-external-loadbalancer
spec:
  selector:
    app: gradle-enterprise
    component: proxy
  ports:
  - name: https
    port: 443
    targetPort: 9443
  type: LoadBalancer

For more details about this kind of service, please refer to your Kubernetes Distribution documentation.

This solution can drastically increase the cost of your Kubernetes installation if you rely on cloud infrastructure. The standard and recommended way to install Gradle Enterprise is by using an Ingress.

HTTPS terminated externally

In many setups, a reverse proxy or load balancer will perform SSL termination. In these cases SSL certificates must be configured with that infrastructure.

Gradle Enterprise needs to know that the application will be accessed over externally terminated HTTPS. This is done with the following configuration:

values.yaml
global:
  hostname: ge.example.com
  externalSSLTermination: true

If you’re using the provided Ingress, you may wish to communicate between the load balancer and the application using HTTP by disabling SSL at the Ingress:

values.yaml
ingress:
  enabled: true
  ssl:
    enabled: false

GRPC

Gradle Enterprise uses the GRPC protocol for Bazel features. To support this, the application has to be exposed with grpc/http2 compatible components (e.g. Ingress-Controller, Firewall, Load balancers…).

Please consult documentation of those infrastructure or Kubernetes components to get more information about support and requirement for grpc compatibility. All example below use example annotations that are not relevant for your installation.

GRPC with Ingress annotations

Some Ingress-Controller require extra implementation-specific annotations on Ingress objects to be compatible with grpc. You can provide them using the following entry in the values.yaml:

values.yaml
ingress:
  enabled: true
  grpc:
    annotations:
      acme.ingress.kubernetes.io/backend-protocol: "GRPC"

GRPC with Service annotations

Some Ingress-Controller require extra implementation-specific annotations on Service objects to be compatible with grpc. You can provide them using the following entry in the values.yaml:

values.yaml
ingress:
  enabled: true
  grpc:
    serviceAnnotations:
      acme.ingress.kubernetes.io/upstream-protocol.h2c: "grpc"

Storage configuration

Storage class configuration

The Kubernetes StorageClass can be configured for data, logs and backup volumes. If omitted, the default storage class for the cluster will be picked, if there is one. Please consult your cluster documentation for available storage classes.

Not all clusters provide a useable default storage class. It’s strongly recommended to consult your cluster documentation and select from the available storage classes explicitly.
values.yaml
global:
  storage:
    data:
      class: some-provisioned-io-storage-class
    backup:
      class: slow-and-cheap-storage-class
    logs:
      class: general-purpose-storage-class
Backup storage class is not necessary when connecting to a user-managed database.

Storage capacity configuration

As described above, Gradle Enterprise needs a certain amount of storage for data, logs and backups. If omitted, the shown default amount of storage will be requested from the cluster.

values.yaml
database:
  storage:
    data:
      capacity: 1000Gi # default 250Gi
    backup:
      capacity: 1000Gi # default 250Gi
buildCacheNode:
  storage:
    data:
      capacity: 100Gi # default 10Gi
enterprise:
  storage:
    logs:
      capacity: 500Mi # default 200Mi
metrics:
  storage:
    data:
      capacity: 1Gi # default 200Mi
testDistribution:
  storage:
    data:
      capacity: 20Gi # default 10Gi
Backup storage capacity is not necessary when connecting to a user-managed database.

Database configuration

By default, Gradle Enterprise will use an embedded database that stores its data in a persistent volume provided by the cluster. In this configuration, there is nothing else to set up regarding the database.

When Gradle Enterprise is configured to store data in a user-managed database, it must be provided with connection settings and credentials for the database. These can be provided either to Helm as configuration or as Kubernetes ConfigMap and Secret resources. Configuring these in Helm is the simplest approach during installation. Providing these values via external Kubernetes resources allows performing update (e.g. changing credentials) without having to rerun helm, and allows integration with other Kubernetes tooling (e.g. for Secret provisioning).

Standard connection settings like host, port and database name must be provided. Optionally, additional JDBC parameters can be specified.

values.yaml
database:
  location: user-managed
  connection:
    host: database.example.com
    port: 5432
    databaseName: gradle_enterprise
    params: "?connectTimeout=60"

These can also be provided as a ConfigMap. The name of the ConfigMap must be provided, and the resource itself must be created prior to starting Gradle Enterprise.

values.yaml
database:
  location: user-managed
  connection:
    configMapName: my-example-db-connection-details

The ConfigMap is then configured like this:

---
apiVersion: v1
kind: ConfigMap
metadata:
  name: my-example-db-connection-details
data:
  host: database.example.com
  port: 5432
  dbname: gradle_enterprise
  jdbcParams: "?connectTimeout=60"

The port, dbname and jdbcParams properties are all optional. The default database name is gradle_enterprise.

The database must have already been created using CREATE DATABASE, the createdb command or an equivalent mechanism in a cloud database interface prior to configuring Gradle Enterprise with connection details.

There are two options for credentials. If provided with credentials for a database superuser (such as the postgres user that is common on PostgreSQL database instances), Gradle Enterprise can perform all further database setup, and can do so for all subsequent Gradle Enterprise upgrades. This is the recommended configuration.

values.yaml
database:
  location: user-managed
  connection: ...
  credentials:
    superuser:
      username: postgres
      password: the_password

Note that in some installations, and often in cloud-based databases, the typical credentials provided by the database provider are not a true superuser, but have many of the same abilities. For example, the supplied postgres account in Amazon RDS Postgres databases is not a true Postgres superuser but has the rds_superuser role. Such accounts are fine to configure Gradle Enterprise to use.

These credentials can also be supplied as a Kubernetes Secret:

values.yaml
database:
  location: user-managed
  connection: ...
  credentials:
    superuser:
      secretName: my-example-db-superuser-credentials

The Secret should have the typical username and password keys, encoded using Base64:

---
apiVersion: v1
kind: Secret
metadata:
  name: my-example-db-superuser-credentials
data:
  username: "cG9zdGdyZXM="
  password: "ZXhhbXBsZS1wYXNzd29yZA=="

In some instances it may be preferable to not supply Gradle Enterprise with database superuser credentials. To set up your database without database superuser credentials, run the setup.sh script from the database setup scripts for your Gradle Enterprise version against it. When upgrading Gradle Enterprise, you will need to run the new version’s database setup script, too. The script will set up less privileged accounts for the application to use, and some privileged functions needed for the application to run.

Failing to run the database setup scripts before the upgrade may cause the application to not work after the upgrade.
You should stop the existing version of Gradle Enterprise before running the new version’s setup scripts.

The credentials for the accounts must then be set by the user and provided to Gradle Enterprise via Helm configuration or Secrets.

values.yaml
database:
  location: user-managed
  connection: ...
  credentials:
    app:
      password: app_password
    migrator:
      password: migrator_password

These credentials can also be supplied as Secret resources. There are two accounts that must be configured, requiring a Secret each.

values.yaml
database:
  location: user-managed
  connection: ...
  credentials:
    app:
      secretName: my-example-db-application-credentials
    migrator:
      secretName: my-example-db-migrator-credentials

In this example stringData is used (which does not require Base64 encoding) to show the correct usernames to use. The usernames are currently not configurable.

---
apiVersion: v1
kind: Secret
metadata:
  name: my-example-db-application-credentials
stringData:
  username: ge_app
  password: "app-example-password"

---
apiVersion: v1
kind: Secret
metadata:
  name: my-example-db-migrator-credentials
stringData:
  username: ge_migrator
  password: "migrator-example-password"

Please contact Gradle support for assistance in running the script or questions about when to use non-superuser credentials.

Horizontal scaling configuration

A number of Gradle Enterprise components can be scaled horizontally to provide greater performance and availability.

values.yaml
global:
  scaling:
    replicas: 2

Gradle Enterprise’s scalable components are implemented as Kubernetes StatefulSet resources. It is also possible to alter their replica count directly using kubectl scale, allowing integration with other Kubernetes tooling.

It is only recommended to alter the replica count directly when connecting to a user-managed database, as available embedded database connections are configured based on the replica count. The user-managed database should have (160 ✖️ replica count) connections available.

OpenShift configuration

Gradle Enterprise needs to know when it is being deploying to an OpenShift cluster.

values.yaml
global:
  openshiftInstallation: true

Unattended application configuration

Many aspects of Gradle Enterprise’s behaviour can be configured via the Admin user interface. These are described in the Administration Manual.

It is possible to pre-configure Gradle Enterprise with the same settings that can be set via the Admin UI. This is useful to create a working Gradle Enterprise instance in a completely automated way, with no need to configure anything via a user interface.

The settings take the form of a yaml file. This file can be hand-written or exported from Gradle Enterprise. For more details on creating a Gradle Enterprise configuration yaml file, see Unattended configuration in the Administration Manual.

Having prepared a Gradle Enterprise configuration file and optionally an encryption key, you can provide it to Helm in one of two ways.

Example key:

gradle-enterprise-configuration.key
aes256:B0uVHRDhng+PraUI:Aj25DOwJsrXnWYcprreHAS4l66k/7q5CIjFDg5PTR7U=

Example yaml file:

gradle-enterprise-configuration.yaml
version: 4
systemPassword: "ObvvvqQww04Fn2jLBOOgjZDkXGL06fNmpueVcdk1lz0=:dBLNuLA/+qiwOqBQKf5pW89SV5mcQBJ4Vph/7lXerdD+2sLM8jij+2WJbBwXsqJ+mJugsveuUb+DyU3LBgkqcg=="
buildScans:
  keepDays: 30
email:
  administratorAddress: gradle-enterprise-adminstrator@example.com
  fromAddress: gradle-enterprise@example.com
  smtpServer: smtp.example.com:587
  sslProtocol: startTls
  authentication:
    type: login
    username: gradle-enterprise-adminstrator@example.com
    password: "aes256:B0uVHRDhng+PraUI:2bOz71vKTexz0QH5:z7lO+1wOC/tA3izLAwV0BXMugg=="

The file and encryption key can be provided inline in a Helm values file under the global.unattended property:

values.yaml
global:
  unattended:
    key: "aes256:B0uVHRDhng+PraUI:Aj25DOwJsrXnWYcprreHAS4l66k/7q5CIjFDg5PTR7U=" (1)
    configuration: (2)
      version: 4
      systemPassword: "ObvvvqQww04Fn2jLBOOgjZDkXGL06fNmpueVcdk1lz0=:dBLNuLA/+qiwOqBQKf5pW89SV5mcQBJ4Vph/7lXerdD+2sLM8jij+2WJbBwXsqJ+mJugsveuUb+DyU3LBgkqcg=="
      buildScans:
        keepDays: 30
      email:
        administratorAddress: gradle-enterprise-adminstrator@example.com
        fromAddress: gradle-enterprise@example.com
        smtpServer: smtp.example.com:587
        sslProtocol: startTls
        authentication:
          type: login
          username: gradle-enterprise-adminstrator@example.com
          password: "aes256:B0uVHRDhng+PraUI:2bOz71vKTexz0QH5:z7lO+1wOC/tA3izLAwV0BXMugg=="
1 The encryption key used for any secrets in your configuration. In this example, the password for email authentication is encrypted. Not required if the configuration does not contain any encrypted passwords. Note that the systemPassword is a hashed value, not an encrypted one, and so does not require an encryption key.
2 The Gradle Enterprise configuration yaml is put inline underneath this property. Content needs to be indented such that it is all correctly under the global.unattended.configuration property.

Or they can be supplied as helm arguments using --set-file:

$ (helm command) \
    --set-file global.unattended.configuration=path/to/gradle-enterprise-configuration.yaml \
    --set-file global.unattended.key=path/to/gradle-enterprise-configuration.key
Gradle Enterprise configuration that is set in Helm like this will not be reapplied (i.e. overwrite manual changes made in the Admin UI) in subsequent installations unless the contents of the config file(s) has changed.

Unattended configuration of S3 Build Scan storage

If using S3 Build Scan storage, this can be configured as part of an unattended configuration. See the Administration Manual for more details.

Installation

To install Gradle Enterprise, commands will need to be executed on a host with connectivity to your Kubernetes cluster.

As discussed in the network connectivity section, Gradle Enterprise supports two installation approaches.

At a high level, an online installation involves the following steps:

  1. Install Helm.

  2. Add and update the Gradle Helm repository.

  3. Create a Helm values file.

  4. Install Gradle Enterprise.

An airgap installation instead involves the following steps:

  1. Download necessary installation files.

  2. Create a Helm values file.

  3. Transfer files to where they will be used.

  4. Upload images to the internal container registry.

  5. Install Helm and Gradle Enterprise.

Please see the following sections for details.

Online installation

If your cluster has internet connectivity, perform an online installation as described below.

1. Install Helm

Develocity requires Helm version 3.5.x (or later) to install.

It is recommended to use the latest version available as this will have all known security vulnerabilities addressed. This document describes the maximum version skew supported between Helm and Kubernetes.

For more information on installing Helm (including alternate installation approaches), see Installing Helm.

Install Helm by running the command:

$ curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash

Verify that helm is installed and working:

$ helm version

The expected output should be similar to this:

version.BuildInfo{Version:"v3.11.1", GitCommit:"293b50c65d4d56187cd4e2f390f0ada46b4c4737", GitTreeState:"clean", GoVersion:"go1.19.5"}

2. Install the gradle-enterprise Helm chart

Develocity is distributed from the Develocity Helm repository.

Add the Develocity Helm repository to your Helm installation and fetch its contents into the local cache:

$ helm repo add gradle https://helm.gradle.com/
$ helm repo update gradle

Verify that the Develocity chart is accessible:

$ helm search repo gradle-enterprise

This will report the latest versions available for the two Develocity charts:

NAME                               	CHART VERSION	APP VERSION	DESCRIPTION
gradle/gradle-enterprise           	2022.4.7        2022.4.7    Official Develocity chart for Kubernetes cluster installations
gradle/gradle-enterprise-standalone	2022.4.7        2022.4.7    Official Develocity chart for standalone installations
This guide covers installation of the gradle-enterprise chart. Installation of the gradle-enterprise-standalone chart is covered in the Self-Hosted Standalone Installation Guide.

3. Install Gradle Enterprise

The recommended way to install Develocity is letting Helm manage the installation. For alternatives, see Advanced installation.

Install Develocity:

$ helm install \
    --create-namespace --namespace develocity \(1)
    ge \(2)
    gradle/gradle-enterprise \(3)
    --values values.yaml \(4)
    --set-file global.license.file=./develocity.license (5)
1 This example uses develocity as the namespace, but it can be a custom name. If you use a custom name, update all other example commands accordingly. Running with --create-namespace will create the namespace if it does not already exist.
The namespace option may not be required for OpenShift if oc login has been run and the active project for the current context is set.
2 This is the Helm release name. It is used by Helm to identify the Develocity installation.
3 The Develocity chart to install, in this case, gradle/gradle-enterprise.
To install a specific version, use --version 2022.4.7.
4 The Helm values file with configuration values, including items such as the hostname.
5 The Develocity license file (if not already included in your values file).

You may wish to pass other files using --set-file, such as certificates for HTTPS, or unattended configuration files.

Configuration values can be provided to Helm in a variety of ways. This includes:

  • providing a Helm values file (which can contain inline files) with --values

  • providing files (such as the Develocity license or certificates) with --set-file

  • setting individual values with --set

Choose a combination that works for your own configuration management processes.

Airgap installation

Airgap installation involves downloading files, transferring them, installing supporting software, and running helm install.

We recommend you save all the files into a single transfer directory so that it is easy to transfer to other hosts. For example:

$ mkdir develocity-installation-files && cd develocity-installation-files

1. Download Helm

Download and verify the Helm binary:

$ curl -L -o helm-linux-amd64.tar.gz https://get.helm.sh/helm-v3.11.1-linux-amd64.tar.gz
$ echo "0b1be96b66fab4770526f136f5f1a385a47c41923d33aab0dcb500e0f6c1bf7c  helm-linux-amd64.tar.gz" | \
    sha256sum -c

2. Download bundle

Save your Develocity license to the transfer directory as develocity.license.

Download and verify the airgap bundle:

$ curl -LOJd @develocity.license \
    https://registry.gradle.com/airgap/gradle-enterprise-2022.4.7-bundle.tar.gz
$ curl -LOJd @develocity.license \
    https://registry.gradle.com/airgap/gradle-enterprise-2022.4.7-bundle.tar.gz.sha256
$ sha256sum -c gradle-enterprise-2022.4.7-bundle.tar.gz.sha256

If checksum verification fails, check the contents of the downloaded files for error messages. If the error message indicates that your license is invalid/expired/not airgap enabled, you will need to request an updated license file by contacting your customer success representative.

Instead of running the above curl commands, you can download the airgap bundle by navigating to https://registry.gradle.com/airgap in your browser and following the instructions on the page.

3. Download example Helm values file

Download and verify the example Helm values file:

$ curl -L -o example.values.yaml \
    https://docs.gradle.com/enterprise/helm-kubernetes-installation/values-2022.4/gradle-enterprise-values-2022.4.yaml
$ curl -L -o example.values.yaml.sha256 \
    https://docs.gradle.com/enterprise/helm-kubernetes-installation/values-2022.4/gradle-enterprise-values-2022.4.yaml.sha256
$ echo "$(cat example.values.yaml.sha256)  example.values.yaml" | sha256sum -c

4. Configure Helm values file

Before installing Develocity and its prerequisites, we recommend creating your Helm values file on your local workstation (where you can use your favorite text editor). Copy “example.values.yaml” to “values.yaml”, review the available settings, and modify as needed.

5. Transfer files

Check that the transfer directory contains the following files (additional files are fine):

  • helm-linux-amd64.tar.gz

  • develocity.license

  • values.yaml

  • gradle-enterprise-2022.4.7-bundle.tar.gz

  • Optional: SSL certificates

Once you’ve verified that you have the required files, transfer them to the host with connectivity to the internal container registry and Kubernetes cluster.

6. Upload images

Follow these instructions on the host with connectivity to the internal container registry with your transferred files present in the current directory.

You must be logged in to the registry prior to running these commands.

Expand the bundle:

$ tar zxvf gradle-enterprise-2022.4.7-bundle.tar.gz

Upload the images to the internal container registry:

$ ./upload-images.sh --registry=registry.example.com/gradle-enterprise

7. Install Helm

Follow these instructions on the host with connectivity to the Kubernetes cluster with your transferred files present in the current directory.

To install Helm:

$ tar -zxvf helm-linux-amd64.tar.gz
$ sudo mv linux-amd64/helm /usr/local/bin/helm

Verify that helm is installed and working:

$ helm version

The expected output should be similar to this:

version.BuildInfo{Version:"v3.11.1", GitCommit:"293b50c65d4d56187cd4e2f390f0ada46b4c4737", GitTreeState:"clean", GoVersion:"go1.19.5"}

8. Install Gradle Enterprise

The recommended way to install Develocity is letting Helm manage the installation. For alternatives, see Advanced installation.

Follow these instructions on the host with connectivity to the Kubernetes cluster with your transferred files present in the current directory.

Expand the bundle (may have already been expanded if you uploaded the images on this host):

$ tar zxvf gradle-enterprise-2022.4.7-bundle.tar.gz

Install Develocity:

$ helm install \
    --create-namespace --namespace develocity \(1)
    ge \(2)
    ./gradle-enterprise-2022.4.7.tgz \(3)
    --values values.yaml \(4)
    --set-file global.license.file=./develocity.license (5)
1 This example uses develocity as the namespace, but it can be a custom name. If you use a custom name, update all other example commands accordingly. Running with --create-namespace will create the namespace if it does not already exist.
The namespace option may not be required for OpenShift if oc login has been run and the active project for the current context is set.
2 This is the Helm release name. It is used by Helm to identify the Develocity installation.
3 The Develocity chart to install. This is the .tgz file included in the airgap bundle.
4 The Helm values file with configuration values, including items such as the hostname.
5 The Develocity license file (if not already included in values file).

You may wish to pass other files using --set-file, such as certificates for HTTPS, or unattended configuration files.

Configuration values can be provided to Helm in a variety of ways. This includes:

  • providing a Helm values file (which can contain inline files) with --values

  • providing files (such as the Develocity license or certificates) with --set-file

  • setting individual values with --set

Choose a combination that works for your own configuration management processes.

Advanced installation options

See the appendix for discussion of advanced installation options.

Post-installation

System check

At this point, it should be possible to see the Helm release installed:

$ helm --namespace gradle-enterprise list
NAME	NAMESPACE        	REVISION	UPDATED                               	STATUS  	CHART                     	    APP VERSION
ge  	gradle-enterprise	1       	2023-03-28 04:59:38.62408043 +0000 UTC	deployed	gradle-enterprise-2022.4.7  	2022.4.7

You can inspect the status of the Gradle Enterprise pods:

$ kubectl --namespace gradle-enterprise get pods
NAME                                               READY   STATUS    RESTARTS   AGE
gradle-enterprise-operator-76694c949d-md5dh        1/1     Running   0          84m
gradle-proxy-0                                     1/1     Running   0          83m
gradle-database-65d975cf8-dk7kw                    2/2     Running   0          83m
gradle-enterprise-app-0                            1/1     Running   0          83m
gradle-metrics-cfcd8f7f7-zqds9                     1/1     Running   0          83m
gradle-test-distribution-broker-6fd84c6988-x6jvw   1/1     Running   0          83m
gradle-build-cache-node-57b9bdd46d-2txf5           1/1     Running   0          84m
gradle-keycloak-0                                  1/1     Running   0          83m

Verifying network connectivity to Gradle Enterprise

Gradle Enterprise has a /ping endpoint, which can be used to verify network connectivity with Gradle Enterprise.

Connectivity to Gradle Enterprise installation can be tested by running the following command on hosts / computers which need to connect to Gradle Enterprise:

$ curl -sw \\n --fail-with-body --show-error https://«gradle-enterprise-host»/ping

It should return SUCCESS.

Cleanup

For airgap installations, it is recommended to remove the following files after installation:

  • helm-linux-amd64.tar.gz

  • gradle-enterprise-2022.4.7-bundle.tar.gz

  • gradle-enterprise-2022.4.7.tgz

  • gradle-enterprise-2022.4.7-images.tar

Once Gradle Enterprise has been installed, files used during installation are not required at runtime and can be removed if desired. However, the following files may be useful to preserve, as they may aid in future upgrades or maintenance:

  • Helm values files

  • SSL certificates

  • Gradle Enterprise license

Care should be applied when handling these files as they may be considered sensitive.

Post-installation setup

It is strongly recommended to immediately change the system user password from its default after installation to secure your installation.

A number of settings should be reviewed after installation. Please refer to the administration manual.

Upgrading

Before upgrading, be sure to check the upgrade notes section for any special considerations when upgrading from older versions of Gradle Enterprise.

To upgrade Helm and Gradle Enterprise follow the same procedure as for an initial installation. However, there are some notes to consider:

  • Run helm repo update gradle to update locally available charts.

  • Run helm upgrade instead of helm install.

  • Use the same Helm release name (ge in the examples in this guide).

  • When running a helm upgrade, Helm will reuse values that were previously used. This means that you can run:

    $ helm upgrade \
        --namespace gradle-enterprise \
        --reuse-values \
        ge gradle/gradle-enterprise

    or for airgap installations:

    $ helm upgrade \
        --namespace gradle-enterprise \
        --reuse-values \
        ge gradle-enterprise-2022.4.7.tgz

    and not need to specify any values. If you do specify some changed values (for example, to use a new license file), they will be merged with your previous values.

  • Alternatively, to force helm upgrade to use only the values that you set at upgrade time, run with --reset-values instead of --reuse-values.

    Running helm upgrade with --reset-values will cause any previous values to be lost. All values (including license files, SSL certificates, etc.) will need to be set as part of the command.
  • If using helm template, all values must be supplied for each invocation.

  • You will be able to update any configuration values at upgrade time, but keep in mind that some values are immutable after initial installation, such as persistent volume sizes for provisioners that do not support dynamic resizing.

  • For major version upgrades (e.g. 2021.2.4 to 2021.3 or later), if data is stored in a user-managed database and superuser credentials are not supplied, the database setup script must be run prior to the upgrade. The correct script for the major version to which the system is being upgraded can be downloaded from the appendix.

Upgrading with multiple replicas

If you have configured more than one replica (see Horizontal scaling configuration), you must scale the replicas down to one before upgrading to avoid having mixed versions running at the same time.

Before applying the upgrade, run the following command to scale the application down to one replica:

$ helm upgrade \
    --namespace gradle-enterprise \
    --reuse-values \
    --set=global.scaling.replicas=1 \
    --version «deployed-version» \
    ge gradle/gradle-enterprise

Where «deployed-version» is the running version of Gradle Enterprise, not the version being upgraded to.

The upgrade to the new version can now be executed. If your upgrade processes uses --reuse-values, you will need to include a --set=global.scaling.replicas=N argument where N is the desired replica number. The following example performs a version upgrade and changes the replica number in one operation:

$ helm upgrade \
    --namespace gradle-enterprise \
    --reuse-values \
    --set=global.scaling.replicas=N \
    --version 2022.4.7 \
    ge gradle/gradle-enterprise

Changing configuration values

To change configuration values, follow the same procedure as for upgrading, but specify the current version to ensure that a later version does not accidentally get installed.

To check the currently deployed version, run:

$ helm history --namespace gradle-enterprise ge --max 1

To apply new configuration values to a Helm-managed online installation, run:

$ helm upgrade \
    --namespace gradle-enterprise \
    ge gradle/gradle-enterprise \
    --version «deployed-version»
    --reuse-values
    «new values options»

or for a Helm-managed airgap installation:

$ helm upgrade \
    --namespace gradle-enterprise \
    ge gradle-enterprise-2022.4.7.tgz \
    --version «deployed-version»
    --reuse-values
    «new values options»

The above examples reuse previous values by default. Any specified values will override the existing values. Alternatively, if you wish to specify all values explicitly (ignoring any previously set values), run with --reset-values instead of --reuse-values.

Running helm upgrade with --reset-values will cause any previous values to be lost. All values (including license files, SSL certificates, etc.) will need to be set as part of the command.

Configuration values can be provided to Helm in a variety of ways. This includes:

  • providing a Helm values file (which can contain inline files) with --values

  • providing files (such as the Develocity license or certificates) with --set-file

  • setting individual values with --set

Choose a combination that works for your own configuration management processes.

If you have made local modifications to a Helm values file, apply the changes to your installation by running an upgrade command (see above) with a --values «updated-values-file» option.

If you have made local modifications to a file you previously provided via --set-file, apply the changes to running an upgrade command (see above) with a --set-file option.

Changing storage class

Storage classes cannot be changed once a persistent volume claim is in place. If you need to alter any configured Gradle Enterprise storage classes, please contact Gradle support for assistance.

Uninstalling

For Helm-managed installations, Gradle Enterprise can be uninstalled by running helm uninstall «your-release-name»:

Example:

$ helm uninstall --namespace gradle-enterprise ge

If running helm template and applying to your cluster manually, this can be done in a couple of ways:

  • If Gradle Enterprise is the only thing in the namespace that is has been installed to, you can delete the namespace.

    $ kubectl delete namespace gradle-enterprise
  • Alternatively, delete all resources that were created in the generated manifest from the last install or upgrade.

    $ kubectl --namespace gradle-enterprise delete -f gradle-enterprise.yaml
For storage classes with a 'delete' reclaim policy this will mean that all data will also be lost.

If using storage classes with a 'retain' reclaim policy, you will need to review remaining persistent volumes in your cluster after uninstalling to reclaim the space used.

Troubleshooting

Support

If you are experiencing issues with Gradle Enterprise, please see the Administration Manual for information about how to submit a support ticket.

Appendix A: Upgrade notes

It is recommended to upgrade to the latest available Gradle Enterprise prior to migrating to the Helm-based installation.

Pre-upgrade database configuration required for superuser-less user-managed databases

Gradle Enterprise installations can use an embedded database provided as part of the installation, or a user-managed database for superior performance and operability. When using a user-managed database, Gradle Enterprise can be configured with superuser access to the database so that it can initialize and configure the database appropriately. Alternatively, this initialization and configuration can be executed manually, avoiding the need to provide Gradle Enterprise with superuser access to the database.

For installations using the superuser-less approach, new configuration requirements in Gradle Enterprise 2022.4 necessitate manually executing an updated configuration script as part of the upgrade.

If your installation uses the superuser-less approach, please contact Gradle Enterprise support to obtain the new configuration scripts before upgrading. If you are unsure whether your installation uses this approach and requires this extra upgrade step, please contact Gradle Enterprise support.

Predictive Test Selection temporary unavailability after upgrading and build plugin version requirement changes

The improvements to the prediction model in this release require re-analyzing build history, which is initiated automatically after upgrading. Attempting to use Predictive Test Selection while this is occurring will result in all tests being selected for execution. The duration of this process depends on the amount of build history stored by your installation and the availability of CPU resources. It is estimated to take ~24 hours for an installation with a build history of 6 months of 60,000 builds per week and 8 processor cores. Due to this, it is recommended that installations using Predictive Test Selection perform the version upgrade at the end of week to allow this process to complete over the weekend.

Predictive Test Selection with Gradle Enterprise 2022.4 requires Gradle Enterprise Gradle plugin v3.12+ or Gradle Enterprise Maven extension v1.16+. Attempting to use Predictive Test Selection with older versions will result in all tests being selected for execution. The Predictive Test Selection Simulator is unaffected by this requirement.

Temporarily degraded performance due to data reindexing

Upon upgrading, a data reindexing process will be initiated in the background with Gradle Enterprise being usable for its duration. CPU usage will be increased and performance may be slightly degraded. For large installations storing many build scans, the reindexing process may take several hours. During this time, some builds may be omitted from cross-build data visualizations. Recent builds are prioritized, making their data available sooner.

Inclusion of Gradle Test Retry plugin functionality in Gradle Enterprise Gradle plugin

While either can be used with Gradle Enterprise, it is recommended that Gradle Enterprise users adopt the Gradle Enterprise plugin’s retry functionality for ensured future compatibility and easier adoption of future Gradle Enterprise specific features. Please see the migration guide.

Appendix B: Advanced installation

Helm chart download and selection options

When asking Helm to install or process a chart, it is possible to pick a number of sources:

  • A chart from a repository directly.

  • A downloaded chart archive.

  • An expanded chart directory.

Which of these to use depends somewhat on network and policy requirements.

Direct selection

This involves simply running helm commands using the above gradle/gradle-enterprise chart name. Helm will download the chart if necessary during execution, so this option requires internet connectivity on the host that Helm is executed on.

Examples:

$ helm install ge gradle/gradle-enterprise «options»
$ helm template gradle/gradle-enterprise «options»

In the absence of reasons not to, Gradle recommends installing Gradle Enterprise charts in this manner, and this method is used in the main installation section above.

Downloaded archive

Helm can download a chart archive, and can execute using the downloaded archive. This is most useful when the host with internet connectivity does not have access to the Kubernetes cluster.

Example:

$ helm pull gradle/gradle-enterprise

Downloads the latest version to an archive in the current directory e.g. gradle-enterprise-2022.4.7.tgz.

Then later, possibly on a different host:

$ helm install ge gradle-enterprise-2022.4.7.tgz «options»

This is also the installation method used for airgap installations.

Expanded chart directory

It is common for users of Helm to download a chart, edit the included Helm values file, and then commit the result to source control. It is recommended to keep configuration in a separate values file, and install a pristine Gradle Enterprise chart with configuration provided alongside. However, if your organisation’s processes expect Helm charts to be edited inline, it is possible using this method.

Example:

$ helm pull gradle/gradle-enterprise --untar

Downloads the latest version, and expands it into a gradle-enterprise directory in the current directory.

Or for airgap customers, simply extract the chart .tgz file included in the bundle:

$ mkdir -p gradle-enterprise && \
    tar zxvf path/to/gradle-enterprise-2022.4.7.tgz -C gradle-enterprise

At this point gradle-enterprise/values.yaml can be edited, and the directory committed to source control.

Later:

$ helm install ge ./gradle-enterprise «options»

Installs Gradle Enterprise from a directory with the expanded chart.

Helm post-processing

Many organisations require the ability to customise Kubernetes manifests prior to applying them to their cluster. Kustomize is one commonly used tool to do this. It is possible to alter the Helm-generated Kubernetes manifests prior to Helm applying them to the cluster, using Helm Post Rendering via the --post-renderer flag. In this way, manifests can be customised while still allowing Helm to manage what is applied to the cluster.

$ helm install \
    --create-namespace --namespace gradle-enterprise \
    ge gradle/gradle-enterprise \
    --values values.yaml \
    --set-file global.license.file=./gradle-enterprise.license \
    --post-renderer path/to/post-renderer.sh

The --post-renderer option specifies an executable script that is given the Helm-rendered manifests as input (from stdin), and produces manifests with further customisations as output (to stdout).

An example of a Helm post-renderer that uses Kustomize is provided below:

#!/usr/bin/env bash
set -eo pipefail
cat <&0 > /absolute/path/to/kustomizations/gradle-enterprise.yaml
kustomize build /absolute/path/to/kustomizations

In this example, the gradle-enterprise.yaml file contains the output of the Helm rendering step. This file should then be used by the kustomization.yaml as an input, as shown below:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
  - gradle-enterprise.yaml

# Rest of kustomization

User-managed installation

An alternative to running helm install directly on your cluster is to have Helm generate Kubernetes manifests that can then be applied via kubectl or other Kubernetes tooling. In this mode, Helm does not keep track of previously installed versions - the resources in the cluster are not tracked and Helm cannot report on the currently installed version.

The most common reasons to do a user-managed installation are:

  • Requirements that the final applied manifests are committed to source control.

  • Need to modify the manifests prior to applying them to the cluster in a way that is not supported by Helm post-processing.

  • Preference or policy to have manifests applied and managed by other tooling.

  • Need to execute Helm on a host separated from the cluster.

There are some other downsides:

  • Some manual cleanup of resources is required after upgrades or uninstallation.

Running helm template

By default, Helm will generate a combined Kubernetes manifest and write it to standard output.

$ helm template \
    gradle/gradle-enterprise \(1)
    --values values.yaml \(2)
    --set-file global.license.file=./gradle-enterprise.license \(3)
    > gradle-enterprise.yaml (4)
1 The Gradle Enterprise chart to install. As discussed above, this can be:
  • The name of the chart in the Gradle Helm chart repo - gradle/gradle-enterprise. To install a specific version, pass this as additional parameters, e.g. --version 2022.4.7.

  • The path to a downloaded chart archive, e.g. ./gradle-enterprise-2022.4.7.tgz.

  • The path to an expanded chart directory, e.g. ./gradle-enterprise.

2 The Helm values file with configuration values, including items such as the hostname.
3 The Gradle Enterprise license file.
4 Save all the rendered manifests to a single yaml file.

You may wish to pass other files using --set-file, such as certificates for HTTPS.

Configuration values can be provided to Helm in a variety of ways. This includes:

  • providing a Helm values file (which can contain inline files) with --values

  • providing files (such as the Develocity license or certificates) with --set-file

  • setting individual values with --set

Choose a combination that works for your own configuration management processes.

The generated manifest can be committed to version control, copied or processed at this point.

Manifests can also be generated as a set of files in a directory, broken up into logical groupings, by calling Helm with the --output-dir argument:

$ helm template \
    gradle/gradle-enterprise \
    --values values.yaml \
    --set-file global.license.file=./gradle-enterprise.license \
    --output-dir ./my-gradle-enterprise-manifests

The above will create files under the specified directory. This may be a preferred form for customisation, including using tools such as Kustomize.

Invariant output

By default, the Gradle Enterprise Helm chart randomly generates several secrets during rendering. For example, the embedded database user credentials. This means that by default, repeated invocations of helm template will result in non-identical manifests, because the randomly generated secrets will be different each time, as a security precaution. This can cause unnecessary restarts in Gradle Enterprise installations, where automated tooling uses changes in rendering output to trigger redeployment.

It is possible to configure Helm such that the output of helm template is invariant. This can be done by setting secret values directly or by specifying a user-managed secret. You also can combine the two solutions if appropriate for your installation, by specifying some secret values directly, and some using user-managed secrets.

The secrets which you’ll need to specify to get invariant output, are shown in the table below.

Helm value Secret data key(s) Usage

For these Helm value keys, you can set either 'value' or 'secretName' under them, to either directly set the secret’s value, or to instead use a user-managed secret, respectively.

When creating user-managed secrets, the secret should have the verbatim data key(s) given in this column. Dots (.) should not be interpreted as nesting or indentation. For example, a secret data key like 'abc.def' should appear in your created secret like this:

apiVersion: v1
kind: Secret
metadata:
  name: gradle-abc-def-secret
data:
  abc.def: "a-secret-value"

database.credentials.app

username, password

The credentials used by the app to connect to the database

username must be set to ge_app

database.credentials.migrator

username, password

The credentials used when running database migrations

username must be set to ge_migrator

enterprise.session.key

client.side.session.key

The symmetric encryption key used for client-side session data

enterprise.session.token

client.side.session.token

The token used for signing the data associated with client sessions

buildCacheNode.buildCacheBuiltin

build.cache.builtin.secret

The shared secret value used by the app to register the built-in build cache node

authenticationBroker.clientSecret

keycloak.client.secret

The client secret used by the app when connecting to keycloak

authenticationBroker.adminPassword

keycloak.admin.password

The password for the keycloak admin user

Below shows an example Helm values yaml you can use to get invariant output by specifying user-managed secrets.

database:
  credentials:
   app:
     secretName: <user-managed kubernetes secret resource name>
   migrator:
     secretName: <user-managed kubernetes secret resource name>

enterprise:
  session:
   key:
     secretName: <user-managed kubernetes secret resource name>
   token:
     secretName: <user-managed kubernetes secret resource name>

buildCacheNode:
  buildCacheBuiltin:
   secretName: <user-managed kubernetes secret resource name>

authenticationBroker:
  clientSecret:
   secretName: <user-managed kubernetes secret resource name>
  adminPassword:
   secretName: <user-managed kubernetes secret resource name>

To get invariant output by specifying secret values directly in your Helm values yaml:

database:
  credentials:
    app:
      password: <secret value>
    migrator:
      password: <secret value>

enterprise:
  session:
    key:
      value: <secret value>
    token:
      value: <secret value>

buildCacheNode:
  buildCacheBuiltin:
    value: <secret value>

authenticationBroker:
  clientSecret:
    value: <secret value>
  adminPassword:
    value: <secret value>

As with other values, it is also possible to set these by passing --set or --set-file when Helm is invoked.

To create a user-managed secret value, you can use kubectl. Below is an example of using kubectl to create embedded database credential secrets, named db-app-user-credentials and db-migrator-user-credentials:

# Add this to your `helm install` command: `--set database.credentials.app.secretName=db-app-user-credentials`
kubectl -n gradle-enterprise create secret generic db-app-user-credentials --from-literal=username=ge_app --from-literal=password=<your password here>

# Add this to your `helm install` command: `--set database.credentials.migrator.secretName=db-migrator-user-credentials`
kubectl -n gradle-enterprise create secret generic db-migrator-user-credentials --from-literal=username=ge_migrator --from-literal=password=<your password here>
When you create a secret and reference it by name in your Gradle Enterprise installation (e.g. using a secretName value), Helm is not managing this secret. If you change the secret value (e.g. when periodically cycling a password), then Gradle Enterprise may need to be restarted manually into order for the changes to be reflected. To do this, restart all the pods in the Gradle Enterprise Helm installation.

Mirroring the Helm Charts

An internal Helm chart repository can be used to mirror the Gradle Enterprise Helm charts.

Using helm pull to fetch the charts is the recommended approach since the Gradle Enterprise Helm repo index uses relative urls.

$ helm search repo gradle-enterprise --versions --output json | jq -r '"helm pull " + .[].name + " --version " + .[].version' | sort | uniq | bash

Appendix C: Verifying network connectivity

For installation/upgrades of online installations, internet connectivity to https://helm.gradle.com is required on the host where you run Helm.

Connectivity can be tested by running the following command:

$ curl -sw \\n --fail-with-body --show-error https://helm.gradle.com/ping

The command should return SUCCESS.

Online installations require runtime internet connectivity to https://registry.gradle.com and https://harbor.gradle.com from your Kubernetes cluster. Internet connectivity from Kubernetes clusters can be affected by a number of factors - please see your cluster’s documentation for more details.

Typically though, connectivity to these can be tested by running the following commands on a host connected to your cluster:

$ kubectl run gradle-registry-test -q --rm --attach --restart=Never \
    --image=curlimages/curl -- -sw \\n --fail-with-body --show-error \
    https://registry.gradle.com/ping
$ kubectl run gradle-harbor-test -q --rm --attach --restart=Never \
    --image=curlimages/curl -- -sw \\n --fail-with-body --show-error \
    https://harbor.gradle.com/ping

Both should return SUCCESS.

If any errors occur, please review your network setup before installing or upgrading Gradle Enterprise.

Appendix D: Configuring Helm for EKS service account binding

If using Amazon EKS and S3 Build Scan storage, one way allow Gradle Enterprise to access S3 is to use a role associated with a service account.

In addition to the application configuration mentioned at the link above, it is necessary to annotate the service account with the ARN of the role being used. This is done in the Helm values file:

values.yaml
enterprise:
  serviceAccount:
    annotations:
      "eks.amazonaws.com/role-arn": "arn:aws:iam::111122223333:role/your-role-name"

Appendix E: Example Helm values file

Appendix F: Database setup scripts