This tutorial will show how to run Develocity on Google Kubernetes Engine.

Develocity can be installed into an existing Kubernetes cluster. It can also be installed on a standalone virtual machine, as shown in our Google Compute Engine installation tutorial. This guide shows how to set up a cluster installation on a Google Kubernetes Engine cluster.

This guide is for the latest version of Develocity, and may not work with earlier versions.

Develocity can generally be installed on Kubernetes clusters running modern Kubernetes versions. The exact supported versions are listed in Develocity’s Self-Hosted Kubernetes Installation Guide. Later versions may be compatible but have not been verified to work.

The majority of this tutorial is a quick start guide to creating and minimally configuring a cluster in GKE for a Develocity installation. If you already have GKE expertise and are able to provision a cluster, you may wish to skip straight to the Develocity installation instructions.

This tutorial is not guaranteed to work (and has not been tested) using Google Cloud’s Assured Workloads for Government. For assistance installing Develocity using Assured Workloads for Government, please contact your account executive or Develocity customer support if you are already a customer.

Prerequisites

1. A Google Cloud account

You can create an account if you do not already have one.

2. A Develocity license

You can request a Develocity trial here. If you have purchased Develocity, you will already have a license file.

3. A Google Cloud project with billing enabled

You will need a Google Cloud project that has billing enabled for this tutorial. If you have a project you wish to use, ensure it has billing and the GKE APIs enabled. You can create a project by following this guide. To enable billing, follow these instructions for your project.

If you are using a new project, you may be asked to enable various APIs when running commands. Go ahead and do this, they are necessary for the tutorial.

4. A Google Cloud IAM user with GKE permissions

If you are the project owner, you likely have all permissions already.

Your Google Cloud IAM user needs permission to create and manage GKE clusters, static IPs, and managed certificates. You also need permission to create resources in GKE clusters, including RBAC roles (which requires the container.roles.create permission). Your nodes will be created using the Compute Engine default service account, so you also need permission to use it.

You can obtain these permissions using the following predefined roles:

  • Kubernetes Engine admin (roles/container.admin)

  • Certificate Manager Editor (roles/certificatemanager.editor)

  • Compute Public IP Admin (roles/compute.publicIpAdmin)

  • Service Account User (roles/iam.serviceAccountUser) for the Compute Engine default service account

Consult the Develocity installation guide for information about debugging required Kubernetes permissions.
For more details on GKE’s access control model, consult the GKE IAM guide and the GKE access control guide.
If you choose to follow our Cloud SQL appendix, you will need the permissions described in it, too.

5. Hostname (optional)

Google Cloud Platform machines are provisioned with an external IP address but no hostname. Develocity is not accessible directly by IP, but there are services like nip.io that automatically map hostnames to IPs, so using your own hostname is not required.

If you want to access Develocity by a host name of your choosing (e.g. develocity.example.com), you will need the ability to create the necessary DNS record to route this name to the instance IP address.

You will be able to update this later. You can start with a public DNS name based on the IP address, and later reconfigure to use a custom hostname if desired.

Host Requirements

This section outlines cluster and host requirements for the installation.

Currently, Develocity only supports the x86_64 architecture.

1. Database

Develocity installations have two database options:

  1. An embedded database that is highly dependent on disk performance.

  2. A user-managed database that is compatible with PostgreSQL 12, 13, or 14, including Google Cloud SQL.

By default, Develocity stores its data in a PostgreSQL database that is run as part of the application itself, with data being stored in a directory mounted on its host machine.

Cloud SQL Database

There are instructions for using Google Cloud SQL as a user-managed database in Using Cloud SQL as a Develocity user-managed database. This can have a number of benefits, including easier resource scaling, backup management, and failover support.

2. Storage

IIn addition to the database, Develocity needs some storage capacity for configuration files, logs, and build cache artifacts. These storage requirements apply regardless of which type of database you use, although the necessary size varies based on the database type. The majority of data is stored in the "installation directory", which defaults to /opt/gradle unless otherwise specified in your Helm values.yaml file.

Capacity

The minimum capacity required for the installation directory for the embedded database case is 250 GB. The minimum capacity required for the installation directory for the user-managed database case is 30 GB. It is recommended to create a specific volume for the installation directory to avoid consuming the space required for Develocity, and to ensure at least 10% of the volume’s space is free at all times.

The following are additional disk capacity requirements:

Location Storage Size

/tmp

1 GB

/var/lib/rancher/k3s

30 GB

These are not particularly performance sensitive.

Performance

For production workloads, storage volumes should exhibit SSD-class disk performance of at least 3000 IOPS (input/output operations per second). Develocity is not compatible with network-based storage solutions due to limitations of latency and data consistency.

Disk performance has a significant impact on Develocity performance. Develocity is not compatible with network-based storage solutions due to limitations of latency and data consistency.

Object storage

Develocity can be configured to store Build Scan data in an object storage service, such as Google Cloud Storage. This can improve performance in high-traffic installations by reducing the load on the database. See Build Scan object storage in the Develocity Administration Manual for a description of the benefits and limitations.

There are instructions on how to configure Develocity to use Google Cloud Storage as an object store in the Object Storage Configuration section of the Kubernetes Helm Chart Configuration Guide.

3. Network Connectivity

Develocity requires network connectivity for periodic license validation.

An installation of Develocity will not start if it cannot connect to both registry.gradle.com and harbor.gradle.com.

It is strongly recommended that production installations of Develocity are configured to use HTTPS with a trusted certificate.

When installing Develocity, you will need to provide a hostname, such as develocity.example.com.

Preinstallation

You need to use a number of tools to create Google Cloud resources and install Develocity. You can either install them locally, or use Google Cloud’s Cloud Shell, which comes with the tools you will need preinstalled and mostly preconfigured.

You can use Cloud Shell via SSH by following these instructions. This requires installing gcloud locally, and then using it to SSH into Cloud Shell, where you would run the tutorial’s commands.
This guide assumes you are using bash as your shell, although any shell that is fairly compatible (e.g. zsh) should work without much trouble.
When files are referred to in this tutorial, they are assumed to be on the machine you are running the command on. If you are using Cloud Shell, you will need to upload them to the shell machine by following the Cloud Shell documentation.

If you decided to use Cloud Shell, complete 2. Configure gcloud (unless you already have the project and zone configured) and then skip to Creating a Google Kubernetes Engine Cluster.

1. Install gcloud

You will be using the gcloud command line tool to provision and configure the Google Kubernetes Engine cluster. To install gcloud on your local machine, follow the instructions in the Google Cloud documentation.

2. Configure gcloud

gcloud configurations are not saved by default by Cloud Shell. To save your configuration, follow these instructions.

To configure gcloud, run gcloud init and follow the initialization guide. You want to use the project you created or decided on in 3. A Google Cloud project with billing enabled. If your project doesn’t have a default zone, you need to set the zone (and region, which will often be set automatically from the zone) to the zone you wish to install Develocity in. If you don’t know which zone or region to select, consult Google Cloud’s region and zone documentation.

Pick the region geographically closest to you or to any pre-existing compute resources, such as CI agents, to ensure the best performance.

3. Install kubectl

To easily install a gcloud-managed version of kubectl, you can run:

$ gcloud components install kubectl

You can also install kubectl through any other means. The Kubernetes documentation lists some of the most popular options (note that you only need to install kubectl, not any of the other tools listed there).

4. Install the gcloud auth plugin for kubectl

To authenticate with your GKE cluster, kubectl needs a GKE-specific plugin. If you are using an older Kubernetes version, this may not be necessary, but it does no harm.

To install the plugin, run:

$ gcloud components install gke-gcloud-auth-plugin
For more details on installation, and why the plugin is necessary, see Google’s blog post.

5. Miscellaneous command line tools

This guide assumes that various common command line tools such as wget, curl and jq are already available on your host. Please refer to the official documentation of these tools for installation instructions.

Creating a Google Kubernetes Engine Cluster

In this section you will create a Google Kubernetes engine to run a Develocity instance, and create an external IP for Develocity to use.

If you’re using Cloud Shell, remember to run these commands there.

1. Create a cluster

For this tutorial, we will use a Standard cluster with the default three nodes, using the e2-standard-8 machine type with 8 vCPUs and 32 GB memory. See Resource requirements in the Self-Hosted Kubernetes Installation Guide for recommendations.

Develocity is fully compatible with Autopilot clusters which will automatically provision nodes based on your workload. There are also GKE solutions for autoscaling nodes horizontally and vertically in standard clusters.

Name this cluster gradle-enterprise. To create it, run:

$ gcloud container clusters create gradle-enterprise --machine-type  e2-standard-8

This command will take a while to complete. It will automatically add and activate a kubectl configuration.

You should then be able to see your cluster when running kubectl config get-contexts:

$ kubectl config get-contexts
CURRENT   NAME                                                       CLUSTER                                                    AUTHINFO                                                   NAMESPACE
*         gke_gradle-enterprise-sandpit_us-west1_gradle-enterprise   gke_gradle-enterprise-sandpit_us-west1_gradle-enterprise   gke_gradle-enterprise-sandpit_us-west1_gradle-enterprise
For more details on Standard cluster creation, see GKE’s Standard cluster creation documentation.

If you decide to change the number of nodes in the cluster, you can run:

$ gcloud container clusters resize gradle-enterprise --num-nodes «target-nodes»
For more details on cluster management, see GKE’s documentation.

2. Create an external IP

Before installing Develocity, you need to create a static external IP for Develocity to use. While it’s possible to configure the IP and hostname later by running helm upgrade with the updated Helm values file, it is easier to configure them now.

To create the static IP, run:

$ gcloud compute addresses create develocity-static-ip --global

You can see the IP address used by the resource by running:

$ gcloud compute addresses describe develocity-static-ip --global --format='value(address)'
The static external IP resource is managed separately from GKE. It will not be automatically deleted if an Ingress object using it is deleted.

3. Configure the hostname

If you intend to use a custom hostname to access your Develocity instance, you now need to add the appropriate DNS records.

Add an A record for your hostname that points to the IP you created in the previous step. For example:

develocity.example.com A 34.110.226.160

Verify that your DNS record works correctly before installing Develocity, such as by using dig develocity.example.com.

You can configure which certificate authorities may be used to sign your Google-managed certificate using DNS records, as described by Google’s documentation.
Google Cloud has its own Cloud DNS service, which can be used for this tutorial. However, there is no integration with GKE that makes it preferable to other DNS services.

It is not possible to use an IP address directly as the Develocity hostname. If you do not wish to set up a permanent DNS record at this time, you can instead use a service like nip.io to provide one based on the IP address. Any of the dash-based naming schemes on the nip.io web page should work, but the following command will generate a useful short name tied to that IP address:

$ GE_IP_ADDR=$(gcloud compute addresses describe develocity-static-ip --global --format='value(address)')

$ echo "gradle-enterprise-$(printf '%02x' $(echo ${GE_IP_ADDR//./ })).nip.io"
gradle-enterprise-226ee2a0.nip.io

gradle-enterprise-226ee2a0.nip.io can then be used as the hostname.

This tutorial uses the Cloud Load Balancer via the GKE ingress controller. If running Develocity behind a cloud load balancer, use /ping for the load balancer’s health check. It will respond as healthy when Develocity is capable of handling requests, even if limited to the interactive starting page while the instance is starting. Using the load balancer to manage fail-over may be undesirable for highly available installations.

If you are installing Develocity in a highly available setup, we recommend submitting a ticket at support.gradle.com for assistance.

4. Create a managed SSL certificate

You can provision and use a Google-managed trusted SSL certificate using Kubernetes manifests. This will work even with a hostname you don’t own (such as one from nip.io) as long as it redirects to a Google Cloud load balancer using your certificate.

We will use a managed certificate in this tutorial. If you want to use a custom SSL certificate instead, skip this step and follow the instructions in HTTP or HTTPS when creating your Helm values file below.

To provision a Google-managed SSL certificate, create a managed certificate resource on the cluster.

managed-cert.yaml
apiVersion: v1
kind: Namespace
metadata:
  name: develocity
---
apiVersion: networking.gke.io/v1
kind: ManagedCertificate
metadata:
  name: develocity-cert
  namespace: develocity
spec:
  domains:
    - develocity.example.com (1)
1 Use the hostname you decided on in 3. Configure the hostname.

To apply this manifest, run the following command:

$ kubectl apply -f managed-cert.yaml

Note that the certificate won’t start provisioning until Develocity is installed.

For more details, consult Google’s guide to using managed certificates with GKE.

Installing Develocity

In this section you will install Develocity on your newly created instance. For full details on installation options, please see the Develocity Helm Kubernetes Installation Manual.

1. Install helm

To install Helm, run:

$ curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash
See Helm’s installation documentation for more details, and non-Linux instructions.

2. Prepare a Helm values file

Create a Helm values file named values.yaml as shown below:

values.yaml
global:
  hostname: develocity.example.com (1)
  # remove this if not using a managed SSL cert
  externalSSLTermination: true (2)
  storage:
    data:
      class: premium-rwo (3)
    backup:
      class: standard (4)

ingress:
  enabled: true
  annotations:
    "kubernetes.io/ingress.global-static-ip-name": develocity-static-ip (5)
    # remove this if not using a managed SSL cert
    "networking.gke.io/managed-certificates": develocity-cert (6)
  grpc:
    serviceAnnotations:
      cloud.google.com/app-protocols: '{"grpc":"HTTP2"}' (7)
1 Use the hostname you decided on in 3. Configure the hostname or substitute it later as shown below.
2 When using a managed SSL cert, SSL is terminated at the load balancer created by the Ingress, so you don’t want to also terminate it at the ingress. Remove this if not using a managed SSL cert.
3 Use a high-performance volume for data volumes. See Choosing storage classes.
4 Use a low performance volume for backups. See Choosing storage classes.
5 Configure the Ingress’s load balancer to use the static IP you created earlier.
6 Configure the Ingress’s load balancer to use the managed SSL certificate you created. Remove this if not using a managed SSL cert.
7 Configure the Ingress’s load balancer to support gRPC protocol via HTTP2
When adding things to your Helm values file, merge any duplicate blocks. Alternatively, you can use separate files and pass all of them with --values «file» when running Helm commands.

This file configures Develocity and its installation. For more details on what is configurable, see the Kubernetes Helm Chart Configuration Guide.

If you want to use a nip.io hostname as described in 3. Configure the hostname, you can substitute it into the Helm values file by running:

$ GE_IP_ADDR=$(gcloud compute addresses describe develocity-static-ip --global --format='value(address)')

$ GE_HOSTNAME="gradle-enterprise-$(printf '%02x' $(echo ${GE_IP_ADDR//./ })).nip.io"

$ sed -i "s/develocity.example.com/${GE_HOSTNAME}/g" path/to/values.yaml
If you want to use a Cloud SQL PostgreSQL instance as your database instead of the embedded database, follow Using Cloud SQL as a Develocity user-managed database and return to this procedure when finished.

Choosing storage classes

In the example Helm values file, we configure Develocity to use a high-performance SSD (the premium-rwo storage class) for its data volumes. This is optional, but recommended for best performance. See the installation manual’s section on storage requirements for more details.

Similarly, we configure the backup storage and log storage to use non-SSD disks (the standard storage class). This is more cost-efficient and avoids Google Cloud’s default 500 GB SSD quota.

You can see the performance characteristics of the different Google Cloud disk types in Google Cloud’s disk performance docs. Note that Develocity has a 250 GB main data volume by default.

If you are using a user-managed database (such as a Cloud SQL database), then this data volume is not created.

Storage classes do not necessarily map to disk types, but you can see which disk type each storage class uses by running:

$ kubectl describe storageclass

The type will be listed as Parameters: type=pd-«disk-type» for the storage classes that map to persistent disks.

Google Cloud’s default standard-rwo storage class maps to the pd-balanced disk type, which is a SSD, despite the storage class’s name. If you don’t explicitly set a storage class for the backup and logs storage, you will end up using SSDs for backup and log storage.

3. Install the gradle-enterprise Helm chart

First, add the https://helm.gradle.com/ helm repository and update it:

$ helm repo add gradle https://helm.gradle.com/
$ helm repo update gradle
If you’re using an older Helm version (which Cloud Shell may), you may need to run helm repo update instead of helm repo update gradle.

Second, download post-renderer resources into your current working directory:

for f in 'backend-config.yaml' 'kustomization.yaml' 'add-develocity-backend-config.sh' 'patches.yaml'
do
  wget "https://docs.gradle.com/develocity/tutorials/gcp-kubernetes/kustomization/$f"
done
chmod a+x add-develocity-backend-config.sh
This kustomize script creates additional Google Cloud resources and configures service definitions, both of which are not possible with the Helm chart.

Make sure that kustomize is available on your computer. If you have opted for Cloud Shell, kustomize is already pre-installed. Otherwise, you can install it with the following command:

curl -s "https://raw.githubusercontent.com/kubernetes-sigs/kustomize/master/hack/install_kustomize.sh"  | bash

Then run helm install with the following command:

$ helm install \
    --create-namespace --namespace develocity \
    develocity \
    gradle/gradle-enterprise \
    --values path/to/values.yaml \(1)
    --set-file global.license.file=path/to/develocity.license \(2)
    --post-renderer=./add-develocity-backend-config.sh (3)
1 The Helm values file you created in 2. Prepare a Helm values file.
2 The license you obtained in 2. A Develocity license.
3 The post renderer script you downloaded

You should see output similar to this:

NAME: develocity
LAST DEPLOYED: Wed Jul 13 04:08:35 2022
NAMESPACE: develocity
STATUS: deployed
REVISION: 1
TEST SUITE: None

If you instead see an error like:

Error: INSTALLATION FAILED: roles.rbac.authorization.k8s.io is forbidden: User "me@gmail.com" cannot create resource "roles" in API group "rbac.authorization.k8s.io" in the namespace "gradle-enterprise": requires one of ["container.roles.create"] permission(s).

4. Wait for Develocity to start

You can see the status of Develocity starting up by examining its pods.

$ kubectl --namespace develocity get pods
NAME                                              READY   STATUS              RESTARTS   AGE
gradle-enterprise-operator-76694c949d-md5dh       1/1     Running             0          39s
gradle-database-65d975cf8-dk7kw                   0/2     Init:0/2            0          39s
gradle-build-cache-node-57b9bdd46d-2txf5          0/1     Init:0/1            0          39s
gradle-proxy-0                                    0/1     ContainerCreating   0          39s
gradle-metrics-cfcd8f7f7-zqds9                    0/1     Running             0          39s
gradle-test-distribution-broker-6fd84c6988-x6jvw  0/1     Init:0/1            0          39s
gradle-keycloak-0                                 0/1     Pending             0          39s
gradle-enterprise-app-0                           0/1     Pending             0          39s
If you use the GKE web UI, some deployments or stateful sets will show as Does not have minimum availability while their init containers are running.

Eventually the pods should all report as Running:

$ kubectl --namespace develocity get pods
NAME                                               READY   STATUS    RESTARTS   AGE
gradle-enterprise-operator-76694c949d-md5dh        1/1     Running   0          4m
gradle-proxy-0                                     1/1     Running   0          3m
gradle-database-65d975cf8-dk7kw                    2/2     Running   0          3m
gradle-enterprise-app-0                            1/1     Running   0          3m
gradle-metrics-cfcd8f7f7-zqds9                     1/1     Running   0          3m
gradle-test-distribution-broker-6fd84c6988-x6jvw   1/1     Running   0          3m
gradle-build-cache-node-57b9bdd46d-2txf5           1/1     Running   0          4m
gradle-keycloak-0                                  1/1     Running   0          3m

Secondly, check that all backends are healthy:

$ kubectl get ingress gradle-enterprise-ingress --namespace develocity --output jsonpath="{.metadata.annotations}" | jq -r '.["ingress.kubernetes.io/backends"]' | jq .
{
  "k8s1-3ee23613-develocity-gradle-build-cache-node-6011-934fec1f": "HEALTHY",
  "k8s1-3ee23613-develocity-gradle-enterprise-app-6011-b9500a51": "HEALTHY",
  "k8s1-3ee23613-develocity-gradle-proxy-80-6ea00d0e": "HEALTHY",
  "k8s1-3ee23613-kube-system-default-http-backend-80-e36aeaea": "HEALTHY"
}

Once all pods have a status of Running and the backends are HEALTHY, the system is up, and you can interact with it by visiting its URL in a web browser. You can also visit the URL immediately once the backends are up and healthy, and will see a starting screen, which will then redirect to a Build Scan list once the app has started.

If the pods do not all start correctly, please see the troubleshooting section in the administration manual.

Once the pods are ready, it may take up to 60 minutes, but not usually that long, for Google to provision the managed SSL certificate. The status of the certificate can be checked by running:

$ kubectl describe managedcertificate --namespace develocity develocity-cert

Look for the Status, Certificate Status, and Domain Status fields in the output.

Using Develocity

Many features of Develocity, including access control, database backups, and Build Scan retention can be configured in Develocity itself, once it is running. The administration manual walks you through the various features you can configure post-installation - you should give the section a read.

For instructions on how to start using Develocity in your builds, consult the Getting Started with Develocity guide. See Teardown and Cleanup for instructions on uninstalling Develocity and deleting related resources, such as persistent disk volumes.

Further reading

Appendix A: Using Cloud SQL as a Develocity user-managed database

Develocity can use a user-managed database instead of using its own embedded database. This can have a number of benefits, including easier resource scaling (and even autoscaling), easier backup and snapshot management, and failover support. For details on the pros and cons of using a user-managed database with Develocity, see the Database options section of the Kubernetes Helm Chart Configuration Guide. This appendix will walk you through using Google Cloud SQL as a user-managed database.

Obtain the required permissions

You will need permission to create and manage Cloud SQL instances and service accounts, and to add roles to service accounts.

You can obtain these permissions using the following built-in roles: roles/iam.serviceAccountAdmin, roles/resourcemanager.projectIamAdmin, and roles/cloudsql.admin.

If getting roles/iam.serviceAccountAdmin and roles/resourcemanager.projectIamAdmin is difficult, you can have someone else who has permission to create service accounts and add roles to them complete 3. Create a Google Cloud service account with Cloud SQL permissions and 4. Bind the Kubernetes service account to the Google Cloud service account.

Set up a Cloud SQL instance

Before starting, it is a good idea to review Develocity’s supported Postgres versions and storage requirements.

1. Decide on a root username and password

Decide on a root password for the database instance. We will refer to it as «db-root-password». This is the password you will use for your database connection, so save it somewhere secure.

The superuser is only used by Develocity to set up the database and create migrator and application users. You can avoid using the superuser from Develocity by setting up the database yourself, as described in the Database options section of Kubernetes Helm Chart Configuration Guide. Please contact Gradle support for help with this.

2. Create the Cloud SQL instance

Create the Cloud SQL instance:

$ gcloud sql instances create gradle-enterprise-database \
    --database-version=POSTGRES_14 \
    --cpu=2 \
    --memory=8GB \
    --storage-size=250GB \
    --require-ssl \
    --database-flags=max_connections=200 \
    --zone=$(gcloud config get compute/zone) \(1)
    --root-password=«db-root-password»
1 The zone where you created your GKE cluster.

This will create an instance with 2 CPUs and 8 GB of RAM, with 250 GB of storage, without any replication. The storage will automatically increase if necessary, but will not decrease.

Cloud SQL databases are not backed up by default, but this can be easily enabled. See the backup documentation.
Consult the command reference to see all options available when creating the Cloud SQL instance.
It is possible to create an instance with a private (VPC-only) IP. Consult this documentation for details. The connection method we use will still work with a private IP, with some modifications described in the Cloud SQL proxy docs.

3. Create a database

Cloud SQL instances don’t come with a database by default, so you need to create one for Develocity:

$ gcloud sql databases create gradle_enterprise --instance=gradle-enterprise-database

Make the database accessible from your GKE cluster

To connect to the instance, you will use the Cloud SQL Auth proxy running as a standalone service. While the official documentation recommends running the proxy as a sidecar container, this does not work for Develocity because some of our init-containers require database access.

1. Enable workload identity for your cluster

Workload identity allows GKE to map Kubernetes service accounts to Google Cloud service accounts. We will use it to map a Kubernetes service account to a Google Cloud service account with Cloud SQL access. It is enabled by default if you are using an Autopilot cluster. If not, you need to enable it.

Enable it for the cluster and the default node pool by running:

$ gcloud container clusters update gradle-enterprise \
    --workload-pool=$(gcloud config get project).svc.id.goog
$ gcloud container node-pools update default-pool \
    --cluster=gradle-enterprise \
    --workload-metadata=GKE_METADATA

This will take some time since GKE has to re-create the nodes.

For more details on workload identity and how to enable it, consult GKE’s documentation. Develocity is fully compatible with workload identity.
If enabling workload identity is not possible for you, you can instead provide the service account’s key file. To do this, skip 4. Bind the Kubernetes service account to the Google Cloud service account and 5. Use the Google Cloud service account and alter the deployment manifest in 6. Deploy the Cloud SQL Proxy according to GKE’s documentation. Note that the service account must still have the required Cloud SQL permissions.

2. Create a Kubernetes service account for the Cloud SQL proxy

The Cloud SQL Proxy needs a service account to authenticate with your Cloud SQL instance. Create it by applying the following manifest to your cluster (e.g. by using kubectl apply -f - and pasting the manifest):

db-service-account.yaml
apiVersion: v1
kind: Namespace
metadata:
  name: gradle-enterprise
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: ge-database-service-account
  namespace: gradle-enterprise
When writing or pasting to a shell’s stdin, use EOF (usually ctrl+d) to end the input.

3. Create a Google Cloud service account with Cloud SQL permissions

You need to create a Google Cloud service account to authenticate with Cloud SQL with:

$ gcloud iam service-accounts create ge-database-service-account \
    --display-name="Develocity Database Service Account"
$ gcloud projects add-iam-policy-binding $(gcloud config get project) \
    --member="serviceAccount:ge-database-service-account@$(gcloud config get project).iam.gserviceaccount.com" \
    --role="roles/cloudsql.client"

4. Bind the Kubernetes service account to the Google Cloud service account

You need to bind the Kubernetes service account you created to the Google Cloud service account you created. This causes GKE to use the Google Cloud service account as the workload’s identity where the Kubernetes service account is used.

Allow the binding by running:

$ PROJECT_ID=$(gcloud config get project)

$ gcloud iam service-accounts add-iam-policy-binding \
    --role="roles/iam.workloadIdentityUser" \
    --member="serviceAccount:${PROJECT_ID}.svc.id.goog[gradle-enterprise/ge-database-service-account]" \
    "ge-database-service-account@${PROJECT_ID}.iam.gserviceaccount.com"

5. Use the Google Cloud service account

Configure the Kubernetes service account to use the binding you created in the last step:

$ PROJECT_ID=$(gcloud config get project)

$ kubectl annotate serviceaccount \
    ge-database-service-account --namespace develocity \
    iam.gke.io/gcp-service-account="ge-database-service-account@${PROJECT_ID}.iam.gserviceaccount.com"

6. Deploy the Cloud SQL Proxy

Get the connection name of your Cloud SQL instance by running:

$ gcloud sql instances describe gradle-enterprise-database --format='value(connectionName)'

We will refer to this as «connection-name»

Then deploy the proxy and create a service for it by applying the following manifest:

database-proxy.yaml
apiVersion: v1
kind: Namespace
metadata:
  name: gradle-enterprise
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: gradle-database-proxy
  namespace: gradle-enterprise
spec:
  selector:
    matchLabels:
        app.kubernetes.io/part-of: gradle-enterprise
        app.kubernetes.io/component: database-proxy
  template:
    metadata:
      labels:
        app.kubernetes.io/part-of: gradle-enterprise
        app.kubernetes.io/component: database-proxy
    spec:
      serviceAccountName: ge-database-service-account
      containers:
      - name: cloud-sql-proxy
        image: gcr.io/cloud-sql-connectors/cloud-sql-proxy:latest (1)
        ports:
        - containerPort: 6543
        args: (2)
          - "--port"
          - "6543"
          - "--address"
          - "0.0.0.0"
          - "«connection-name»" (3)
        resources:
          requests:
            cpu: "1" (4)
            memory: "2Gi" (5)
          limits:
            cpu: "2" (4)
            memory: "4Gi" (5)
        securityContext:
          runAsNonRoot: true
---
apiVersion: v1
kind: Service
metadata:
  name: gradle-database-proxy
  namespace: gradle-enterprise
spec:
  selector:
    app.kubernetes.io/part-of: gradle-enterprise
    app.kubernetes.io/component: database-proxy
  ports:
  - port: 5432
    targetPort: 6543
1 This uses the latest version of the Cloud SQL proxy. It is recommended to use a specific version for long-lived environments.
2 The arguments to the Cloud SQL proxy.
3 The connection name of your Cloud SQL instance, which you can substitute as shown below.
4 While it is fairly lightweight, the proxy’s CPU use scales linearly with the amount of database IO.
5 While it is fairly lightweight, the proxy’s memory use scales linearly with the amount of active connections.
In any long-lived deployments, you should use a specific version of the Cloud SQL Proxy instead of latest. See item 1 above.
The arguments used here are only valid for major version 2 of the Cloud SQL Proxy. If you’re using version 1, you can reverse-migrate the arguments using the migration guide.

To easily apply this manifest while substituting in the correct «connection-name», run (verbatim):

$ CONNECTION_NAME=$(gcloud sql instances describe gradle-enterprise-database --format='value(connectionName)')

$ sed "s/«connection-name»/${CONNECTION_NAME}/g" | kubectl apply -f -

And then paste the above manifest (verbatim) into stdin.

When writing or pasting to a shell’s stdin, use EOF (usually ctrl+d) to end the input.
Consult the GKE and Cloud SQL tutorial and Connect from GKE documentation for more details on setting up Cloud SQL access from GKE.

Configure Develocity to use your Cloud SQL instance

Add the following configuration snippet to your Helm values file:

values.yaml
database:
  location: user-managed
  connection:
    host: gradle-database-proxy
    port: 5432
    databaseName: gradle_enterprise
    params: "?ssl=false"
  credentials:
    superuser:
      username: postgres
      password: «db-root-password»

If you skipped to this appendix from 2. Prepare a Helm values file while installing Develocity, continue at 3. Install the gradle-enterprise Helm chart.

When adding things to your Helm values file, merge any duplicate blocks. Alternatively, you can use separate files and pass all of them with --values «file» when running Helm commands.
The superuser is only used to set up the database and create migrator and application users. You can avoid using the superuser by setting up the database yourself, as described in the Database options section of Kubernetes Helm Chart Configuration Guide. Please contact Gradle support for help with this.
This embeds your database superuser credentials in your Helm values file, meaning it must be kept secure. If you prefer to provide the credentials as a Kubernetes secret, consult Develocity’s Database options.
While we recommend completing this appendix before installing the Develocity Helm chart, it is possible to do it afterward and then update the Helm release. To do this, follow the instructions in the installation manual.
Switching to a user-managed database after installing Develocity will result in the loss of any data stored prior to the switch. This may not be an issue for new installations. If it is, follow the user-managed database migration guide.

Appendix B: Teardown and Cleanup

This appendix will walk you through tearing down Develocity and deleting any resources created by following this tutorial. Before deleting your cluster, you should uninstall the Develocity helm chart. Otherwise, the persistent storage disks will not be deleted.

To uninstall Develocity, run:

$ helm uninstall --namespace develocity develocity

After executing this command, wait a minute or so for the disks to be deleted. The disks backing the three log volumes and the nodes' boot disks will remain. The log disks can be deleted manually if you wish, and the node boot disks will be deleted when you delete the cluster. You can list the disks by running:

$ gcloud compute disks list

To delete the managed certificate and static IP address, run:

$ kubectl delete managedcertificate --namespace develocity develocity-cert
$ gcloud compute addresses delete develocity-static-ip --global
If you plan on re-installing Develocity at a later date, you may wish to keep and re-use the IP address.

To delete the cluster, you simply need to enter the command:

$ gcloud container clusters delete gradle-enterprise

This will ask if you want to continue. Enter y to delete the cluster.

If you’re using other resources, like a Cloud SQL database, remember to delete them too. Cloud SQL teardown instructions are in the section below.

Cloud SQL

If you followed Using Cloud SQL as a Develocity user-managed database, you have some additional cleanup to do.

Deleting a Cloud SQL instance also deletes any automated backups of its database.

To delete your Cloud SQL instance, run:

$ gcloud sql instances delete gradle-enterprise-database

To also delete the service account you created, run:

$ gcloud iam service-accounts delete \
    "ge-database-service-account@$(gcloud config get project).iam.gserviceaccount.com"
If you didn’t get full permissions for service account management, you may not be able to do this yourself.