This tutorial will show how to run Gradle Enterprise on Amazon EC2.

This tutorial requires using Gradle Enterprise 2022.3.1 or later.

You can complete this tutorial in:

  • 10 minutes (read the tutorial)

  • 30 minutes (read the tutorial and perform the installation)

Gradle Enterprise can be installed as a standalone installation on a single server. It can also be installed in an existing Kubernetes cluster, including Amazon EKS, as shown in our EKS installation tutorial. This tutorial shows how to set up a standalone installation on an Amazon EC2 instance.

Gradle Enterprise can be installed on most modern Linux servers. The majority of this tutorial is a quick start guide to creating and minimally configuring an instance in EC2 for a Gradle Enterprise installation. If you already have EC2 expertise and are able to provision an instance, you may wish to skip straight to the Gradle Enterprise installation instructions.

Other installation tutorials

Prerequisites

1. An AWS Account

You can create a free account if you do not already have one. However, you will not be able to complete this tutorial on the free tier.

2. A Gradle Enterprise license

You can request a Gradle Enterprise trial here. If you have purchased Gradle Enterprise, you will already have a license file.

You will need permission to create and manage EC2 instances, security groups, and SSH keys. This can be easily granted using the AmazonEC2FullAccess AWS managed policy.

If you want to use AWS’s Cloud Shell (see Setting up your shell environment), you will also need Cloud Shell permissions, which can be easily granted with the AWSCloudShellFullAccess AWS managed policy.

If you choose to follow our RDS or S3 appendices, you will need the permissions described in them, too.

4. Hostname (optional)

Amazon EC2 machines are provisioned with generated hostnames such as ec2-203-0-113-25.compute-1.amazonaws.com. You can use this hostname to access Gradle Enterprise.

If you want to access Gradle Enterprise by a host name of your choosing (e.g. ge.example.com), you will need the ability to create the necessary DNS record to route this name to the AWS-created hostname.

You will be able to update this later. You can start with AWS’s hostname, and later reconfigure to use a custom hostname if desired.

AWS provides a guide to using their DNS service with EC2. It requires a running EC2 instance, so you should follow it after finishing Creating an EC2 Instance.

Setting up your shell environment

You need to use a number of tools to create AWS resources and install Gradle Enterprise. You can either install them locally, or use AWS’s Cloud Shell, which comes with the tools you will need preinstalled and preconfigured.

We assume you are using bash as your shell, although any shell that is fairly compatible (e.g. zsh) should work without much trouble.

Some tools need to be installed on your EC2 VM. You will install those later.

When files are referred to in this tutorial, they are assumed to be on the machine you are running the command on. When running commands on your VM, any necessary files will be explicitly uploaded. When using Cloud Shell, use the "Upload File" action as explained in AWS’s guide.
The only persistent storage in Cloud Shell is $HOME.

If you decide to use Cloud Shell, skip to Creating an EC2 Instance.

1. Install the AWS CLI

You will be using the aws command line tool to provision and configure your server. To install it on your local machine, follow the instructions in the AWS documentation.

2. Configure the AWS CLI

The aws CLI must be configured with an access key to be able to access your AWS account. If you have already configured the CLI, you can skip this step.

If you have an access key already, but have not configured the aws CLI, you can follow the AWS CLI quick setup guide. Choose the region you wish to install Gradle Enterprise in. You should generally pick the region closest to you geographically to ensure the best performance. AWS provides a list of all available EKS regions.

If you do not have an access key, follow the AWS CLI Prerequisites guide, and then the quick setup guide.

Creating an EC2 Instance

In this section you will create an EC2 instance to run Gradle Enterprise on.

If you’re using Cloud Shell, remember to run these commands there.

1. Generate a key pair for SSH access

If you do not already have a key pair for accessing AWS EC2 resources, you will need to create one. If you already have a key pair that you wish to use, or wish to use one of the other methods for connecting to EC2 instances, you can skip this step. The rest of this tutorial assumes you will access the instance via SSH.

Before creating an EC2 agent, you will need to create a key pair for it. Don’t lose the generated private key. It is only accessible when created, and will be required to access the instance. If you do lose it, consult AWS’s recovery instructions.

To create your key pair, which we will call GradleEnterpriseKeyPair, use the command:

$ aws ec2 create-key-pair \
    --key-name GradleEnterpriseKeyPair \
    --query 'KeyMaterial' \
    --output text > GradleEnterpriseKeyPair.pem

If you are using Powershell, its default UTF-8 encoding for file redirections can cause issues, so you should use this instead:

PS C:\> aws ec2 create-key-pair --key-name GradleEnterpriseKeyPair --query 'KeyMaterial' --output text |
    out-file -encoding ascii -filepath GradleEnterpriseKeyPair.pem

SSH requires that only your user has permissions to read PEM files, so you need to change the file permissions. Under Unix/Linux/OS X, this can be done by using the chmod command:

$ chmod 400 path/to/GradleEnterpriseKeyPair.pem
For more details, consult the AWS CLI key pair guide.

2. Create a security group

AWS comes with a default VPC, so you do not need to create new one to follow this tutorial. You need to create a security group in the default VPC that exposes the networking ports required to access Gradle Enterprise.

To create a security group, use the following command:

$ aws ec2 create-security-group --group-name ge-sg \
    --description "Gradle Enterprise security group" \
    --vpc-id $(aws ec2 describe-vpcs --filters Name=is-default,Values=true --query 'Vpcs[0].VpcId' --output text) (1)
1 The ID of the default VPC.

You will need to expose port 443 globally, for HTTPS. If you are using HTTP, you will need to expose port 80 instead.

To expose port 443, run:

$ SECURITY_GROUP_ID=$(
    aws ec2 describe-security-groups \
    --filters Name=group-name,Values=ge-sg \
    --query 'SecurityGroups[0].GroupId' --output text
  )

$ aws ec2 authorize-security-group-ingress \
    --protocol tcp --port 443 \
    --cidr 0.0.0.0/0 \
    --group-id ${SECURITY_GROUP_ID} (1)
1 The ID of the security group you just created.
If you also expose port 80 but HTTPS is configured, Gradle Enterprise will redirect HTTP access to HTTPS.

You will also want to expose port 22 for SSH access, but usually only to select IPs. To do this for «target-ip», run:

$ SECURITY_GROUP_ID=$(
    aws ec2 describe-security-groups \
    --filters Name=group-name,Values=ge-sg \
    --query 'SecurityGroups[0].GroupId' --output text
  )

$ aws ec2 authorize-security-group-ingress \
    --protocol tcp --port 22 \
    --group-id ${SECURITY_GROUP_ID} \(1)
    --cidr «target-ip»/32
1 The ID of the security group you just created.
The command requires IPs be specified as CIDR blocks. Appending /32 to your IP denotes a CIDR block with only that IP.

You can easily find your own IP using an AWS utility: run curl https://checkip.amazonaws.com.

To expose SSH globally (note that it still requires the key generated earlier), use 0.0.0.0/0 instead of «target-ip»/32.

For more details, consult the AWS CLI security group guide.

3. Choose a virtual server image

Before creating your EC2 instance, you will need to choose which server image to use. Choose a server image appropriate for your environment after consulting the Gradle Enterprise operating system requirements.

You can use the AWS CLI to see the latest EBS-backed Amazon Linux 2 AMIs available in your region by running a command like:

$ aws ec2 describe-images \
    --owners self amazon \
    --filters Name=architecture,Values=x86_64 \
    Name=image-type,Values=machine \
    'Name=description,Values=*Amazon Linux 2 AMI*' \
    Name=root-device-type,Values=ebs

Images are sorted by descending creation date, so you should choose the first one that meets your needs. If you do not have a preference, choose the most recent minimal HVM “Amazon Linux 2 AMI” version. The image name for these images looks something like amzn2-ami-minimal-hvm-2.*-x86_64-ebs.

For more details, consult the AWS guide for finding suitable images.

We will refer to the chosen image id as the «image-id». Note the image’s RootDeviceName, we will refer to this as «root-device-name».

4. Choose an instance type

At the time of writing, the simplest instance type that meets Gradle Enterprise’s minimum requirements is t2.xlarge. This instance type will be adequate for most installations, unless the instance is very heavily used, and is what we will use for this tutorial.

5. Choose storage

Gradle Enterprise installations that use the embedded database are highly dependent on disk performance. The minimum storage space for Gradle Enterprise’s data volume is 250 GB when using an embedded database, with 3,000 or more IOPS recommended.

Alternatively, Gradle Enterprise can use a user-managed database that is compatible with PostgreSQL 12, 13, or 14, including Amazon RDS and Aurora. For simplicity, this tutorial will stick with the embedded database. However, there are instructions for using Amazon RDS as a user-managed database in an appendix.

Gradle Enterprise can also be configured to store Build Scans in a S3 bucket. This can help performance in high-traffic installations by taking load off the database. See the S3 appendix for details.

Regardless of the above options, you need to ensure you provision at least 30 GB for /var/lib/rancher/k3s and 1 GB for /tmp. These are not particularly performance sensitive.

You need to choose the disk type for the data volumes. Consult the Gradle Enterprise AWS storage requirements and AWS’s volume type guide. gp3 is a good starting point, and defaults to 3,000 IOPS.

For simplicity, in this tutorial we will expand the root volume to 300 GB and change the type to gp3, rather than mounting individual volumes. This is not advisable on a production instance.

6. Create your instance

To create your instance, run:

$ SECURITY_GROUP_ID=$(
    aws ec2 describe-security-groups \
    --filters Name=group-name,Values=ge-sg \
    --query 'SecurityGroups[0].GroupId' --output text
  )

$ aws ec2 run-instances \
    --count 1 \
    --instance-type t2.xlarge \(1)
    --key-name GradleEnterpriseKeyPair \(2)
    --security-group-ids ${SECURITY_GROUP_ID} \(3)
    --associate-public-ip-address \(4)
    --tag-specifications 'ResourceType=instance,Tags=[{Key=gradle-enterprise,Value=true}]' \(5)
    --image-id «image-id» \(6)
    --block-device-mappings \
    '[{"DeviceName":"«root-device-name»","Ebs":{"VolumeSize":300,"VolumeType":"gp3","DeleteOnTermination":true}}]'(7)
1 The instance type from 4. Choose an instance type.
2 The name of the key pair you want to use to access your EC2 instance.
3 The security group ID from 2. Create a security group.
4 If you are using a load balancer, you probably do not want the instance’s IP to be public. This depends on your networking setup.
5 Tag your instance so we can easily identify it later.
6 The «image-id» from 3. Choose a virtual server image.
7 The «root-device-name» from 3. Choose a virtual server image (usually /dev/xvda). Note that if you are using a user-managed database, the volume size may be decreased to 50 GB.
When using an embedded database, you may want to use the --ebs-optimized argument. It will result in better disk performance, but will add additional costs and only supports some instance types. See AWS’s documentation for details.

You can view your instance’s details by running:

$ aws ec2 describe-instances --filters Name=tag-key,Values=gradle-enterprise
If you have multiple instances created by following this tutorial, or multiple instances with the gradle-enterprise tag, the commands in this tutorial that look up your instance’s hostname or instance ID using aws ec2 describe-instances will not work out of the box. Alter the Reservations index in these commands to select the correct instance.

To view just the hostname, run:

$ aws ec2 describe-instances \
    --filters Name=tag-key,Values=gradle-enterprise Name=instance-state-name,Values=running \
    --query Reservations[0].Instances[0].PublicDnsName \
    --output text

This will be blank until your instance has started.

For more details, consult the AWS CLI EC2 instances guide.

7. Configure the hostname

If you intend to use a custom hostname to access your Gradle Enterprise instance, you now need to add the appropriate DNS records.

Add an A record for your hostname that points to the public IP address of your EC2 instance. You can find this IP address by describing the instance as shown in the previous step and looking for the PublicIp field(s).

ge.example.com A 12.34.56.78

You should verify that your DNS record works correctly before installing Gradle Enterprise, such as by using dig ge.example.com.

Alternatively, you can use the hostname generated by AWS. You can find the generated hostname by describing the instance as shown in the previous step and looking for the PublicDnsName field(s).

You can run Gradle Enterprise behind a cloud load balancer such as Elastic Load Balancing. If you do this, you should generally use /ping for the load balancer’s health check. It will respond as healthy when Gradle Enterprise is capable of handling requests, even if that is limited to showing the interactive starting page while the instance is starting.

This may be unwanted for highly available installations using the load balancer to manage failover. If you are installing Gradle Enterprise in a highly available setup, we recommend submitting a ticket at support.gradle.com for assistance.

You can use an AWS-managed SSL certificate by configuring Gradle Enterprise to use external SSL termination, putting your Gradle Enterprise installation behind an Elastic Load Balancer, and terminating SSL at the load balancer using the managed certificate. Consult the ALB and NLB HTTPS/TLS documentation for more details.

8. Set up your connection to the EC2 instance

There are various ways to connect to an EC2 instance. As explained in 1. Generate a key pair for SSH access, we are using SSH for this tutorial. If you are using a different method, you can skip this step. Whichever method you use, you need to have access to the instance’s command line and be able to upload files.

Once your instance starts (you can see the status by describing the instance as explained above), SSH into your instance with the following command:

$ GE_HOSTNAME=$(
    aws ec2 describe-instances \
    --filters Name=tag-key,Values=gradle-enterprise Name=instance-state-name,Values=running  \
    --query Reservations[0].Instances[0].PublicDnsName \
    --output text
  )

$ ssh -i path/to/GradleEnterpriseKeyPair.pem ec2-user@${GE_HOSTNAME}
The username (ec2-user in the command above) differs depending on which virtual server image you chose. For “Amazon Linux 2 AMI” images it is ec2-user, but may be different if you chose a different AMI.

You should then be able to run commands in the instance’s shell. Once you have ensured SSH access works, return to your machine.

Installing Gradle Enterprise

In this section you will install Gradle Enterprise on your newly created instance. For full details on installation options, please see the Gradle Enterprise Helm Standalone Installation Manual.

1. Prepare a Helm values file

Create a Helm values file named values.yaml as shown below:

values.yaml
global:
  hostname: ge.example.com (1)
1 Use the hostname you decided on in 7. Configure the hostname or substitute it later as shown below.
When adding things to your Helm values file, merge any duplicate blocks. Alternatively, you can use separate files and pass all of them with --values «file» when running Helm commands.

This file configures Gradle Enterprise and its installation. You can find more details on what is configurable in the Gradle Enterprise installation manual’s Helm configuration section.

If you want to use AWS’s autogenerated hostname, you can substitute it in the Helm values file by running:

$ GE_HOSTNAME=$(
    aws ec2 describe-instances \
    --filters Name=tag-key,Values=gradle-enterprise Name=instance-state-name,Values=running  \
    --query Reservations[0].Instances[0].PublicDnsName \
    --output text
  )

$ sed -i "s/ge.example.com/${GE_HOSTNAME}/g" path/to/values.yaml
On macOS, you may need to replace -i with -i '' when using sed.
If you want to provide an SSL certificate instead of having Gradle Enterprise generate a self-signed one, follow the instructions in the installation manual.
If you want to use Amazon RDS as your database instead of the embedded database, follow Using Amazon RDS as a Gradle Enterprise user-managed database.
You can use S3 to store Build Scans, which can improve performance in high-traffic instances. To do so, follow Storing Build Scans in Amazon S3.

2. Copy configuration files to the instance

Copy the Helm values file you created and your Gradle Enterprise license file to the instance:

$ GE_HOSTNAME=$(
    aws ec2 describe-instances \
    --filters Name=tag-key,Values=gradle-enterprise Name=instance-state-name,Values=running  \
    --query Reservations[0].Instances[0].PublicDnsName \
    --output text
  )

$ scp -i path/to/GradleEnterpriseKeyPair.pem path/to/values.yaml ec2-user@${GE_HOSTNAME}:~
$ GE_HOSTNAME=$(
  aws ec2 describe-instances \
  --filters Name=tag-key,Values=gradle-enterprise Name=instance-state-name,Values=running  \
  --query Reservations[0].Instances[0].PublicDnsName \
  --output text
)

$ scp -i path/to/GradleEnterpriseKeyPair.pem path/to/gradle-enterprise.license ec2-user@${GE_HOSTNAME}:~

If you are using a custom SSL certificate, copy the certificate and key files to the instance as well.

3. Install K3s and Helm on your VM

SSH into the instance:

$ GE_HOSTNAME=$(
    aws ec2 describe-instances \
    --filters Name=tag-key,Values=gradle-enterprise Name=instance-state-name,Values=running  \
    --query Reservations[0].Instances[0].PublicDnsName \
    --output text
  )

$ ssh -i path/to/GradleEnterpriseKeyPair.pem ec2-user@${GE_HOSTNAME}

Once you’re on the machine, install K3s and make it available to the current user:

$ curl -sfL https://get.k3s.io | sh -
$ sudo chown ${UID} /etc/rancher/k3s/k3s.yaml
$ mkdir -p "${HOME}/.kube"
$ ln -s /etc/rancher/k3s/k3s.yaml "${HOME}/.kube/config"

Then install Helm:

$ curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash

4. Install the gradle-enterprise-standalone Helm chart

First, add the https://helm.gradle.com/ helm repository and update it:

$ helm repo add gradle https://helm.gradle.com/
$ helm repo update gradle

Then run helm install with the following command:

$ helm install \
    --create-namespace --namespace gradle-enterprise \
    ge-standalone \
    gradle/gradle-enterprise-standalone \
    --values ./values.yaml \(1)
    --set-file global.license.file=./gradle-enterprise.license(2)
1 The Helm values file you created in 1. Prepare a Helm values file and uploaded in 2. Copy configuration files to the instance.
2 The license you obtained in 2. A Gradle Enterprise license and uploaded in 2. Copy configuration files to the instance.

You should see output similar to this:

NAME: ge-standalone
LAST DEPLOYED: Wed Jul 13 04:08:35 2022
NAMESPACE: gradle-enterprise
STATUS: deployed
REVISION: 1
TEST SUITE: None

5. Wait for Gradle Enterprise to start

You can see the status of Gradle Enterprise starting up by examining its pods.

$ kubectl --namespace gradle-enterprise get pods
NAME                                              READY   STATUS              RESTARTS   AGE
gradle-enterprise-operator-76694c949d-md5dh       1/1     Running             0          39s
gradle-database-65d975cf8-dk7kw                   0/2     Init:0/2            0          39s
gradle-build-cache-node-57b9bdd46d-2txf5          0/1     Init:0/1            0          39s
gradle-proxy-0                                    0/1     ContainerCreating   0          39s
gradle-metrics-cfcd8f7f7-zqds9                    0/1     Running             0          39s
gradle-test-distribution-broker-6fd84c6988-x6jvw  0/1     Init:0/1            0          39s
gradle-keycloak-0                                 0/1     Pending             0          39s
gradle-enterprise-app-0                           0/1     Pending             0          39s

Eventually the pods should all report as Running:

$ kubectl --namespace gradle-enterprise get pods
NAME                                               READY   STATUS    RESTARTS   AGE
gradle-enterprise-operator-76694c949d-md5dh        1/1     Running   0          4m
gradle-proxy-0                                     1/1     Running   0          3m
gradle-database-65d975cf8-dk7kw                    2/2     Running   0          3m
gradle-enterprise-app-0                            1/1     Running   0          3m
gradle-metrics-cfcd8f7f7-zqds9                     1/1     Running   0          3m
gradle-test-distribution-broker-6fd84c6988-x6jvw   1/1     Running   0          3m
gradle-build-cache-node-57b9bdd46d-2txf5           1/1     Running   0          4m
gradle-keycloak-0                                  1/1     Running   0          3m

Once all pods have a status of Running, the system is up and can you can interact with it by visiting its URL in a web browser.

You can also visit the URL immediately, and will see a starting screen, which will then redirect to a Build Scan list once the app has started.

Gradle Enterprise uses a self-signed SSL certificate by default, so most browsers will warn about an untrusted certificate. To avoid this, use a managed SSL certificate as described in 7. Configure the hostname or use a custom trusted SSL certificate as described in 1. Prepare a Helm values file.

If the pods do not all start correctly, please see the troubleshooting section in the administration manual.

6. Update the system user password

Unless you have configured the system password using unattended configuration, such as when configuring S3 scans storage, Gradle Enterprise will create a system user with elevated permissions and a default password. This is insecure, and you should change the password as soon as possible.

To change the password, simply visit Gradle Enterprise using a web browser and sign in (using the "Sign In" button at the top right of the page) with the system user (username system, password default). You will then be prompted to select a new password for the system user account. You should record the new password and keep it secret.

It is recommended to not use the system user account regularly. Instead, real administrator user accounts should be created by configuring access control (see the next section).

Using Gradle Enterprise

Many features of Gradle Enterprise, including access control, database backups, and Build Scan retention can be configured in Gradle Enterprise itself, once it is running. The administration manual walks you through the various features you can configure post-installation - you should give the section a read.

For instructions on how to start using Gradle Enterprise in your builds, consult the Getting Started with Gradle Enterprise guide.

Deleting your instance

Further reading

Appendix A: Using Amazon RDS as a Gradle Enterprise user-managed database

Gradle Enterprise can use a user-managed database instead of using its own embedded database. This can have a number of benefits, including easier resource scaling (and even autoscaling), easier backup and snapshot management, and failover support. For details on the pros and cons of using a user-managed database with Gradle Enterprise, see the database section of the installation manual. This appendix will walk you through using an Amazon RDS PostgreSQL instance as your database.

Obtain the required permissions

You will need permission to create and manage Amazon RDS instances. You will also need to create a security group, but you already have permissions to do that.

The necessary permissions can be easily granted by using the AmazonRDSFullAccess AWS managed policy.

Set up an RDS instance

Before starting, it is a good idea to review Gradle Enterprise’s supported Postgres versions and storage requirements.

1. Decide on a root username and password

Decide on a root username and password for the database instance. We will refer to them as «db-root-username» and «db-root-password», respectively. These are the credentials you will use for your database connection, so save them somewhere secure.

The superuser is only used by Gradle Enterprise to set up the database and create migrator and application users. You can avoid using the superuser from Gradle Enterprise by setting up the database yourself, as described in the database configuration section of Gradle Enterprise’s installation manual. Please contact Gradle support for help with this.

2. Create a security group and enable ingress

Before creating the database, you have to create a security group in the VPC you want to use. In this tutorial, you will use the default VPC, which is the same VPC our EC2 instance is in.

To create the security group, run:

$ aws ec2 create-security-group --group-name ge-db-sg \
    --description "Gradle Enterprise DB security group" \
    --vpc-id $(aws ec2 describe-vpcs --filters Name=is-default,Values=true --query 'Vpcs[0].VpcId' --output text) (1)
1 The ID of the default VPC.

Then enable ingress for port 5432 from your EC2 agent’s security group:

$ RDS_SECURITY_GROUP_ID=$(
    aws ec2 describe-security-groups \
    --filters Name=group-name,Values=ge-db-sg \
    --query 'SecurityGroups[0].GroupId' --output text
  )

$ aws ec2 authorize-security-group-ingress \
    --protocol tcp --port 5432 \
    --source-group ge-sg \(1)
    --group-id ${RDS_SECURITY_GROUP_ID} (2)
1 The security group of your EC2 instance.
2 The security group of your RDS instance.

3. Create the RDS instance

Now create the RDS instance:

$ RDS_SECURITY_GROUP_ID=$(
    aws ec2 describe-security-groups \
    --filters Name=group-name,Values=ge-db-sg \
    --query 'SecurityGroups[0].GroupId' --output text
  )

$ aws rds create-db-instance \
    --engine postgres \
    --engine-version 14.3 \
    --db-instance-identifier gradle-enterprise-database \
    --db-name gradle_enterprise \
    --allocated-storage 250 \(1)
    --iops 3000 \(2)
    --db-instance-class db.m5.large \
    --backup-retention-period 3 \(3)
    --no-publicly-accessible \
    --vpc-security-group-ids ${RDS_SECURITY_GROUP_ID} \(4)
    --master-username «db-root-username» \
    --master-user-password «db-root-password»
1 Gradle Enterprise should be installed with 250GB of database storage to start with.
2 As discussed in 5. Choose storage, Gradle Enterprise’s data volumes and database should support at least 3,000 IOPS.
3 The backup retention period, in days.
4 The security group you created in the previous step.
While you don’t configure it here, RDS supports storage autoscaling.
Consult AWS’s database creation guide and the CLI command reference for more details on RDS instance creation.

You can then view the status of your instance with:

$ aws rds describe-db-instances --db-instance-identifier gradle-enterprise-database

Wait until the DBInstanceStatus is available. You should then see the hostname of the instance under Endpoint. This is the address you will use to connect to the instance. We will refer to it as «database-address».

Configure Gradle Enterprise to use your RDS instance

Add the following configuration snippet to your Helm values file:

values.yaml
database:
  location: user-managed
  connection:
    host: «database-address»
    databaseName: gradle_enterprise
  credentials:
    superuser:
      username: «db-root-username»
      password: «db-root-password»
When adding things to your Helm values file, merge any duplicate blocks. Alternatively, you can use separate files and pass all of them with --values «file» when running Helm commands.

You can substitute «database-address» in the Helm values file by running (verbatim):

$ DATABASE_ADDRESS=$(
    aws rds describe-db-instances \
    --db-instance-identifier gradle-enterprise-database \
    --query 'DBInstances[0].Endpoint.Address' \
    --output text
  )

$ sed -i "s/«database-address»/${DATABASE_ADDRESS}/g" path/to/values.yaml
The superuser is only used to set up the database and create migrator and application users. You can avoid using the superuser by setting up the database yourself, as described in the database configuration section of Gradle Enterprise’s installation manual. Please contact Gradle support for help with this.
This embeds your database superuser credentials in your Helm values file, meaning it must be kept secure. If you prefer to provide the credentials as a Kubernetes secret, consult Gradle Enterprise’s database configuration instructions.
While we recommend completing this appendix before installing the Gradle Enterprise Helm chart, it is possible to do it afterwards and then update the the Helm release. To do this, follow the instructions in the installation manual.
Switching to a user-managed database after installing Gradle Enterprise will result in the lose of any data stored prior to the switch. This may not be an issue for new installations. If it is, follow the user-managed database migration guide.

Appendix B: Storing Build Scans in Amazon S3

Gradle Enterprise can be configured to store most Build Scan data in an S3 bucket rather than its database. This can have a number of benefits, including better performance for very high-traffic instances, high scalability and fault tolerance, and cheaper storage costs. For details on the pros and cons of using S3 Build Scan storage with Gradle Enterprise, see the S3 storage section of the administration manual. This appendix will walk you through using an Amazon S3 bucket to store Build Scans.

Gradle Enterprise will still use its database to store other information. However, the size and performance requirements of the database will be much smaller.

Obtain the required permissions

You will need permission to create and manage Amazon S3 buckets. You will also need to be able to create IAM policies, roles, and instance profiles.

The necessary permissions can be easily granted by using the AmazonS3FullAccess and IAMFullAccess AWS managed policies.

If obtaining IAMFullAccess is difficult, you can have someone who has it run 2. Create a policy allowing bucket access, 3. Create an EC2 instance profile with your policy, and 4. Attach the instance profile to your instance. You can also use a more limited set of IAM permissions that allows creating profiles, roles, and instance profiles, and attaching policies to roles, roles to instance profiles, and instance profiles to EC2 instances.

Set up a S3 bucket and allow access from your EC2 instance

You need to create an S3 bucket and create an IAM policy that allows access to it. Then you need to associate that policy with your EC2 instance.

1. Create a S3 bucket

To create the S3 bucket, run:

$ ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)

$ aws s3 mb s3://gradle-enterprise-build-scans-${ACCOUNT_ID} (1)
1 S3 bucket names must be unique across all AWS accounts, within groups of regions. To comply with this, we use your account ID as a suffix. If you have multiple installations of Gradle Enterprise you want to use S3 storage with, either add a suffix or use the same bucket with a different scans object prefix.

2. Create a policy allowing bucket access

To create a role allowing access to your bucket, first create a policy.json file with the following content:

policy.json
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::gradle-enterprise-build-scans-«account-id»"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:DeleteObject",
                "s3:AbortMultipartUpload"
            ],
            "Resource": [
                "arn:aws:s3:::gradle-enterprise-build-scans-«account-id»/*"
            ]
        }
    ]
}

Then run the following commands:

$ ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)

$ sed -i "s/«account-id»/${ACCOUNT_ID}/g" ./policy.json (1)

$ aws iam create-policy \
    --policy-name "gradle-enterprise-build-scan-access" \
    --policy-document file://policy.json (1)
1 The policy.json file you created.

3. Create an EC2 instance profile with your policy

To allow your EC2 instance to use the policy you just created, you need to create an EC2 instance policy that can use it.

First, create a ec2-role-trust-policy.json file with the following contents:

ec2-role-trust-policy.json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": { "Service": "ec2.amazonaws.com"},
      "Action": "sts:AssumeRole"
    }
  ]
}

Then run the following commands:

$ aws iam create-role \
    --role-name "GradleEnterprise_BuildScans_S3_Role" \
    --assume-role-policy-document file://ec2-role-trust-policy.json

$ ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)

$ aws iam attach-role-policy \
    --role-name "GradleEnterprise_BuildScans_S3_Role" \
    --policy-arn "arn:aws:iam::${ACCOUNT_ID}:policy/gradle-enterprise-build-scan-access" (1)
$ aws iam create-instance-profile \
    --instance-profile-name "GradleEnterprise_BuildScans_S3_InstanceProfile"

$ aws iam add-role-to-instance-profile \
    --instance-profile-name "GradleEnterprise_BuildScans_S3_InstanceProfile" \
    --role-name "GradleEnterprise_BuildScans_S3_Role"
1 The ARN of the policy you created in the previous step.

4. Attach the instance profile to your instance

Now that you have an instance profile that grants access to your s3 bucket, you need to attach it to your EC2 instance.

An instance can only have one instance profile. If you are already using an instance profile for your Gradle Enterprise instance, either add your new role to it, or replace it with the new instance profile. See AWS’s guide for details.

To do this, run:

$ INSTANCE_ID=$(
    aws ec2 describe-instances \
    --filters Name=tag-key,Values=gradle-enterprise \
    --query Reservations[0].Instances[0].InstanceId \
    --output text
  )

$ aws ec2 associate-iam-instance-profile \
    --iam-instance-profile Name=GradleEnterprise_BuildScans_S3_InstanceProfile \
    --instance-id ${INSTANCE_ID} (1)
1 The instance ID of your EC2 instance, from 6. Create your instance.

Update your Helm values file

You need to increase the Gradle Enterprise pod’s memory request and limit. This is done via the Helm values file. Add the following to it:

values.yaml
enterprise:
  resources:
    requests:
      memory: 6Gi (1)
    limits:
      memory: 6Gi (1)
1 If you have already set a custom value here, instead increase it by 2Gi.
When adding things to your Helm values file, merge any duplicate blocks. Alternatively, you can use separate files and pass all of them with --values «file» when running Helm commands.
While we recommend completing this appendix before installing the Gradle Enterprise Helm chart, it is possible to do it afterwards and then update the the Helm release. To do this, follow the instructions in the installation manual.

Configure Gradle Enterprise to store Build Scans in your S3 bucket

Now that we have created a S3 bucket and allowed our EC2 instance to access it, we need to configure Gradle Enterprise to actually use it. There are two ways to do this: using Gradle Enterprise’s web UI or using the unattended configuration mechanism. While using the web UI is often easier, it requires starting your Gradle Enterprise instance and then configuring the Build Scans storage. The unattended configuration mechanism lets you configure it as part of your Helm values file. We will provide instructions for both methods.

Using the web UI

To configure Gradle Enterprise to use your bucket using the web UI, follow the instructions in the administration manual with the following configuration:

  • Bucket: gradle-enterprise-build-scans-«account-id». «account-id» can be found by running aws sts get-caller-identity --query Account --output text.

  • Region: your current region. Viewable by running aws configure list | grep region.

  • S3 credentials: Obtain from environment

To print the actual name of your bucket, run:

$ ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)

$ echo "gradle-enterprise-build-scans-${ACCOUNT_ID}"

Using unattended configuration

Before using the unattended configuration mechanism, you should read the relevant section of the administration manual.

1. Choose and hash system password

First, you need to choose a system password and hash it. To do this, install the Gradle Enterprise Admin CLI. Then run:

$ gradle-enterprise-admin config-file hash -o secret.txt -s -

To hash your password from stdin and write it to secret.txt. We will refer to the hashed password as «hashed-system-password».

2. Modify your Helm values file

To use your S3 bucket, add the following to your Helm values file:

values.yaml
global:
  unattended:
    configuration:
      version: 5
      systemPassword: "«hashed-system-password»" (1)
      buildScans:
        storage:
          incomingStorageType: s3
          s3:
            bucket: gradle-enterprise-build-scans-«account-id» (2)
            region: «region» (3)
            credentials:
              source: environment
      advanced:
        app:
          heapMemory: 5632 (4)
1 Your hashed system password.
2 Your account ID, which we will substitute in below.
3 The region of your S3 bucket, which should be your current region. Viewable by running aws configure list | grep region.
4 If you have already set a custom value here, instead increase it by 2048.
When adding things to your Helm values file, merge any duplicate blocks. Alternatively, you can use separate files and pass all of them with --values «file» when running Helm commands.

Then substitute «account-id» in the Helm values file by running (verbatim):

$ ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)

$ sed -i "s/«account-id»/${ACCOUNT_ID}/g" path/to/values.yaml
While we recommend completing this appendix before installing the Gradle Enterprise Helm chart, it is possible to do it afterwards and then update the the Helm release. To do this, follow the instructions in the installation manual.

Verifying S3 storage is used

Gradle Enterprise will start even if your S3 configuration is incorrect. Once Gradle Enterprise has started, you can verify that the S3 configuration is correct by using the "Test S3 Connection" button on the /admin/build-scans page, or by uploading a scan and then checking for its presence in your S3 bucket.

To view the build scans stored in your S3 bucket, run:

$ ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)

$ aws s3 ls s3://gradle-enterprise-build-scans-${ACCOUNT_ID}/build-scans/ \(1)
    --recursive --human-readable --summarize
2022-09-27 19:11:06    6.6 KiB build-scans/2022/09/27/aprvi3bnnxyzm

Total Objects: 1
   Total Size: 6.6 KiB
1 If you used a custom prefix, use it here instead of build-scans.

You won’t see anything at first, but once you upload a build scan (see Using Gradle Enterprise), you should see it there.

Appendix C: Teardown and Cleanup

This appendix will walk you through tearing down Gradle Enterprise and deleting any resources created by following this tutorial.

To delete your instance, run:

$ INSTANCE_ID=$(
    aws ec2 describe-instances \
    --filters Name=tag-key,Values=gradle-enterprise \
    --query Reservations[0].Instances[0].InstanceId \
    --output text
  )

$ aws ec2 terminate-instances --instance-ids ${INSTANCE_ID}
This command assumes that only one instance was created using this tutorial. If you have other instances with the tag gradle-enterprise, you may need to alter the Reservations index.

To also remove the security group you created, run:

$ SECURITY_GROUP_ID=$(
    aws ec2 describe-security-groups \
    --filters Name=group-name,Values=ge-sg \
    --query 'SecurityGroups[0].GroupId' --output text
  )

$ aws ec2 delete-security-group --group-id ${SECURITY_GROUP_ID}
This will fail until deletion of the EC2 instance has finished.

And for the SSH key pair:

$ aws ec2 delete-key-pair --key-name GradleEnterpriseKeyPair

If you have other resources, like a user-managed database or S3 Build Scan storage, remember to delete those too. RDS and S3 teardown instructions are in their respective sections, below.

RDS

If you followed Using Amazon RDS as a Gradle Enterprise user-managed database, you have some additional cleanup to do.

To delete the RDS instance, run:

Deleting an RDS instance also deletes any automated backups of its database. However, by default, the deletion command will create a final snapshot of the database.
$ aws rds delete-db-instance \
    --db-instance-identifier gradle-enterprise-database \
    --final-db-snapshot-identifier gradle-enterprise-db-snapshot
The delete-db-instance command requires that your instance is running so that it can create a final snapshot. You can skip the final snapshot and avoid this by passing --skip-final-snapshot instead of --final-db-snapshot-identifier gradle-enterprise-db-snapshot.

The command will complete immediately, but deletion will likely take some time.

For more details on RDS instance deletion, consult AWS’s guide.

To also delete the security group, run:

$ RDS_SECURITY_GROUP_ID=$(
    aws ec2 describe-security-groups \
    --filters Name=group-name,Values=ge-db-sg \
    --query 'SecurityGroups[0].GroupId' --output text
  )

$ aws ec2 delete-security-group --group-id ${RDS_SECURITY_GROUP_ID}
This will fail until deletion of the database instance has finished.

S3

If you followed Storing Build Scans in Amazon S3, you have some additional cleanup to do.

Deleting your S3 bucket will delete all stored Build Scans. Gradle Enterprise features that rely on historical analysis (e.g. test analytics, predictive test selection) will fail or will provide less useful information.

To delete your S3 bucket, run:

$ ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)

$ aws s3 rb s3://gradle-enterprise-build-scans-${ACCOUNT_ID} --force

Once you have deleted your EC2 instance, you can also delete your IAM resources by running the following commands:

$ aws iam remove-role-from-instance-profile \
    --instance-profile-name "GradleEnterprise_BuildScans_S3_InstanceProfile" \
    --role-name "GradleEnterprise_BuildScans_S3_Role"

$ aws iam delete-instance-profile --instance-profile-name GradleEnterprise_BuildScans_S3_InstanceProfile
$ ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)

$ aws iam detach-role-policy \
    --role-name "GradleEnterprise_BuildScans_S3_Role" \
    --policy-arn "arn:aws:iam::${ACCOUNT_ID}:policy/gradle-enterprise-build-scan-access" (1)

$ aws iam delete-role --role-name GradleEnterprise_BuildScans_S3_Role

$ aws iam delete-policy --policy-arn "arn:aws:iam::${ACCOUNT_ID}:policy/gradle-enterprise-build-scan-access" (1)
1 The ARN of the policy you created.
If you didn’t get full IAM permissions, you may not be able to do this yourself.