Untitled

Using Amazon RDS

This section will walk you through using an Amazon RDS PostgreSQL instance as your database.

1. Obtain the Required Permissions

You need permission to create and manage Amazon RDS instances and security groups.

The necessary permissions are granted using the AmazonRDSFullAccess AWS managed policy.

2. Set up an RDS Instance

Develocity is compatible with PostgreSQL versions 14 through 17. The minimum storage space required is 250 GB with 3,000 or more IOPS.

A. Create a root username and password

Create a root username and password for the database instance, referred to below as «db-root-username» and «db-root-password», respectively. These are the credentials you will use for your database setup; save them somewhere secure.

B. Create a Security Group

Before creating the database, you have to create a security group in the VPC you want to use.

In this tutorial you will use the eksctl created VPC used by your cluster.

You can use a different VPC, but you will need to make the RDS instance accessible from your cluster (for example, by peering the VPCs).

To create the Security Group, run:

CLUSTER_VPC_ID=$(
  aws ec2 describe-vpcs \
  --filters Name=tag:aws:cloudformation:stack-name,Values=eksctl-develocity-cluster \
  --query 'Vpcs[0].VpcId' \
  --output text
)

aws ec2 create-security-group --group-name develocity-database \
  --description "Develocity DB security group" \
  --vpc-id ${CLUSTER_VPC_ID}

C. Enable Ingress

Then enable ingress to the RDS instance from your cluster for port 5432 by running:

CLUSTER_SECURITY_GROUP_ID=$(
  aws eks describe-cluster --name develocity \
  --query cluster.resourcesVpcConfig.clusterSecurityGroupId \
  --output text
)

RDS_SECURITY_GROUP_ID=$(
  aws ec2 describe-security-groups \
  --filters Name=group-name,Values=develocity-database \
  --query 'SecurityGroups[0].GroupId' \
  --output text
)

aws ec2 authorize-security-group-ingress \
  --protocol tcp --port 5432 \
  --source-group ${CLUSTER_SECURITY_GROUP_ID} \
  --group-id ${RDS_SECURITY_GROUP_ID}

D. Create a subnet group

Before creating the database, you need to create a subnet group to specify how the RDS instance will be networked.

This subnet group must have subnets in two availability zones, and typically should use private subnets.

eksctl has already created private subnets you can use.

Create a subnet group containing them by running:

CLUSTER_VPC_ID=$(
  aws ec2 describe-vpcs \
  --filters Name=tag:aws:cloudformation:stack-name,Values=eksctl-develocity-cluster \
  --query 'Vpcs[0].VpcId' \
  --output text
)

SUBNET_IDS=$(
  aws ec2 describe-subnets \
  --query 'Subnets[?!MapPublicIpOnLaunch].SubnetId' \
  --filters Name=vpc-id,Values=${CLUSTER_VPC_ID} \
  --output json
)

aws rds create-db-subnet-group --db-subnet-group-name develocity-database \
  --db-subnet-group-description "Develocity DB subnet group" \
  --subnet-ids ${SUBNET_IDS}

Consult RDS’s subnet group documentation for more details on subnet groups and their requirements.

E. Create the RDS instance

Create the RDS instance:

RDS_SECURITY_GROUP_ID=$(
  aws ec2 describe-security-groups \
  --filters Name=group-name,Values=develocity-database \
  --query 'SecurityGroups[0].GroupId' \
  --output text
)

RDS_POSTGRES_VERSION=$(
  aws rds describe-db-engine-versions \
    --engine postgres \
    --engine-version 17 \(1)
    --default-only \
    --query 'DBEngineVersions[0].EngineVersion' \
    --output text
  )

1	The latest major version of PostgreSQL that Develocity supports.

aws rds create-db-instance \
  --engine postgres \
  --engine-version ${RDS_POSTGRES_VERSION} \
  --db-instance-identifier develocity-database \
  --db-name gradle_enterprise \
  --allocated-storage 250 \(1)
  --iops 3000 \(2)
  --db-instance-class db.m5.large \
  --db-subnet-group-name develocity-database \
  --backup-retention-period 3 \(3)
  --no-publicly-accessible \
  --vpc-security-group-ids ${RDS_SECURITY_GROUP_ID} \
  --master-username «db-root-username» \
  --master-user-password «db-root-password»

1	Develocity should be installed with 250GB of database storage to start with.
2	Develocity’s data volumes and database should support at least 3,000 IOPS.
3	The backup retention period, in days.

While you don’t configure it here, RDS supports storage autoscaling.

Consult AWS’s database creation guide and the CLI command reference for more details on RDS instance creation.

You can view the status of your instance with:

aws rds describe-db-instances --db-instance-identifier develocity-database

Wait until the DBInstanceStatus is available.

Once available, you can see the hostname of the instance under Endpoint > Address. This is the hostname you will use to connect to the instance, subsequently referred to as «database-hostname».

3. Configure the RDS instance for IAM authentication

Develocity supports connecting to the database using IAM authentication. This step describes how to configure your RDS instance to allow IAM database authentication for all the database users that Develocity will connect as (including the superuser, although it is possible for Develocity to not require superuser access, as explained in the Database setup with IAM database authentication section of the Kubernetes Helm chart Configuration Guide).

A. Enable IAM database authentication on the RDS instance

To modify your created RDS database instance to allow IAM database authentication, run:

aws rds modify-db-instance \
  --db-instance-identifier develocity-database \
  --apply-immediately \
  --enable-iam-database-authentication

B. Create a policy allowing database access

To create a role that is permitted to connect to the RDS database as the required database users, using IAM authentication, create a database-policy.json file with the following content:

CONFIGURED_REGION=$(aws configure list | grep region | awk '{print $2}')

ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)

DBI_RESOURCE_ID=$(
  aws rds describe-db-instances \
  --db-instance-identifier develocity-database \
  --query 'DBInstances[0].DbiResourceId' \
  --output text
)

database-policy.json

cat <<EOF > database-policy.json
{
   "Version": "2012-10-17",
   "Statement": [
      {
         "Effect": "Allow",
         "Action": [
             "rds-db:connect"
         ],
         "Resource": [
             "arn:aws:rds-db:${CONFIGURED_REGION}:${ACCOUNT_ID}:dbuser:${DBI_RESOURCE_ID}/ge_app",
             "arn:aws:rds-db:${CONFIGURED_REGION}:${ACCOUNT_ID}:dbuser:${DBI_RESOURCE_ID}/ge_migrator",
             "arn:aws:rds-db:${CONFIGURED_REGION}:${ACCOUNT_ID}:dbuser:${DBI_RESOURCE_ID}/ge_monitor",
             "arn:aws:rds-db:${CONFIGURED_REGION}:${ACCOUNT_ID}:dbuser:${DBI_RESOURCE_ID}/«db-root-username»" (1)
         ]
      }
   ]
}
EOF

1	Replace `«db-root-username»` with the username you chose when creating your RDS instance’s root credentials.

Then run the following command:

aws iam create-policy \
  --policy-name "develocity-rds-access" \
  --policy-document file://database-policy.json

C. Create a role for EKS

To allow Develocity to connect to RDS using the policy you just created, you need to create an IAM role that has attached to the policy you just created. Kubernetes service accounts in EKS need to be able to assume this role.

To associate a Kubernetes service account with an AWS IAM role, we need to use an AWS OIDC provider. We already installed one when setting up the Storage Class EBS CSI driver, so we can use it here.

To create an IAM role that can be assumed by Kubernetes service accounts in EKS using the policy you just created, run the following commands:

ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)

POLICY_ARN="arn:aws:iam::${ACCOUNT_ID}:policy/develocity-rds-access"

eksctl create iamserviceaccount \
  --name develocity-database-account \
  --namespace develocity \
  --cluster develocity \
  --approve \
  --role-only \
  --role-name Develocity_Database_Role \
  --attach-policy-arn ${POLICY_ARN}

Change the trust relationship policy to allow the service accounts to assume the role:

OIDC_PROVIDER=$(aws eks describe-cluster --name develocity \
  --query "cluster.identity.oidc.issuer" \
  --output text | sed -e "s/^https:\/\///")

cat <<EOF > database-trust-relationship.json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "arn:aws:iam::$ACCOUNT_ID:oidc-provider/$OIDC_PROVIDER"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "${OIDC_PROVIDER}:sub": "system:serviceaccount:develocity:gradle-database",
          "${OIDC_PROVIDER}:aud": "sts.amazonaws.com"
        },
        "StringEquals": {
          "${OIDC_PROVIDER}:sub": "system:serviceaccount:develocity:gradle-enterprise-app",
          "${OIDC_PROVIDER}:aud": "sts.amazonaws.com"
        },
        "StringEquals": {
          "${OIDC_PROVIDER}:sub": "system:serviceaccount:develocity:gradle-enterprise-app-background-processor",
          "${OIDC_PROVIDER}:aud": "sts.amazonaws.com"
        },
        "StringEquals": {
          "${OIDC_PROVIDER}:sub": "system:serviceaccount:develocity:gradle-keycloak",
          "${OIDC_PROVIDER}:aud": "sts.amazonaws.com"
        },
        "StringEquals": {
          "${OIDC_PROVIDER}:sub": "system:serviceaccount:develocity:gradle-test-distribution-broker",
          "${OIDC_PROVIDER}:aud": "sts.amazonaws.com"
        }
      }
    }
  ]
}
EOF

aws iam update-assume-role-policy \
  --role-name Develocity_Database_Role \
  --policy-document file://database-trust-relationship.json

4. Configure Develocity with RDS

The superuser credentials are only required to set up the database and create the migrator and application users. IAM authentication does not work with the superuser without the role rds_iam. For the sake of simplicity, we still use password authentication for the superuser in this tutorial. Consider setting up the database yourself, as described in the Database options section of Develocity’s installation manual, to avoid superuser usage. For this option to work, you must follow the instructions above to enable and configure IAM database authentication for the migrator and application users.

Add the following configuration snippet to your Helm values file:

values.yaml

database:
  location: user-managed
  provider: aws-rds
  connection:
    host: «database-hostname»
    databaseName: gradle_enterprise
  credentials:
    type: irsa
    irsa:
      serviceAccountAnnotations:
        "eks.amazonaws.com/role-arn": "arn:aws:iam::«account-id»:role/Develocity_Database_Role"
    superuser:
      username: «db-root-username» (1)
      password: «db-root-password» (1)

1	The RDS root credentials you chose earlier.

If you have any questions or need any assistance don’t hesitate to get in touch with the Develocity support team or your customer success representative.