EKS Pod Identity Authentication for Amazon RDS and S3
This documentation provides step-by-step instructions for configuring EKS Pod Identity authentication for Amazon RDS, S3, or both. Each section covers the required setup and configuration details for the respective service, select the section that matches your use case to get started:
Using Amazon RDS
This section will walk you through using an Amazon RDS PostgreSQL instance as your database.
1. Obtain the Required Permissions
You need permission to create and manage Amazon RDS instances and security groups.
The necessary permissions are granted using the AmazonRDSFullAccess AWS managed policy.
2. Set up an RDS Instance
Develocity is compatible with PostgreSQL versions 14 through 17. The minimum storage space required is 250 GB with 3,000 or more IOPS.
A. Create a root username and password
Create a root username and password for the database instance, referred to below as «db-root-username» and «db-root-password», respectively.
These are the credentials you will use for your database setup; save them somewhere secure.
B. Create a Security Group
Before creating the database, you have to create a security group in the VPC you want to use.
In this tutorial you will use the eksctl created VPC used by your cluster.
You can use a different VPC, but you will need to make the RDS instance accessible from your cluster (for example, by peering the VPCs).
To create the Security Group, run:
CLUSTER_VPC_ID=$(
aws ec2 describe-vpcs \
--filters Name=tag:aws:cloudformation:stack-name,Values=eksctl-develocity-cluster \
--query 'Vpcs[0].VpcId' \
--output text
)
aws ec2 create-security-group --group-name develocity-database \
--description "Develocity DB security group" \
--vpc-id ${CLUSTER_VPC_ID}
C. Enable Ingress
Then enable ingress to the RDS instance from your cluster for port 5432 by running:
CLUSTER_SECURITY_GROUP_ID=$(
aws eks describe-cluster --name develocity \
--query cluster.resourcesVpcConfig.clusterSecurityGroupId \
--output text
)
RDS_SECURITY_GROUP_ID=$(
aws ec2 describe-security-groups \
--filters Name=group-name,Values=develocity-database \
--query 'SecurityGroups[0].GroupId' \
--output text
)
aws ec2 authorize-security-group-ingress \
--protocol tcp --port 5432 \
--source-group ${CLUSTER_SECURITY_GROUP_ID} \
--group-id ${RDS_SECURITY_GROUP_ID}
D. Create a subnet group
Before creating the database, you need to create a subnet group to specify how the RDS instance will be networked.
This subnet group must have subnets in two availability zones, and typically should use private subnets.
eksctl has already created private subnets you can use.
Create a subnet group containing them by running:
CLUSTER_VPC_ID=$(
aws ec2 describe-vpcs \
--filters Name=tag:aws:cloudformation:stack-name,Values=eksctl-develocity-cluster \
--query 'Vpcs[0].VpcId' \
--output text
)
SUBNET_IDS=$(
aws ec2 describe-subnets \
--query 'Subnets[?!MapPublicIpOnLaunch].SubnetId' \
--filters Name=vpc-id,Values=${CLUSTER_VPC_ID} \
--output json
)
aws rds create-db-subnet-group --db-subnet-group-name develocity-database \
--db-subnet-group-description "Develocity DB subnet group" \
--subnet-ids ${SUBNET_IDS}
| Consult RDS’s subnet group documentation for more details on subnet groups and their requirements. |
E. Create the RDS instance
Create the RDS instance:
RDS_SECURITY_GROUP_ID=$(
aws ec2 describe-security-groups \
--filters Name=group-name,Values=develocity-database \
--query 'SecurityGroups[0].GroupId' \
--output text
)
RDS_POSTGRES_VERSION=$(
aws rds describe-db-engine-versions \
--engine postgres \
--engine-version 17 \(1)
--default-only \
--query 'DBEngineVersions[0].EngineVersion' \
--output text
)
| 1 | The latest major version of PostgreSQL that Develocity supports. |
aws rds create-db-instance \
--engine postgres \
--engine-version ${RDS_POSTGRES_VERSION} \
--db-instance-identifier develocity-database \
--db-name gradle_enterprise \
--allocated-storage 250 \(1)
--iops 3000 \(2)
--db-instance-class db.m5.large \
--db-subnet-group-name develocity-database \
--backup-retention-period 3 \(3)
--no-publicly-accessible \
--vpc-security-group-ids ${RDS_SECURITY_GROUP_ID} \
--master-username «db-root-username» \
--master-user-password «db-root-password»
| 1 | Develocity should be installed with 250GB of database storage to start with. |
| 2 | Develocity’s data volumes and database should support at least 3,000 IOPS. |
| 3 | The backup retention period, in days. |
While you don’t configure it here, RDS supports storage autoscaling.
| Consult AWS’s database creation guide and the CLI command reference for more details on RDS instance creation. |
You can view the status of your instance with:
aws rds describe-db-instances --db-instance-identifier develocity-database
Wait until the DBInstanceStatus is available.
Once available, you can see the hostname of the instance under Endpoint > Address.
This is the hostname you will use to connect to the instance, subsequently referred to as «database-hostname».
3. Configure the RDS instance for IAM authentication
Develocity supports connecting to the database using IAM authentication. This step describes how to configure your RDS instance to allow IAM database authentication for all the database users that Develocity will connect as (including the superuser, although it is possible for Develocity to not require superuser access, as explained in the Database setup with IAM database authentication section of the Kubernetes Helm chart Configuration Guide).
A. Enable IAM database authentication on the RDS instance
To modify your created RDS database instance to allow IAM database authentication, run:
aws rds modify-db-instance \
--db-instance-identifier develocity-database \
--apply-immediately \
--enable-iam-database-authentication
B. Create a policy allowing database access
To create a role that is permitted to connect to the RDS database as the required database users, using IAM authentication, create a database-policy.json file with the following content:
CONFIGURED_REGION=$(aws configure list | grep region | awk '{print $2}')
ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
DBI_RESOURCE_ID=$(
aws rds describe-db-instances \
--db-instance-identifier develocity-database \
--query 'DBInstances[0].DbiResourceId' \
--output text
)
cat <<EOF > database-policy.json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"rds-db:connect"
],
"Resource": [
"arn:aws:rds-db:${CONFIGURED_REGION}:${ACCOUNT_ID}:dbuser:${DBI_RESOURCE_ID}/ge_app",
"arn:aws:rds-db:${CONFIGURED_REGION}:${ACCOUNT_ID}:dbuser:${DBI_RESOURCE_ID}/ge_migrator",
"arn:aws:rds-db:${CONFIGURED_REGION}:${ACCOUNT_ID}:dbuser:${DBI_RESOURCE_ID}/ge_monitor",
"arn:aws:rds-db:${CONFIGURED_REGION}:${ACCOUNT_ID}:dbuser:${DBI_RESOURCE_ID}/«db-root-username»" (1)
]
}
]
}
EOF
| 1 | Replace «db-root-username» with the username you chose when creating your RDS instance’s root credentials. |
Then run the following command:
aws iam create-policy \
--policy-name "develocity-rds-access" \
--policy-document file://database-policy.json
C. Create a role for EKS
To allow Develocity to connect to RDS using the policy you just created, you need to create an IAM role that has attached to the policy you just created. Kubernetes service accounts in EKS need to be able to assume this role.
To create an IAM role that can be assumed by Kubernetes service accounts in EKS using the policy you just created, run the following commands:
ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
POLICY_ARN="arn:aws:iam::${ACCOUNT_ID}:policy/develocity-rds-access"
eksctl create iamserviceaccount \
--name develocity-database-account \
--namespace develocity \
--cluster develocity \
--approve \
--role-only \
--role-name Develocity_Database_Role \
--attach-policy-arn ${POLICY_ARN}
Change the trust relationship policy to allow the service accounts to assume the role:
cat <<EOF > pod-identity-trust-relationship.json
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowEksAuthToAssumeRoleForPodIdentity",
"Effect": "Allow",
"Principal": {
"Service": "pods.eks.amazonaws.com"
},
"Action": [
"sts:AssumeRole",
"sts:TagSession"
]
}
]
}
EOF
aws iam update-assume-role-policy \
--role-name Develocity_Database_Role \
--policy-document file://pod-identity-trust-relationship.json
4. Configure Develocity with RDS
The superuser credentials are only required to set up the database and create the migrator and application users. IAM authentication does not work with the superuser without the role rds_iam. For the sake of simplicity, we still use password authentication for the superuser in this tutorial. Consider setting up the database yourself, as described in the Database options section of Develocity’s installation manual, to avoid superuser usage.
For this option to work, you must follow the instructions above to enable and configure IAM database authentication for the migrator and application users.
|
First, you need to add the Pod Identity add-on to your cluster.
eksctl create addon --cluster develocity --name eks-pod-identity-agent
Then you need to associate the service accounts with the IAM role you created earlier.
for KUBERNETES_SERVICE_ACCOUNT in 'gradle-enterprise-app' 'gradle-enterprise-app-background-processor' 'gradle-database' 'gradle-test-distribution-broker'
do
eksctl create podidentityassociation \
--cluster develocity \
--namespace develocity \
--service-account-name ${KUBERNETES_SERVICE_ACCOUNT} \
--role-arn="arn:aws:iam::${ACCOUNT_ID}:role/Develocity_Database_Role"
done
Add the following configuration snippet to your Helm values file:
database:
location: user-managed
provider: aws-rds
connection:
host: «database-hostname»
databaseName: gradle_enterprise
credentials:
type: podIdentity
superuser:
username: «db-root-username» (1)
password: «db-root-password» (1)
| 1 | The RDS root credentials you chose earlier. |
Using Amazon S3
Develocity can use a user-managed Object Storage instead of its own embedded version. This has several benefits, from scalable storage to reduced operation burden and better backup and failover management. This appendix will walk you through using an Amazon S3 buckets to store Build Scan® data and monitoring data such as metrics.
1. Obtain the required permissions
You will need permission to create and manage Amazon S3 buckets.
You also need to create IAM policies and roles, but you already have permission to do that from the eksctl policies.
The necessary permissions can be granted by using the AmazonS3FullAccess AWS managed policy.
2. Set up S3 Buckets
To create the S3 buckets, run:
ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text) \
aws s3 mb s3://develocity-application-data-${ACCOUNT_ID} && \(1)
aws s3 mb s3://develocity-monitoring-data-${ACCOUNT_ID} (2)
| 1 | The name of the bucket meant to store application data, like Build Scan data or Build Cache entries |
| 2 | The name of the bucket meant to store monitoring data, like metrics collected during application lifetime |
|
Storing data in different buckets allows you to apply various strategies, such as access control, replication, soft-delete, backup, and more. However, you can use only one bucket for both application and monitoring data; this is an operation’s decision based on your practices. |
3. Set up roles for Elastic Kubernetes Service
To create a role allowing access to the application data bucket, execute the following commands in the console:
ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text) \
cat <<EOF > application-data-bucket-policy.json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::develocity-application-data-${ACCOUNT_ID}"
]
},
{
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:DeleteObject",
"s3:AbortMultipartUpload"
],
"Resource": [
"arn:aws:s3:::develocity-application-data-${ACCOUNT_ID}/*"
]
}
]
}
EOF
aws iam create-policy \
--policy-name "develocity-s3-application-data-access" \
--policy-document file://application-data-bucket-policy.json
To create a role allowing access to the monitoring data bucket, execute the following commands in the console:
ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text) \
cat <<EOF > monitoring-data-bucket-policy.json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::develocity-monitoring-data-${ACCOUNT_ID}"
]
},
{
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:DeleteObject",
"s3:AbortMultipartUpload"
],
"Resource": [
"arn:aws:s3:::develocity-monitoring-data-${ACCOUNT_ID}/*"
]
}
]
}
EOF
aws iam create-policy \
--policy-name "develocity-s3-monitoring-data-access" \
--policy-document file://monitoring-data-bucket-policy.json
4. Attach Roles to Elastic Kubernetes Service
To allow Develocity to access the S3 buckets using the policies you just created, you need to create IAM roles and attach those policies to them. Then, you need to allow Kubernetes service accounts in your Develocity instance’s EKS cluster’s namespace to assume these roles.
To create IAM roles that can be assumed by Kubernetes service accounts in EKS using the policies you just created, run the following commands:
ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
Create the IAM role for the monitoring data access:
eksctl create iamserviceaccount \
--name develocity-monitoring-account \
--namespace develocity \
--cluster develocity \
--approve \
--role-only \
--role-name Develocity_Monitoring_Role \
--attach-policy-arn arn:aws:iam::${ACCOUNT_ID}:policy/develocity-s3-monitoring-data-access
Create the IAM role for the application data access:
eksctl create iamserviceaccount \
--name develocity-application-account \
--namespace develocity \
--cluster develocity \
--approve \
--role-only \
--role-name Develocity_Application_Role \
--attach-policy-arn arn:aws:iam::${ACCOUNT_ID}:policy/develocity-s3-application-data-access
Change the trust relationship policy to allow the service accounts to assume the roles:
cat <<EOF > pod-identity-trust-relationship.json
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowEksAuthToAssumeRoleForPodIdentity",
"Effect": "Allow",
"Principal": {
"Service": "pods.eks.amazonaws.com"
},
"Action": [
"sts:AssumeRole",
"sts:TagSession"
]
}
]
}
EOF
aws iam update-assume-role-policy \
--role-name Develocity_Monitoring_Role \
--policy-document file://pod-identity-trust-relationship.json
aws iam update-assume-role-policy \
--role-name Develocity_Application_Role \
--policy-document file://pod-identity-trust-relationship.json
5. Update your installation’s resource requirements
When using S3 Build Scan storage, Develocity uses more memory.
To update Develocity’s memory usage specification, update your Helm values file with the following values:
enterprise:
resources:
requests:
memory: 6Gi (1)
limits:
memory: 6Gi (1)
| 1 | If you have already set a custom value here, instead increase it by 2Gi. |
When adding items to your Helm values file, merge any duplicate blocks. Alternatively, you can use separate files and pass all of them with --values «file» when running Helm commands.
|
If you are additionally using the background processor component, you should also update its values:
enterpriseBackgroundProcessor:
resources:
requests:
memory: 6Gi (1)
limits:
memory: 6Gi (1)
| 1 | If you have already set a custom value here, instead increase it by 2Gi. |
6. Configure Develocity with S3
Develocity must now be configured to use S3. To do this, you must use the unattended configuration mechanism.
Develocity can store Build Scan data in either the configured database or in the configured object store.
The unattended configuration mechanism lets you configure which of these is used to store Build Scan data as part of a configuration file, which can be embedded in your Helm values file as described in the unattended configuration guide.
This section will describe how to extend your Helm values file to include the correct unattended configuration block for S3 Build Scan storage.
First, we need to create a minimal unattended configuration file. This requires you to choose a password for the system user and hash it. To do this, install Develocityctl.
The following command will prompt you to enter a password and write the hash into the secret.txt.
We will refer to the hashed password as «hashed-system-password»
develocityctl config-file hash -o secret.txt
First, you need to add the Pod Identity add-on to your cluster.
eksctl create addon --cluster develocity --name eks-pod-identity-agent
Then you need to associate the service accounts with the Application Role you created earlier.
for KUBERNETES_SERVICE_ACCOUNT in 'gradle-enterprise-app' 'gradle-enterprise-app-background-processor'
do
eksctl create podidentityassociation \
--cluster develocity \
--namespace develocity \
--service-account-name ${KUBERNETES_SERVICE_ACCOUNT} \
--role-arn="arn:aws:iam::${ACCOUNT_ID}:role/Develocity_Application_Role"
done
And you need to associate the service accounts with the Monitoring Role you created earlier.
for KUBERNETES_SERVICE_ACCOUNT in 'gradle-enterprise-operator' 'gradle-monitoring'
do
eksctl create podidentityassociation \
--cluster develocity \
--namespace develocity \
--service-account-name ${KUBERNETES_SERVICE_ACCOUNT} \
--role-arn="arn:aws:iam::${ACCOUNT_ID}:role/Develocity_Monitoring_Role"
done
To use your S3 bucket, add the following to your Helm values file:
global:
unattended:
configuration:
version: 12 (1)
systemPassword: "«hashed-system-password»" (2)
buildScans:
incomingStorageType: objectStorage
enterprise:
resources:
requests:
memory: 8Gi (3)
limits:
memory: 8Gi (3)
objectStorage:
type: s3
s3:
bucket: develocity-application-data-«account-id» (4)
region: «region» (5)
credentials:
type: podIdentity (6)
monitoring:
bucket: develocity-monitoring-data-«account-id» (4)
region: «region» (5)
credentials:
type: podIdentity
| 1 | The version of the unattended configuration. |
| 2 | Your hashed system password. |
| 3 | If you have already set a custom value here, instead increase it by 2Gi. |
| 4 | «account-id» is the ID of your AWS account. |
| 5 | The region where your S3 bucket resides, which should be your current region. |
Once you have updated your Helm values file as described above, you need to reapply it using the method described in Changing Configuration Values. This will update your Develocity installation to use the unattended configuration you created above, and Develocity will restart.
| Switching between embedded Object Storage and user-managed Object Storage is not supported, starting with S3 is recommended as a more scalable solution, as data migration will not be possible later on. |
7. Verify S3 Storage is Used
| Develocity will start even if your S3 configuration is incorrect. |
To confirm that Develocity is storing incoming Build Scans in S3 and also able to read Build Scan data from S3, you should first upload a new Build Scan to your Develocity instance. Second, confirm that you can view the Build Scan. Finally, confirm that the Build Scan is stored in your S3 bucket, by running:
ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
aws s3 ls s3://develocity-application-data-${ACCOUNT_ID}/build-scans/ \(1)
--recursive --human-readable --summarize
| 1 | If you used a custom prefix, use it here instead of build-scans. |
2025-05-27 19:11:06 6.6 KiB build-scans/2025/05/27/aprvi3bnnxyzm Total Objects: 1 Total Size: 6.6 KiB
Using Amazon RDS and S3
When using both Amazon RDS and S3 with Develocity, you should follow the principle of least privilege by assigning dedicated IAM roles to each component based on its specific access requirements.
-
The
gradle-database,test-distribution-broker, andgradle-keycloakcomponents require access only to Amazon RDS. -
The
gradle-monitoringandgradle-enterprise-operatorcomponents require access only to the S3 monitoring data bucket. -
The
gradle-enterprise-appandgradle-enterprise-app-background-processorcomponents require access to both Amazon RDS and the S3 application data bucket.
This page provides step-by-step instructions for creating and assigning these fine-grained IAM roles, ensuring each Develocity component has only the permissions it needs to interact securely with AWS services.
Using Amazon RDS
This section will walk you through using an Amazon RDS PostgreSQL instance as your database.
1. Obtain the Required Permissions
You need permission to create and manage Amazon RDS instances and security groups.
The necessary permissions are granted using the AmazonRDSFullAccess AWS managed policy.
2. Set up an RDS Instance
Develocity is compatible with PostgreSQL versions 14 through 17. The minimum storage space required is 250 GB with 3,000 or more IOPS.
A. Create a root username and password
Create a root username and password for the database instance, referred to below as «db-root-username» and «db-root-password», respectively.
These are the credentials you will use for your database setup; save them somewhere secure.
B. Create a Security Group
Before creating the database, you have to create a security group in the VPC you want to use.
In this tutorial you will use the eksctl created VPC used by your cluster.
You can use a different VPC, but you will need to make the RDS instance accessible from your cluster (for example, by peering the VPCs).
To create the Security Group, run:
CLUSTER_VPC_ID=$(
aws ec2 describe-vpcs \
--filters Name=tag:aws:cloudformation:stack-name,Values=eksctl-develocity-cluster \
--query 'Vpcs[0].VpcId' \
--output text
)
aws ec2 create-security-group --group-name develocity-database \
--description "Develocity DB security group" \
--vpc-id ${CLUSTER_VPC_ID}
C. Enable Ingress
Then enable ingress to the RDS instance from your cluster for port 5432 by running:
CLUSTER_SECURITY_GROUP_ID=$(
aws eks describe-cluster --name develocity \
--query cluster.resourcesVpcConfig.clusterSecurityGroupId \
--output text
)
RDS_SECURITY_GROUP_ID=$(
aws ec2 describe-security-groups \
--filters Name=group-name,Values=develocity-database \
--query 'SecurityGroups[0].GroupId' \
--output text
)
aws ec2 authorize-security-group-ingress \
--protocol tcp --port 5432 \
--source-group ${CLUSTER_SECURITY_GROUP_ID} \
--group-id ${RDS_SECURITY_GROUP_ID}
D. Create a subnet group
Before creating the database, you need to create a subnet group to specify how the RDS instance will be networked.
This subnet group must have subnets in two availability zones, and typically should use private subnets.
eksctl has already created private subnets you can use.
Create a subnet group containing them by running:
CLUSTER_VPC_ID=$(
aws ec2 describe-vpcs \
--filters Name=tag:aws:cloudformation:stack-name,Values=eksctl-develocity-cluster \
--query 'Vpcs[0].VpcId' \
--output text
)
SUBNET_IDS=$(
aws ec2 describe-subnets \
--query 'Subnets[?!MapPublicIpOnLaunch].SubnetId' \
--filters Name=vpc-id,Values=${CLUSTER_VPC_ID} \
--output json
)
aws rds create-db-subnet-group --db-subnet-group-name develocity-database \
--db-subnet-group-description "Develocity DB subnet group" \
--subnet-ids ${SUBNET_IDS}
| Consult RDS’s subnet group documentation for more details on subnet groups and their requirements. |
E. Create the RDS instance
Create the RDS instance:
RDS_SECURITY_GROUP_ID=$(
aws ec2 describe-security-groups \
--filters Name=group-name,Values=develocity-database \
--query 'SecurityGroups[0].GroupId' \
--output text
)
RDS_POSTGRES_VERSION=$(
aws rds describe-db-engine-versions \
--engine postgres \
--engine-version 17 \(1)
--default-only \
--query 'DBEngineVersions[0].EngineVersion' \
--output text
)
| 1 | The latest major version of PostgreSQL that Develocity supports. |
aws rds create-db-instance \
--engine postgres \
--engine-version ${RDS_POSTGRES_VERSION} \
--db-instance-identifier develocity-database \
--db-name gradle_enterprise \
--allocated-storage 250 \(1)
--iops 3000 \(2)
--db-instance-class db.m5.large \
--db-subnet-group-name develocity-database \
--backup-retention-period 3 \(3)
--no-publicly-accessible \
--vpc-security-group-ids ${RDS_SECURITY_GROUP_ID} \
--master-username «db-root-username» \
--master-user-password «db-root-password»
| 1 | Develocity should be installed with 250GB of database storage to start with. |
| 2 | Develocity’s data volumes and database should support at least 3,000 IOPS. |
| 3 | The backup retention period, in days. |
While you don’t configure it here, RDS supports storage autoscaling.
| Consult AWS’s database creation guide and the CLI command reference for more details on RDS instance creation. |
You can view the status of your instance with:
aws rds describe-db-instances --db-instance-identifier develocity-database
Wait until the DBInstanceStatus is available.
Once available, you can see the hostname of the instance under Endpoint > Address.
This is the hostname you will use to connect to the instance, subsequently referred to as «database-hostname».
3. Configure the RDS instance for IAM authentication
Develocity supports connecting to the database using IAM authentication. This step describes how to configure your RDS instance to allow IAM database authentication for all the database users that Develocity will connect as (including the superuser, although it is possible for Develocity to not require superuser access, as explained in the Database setup with IAM database authentication section of the Kubernetes Helm chart Configuration Guide).
A. Enable IAM database authentication on the RDS instance
To modify your created RDS database instance to allow IAM database authentication, run:
aws rds modify-db-instance \
--db-instance-identifier develocity-database \
--apply-immediately \
--enable-iam-database-authentication
Using Amazon S3
Develocity can use a user-managed Object Storage instead of its own embedded version. This has several benefits, from scalable storage to reduced operation burden and better backup and failover management. This appendix will walk you through using an Amazon S3 buckets to store Build Scan® data and monitoring data such as metrics.
1. Obtain the required permissions
You will need permission to create and manage Amazon S3 buckets.
You also need to create IAM policies and roles, but you already have permission to do that from the eksctl policies.
The necessary permissions can be granted by using the AmazonS3FullAccess AWS managed policy.
2. Set up S3 Buckets
To create the S3 buckets, run:
ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text) \
aws s3 mb s3://develocity-application-data-${ACCOUNT_ID} && \(1)
aws s3 mb s3://develocity-monitoring-data-${ACCOUNT_ID} (2)
| 1 | The name of the bucket meant to store application data, like Build Scan data or Build Cache entries |
| 2 | The name of the bucket meant to store monitoring data, like metrics collected during application lifetime |
|
Storing data in different buckets allows you to apply various strategies, such as access control, replication, soft-delete, backup, and more. However, you can use only one bucket for both application and monitoring data; this is an operation’s decision based on your practices. |
Create IAM Policies and Roles
1. Create policies allowing RDS and S3 access
A. Create a policy allowing RDS access
To create a role that is permitted to connect to the RDS database as the required database users, using IAM authentication, create a database-policy.json file with the following content:
CONFIGURED_REGION=$(aws configure list | grep region | awk '{print $2}')
ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
DBI_RESOURCE_ID=$(
aws rds describe-db-instances \
--db-instance-identifier develocity-database \
--query 'DBInstances[0].DbiResourceId' \
--output text
)
cat <<EOF > database-policy.json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"rds-db:connect"
],
"Resource": [
"arn:aws:rds-db:${CONFIGURED_REGION}:${ACCOUNT_ID}:dbuser:${DBI_RESOURCE_ID}/ge_app",
"arn:aws:rds-db:${CONFIGURED_REGION}:${ACCOUNT_ID}:dbuser:${DBI_RESOURCE_ID}/ge_migrator",
"arn:aws:rds-db:${CONFIGURED_REGION}:${ACCOUNT_ID}:dbuser:${DBI_RESOURCE_ID}/ge_monitor",
"arn:aws:rds-db:${CONFIGURED_REGION}:${ACCOUNT_ID}:dbuser:${DBI_RESOURCE_ID}/«db-root-username»" (1)
]
}
]
}
EOF
| 1 | Replace «db-root-username» with the username you chose when creating your RDS instance’s root credentials. |
Then run the following command:
aws iam create-policy \
--policy-name "develocity-rds-access" \
--policy-document file://database-policy.json
B. Create a policy allowing application data bucket access
To create a role allowing access to the application data bucket, execute the following commands in the console:
ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text) \
cat <<EOF > application-data-bucket-policy.json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::develocity-application-data-${ACCOUNT_ID}"
]
},
{
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:DeleteObject",
"s3:AbortMultipartUpload"
],
"Resource": [
"arn:aws:s3:::develocity-application-data-${ACCOUNT_ID}/*"
]
}
]
}
EOF
aws iam create-policy \
--policy-name "develocity-s3-application-data-access" \
--policy-document file://application-data-bucket-policy.json
C. Create a policy allowing monitoring data bucket access
To create a role allowing access to the monitoring data bucket, execute the following commands in the console:
ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text) \
cat <<EOF > monitoring-data-bucket-policy.json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::develocity-monitoring-data-${ACCOUNT_ID}"
]
},
{
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:DeleteObject",
"s3:AbortMultipartUpload"
],
"Resource": [
"arn:aws:s3:::develocity-monitoring-data-${ACCOUNT_ID}/*"
]
}
]
}
EOF
aws iam create-policy \
--policy-name "develocity-s3-monitoring-data-access" \
--policy-document file://monitoring-data-bucket-policy.json
2. Create Roles for Develocity components
We create 3 IAM roles for the Develocity components:
-
Develocity_Database_Rolefor the components that require access only to Amazon RDS. -
Develocity_Monitoring_Rolefor the components that require access only to the S3 monitoring data bucket. -
Develocity_Application_Rolefor the components that require access to both Amazon RDS and the S3 application data bucket.
To create the IAM roles, run the following commands:
Create the IAM role for the database access:
eksctl create iamserviceaccount \
--name develocity-database-account \
--namespace develocity \
--cluster develocity \
--approve \
--role-only \
--role-name Develocity_Database_Role \
--attach-policy-arn arn:aws:iam::${ACCOUNT_ID}:policy/develocity-rds-access
Create the IAM role for the monitoring data access:
eksctl create iamserviceaccount \
--name develocity-monitoring-account \
--namespace develocity \
--cluster develocity \
--approve \
--role-only \
--role-name Develocity_Monitoring_Role \
--attach-policy-arn arn:aws:iam::${ACCOUNT_ID}:policy/develocity-s3-monitoring-data-access
Create the IAM role for the application data and database access:
eksctl create iamserviceaccount \
--name develocity-application-account \
--namespace develocity \
--cluster develocity \
--approve \
--role-only \
--role-name Develocity_Application_Role \
--attach-policy-arn arn:aws:iam::${ACCOUNT_ID}:policy/develocity-rds-access
--attach-policy-arn arn:aws:iam::${ACCOUNT_ID}:policy/develocity-s3-application-data-access
3. Update the trust relationship policies
Update the trust relationship policy for the roles you created above to allow the pods to assume the roles.
cat <<EOF > trust-relationship.json
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowEksAuthToAssumeRoleForPodIdentity",
"Effect": "Allow",
"Principal": {
"Service": "pods.eks.amazonaws.com"
},
"Action": [
"sts:AssumeRole",
"sts:TagSession"
]
}
]
}
EOF
aws iam update-assume-role-policy \
--role-name Develocity_Database_Role \
--policy-document file://trust-relationship.json
aws iam update-assume-role-policy \
--role-name Develocity_Monitoring_Role \
--policy-document file://trust-relationship.json
aws iam update-assume-role-policy \
--role-name Develocity_Application_Role \
--policy-document file://trust-relationship.json
4. Configure Develocity to use RDS and S3
A. Update your installation’s resource requirements
When using S3 Build Scan storage, Develocity uses more memory.
To update Develocity’s memory usage specification, update your Helm values file with the following values:
enterprise:
resources:
requests:
memory: 6Gi (1)
limits:
memory: 6Gi (1)
| 1 | If you have already set a custom value here, instead increase it by 2Gi. |
When adding items to your Helm values file, merge any duplicate blocks. Alternatively, you can use separate files and pass all of them with --values «file» when running Helm commands.
|
If you are additionally using the background processor component, you should also update its values:
enterpriseBackgroundProcessor:
resources:
requests:
memory: 6Gi (1)
limits:
memory: 6Gi (1)
| 1 | If you have already set a custom value here, instead increase it by 2Gi. |
B. Associate Service Accounts with IAM Roles
First, you need to add the Pod Identity add-on to your cluster.
eksctl create addon --cluster develocity --name eks-pod-identity-agent
Then you need to associate the service accounts with the IAM roles you created earlier.
components=(
"gradle-enterprise-app:Develocity_Application_Role"
"gradle-enterprise-app-background-processor:Develocity_Application_Role"
"gradle-database:Develocity_Database_Role"
"gradle-test-distribution-broker:Develocity_Database_Role"
"gradle-keycloak:Develocity_Database_Role"
"gradle-monitoring:Develocity_Monitoring_Role"
"gradle-enterprise-operator:Develocity_Monitoring_Role"
)
for entry in "${components[@]}"; do
KUBERNETES_SERVICE_ACCOUNT="${entry%%:*}"
IAM_ROLE="${entry##*:}"
eksctl create podidentityassociation \
--cluster develocity \
--namespace develocity \
--service-account-name ${KUBERNETES_SERVICE_ACCOUNT} \
--role-arn="arn:aws:iam::${ACCOUNT_ID}:role/${IAM_ROLE}"
done
C. Create Unattended Configuration
Develocity must now be configured to use S3. To do this, you must use the unattended configuration mechanism.
Develocity can store Build Scan data in either the configured database or in the configured object store.
The unattended configuration mechanism lets you configure which of these is used to store Build Scan data as part of a configuration file, which can be embedded in your Helm values file as described in the unattended configuration guide.
This section will describe how to extend your Helm values file to include the correct unattended configuration block for S3 Build Scan storage.
First, we need to create a minimal unattended configuration file. This requires you to choose a password for the system user and hash it. To do this, install Develocityctl.
The following command will prompt you to enter a password and write the hash into the secret.txt.
We will refer to the hashed password as «hashed-system-password»
develocityctl config-file hash -o secret.txt
D. Update your Helm values file
The superuser credentials are only required to set up the database and create the migrator and application users.
IAM authentication does not work with the superuser without the role rds_iam.
For the sake of simplicity, we still use password authentication for the superuser in this tutorial.
Consider setting up the database yourself, as described in the Database options section of Develocity’s installation manual, to avoid superuser usage.
For this option to work, you must follow the instructions above to enable and configure IAM database authentication for the migrator and application users.
|
global:
unattended:
configuration:
version: 12 (1)
systemPassword: "«hashed-system-password»" (2)
buildScans:
incomingStorageType: objectStorage
enterprise:
resources:
requests:
memory: 8Gi (3)
limits:
memory: 8Gi (3)
database:
location: user-managed
provider: aws-rds
connection:
host: «database-hostname»
databaseName: gradle_enterprise
credentials:
type: podIdentity
superuser:
username: «db-root-username» (4)
password: «db-root-password» (4)
objectStorage:
type: s3
s3:
bucket: develocity-application-data-«account-id» (5)
region: «region» (5)
credentials:
type: podIdentity (7)
monitoring:
bucket: develocity-monitoring-data-«account-id» (6)
region: «region» (6)
credentials:
type: podIdentity
| 1 | The version of the unattended configuration. |
| 2 | Your hashed system password. |
| 3 | If you have already set a custom value here, instead increase it by 2Gi. |
| 4 | The RDS root credentials you chose earlier. |
| 5 | The name and region of the S3 bucket you created for application data. |
| 6 | The name and region of the S3 bucket you created for monitoring data. |
Once you have updated your Helm values file as described above, you need to reapply it using the method described in Changing Configuration Values. This will update your Develocity installation to use the unattended configuration you created above, and Develocity will restart.
| Switching between embedded Object Storage and user-managed Object Storage is not supported, starting with S3 is recommended as a more scalable solution, as data migration will not be possible later on. |
D. Verify S3 Storage is Used
| Develocity will start even if your S3 configuration is incorrect. |
To confirm that Develocity is storing incoming Build Scans in S3 and also able to read Build Scan data from S3, you should first upload a new Build Scan to your Develocity instance. Second, confirm that you can view the Build Scan. Finally, confirm that the Build Scan is stored in your S3 bucket, by running:
ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
aws s3 ls s3://develocity-application-data-${ACCOUNT_ID}/build-scans/ \(1)
--recursive --human-readable --summarize
| 1 | If you used a custom prefix, use it here instead of build-scans. |
2025-05-27 19:11:06 6.6 KiB build-scans/2025/05/27/aprvi3bnnxyzm Total Objects: 1 Total Size: 6.6 KiB
If you have any questions or need any assistance don’t hesitate to get in touch with the Develocity support team or your customer success representative.