<!-- llms-index: https://docs.gradle.com/develocity/llms.txt -->

<a id="component-eol-banner"></a>

You are viewing **Develocity Documentation 2025.4**. To view the latest available version of the docs, see [2026.1](https://docs.gradle.com/develocity/2026.1/installation/aws/irsa-s3/).

# Using Amazon S3

<a id="preamble"></a>

Develocity can use a user-managed Object Storage instead of its own embedded version. This has several benefits, from scalable storage to reduced operation burden and better backup and failover management. This appendix will walk you through using an [Amazon S3](https://docs.aws.amazon.com/AmazonS3/latest/userguide/Welcome.html) buckets to store Build Scan® data and monitoring data such as metrics.

<a id="irsa_s3_obtain_permissions"></a>

## Obtain the Required Permissions

You will need permission to create and manage Amazon S3 buckets. You also need to create IAM policies and roles, but you already have permission to do that from the `eksctl` policies.

The necessary permissions can be granted by using the `AmazonS3FullAccess` AWS managed policy.

<a id="irsa_s3_access"></a>

## Set Up S3 Buckets

To create the S3 buckets, run:

```shell
ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text) \
  aws s3 mb s3://develocity-application-data-${ACCOUNT_ID} && \(1)
  aws s3 mb s3://develocity-monitoring-data-${ACCOUNT_ID} (2)
```

1. The name of the bucket meant to store application data, like Build Scan data or Build Cache entries
2. The name of the bucket meant to store monitoring data, like metrics collected during application lifetime

> [!NOTE]
> Storing data in different buckets allows you to apply various strategies, such as access control, replication, soft-delete, backup, and more. However, you can use only one bucket for both application and monitoring data; this is an operation’s decision based on your practices.

<a id="set-up-a-role-for-elastic-kubernetes-service"></a>

## Set Up a Role for Elastic Kubernetes Service

To create a role allowing access to the application data bucket, execute the following commands in the shell:

**create-application-data-policy.sh:**

```
ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text) \
  cat <<EOF > application-data-bucket-policy.json
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::develocity-application-data-${ACCOUNT_ID}"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:DeleteObject",
                "s3:AbortMultipartUpload"
            ],
            "Resource": [
                "arn:aws:s3:::develocity-application-data-${ACCOUNT_ID}/*"
            ]
        }
    ]
}
EOF
aws iam create-policy \
    --policy-name "develocity-s3-application-data-access" \
    --policy-document file://application-data-bucket-policy.json
```

To create a role allowing access to the monitoring data bucket, execute the following commands in the shell:

**create-monitoring-data-policy.sh:**

```
ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text) \
  cat <<EOF > monitoring-data-bucket-policy.json
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::develocity-monitoring-data-${ACCOUNT_ID}"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:DeleteObject",
                "s3:AbortMultipartUpload"
            ],
            "Resource": [
                "arn:aws:s3:::develocity-monitoring-data-${ACCOUNT_ID}/*"
            ]
        }
    ]
}
EOF
aws iam create-policy \
    --policy-name "develocity-s3-monitoring-data-access" \
    --policy-document file://monitoring-data-bucket-policy.json
```

<a id="irsa_s3_create_eks_role"></a>

## Attach Roles to Elastic Kubernetes Service

To allow Develocity to access the S3 buckets using the policies you just created, you need to create IAM roles and attach those policies to them. Then, you need to allow Kubernetes service accounts in your Develocity instance’s EKS cluster’s namespace to assume these roles.

To associate the service account with an AWS IAM role, we need to use an AWS OIDC provider. We already installed one when setting up the [Storage Class EBS CSI driver](https://docs.gradle.com/develocity/2025.4/installation/aws/aws-eks-cluster/#storage_class), so we can use it here.

To create IAM roles that can be assumed by Kubernetes service accounts in EKS using the policies you just created, run the following commands:

```shell
ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
```

Create the IAM role for the monitoring data access:

```shell
eksctl create iamserviceaccount \
  --name develocity-monitoring-account \
  --namespace develocity \
  --cluster develocity \
  --approve \
  --role-only \
  --role-name Develocity_Monitoring_Role \
  --attach-policy-arn arn:aws:iam::${ACCOUNT_ID}:policy/develocity-s3-monitoring-data-access
```

Create the IAM role for the application data access:

```shell
eksctl create iamserviceaccount \
  --name develocity-application-account \
  --namespace develocity \
  --cluster develocity \
  --approve \
  --role-only \
  --role-name Develocity_Application_Role \
  --attach-policy-arn arn:aws:iam::${ACCOUNT_ID}:policy/develocity-s3-application-data-access
```

Change the trust relationship policies to allow the service accounts to assume the roles:

```shell
OIDC_PROVIDER=$(aws eks describe-cluster --name develocity \
  --query "cluster.identity.oidc.issuer" \
  --output text | sed -e "s/^https:\/\///")
```

```shell
cat <<EOF > application-data-trust-relationship.json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "arn:aws:iam::$ACCOUNT_ID:oidc-provider/$OIDC_PROVIDER"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "${OIDC_PROVIDER}:sub": "system:serviceaccount:develocity:gradle-enterprise-app",
          "${OIDC_PROVIDER}:aud": "sts.amazonaws.com"
        },
        "StringEquals": {
          "${OIDC_PROVIDER}:sub": "system:serviceaccount:develocity:gradle-enterprise-app-background-processor",
          "${OIDC_PROVIDER}:aud": "sts.amazonaws.com"
        }
      }
    }
  ]
}
EOF
```

```shell
cat <<EOF > monitoring-data-trust-relationship.json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "arn:aws:iam::$ACCOUNT_ID:oidc-provider/$OIDC_PROVIDER"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "${OIDC_PROVIDER}:sub": "system:serviceaccount:develocity:gradle-enterprise-operator",
          "${OIDC_PROVIDER}:aud": "sts.amazonaws.com"
        },
        "StringEquals": {
          "${OIDC_PROVIDER}:sub": "system:serviceaccount:develocity:gradle-monitoring",
          "${OIDC_PROVIDER}:aud": "sts.amazonaws.com"
        }
      }
    }
  ]
}
EOF
```

```shell
aws iam update-assume-role-policy \
  --role-name Develocity_Monitoring_Role \
  --policy-document file://monitoring-data-trust-relationship.json
```

```shell
aws iam update-assume-role-policy \
  --role-name Develocity_Application_Role \
  --policy-document file://application-data-trust-relationship.json
```

<a id="irsa_s3_helm_values_file"></a>

## Update Your Installation’s Resource Requirements

When using S3 Build Scan storage, Develocity uses more memory.

To update Develocity’s memory usage specification, update your Helm values file with the following values:

**values.yaml:**

```
enterprise:
  resources:
    requests:
      memory: 6Gi (1)
    limits:
      memory: 6Gi (1)
```

1. If you have already set a custom value here, instead increase it by 2Gi.

> [!IMPORTANT]
> When adding items to your Helm values file, merge any duplicate blocks. Alternatively, you can use separate files and pass all of them with `--values «file»` when running Helm commands.

> [!WARNING]
> You may need to scale up your cluster or use nodes with more memory to be able to satisfy the increased memory requirements. See the section [create cluster](https://docs.gradle.com/develocity/2025.4/installation/aws/aws-eks-cluster/#create_cluster) for scaling instructions.

If you are additionally using the background processor component, you should also update its values:

**values.yaml:**

```
enterpriseBackgroundProcessor:
  resources:
    requests:
      memory: 6Gi (1)
    limits:
      memory: 6Gi (1)
```

1. If you have already set a custom value here, instead increase it by 2Gi.

<a id="irsa_s3_build_scans"></a>

## Configure Develocity with S3

Develocity must now be configured to use S3. To do this, you must use the [unattended configuration mechanism](https://docs.gradle.com/develocity/2025.4/administration/admin-manual/#unattended_configuration).

Develocity can [store Build Scan data in either the configured database or in the configured object store](https://docs.gradle.com/develocity/2025.4/administration/admin-manual/#build_scan_storage).

The unattended configuration mechanism lets you configure which of these is used to store Build Scan data as part of a configuration file, which can be embedded in your Helm values file as described in the [unattended configuration guide](https://docs.gradle.com/develocity/2025.4/administration/admin-manual/#configuring_develocity_for_unattended_configuration).

This section will describe how to extend your Helm values file to include the correct unattended configuration block for S3 Build Scan storage.

First, we need to create a minimal unattended configuration file. This requires you to choose a password for the system user and hash it. To do this, install [Develocityctl](https://docs.gradle.com/develocity/develocityctl/1.22/).

The following command will prompt you to enter a password and write the hash into the `secret.txt`. We will refer to the hashed password as `«hashed-system-password»`

```shell
develocityctl config-file hash -o secret.txt
```

To use your S3 bucket, add the following to your Helm values file:

**values.yaml:**

```
global:
  unattended:
    configuration:
      version: 14 (1)
      systemPassword: "«hashed-system-password»" (2)
      buildScans:
        incomingStorageType: objectStorage

enterprise:
  resources:
    requests:
      memory: 8Gi (3)
    limits:
      memory: 8Gi (3)

objectStorage:
  type: s3
  s3:
    bucket: develocity-application-data-«account-id» (4)
    region: «region» (5)
    credentials:
      type: irsa (6)
      irsa:
        serviceAccountAnnotations:
          "eks.amazonaws.com/role-arn": "arn:aws:iam::«account-id»:role/Develocity_Application_Role" (4)
    monitoring:
      bucket: develocity-monitoring-data-«account-id» (4)
      region: «region» (5)
      credentials:
        type: irsa (6)
        irsa:
          serviceAccountAnnotations:
            "eks.amazonaws.com/role-arn": "arn:aws:iam::«account-id»:role/Develocity_Monitoring_Role" (4)
```

1. The version of the unattended configuration.
2. Your hashed system password.
3. If you have already set a custom value here, instead increase it by 2Gi.
4. «account-id» is the ID of your AWS account.
5. The region where your S3 bucket resides, which should be your current region.
6. The type of AWS credentials, in this example, the IAM Roles for Service Account, as described in IAM Roles for Service Account credentials configuration (IRSA).

Once you have updated your Helm values file as described above, you need to reapply it using the method described in [Changing Configuration Values](https://docs.gradle.com/develocity/2025.4/reference/kubernetes-chart/#changing_configuration_values). This will update your Develocity installation to use the unattended configuration you created above, and Develocity will restart.

> [!IMPORTANT]
> Switching between embedded Object Storage and user-managed Object Storage is not supported, starting with S3 is recommended as a more scalable solution, as data migration will not be possible later on.

<a id="irsa_s3_usage"></a>

## Verify S3 Storage Is Used

> [!WARNING]
> Develocity will start even if your S3 configuration is incorrect.

To confirm that Develocity is storing incoming Build Scans in S3 and also able to read Build Scan data from S3, you should first upload a new Build Scan to your Develocity instance. Second, confirm that you can view the Build Scan. Finally, confirm that the Build Scan is stored in your S3 bucket, by running:

```shell
ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
```

```shell
aws s3 ls s3://develocity-application-data-${ACCOUNT_ID}/build-scans/ \(1)
  --recursive --human-readable --summarize
```

1. If you used a custom prefix, use it here instead of build-scans.

**Output:**

```
2025-05-27 19:11:06    6.6 KiB build-scans/2025/05/27/aprvi3bnnxyzm

Total Objects: 1
   Total Size: 6.6 KiB
```