Develocity Bazel Configuration Guide | Develocity Documentation

Getting set up

To publish Build Scans to your Develocity server, you need to pass a few additional command line options to Bazel. The simplest way to do so is to add the following to the .bazelrc file in your build workspace.

common:develocity --remote_cache=grpcs://«develocity-server»
common:develocity --bes_results_url=https://«develocity-server»/build/
common:develocity --bes_backend=grpcs://«develocity-server»

build --config=develocity

Replacing «develocity-server» with the hostname of your server. If you use a non-standard port for HTTPS traffic, this must also include the port number. The last line makes Bazel publish a Build Scan for every build. See Controlling when Build Scans are published for more details.

While not strictly needed, consider adding the following options as well, which increase the level of detail you will see in your builds.

common:develocity --build_event_publish_all_actions=true
common:develocity --noslim_profile

Authenticating

Develocity installations may be configured to require Build Scan publishing to be authenticated. Additionally, installations may be configured to only allow certain users to publish Build Scans. If your instance doesn’t require authentication, you can skip to Configuring Build Scans.

Develocity access keys should be treated with the same secrecy as passwords. They are used to authorize access to Develocity from a build.

Log into your Develocity server, from the upper-right corner click your user icon, then "My Settings".
Ensure that your user has at least "Publish Build Scans" and either "Read and write build cache data" or "Read build cache data and write Bazel CAS data" permissions. See Cache Permissions.
Select "Access keys" on the left and then generate a new access key.
In a .bazelrc file in your user’s home directory, add the following:

common:develocity --remote_cache_header=Authorization="Bearer «the access key you just generated»"
common:develocity --bes_header=Authorization="Bearer «the access key you just generated»"

If you have an existing access key, it can be reused. Generally, it is considered best practice to use each access key for a single purpose, so that if it is revoked there is no unexpected collateral damage.

This configuration can also be specified on the command line as part of the build invocation, or via other mechanisms that Bazel provides. Please see the Bazel documentation for more information.

At the beginning and end of your build, you will see output similar to:

INFO: Streaming build results to: https://«develocity-server»/build/66e6805b-7ce1-4e32-bc26-c0fa129c76a7

The link shown is your Build Scan® link and can be used to observe the results of the build.

Short-lived access tokens

Develocity access keys are long-lived, creating risks if they are leaked. To avoid this, users can use short-lived access tokens to authenticate with Develocity. Access tokens can be used wherever an access key would be used. Access tokens are only valid for the Develocity instance that created them.

Develocity server version 2024.1+ supports access tokens.

Changing a Develocity instance’s hostname will cause all existing access tokens to become invalid.

To create an access token:

Get an access key or access token for the user you wish to create a token for.
Decide which permissions the new access token should have.
If project-level access control is enabled, decide which projects the new access token should be able to access.
Decide how long the new access token should live.
Make a POST request to /api/auth/token, optionally with the following parameters. The response will be the access token.
1. A permissions= query parameter with the config values of each permission you wish to grant. By default, all permissions for the credential used to authenticate the token request are granted to the created token.
2. If project-level access control is enabled, a projectIds= query parameter with the ID of each project you wish to grant access to. By default, all projects for the credential used to authenticate the token request are granted to the created token.
3. An expiresInHours= query parameter with the token’s intended lifetime in hours, with a maximum of 24. The default is two hours, or the remaining lifetime of the credential used to authenticate the request, whichever is smaller.

The requested permissions and project ids can be specified as comma-seperated lists or repeated parameters. For example, ?projectIds=a,b&projectIds=c is valid and will request projects a, b, and c.

If project-level access control is not enabled, all access tokens will be granted the “Access all data without an associated project” permission even if it is not explicitly requested.

If the user creating the token does not have one of the requested permissions or projects, Develocity will respond with a 403 Forbidden error. If an access token is used to authenticate the creation request, its permissions and projects will be used for this check instead of the user’s. The request will also error if the requested lifetime would cause the new access token to expire after the one used to authenticate the request. Together, this means you cannot create an access token with more access or a later expiration than the credentials used to authenticate the request.

See the API documentation for more details on the /api/auth/token endpoint.

Here is an example using CURL to create an access token:

$ curl -X POST https://ge.mycompany.com/api/auth/token?permissions=publishScan,writeCache,accessDataWithoutAssociatedProject&projectIds=project-a,project-b&expiresInHours=1 \
    -H "Authorization: Bearer 7asejatf24zun43yshqufp7qi4ovcefxpykbwzqbzilcpwzb52ja"

eyJraWQiOiJ0ZXN0LWtleSIsImFsZyI6IlJTMjU2IiwidHlwIjoiSldUIn0.eyJpc19hbm9ueW1vdXMiOmZhbHNlLCJwZXJtaXNzaW9ucyI6WyJSRUFEX1ZFUlNJT04iLCJFWFBPUlRfREFUQSIsIkFDQ0VTU19EQVRBX1dJVEhPVVRfQVNTT0NJQVRFRF9QUk9KRUNUIl0sInByb2plY3RzIjp7ImEiOjEsImIiOjJ9LCJ1c2VyX2lkIjoic29tZS1pZCIsInVzZXJuYW1lIjoidGVzdCIsImZpcnN0X25hbWUiOiJhIiwibGFzdF9uYW1lIjoidXNlciIsImVtYWlsIjoiYkBncmFkbGUuY29tIiwic3ViIjoidGVzdCIsImV4cCI6NzIwMCwibmJmIjowLCJpYXQiOjAsImF1ZCI6ImV4YW1wbGUuZ3JhZGxlLmNvbSIsImlzcyI6ImV4YW1wbGUuZ3JhZGxlLmNvbSIsInRva2VuX3R5cGUiOiJhY2Nlc3NfdG9rZW4ifQ.H1_NEG1xuleP-WIAY_uvSmdd2o7i_-Ko3qhlo04zvCgrElJe7_F5jNuqsyDfnb5hvKlOe5UKG_7QPTgY9-3pFQ

The resulting token would have the following permissions:

“Publish Build Scans”
“Read and write Build Cache data”
“Access all data without an associated project”

And it would have access to these projects:

“project-a”
“project-b”

The token would only be usable for one hour.

Using a Bazel remote cache

Bazel Build Scans provide more information when the build uses a remote cache that is also accessible by the Develocity server.

Bazel uploads build artifacts, test results and diagnostic information to the remote cache. When Build Scans are published to Develocity, such diagnostic information is read from the remote cache and included in Build Scans. This allows Build Scans to provide more information about the build. In particular, action profile information and detailed test results are included.

When viewing Build Scans, the build artifacts and diagnostic logs of targets and failed actions can be downloaded from the user interface if the remote cache is accessible.

If you’re using the remote cache that is part of Develocity, this should function automatically, though note that network connectivity from Develocity to the cache is required.

Using an external Bazel cache

Develocity has the ability to reach out to any Bazel cache that is exposed over the GRPC protocol. To use an external Bazel cache with Develocity, see the Kubernetes Helm Chart Configuration Guide or Standalone Helm Chart Configuration Guide for details on how to configure access to the external cache.

Develocity’s cache permissions

With most build systems, it is generally considered a best practice to not allow all users write access to the build cache due to concerns of cache poisoning. However, in order for Develocity to generate an enriched Build Scan® for Bazel, the required artifacts containing build metadata as mentioned above are only published by Bazel to the remote cache, so pure read-only cache access is not appropriate.

It is important to note that Bazel has two types of cache - Action Cache and a Content-Addressable Storage (CAS). It is generally considered safe for developers to write to the CAS, as these writes are also more secure than arbitrary cache writes, as all keys to the content-addressable storage are digests of the value that they point to and are verified server-side, making cache poisoning less of a concern. All files linked to a Build Scan® are CAS data. Thus, cache pollution or potential build security concerns are with regards to providing access to writing to the Action Cache.

Develocity’s remote cache offers a more granular role permission in the Develocity administration console - "Read build cache data and write Bazel CAS data". Consider granting this role permission by default to developers using Bazel, and reserving "Read and write build cache data" for trusted build systems.

Role Permission	Action Cache	CAS
Read build cache data	Read-Only	Read-Only
Read build cache and write Bazel CAS data	Read-Only	Write
Read and write build cache data	Write	Write

Role Permission

Action Cache

CAS

Read build cache data

Read-Only

Read build cache and write Bazel CAS data

Read-Only

Write

Read and write build cache data

Write

If multiple of these permissions are granted, the most permissive selection will take effect.

Reducing Remote Cache Uploads

Regardless of whether using Develocity’s remote cache or an external cache, it might be desirable for certain users or systems to not upload all CAS artifacts. This could be for build performance reasons due to slow network upload speeds, or for network / storage resource savings concerns.

Setting the --noremote_upload_local_results Bazel flag via command line or .bazelrc file will configure Bazel to only upload build metadata files such as the profile.json & test.xml files to the remote cache, but NOT upload all build artifacts, such as compilation outputs. This will save a substantial amount of bandwidth, but still generate an enhanced Bazel Build Scan®.

Configuring Build Scans

Controlling when Build Scans are published

Once you’ve gone through the initial setup of the previous section, you are ready to start publishing Build Scans. To control when Build Scans are published, you need to enable or disable the default configuration in your .bazelrc file. If you followed the setting up steps, you should find a line like the one below:

build --config=develocity

You can remove this line and instead add --config=develocity on the command line when you want to publish a Build Scan.

Background uploading

By default, Bazel will wait for all the build events and corresponding cache entries to be uploaded. This can be changed by adding the following to .bazelrc

common:develocity --bes_upload_mode=fully_async

Setting the project name

Develocity allows analyzing sets of builds based on search criteria. One such criterion is the project name, which is useful to set to a well known name for the project to allow for more effective searching.

By default, the project name is the same as the directory name of the build workspace. To configure it to a different value, specify the following configuration options:

common:develocity --bes_instance_name=«project-name»
common:develocity --remote_instance_name=«project-name»

If Project-level Access Control is enabled, setting the project name to a custom value will also set that value as the project identifier.

Extending Build Scans

You can easily include extra information in your Build Scans in the form of tags, links and values. This is a very powerful mechanism for capturing and sharing information that is important to your build and development process.

Such information is included by using Bazel’s workspace status and/or build metadata mechanisms, with specific value name prefixes.

The workspace status mechanism is most appropriate when a tag, value or link needs to be computed. The build metadata mechanism can be used when it is static, or is specified as part of the command line invocation.

Tags, values and links use the following prefixes respectively:

SCAN_TAG_
SCAN_VALUE_
SCAN_LINK_

The following are examples of creating a tag, value and link using build metadata at build invocation time:

bazel build ... \
--build_metadata=SCAN_TAG_LOCAL_BUILD= \
--build_metadata=SCAN_VALUE_GIT_COMMIT_ID=$(git rev-parse HEAD) \
--build_metadata=SCAN_LINK_CI_BUILD=https://ci-server.company.com/build/${BUILD_ID}

The workspace status mechanism works by running a program that prints status item values to stdout. The following is an example of using a shell script named scan_data.sh in the workspace directory to create a tag, value and link:

scan_data.sh

#!/bin/sh
echo SCAN_TAG_LOCAL_BUILD ""
echo "SCAN_VALUE_GIT_COMMIT_ID $(git rev-parse HEAD)"
echo "SCAN_LINK_CI_BUILD https://ci-server.company.com/build/${BUILD_ID}"

This script must be specified as a workspace status command, which is typically achieved by adding the following to the workspace .bazelrc file.

.bazelrc

build --workspace_status_command=$(pwd)/scan_data.sh

If your workspace status command issues a SCAN_TAG as the very last line of output, Bazel will truncate it unless it also has a dummy value. See this section for more details

Project-level access control

(Develocity 2023.4+)

To leverage Develocity’s project-level access control, builds must specify the following configuration options with the desired Develocity project ID:

common:develocity --bes_instance_name=«project-name»
common:develocity --remote_instance_name=«project-name»

These are typically set in the workspace .bazelrc file and checked in to version control.

Troubleshooting

Failed to query remote execution capabilities

ERROR: Failed to query remote execution capabilities: CANCELLED: Bazel compatibility has not been enabled for your Develocity installation

If you notice an error like the above, it means that Bazel support was not enabled on your Develocity instance. Contact your administrator to enable Bazel support.

A tag is missing from the Build Scan

If your workspace status command issues a SCAN_TAG as the very last line of output, Bazel will truncate it unless it also has a dummy value, leading to missing the tag in the Build Scan. The value is not going to be used, but it needs to be a non-empty string to ensure the tag is picked up correctly, like below

scan_data.sh

#!/bin/sh
# ..
echo SCAN_TAG_LOCAL_BUILD "dummy"

Appendix A: Captured information

Build Scans for Bazel work differently from those for Gradle, Maven, or sbt: those build tools rely on a plugin developed by Gradle to determine what information is captured by Develocity, but in Bazel this is all controlled by client-side settings, and built-in Bazel functionality. Because of this, we cannot directly control which information will be captured, but this document will cover the most typical cases.

In general, assume that all data sent to Develocity by way of either the Bazel JSON Trace Profile or the Build Event Protocol (BEP) will be captured by Develocity.

Enumerated below is the data that will be sent based on a few common Bazel options being set for a Java build. The exact information captured will vary based on which Bazel rules, configuration options, and client version are used.

--noslim_profile Generating a profile is the default behavior, but by default a slim profile is generated, which contains less information. This is the information contained in a fat profile:

--build_event_publish_all_actions By default, Bazel summarizes short actions, which makes it hard for Develocity to construct a complete build timeline.

--remote_cache The Build Events will frequently reference files from the local filesystem (test.log, test.xml, failed actions, etc.) and if they are not uploaded to the remote cache, they will be unavailable for analysis.

Environment
- All client environment variables are captured
Build
- Build invocation options
- Bazel version
- Date & timestamps
- Output directory on the client Machine
- CPU Usage for both client overhead & build
- Memory usage for both client overhead & build
- Target names
- Source file names
- Remote output downloads
- Platform information
- Target configuration
- Action inputs and outputs
- Client environment variables
- Actions
  - Successful actions are uploaded to the action cache
  - Failed actions are stored in the CAS and referenced by the BEP to ease debugging test logs, test results, etc.

Redacted information

In order to protect secrets which may likely appear in environment variables or Bazel flags, Develocity, by default, displays the following fields as "<REDACTED>" when displaying a Bazel Build Scan®:

Environment Variables

ALL environment variables except for:

BITRISE_GIT_BRANCH
BITRISE_GIT_COMMIT
BUILDKITE_BRANCH
BUILDKITE_BUILD_URL
BUILDKITE_COMMIT
BUILDKITE_JOB_ID
BUILDKITE_REPO
CIRCLE_BRANCH
CIRCLE_REPOSITORY_URL
CIRCLE_SHA1
CI_COMMIT_BRANCH
CI_COMMIT_SHA
CI_REPOSITORY_URL
CI_RUNNER
CI
COMMIT_SHA
GITHUB_ACTOR
GITHUB_HEAD_REF
GITHUB_REF
GITHUB_REPOSITORY
GITHUB_RUN_ID
GITHUB_SHA
GIT_BRANCH
GIT_COMMIT
GIT_REPOSITORY_URL
GIT_URL
REPO_URL
TRAVIS_BRANCH
TRAVIS_COMMIT
TRAVIS_REPO_SLUG
USER

*_header Flags

There are several Bazel flags that designate headers of requests to remote servers. As some of these headers may contain sensitive information, the values of certain header names will be redacted.

The flags with redacted headers are:

--bes_header
--remote_cache_header
--remote_downloader_header
--remote_exec_header
--remote_header

The header names for which values are redacted are:

Authorization
X-Buildbuddy-Api-Key
X-Engflow-Auth-Token

Header names are checked in a case-insensitive fashion.