The information needed to query your specific environment, to optimize your searches, and provide the custom context needed to diagnose failures and rapidly gain insights can be easily added to Build Scans.

You can complete this tutorial in:

  • 1 minute (read the Introduction)

  • 5-10 minutes (read the Introduction and Tour)

  • 15-20 minutes (read the Introduction and Tour and perform the Hands-on Lab)

Introduction

Develocity Build Scans already contain a vast amount of information about what is happening in your builds.

Because every environment is different, we also understand that there typically is additional metadata that you will want to have included in your Build Scans.

This information helps you to more quickly diagnose failed builds, identify ownership or other custom attributes for your builds, and are especially useful for providing integration links to other federated systems involved in your CI/CD pipeline.

There are three different mechanisms to allow you to add data to your Build Scans:

  • links — hyperlinks to things like CI build records, ticketing systems, source code control systems.

  • tags — used for attributes such as build type [CI or dev], environment, owner, application, etc.

  • custom values — name/value pairs that are used for things like code scanning results, attaching metric values, posting computed results from querying some other system.

The ability to attach extended metadata to Build Scans can make their application to use cases such as reducing debugging time, reducing the occurrence of flaky or false alarms, and identifying trends to help keep builds fast that much more powerful and effective - saving you and your team valuable time and money.

Tour: Extending Build Scans

Distinguish CI Build Scans from developer Build Scans

This tutorial assumes some basic familiarity with the Gradle Build Scan plugin and its configuration model. If you need to familiarize yourself with that, we recommend you read this before continuing.

In most situations, a tag is used to distinguish CI builds from local developer builds. The management of CI builds is sometimes done mostly by different people, and in many cases, it is useful to filter scan lists to show only CI builds.

Most CI servers (e.g. Jenkins) will set standard environment variables and the standard technique is to set a tag conditional to the presence of one of those variables in the build.

Here is a trivial code example of doing this:

// Local or CI
if (System.getenv("CI") != null) {
    buildScan.tag("CI")
    buildScan.link("CI Build", System.getenv("BUILD_URL"))
} else {
    buildScan.tag("LOCAL")
}
// Local or CI
if (System.getenv('CI')) {
    buildScan.tag 'CI'
    buildScan.link 'CI Build', System.getenv('BUILD_URL')
} else {
    buildScan.tag 'LOCAL'
}

Here we set a tag (and also a link) based on an environment variable. The usage of links in this fashion demonstrates linking back to the CI server from the Build Scan, which is a common use case.

Let’s see how we can use such tags after being populated. Open a scan list by clicking here.

We can find out the builds that have the CI tag by entering CI in the Tags filter at the top of the Build Scan list and clicking the Search button. Go ahead and do that now.

You should see that 8 of the builds are so tagged.

You can optionally repeat these steps for the tag LOCAL.

For either case, click on one of the Build Scans that come back as the result of the search, and you will see the tag prominently displayed at the very top of the scan page.

Add source control information to your Build Scans

A very useful addition to your Build Scans is to embed source control information. For example, if you are using Git, you can attach the commit ID, branch, repository pointers, and workspace status to the Build Scan. Using GitHub API’s, you can also do things like post a Gist that contains the diffs between the workspace and the last pushed commit.

Open this Build Scan to learn more.

First, notice that the tag dirty is visible at the top. This indicates that there are un-committed changes in the workspace.

You will also notice the Source link, which allows for navigation to the Github repository (even though this one is private to Gradle Inc.).

Using the left-hand navigation, select the Custom Values Build Scan section, and you will see that we have also computed and written as custom values the Git branch name, commit ID, and the output of the git status command.

Here is a code sample that shows how some of these values can be inserted into your Build Scans through your Gradle build scripts:

// Execute a CLI command and return the output
val execute = { p: String ->
    ProcessBuilder(p.split(" ")).start().apply { waitFor() }.inputStream.bufferedReader().use { it.readText().trim() }
}

// Git commit id
val commitId = execute("git rev-parse --verify HEAD")
if (!commitId.isNullOrEmpty()) {
    buildScan.value("Git Commit ID", commitId)
    buildScan.link("Source", "$GITHUB_REPO/tree/$commitId")
}

// Git branch name
val branchName = execute("git rev-parse --abbrev-ref HEAD")
if (!branchName.isNullOrEmpty()) {
    buildScan.value("Git Branch Name", branchName)
}

// Git dirty local state
val status = execute("git status --porcelain")
if (!status.isNullOrEmpty()) {
    buildScan.tag("dirty")
    buildScan.value("Git Status", status)
}
// Git commit id
def commitId = 'git rev-parse --verify HEAD'.execute().text.trim()
if (commitId) {
    buildScan.value 'Git Commit ID', commitId
    buildScan.link 'Source', "$GITHUB_REPO/tree/$commitId"
}

// Git branch name
def branchName = 'git rev-parse --abbrev-ref HEAD'.execute().text.trim()
if (branchName) {
    buildScan.value 'Git Branch Name', branchName
}

// Git dirty local state
def status = 'git status --porcelain'.execute().text
if (status) {
    buildScan.tag 'dirty'
    buildScan.value 'Git Status', status
}

Surface static code analysis issues in Build Scans

Your build might automate many different tools that generate output about your code. Checkstyle is an example of a code checker that is not (currently) fully integrated into a Gradle core plugin. Let’s use that to demonstrate how to run an external tool and then brand the Build Scan with a set of values; in this case we will annotate the Build Scan with the output of the checkstyle command.

This Build Scan deep link takes you directly to the custom values for this Build Scan that shows you the checkstyle output.

Here is a code sample that demonstrates all it takes to cause this to happen:

import groovy.util.XmlSlurper
import groovy.util.slurpersupport.NodeChildren
import groovy.util.slurpersupport.NodeChild

...

gradle.taskGraph.afterTask {
    if (this is Checkstyle && state.failure != null) {
        val x = XmlSlurper().parse(reports.xml.destination)
        val files = x.getProperty("file") as NodeChildren
        files.forEach { f ->
            val file = f as NodeChild
            val filePath = rootProject.relativePath(file.attributes()["name"])
            val errors = file.getProperty("error") as NodeChildren
            errors.forEach { e ->
                val error = e as NodeChild
                val attr = error.attributes()
                val msg = "${filePath}:${attr["line"]}:${attr["column"]} \u2192 ${attr["message"]}"
                project.buildScan.value("Checkstyle Issue", msg)
            }
        }
    }
}
// Checkstyle violations
gradle.taskGraph.afterTask { Task task, TaskState state ->
    if (task instanceof Checkstyle) {
        if (state.failure) {
            def checkstyle = new XmlSlurper().parse(task.reports.xml.destination)
            def errors = checkstyle.file.collect {
                String filePath = task.project.rootProject.relativePath(it.@name.text())
                it.error.collect { "${filePath}:${it.@line}:${it.@column} \u2192 ${it.@message}" }
            }.flatten()
            errors.each { task.project.buildScan.value 'Checkstyle Issue', it }
        }
    }
}

Reach out for help when local build fails to succeed

Expanding on the earlier example where we annotate the Build Scan with Git properties, we now go a step further and actually take the diffs and create a Gist from them.

If a local build fails, a build team engineer can now look at possible build script changes that are in the developer’s workspace and determine in any of those diffs played a role in the failure, potentially saving a lot of time to determine root cause.

Examine this Build Scan.

Notice the Diff link at the header of the main area. If you click on that link, you will go to a Gist which clearly shows that a lot of local changes to tests occurred, and possibly those changes are what is causing some test failures to appear in this particular Build Scan.

Categorize build failures

Your workflow for build failures might be different depending on whether the failure was a true build failure or if it is a test failure.

But furthermore, it can be interesting to further classify failures and perform queries based on the failure type. For example, to know how many Java compilation failures we are seeing versus other types of failures.

To see this in action, view this Build Scan.

We have used the deep link to take you directly to the custom values section, and you can see that this failure is a compilation failure.

You can get further details about the compile failure by navigating to the Failure section of the Build Scan via the left-hand navigation.

Here is the code snippet from a build script that computes these custom values:

buildScan.buildFinished {
    if (failure is LocationAwareException) {
        if (failure.cause is TaskExecutionException) {
        	val cause = failure.cause as TaskExecutionException
            val value = if (cause.task is Test) {
                "TEST"
            } else if (cause.cause is CompilationFailedException) {
                "COMPILATION"
            } else {
                "UNKNOWN_TASK"
            }
            buildScan.value("FAILURE", value)
        } else {
            buildScan.value("FAILURE", "OTHER")
        }
    }
}
buildScan.buildFinished { result ->
    if (result.failure instanceof LocationAwareException) {
        if (result.failure.cause instanceof TaskExecutionException) {
            def value
            if (result.failure.cause.task instanceof Test) {
                value = 'TEST'
            } else if (result.failure.cause.cause instanceof CompilationFailedException) {
                value = 'COMPILATION'
            } else {
                value = 'UNKNOWN_TASK'
            }
            buildScan.value 'FAILURE', value
        } else {
            buildScan.value 'FAILURE', 'OTHER'
        }
    }
}

Let’s finish this by seeing how many compilation failures we have seen, open the scan list.

type FAILURE=COMPILATION into the search box for Custom values.

You should see that we have encountered so far 4 builds that suffered a compilation error.

Hands-on Lab: Extending Build Scans

Read on to go one level deeper.

Prerequisites

To follow these steps, you will need:

  1. A local JVM installed.
    JVM requirements are specified at https://gradle.org/install. Note that a local Gradle Build Tool install is optional and not required. All hands-on labs will execute the Gradle Build Tool using the Gradle Wrapper.

  2. An instance of a Develocity server.
    You can request a Develocity trial here.

For the rest of the document, we will assume you have a Develocity instance, available at https://develocity.mycompany.com.

Lab: Generating extended metadata in Develocity Build Scans

Open a terminal window and clone the GitHub repository Build Scan quickstart.

Check out the branch custom-values with git checkout custom-values.

Using the text editor of your choice, modify the Build Scan configuration in settings.gradle to point to your Develocity server instance:

Replace 
buildScan {
  server = '<<your Develocity instance>>'
  publishAlways()
}
with 
buildScan {
  // Your personal Develocity instance
  server = 'https://develocity.mycompany.com'
  publishAlways()
}

All of the examples we discussed in the tour are combined in this example, and the code can be viewed in the build script customData.gradle in the root directory of the repository now.

Run the build with gradle build and open the Build Scan that logs at the conclusion of the build. You will be able to see the custom values for checkstyle, links, CI tags, etc.

Conclusion

This tutorial has covered a few common scenarios for adding extended data to Build Scans. This information can greatly assist you and your team in fully leveraging the power of Develocity.

We would welcome seeing your use cases for extended data and hope to see your suggestions and code in the public GitHub repository!