---
component: provenance-governor
version: "1.7"
slug: provenance-governor/monitoring-services
canonical_url: "https://docs.gradle.com/develocity/provenance-governor/1.7/monitoring-services/"
title: "Monitoring"
description: "Monitoring endpoints and metrics for Develocity Provenance Governor."
keywords:
  - "admin"
  - "Grafana"
  - "Kubernetes"
status: current
---

<!-- llms-index: https://docs.gradle.com/develocity/llms.txt -->

# Monitoring

<a id="preamble"></a>

<a id="monitoring-services"></a>

You can monitor the health and performance of your Develocity Provenance Governor deployment using built-in monitoring endpoints. These are accessible on a dedicated port (9090) and protected by basic authentication by default. Separation from the main application port improves security for production traffic.

<a id="available-endpoints"></a>

## Available Endpoints

<table class="tableblock frame-all grid-all stretch"><colgroup><col style="width: 25%;"> <col style="width: 75%;"></colgroup><tbody><tr><td class="tableblock halign-left valign-top">Endpoint</td><td class="tableblock halign-left valign-top">Description</td></tr><tr><td class="tableblock halign-left valign-top">/actuator/health</td><td class="tableblock halign-left valign-top">Overall health status</td></tr><tr><td class="tableblock halign-left valign-top">/actuator/health/liveness</td><td class="tableblock halign-left valign-top">Kubernetes liveness probe</td></tr><tr><td class="tableblock halign-left valign-top">/actuator/health/readiness</td><td class="tableblock halign-left valign-top">Kubernetes readiness probe</td></tr><tr><td class="tableblock halign-left valign-top">/actuator/prometheus</td><td class="tableblock halign-left valign-top">Prometheus scrapeable metrics</td></tr><tr><td class="tableblock halign-left valign-top">/actuator/info</td><td class="tableblock halign-left valign-top">General application info</td></tr></tbody></table>

<a id="accessing-endpoints"></a>

## Accessing Endpoints

Use port-forwarding for local machine access, or direct service URLs for in-cluster.

```shell
kubectl port-forward svc/api-http-monitoring 9090:9090 -n develocity-provenance-governor
```

<a id="security-configuration"></a>

## Security & Configuration

All monitoring endpoints on port 9090 require basic authentication; see application config for username/password management. For load balancer/liveness, unauthenticated `/livez` and `/readyz` are offered on app port (8080).

<a id="prometheus-metrics"></a>

## Prometheus Metrics

The `/actuator/prometheus` endpoint exposes metrics in Prometheus format, which can be scraped by a Prometheus server for monitoring and alerting.

<a id="available-metrics"></a>

### Available Metrics

The Prometheus endpoint (`/actuator/prometheus`) provides metrics including:

*   **JVM Metrics**: Memory usage, garbage collection, thread counts
    
*   **HTTP Metrics**: Request rates, response times, error rates
    
*   **Application Metrics**: Custom business metrics for attestation publishing and policy evaluation
    
*   **System Metrics**: CPU and system resource utilization
    

<a id="enabling-histogram-percentiles"></a>

## Enabling Histogram Percentiles

By default, only basic timing metrics (`_sum`, `_count`, `_max`) are collected. To enable percentile calculations (p50, p95, p99), configure histogram buckets in your application properties:

```
# Enable histogram buckets for HTTP server requests
management.metrics.distribution.percentiles-histogram.http.server.requests=true

# Enable histogram buckets for HTTP client requests (upstream calls)
management.metrics.distribution.percentiles-histogram.http.client.requests=true

# Optionally, define specific percentiles to publish directly
management.metrics.distribution.percentiles.http.server.requests=0.5,0.95,0.99
management.metrics.distribution.percentiles.http.client.requests=0.5,0.95,0.99

# Define SLO buckets for alerting
management.metrics.distribution.slo.http.server.requests=100ms,250ms,500ms,1s,5s
```

> [!WARNING]
> Histogram buckets increase metric cardinality and storage requirements. Each histogram creates multiple time series (one per bucket boundary). Enable only for endpoints where percentile analysis is needed, or accept the additional storage cost for comprehensive monitoring.

With histogram buckets enabled, you can use `histogram_quantile()` in Prometheus to calculate arbitrary percentiles without pre-defining them at collection time.

<a id="prometheus-query-examples"></a>

## Prometheus Query Examples

The following queries help monitor the health and performance of Develocity Provenance Governor.

<a id="request-rate"></a>

### Request Rate

Monitor the rate of incoming HTTP requests per second:

```text
sum(rate(http_server_requests_seconds_count{namespace="develocity-provenance-governor"}[5m])) by (uri, status)
```

Filter to specific endpoints:

```text
sum(rate(http_server_requests_seconds_count{namespace="develocity-provenance-governor", uri=~"/packages.*"}[5m])) by (uri, status)
```

<a id="success-rate"></a>

### Success Rate

Calculate the percentage of successful requests (2xx responses):

```text
sum(rate(http_server_requests_seconds_count{namespace="develocity-provenance-governor", status=~"2.."}[5m]))
/
sum(rate(http_server_requests_seconds_count{namespace="develocity-provenance-governor"}[5m]))
* 100
```

<a id="error-rate"></a>

### Error Rate

Track client errors (4xx) and server errors (5xx):

```text
# Client errors (4xx)
sum(rate(http_server_requests_seconds_count{namespace="develocity-provenance-governor", status=~"4.."}[5m])) by (uri)

# Server errors (5xx)
sum(rate(http_server_requests_seconds_count{namespace="develocity-provenance-governor", status=~"5.."}[5m])) by (uri)
```

<a id="request-latency"></a>

### Request Latency

**Average latency** per endpoint:

```text
sum(rate(http_server_requests_seconds_sum{namespace="develocity-provenance-governor"}[5m])) by (uri)
/
sum(rate(http_server_requests_seconds_count{namespace="develocity-provenance-governor"}[5m])) by (uri)
```

**Maximum latency** (note: this value resets periodically):

```text
http_server_requests_seconds_max{namespace="develocity-provenance-governor"}
```

**Percentile latency** (requires histogram configuration):

```text
# 95th percentile (p95)
histogram_quantile(0.95, sum(rate(http_server_requests_seconds_bucket{namespace="develocity-provenance-governor"}[5m])) by (le, uri))

# 99th percentile (p99)
histogram_quantile(0.99, sum(rate(http_server_requests_seconds_bucket{namespace="develocity-provenance-governor"}[5m])) by (le, uri))
```

<a id="upstream-service-latency"></a>

### Upstream Service Latency

Monitor latency to external services (Develocity, Artifactory):

```text
# Average latency to Develocity
sum(rate(http_client_requests_seconds_sum{namespace="develocity-provenance-governor", client_name="develocity.example.com"}[5m]))
/
sum(rate(http_client_requests_seconds_count{namespace="develocity-provenance-governor", client_name="develocity.example.com"}[5m]))
```

<a id="jvm-metrics"></a>

### JVM Metrics

Monitor JVM health:

```text
# Heap memory usage
jvm_memory_used_bytes{namespace="develocity-provenance-governor", area="heap"}

# Live thread count
jvm_threads_live_threads{namespace="develocity-provenance-governor"}

# CPU usage (0-1 ratio)
process_cpu_usage{namespace="develocity-provenance-governor"}
```