Visualize metrics using Grafana

The metrics collected in a workspace within Amazon Managed Service for Prometheus can be visualized using Grafana. Grafana v7.3.x has added a new feature to support AWS Signature Version 4 (SigV4) authentication and we will be using that version here.

Deploy a self-managed Grafana installation using Helm charts using the commands below

helm repo add grafana https://grafana.github.io/helm-charts
kubectl create ns grafana
helm install grafana-for-amp grafana/grafana -n grafana

Update your Grafana server to use the AWS signing proxy

Create a new file and name it amp_query_override_values.yaml. This file will be used to update your Grafana deployment to enable the Sigv4 protocol which the AWS signing proxy uses to authenticate.

serviceAccount:
    name: "amp-iamproxy-service-account"
    annotations:
        eks.amazonaws.com/role-arn: "${IAM_PROXY_PROMETHEUS_ROLE_ARN}"
grafana.ini:
  auth:
    sigv4_auth_enabled: true

Now execute the following command to update your Grafana environment.

helm upgrade --install grafana-for-amp grafana/grafana -n grafana -f ./amp_query_override_values.yaml

Login to Grafana console to visualize the metrics

Replace the string GRAFANA_POD_NAME with the actual Grafana pod name you just created and execute the command

kubectl port-forward -n grafana pods/GRAFANA_POD_NAME 5001:3000

Next, open Grafana from an internet browser using http://localhost:5001 URL and login with the admin username.

The password is obtained from the Kubernetes secret as follows:

kubectl get secrets grafana-for-amp -n grafana -o jsonpath='{.data.admin-password}'|base64 --decode

Configure AMP data source

Select Prometheus from the list of data sources and in the URL field, specify the Endpoint – query URL displayed in the AMP workspace details page without the /api/v1/query string at the end of the URL. Select SigV4 for authentication and select the AWS Region

Simply select the AMP workspace from the list and click Add data sources. Once added you will able to see that the AMP data source is authenticated through SigV4 protocol. Grafana (7.3.5 and above) has the AWS SigV4 proxy built-in as a plugin which makes this possible.

Make sure you rename the datasource as AMP-Datasource

AMP configuration

Query Metrics

Visualization in self-hosted Grafana is similar to Amazon Managed Service for Grafana. Go here and follow the instructions to visualize metrics