Visualize metrics using Grafana

The metrics collected in a workspace within Amazon Managed Service for Prometheus can be visualized using Grafana. Grafana v7.3.x has added a new feature to support AWS Signature Version 4 (SigV4) authentication, and we will be using that version here.

  1. In the AWS Management Console on the Services menu, click Cloud9.
  2. Click Open IDE on the observabilityworkshop Cloud9 instance.

Deploy a self-managed Grafana installation using Helm charts using the commands below

  1. Execute the following commands in the terminal:
helm repo add grafana https://grafana.github.io/helm-charts
kubectl create ns grafana
helm install grafana-for-amp grafana/grafana -n grafana

Update your Grafana server to use the AWS signing proxy

  1. Execute the following commands in the terminal:

These commands create a new file named amp_query_override_values.yaml and fill it with the Service Account Role ARN. This file will be used to update your Grafana deployment to enable the Sigv4 protocol which the AWS signing proxy uses to authenticate.

cat << EOF > amp_query_override_values.yaml
serviceAccount:
    name: "amp-iamproxy-service-account"
    annotations:
        eks.amazonaws.com/role-arn: "${IAM_PROXY_PROMETHEUS_ROLE_ARN}"
grafana.ini:
  auth:
    sigv4_auth_enabled: true
EOF

Update your Grafana environment

  1. Execute the following commands in the terminal:
helm upgrade --install grafana-for-amp grafana/grafana -n grafana -f ./amp_query_override_values.yaml

Login to Grafana console to visualize the metrics

  1. Execute the following commands in the terminal:

In order to login into Grafana, you’ll need the admin password that was created during the installation.The following command retrieves it from the Kubernetes secret.

kubectl get secrets grafana-for-amp -n grafana -o jsonpath='{.data.admin-password}'|base64 --decode
  1. Execute the following commands in the terminal:

This command allows you to access the Grafana interface.

export POD_NAME=$(kubectl get pods --namespace grafana -l "app.kubernetes.io/name=grafana,app.kubernetes.io/instance=grafana-for-amp" -o jsonpath="{.items[0].metadata.name}")
kubectl --namespace grafana port-forward $POD_NAME 8080:3000
  1. Open Grafana from an internet browser using http://localhost:8080 URL and login with the admin username.

If you are on Cloud9, you can open the preview browser by clicking on Preview Running Application as shown below.

Prom dashboard

Configure AMP data source

  1. Select Prometheus from the list of data sources and in the URL field

  2. Specify the Endpoint – query URL displayed in the AMP workspace details page without the /api/v1/query string at the end of the URL.

  3. Select SigV4 for authentication and select the AWS Region.

  4. Select the AMP workspace from the list and click Add data sources.

Once added, you should able to see that the AMP data source is authenticated through SigV4 protocol. Grafana (7.3.5 and above) has the AWS SigV4 proxy built-in as a plugin which makes this possible.

Make sure you rename the datasource as observability-workshop

AMP configuration

Query Metrics

Visualization in self-hosted Grafana is similar to Amazon Managed Service for Grafana. Go here and follow the instructions to visualize metrics.

This concludes the Amazon Managed Service for Prometheus module.