CloudWatch Container Insights monitoring for Prometheus automates the discovery of Prometheus metrics from containerized systems and workloads. Prometheus is an open-source systems monitoring and alerting toolkit.
The front end application is already exposing metrics in Prometheus exposition format. By adding /metrics
to the application URL, you can view the metrics in the browser.
Cloud9
.Open IDE
on the observabilityworkshop Cloud9 instance.This command ensures that the CloudWatch Prometheus agent is running.
kubectl get pods -n amazon-cloudwatch
Your output should look similar to the one below. (Notice the 3rd line indicating the CloudWatch Prometheus agent running. In the default configuration, there will be just one agent running per cluster.)
NAME READY STATUS RESTARTS AGE
cloudwatch-agent-fnh25 1/1 Running 0 172m
cloudwatch-agent-tfgk7 1/1 Running 0 172m
cwagent-prometheus-5bbf77558b-sqkfm 1/1 Running 0 172m
fluent-bit-7p67r 1/1 Running 0 172m
fluent-bit-cqphf 1/1 Running 0 172m
Open the file called prometheus-eks.yaml
located at one-observability-demo/PetAdoptions/cdk/pet_stack/resources/prometheus-eks.yaml
(or alternatively, click here).
Search the file for the term `PetSite.
Here you will find the configurations used to collect metrics from the application endpoint. The following screenshot shows a portion of the configuration where we specify metric selectors and appropriate dimensions under which the metrics should be aggregated.
CloudWatch
in the AWS Console.Metrics
.You should see a new namespace called ContainerInsights/Prometheus
under Custom Namespaces
.
ContainerInsights/Prometheus
that was just created.ClusterName, Namespace, app
.Just like regular Container Insights metrics, Prometheus metrics are also collected through Embedded Metric Format.
logs
, click log groups
.filter log groups
search box, type prometheus
./aws/containerinsights/PetSite/prometheus
.Alternatively, you can click here to go directly to the log group under which the Prometheus metrics are collected.
PetSite
log stream.You should see the Prometheus metrics that were collected from the environment.
Read more about this topic in this blog post - https://aws.amazon.com/blogs/containers/using-prometheus-metrics-in-amazon-cloudwatch/
Take a look at this blog post from Viji Sarathy that talks about performing autoscaling actions using Prometheus metrics collected using the Cloudwatch Prometheus agent.