Logs from EKS cluster

Logs are collected by the fluentd daemonset running in the EKS nodes. The following CloudWatch log groups are created by default when Container Insights is setup:

  • /aws/containerinsights/cluster-name/application
  • /aws/containerinsights/cluster-name/dataplane
  • /aws/containerinsights/cluster-name/host
  • /aws/containerinsights/cluster-name/performance

Go to CloudWatch Logs Log Groups page and enter the cluster name in the filter log groups textbox which will filter the log groups and show the results similar to the one below

EKS Metrics

Click on application log group which will show you different log streams for each pods in the cluster originating from different pods.

Likewise, you will see 2 log streams one for each node in the cluster under performance log group. Select one of the log streams to see performance logs originating from the nodes.

Along with performance log data, you will also see that there are metrics present in Embedded Metric Format.

Go to Embedded Metric Format (EMF) workshop module in case you want to dive deep into that topic.

To see Metric log data simply enter CloudWatchMetrics (case sensitive) in the Filter events textbox as shown below.

EKS Metrics

Using Logs Insights to query log data

Go to Logs Insights module for a deep dive on this topic.

CloudWatch Logs Insights offers a simple to use, purpose built query language that you can use to query log data from CloudWatch Logs.

The following is a sample query that you can use to query log data from the application log group.

fields @timestamp, @message
| filter @message like 'puppy'
| sort @timestamp desc
| limit 200

See the following screenshot showing log insights query execution results. The query above selects log data that contain the string puppy in the log message.

EKS LogInsightsQuery