Explore X-Ray

X-Ray Setup

Take a look at the setup architecture diagrams for EC2 and container environments such as ECS/EKS. X-Ray has an agent that runs as a separate process from your application. The X-Ray SDK collects traces from the application and sends them to the local agent over UDP. The agent then sends the traces to the X-Ray service using X-Ray APIs.

X-Ray setup

Alternatively, you can also collect traces instrumented with OpenTelemetry to be analysed with X-Ray using the following setup.

ADOT setup

Here is a dedicated section on AWS Distro for OpenTelemetry and how it is implemented in this lab.

Service map on X-Ray

  1. In the AWS Management Console on the Services menu, navigate to X-Ray. You should see a service map similar to the one shown below.

The Service map shows all the microservices and other components the application is making use of. When None is selected at the bottom right, you will be able to see the average response times and requests served by each service.

  1. Switch between None, Traffic, and Health modes by clicking the buttons in the bottom right corner. You will see the size of the circles (nodes) adjust to reflect the traffic and health status relative to other services in the application.

Trace duration

  1. Select a duration from the time drop down at the top right as shown below.

You can view upto 6 hours of trace data at any given point.

Duration

If you are not familiar with X-Ray basic concepts, it would help to spend a few minutes to learn about Segments, Subsegments, Traces and more here before proceeding further.

View traces

  1. Click on a any node.

This will cause a modal to appear with service details. The service details modal has a response time distribution graph for the requests that the node has processed.

The screenshot below shows that the S3 node has been selected. It also indicates that Faults were recorded as well.

Duration

  1. Click on the arrow between two nodes.

The edge details modal (previously the service details modal) shows the requests that are being processed between the two services joined by the arrow.

Duration

  1. Click on View traces from the edge details modal.

This will show all the traces that are captured between these two services. (Note: if you are interested in only seeing traces that have a response time greater than 3 seconds, you can filter those traces by selecting that area of the graph with the mouse before clicking view traces.)

If you are interested in filtering by traces that complete within a certain duration, you can filter those traces by selecting that area of the graph with the mouse before clicking view traces.

Duration

You should see a screen similar to the one below.

Clicking on a specific URL will filter the traces for that URL and display them in the trace list table.

Duration

Learn about filter expressions here

  1. Select a trace from the trace list table by clicking on a trace ID. This will show you a screen similar to the one below. (The errors may not be present all the time, as it has been randomized)

Notice the Trace Map that shows the services that particular request traversed through. Below the Trace Map is the Segment timeline which shows the different Segments and Subsegments the trace contains.

Notice that the subsegments that have error responses have a yellow error icon.

Duration

  1. If possible, click on one of the error icons.

This will take you to the exceptions tab of the subsegment and show you the captured exception details.

Exception

X-Ray annotations

Learn about Annotations - https://docs.aws.amazon.com/xray/latest/devguide/xray-concepts.html#xray-concepts-annotations

  1. Return to the Service map by navigating to the X-Ray home page from the AWS Console.

  2. Select the payforadoption node, then click View traces.

  3. Click the URL that ends with /Payment/MakePayment and select any one of the traces.

  4. As shown on the screenshot below, if you click on a node on the trace, the associated segment will be selected.

MakePayment

  1. Click on the entire segment payforadoption as indicated above.

This will show you the Subsegment details.

  1. Click on the Annotations tab. You will be able to see the custom Annotations that were instrumented through the X-Ray SDK. You can see PetId and PetType being annotated in the trace.

Annotations are indexed and can be used to filter traces. For example, a Filter expression like annotation.PetType="bunny" will show me the traces that have the annotations of PetType=bunny.

Annotations

  1. Switch to the Metadata tab to see the metadata information that was added to the trace. Annotations

X-Ray can also show details about the AWS service being used in the trace.

  1. Close the current window and open the database subsegment called adoptions. For this subsgement, you will see a new tab called SQL with the prepared SQL query and various informations about the target database.

SQL

Wrapping SQL queries with X-Ray will automatically create subsgemnts with additional structured metadata. https://docs.aws.amazon.com/xray/latest/devguide/xray-sdk-go-sqlclients.html

  1. Select a DynamoDB subsegment.
  2. Go to the Resources tab and check the details such as the table name, the operation that was performed on the table, and the region.

You can check this out for other services as well to see service specific information.

ServiceDetails

This concludes this section. You may continue on to the next section.