Take a look at the setup architecture diagrams for EC2 and container environments such as ECS/EKS. X-Ray has an agent that runs as a separate process from your application. The X-Ray SDK collects traces from the application and sends them to the local agent over UDP. The agent then sends the traces to the X-Ray service using X-Ray APIs.
Alternatively, you can also collect traces instrumented with OpenTelemetry to be analysed with X-Ray using the following setup.
Here is a dedicated section on AWS Distro for OpenTelemetry and how it is implemented in this lab.
X-Ray
. You should see a service map similar to the one shown below.
The Service map shows all the microservices and other components the application is making use of. When None is selected at the bottom right, you will be able to see the average response times and requests served by each service.
None
, Traffic
, and Health
modes by clicking the buttons in the bottom right corner. You will see the size of the circles (nodes) adjust to reflect the traffic and health status relative to other services in the application.You can view upto 6 hours of trace data at any given point.
If you are not familiar with X-Ray basic concepts, it would help to spend a few minutes to learn about Segments, Subsegments, Traces and more here before proceeding further.
node
.This will cause a modal to appear with service details. The service details modal has a response time distribution graph for the requests that the node has processed.
The screenshot below shows that the S3 node has been selected. It also indicates that Faults were recorded as well.
arrow
between two nodes.The edge details modal (previously the service details modal) shows the requests that are being processed between the two services joined by the arrow.
View traces
from the edge details modal.This will show all the traces that are captured between these two services. (Note: if you are interested in only seeing traces that have a response time greater than 3 seconds, you can filter those traces by selecting that area of the graph with the mouse before clicking view traces.)
If you are interested in filtering by traces that complete within a certain duration, you can filter those traces by selecting that area of the graph with the mouse before clicking view traces.
You should see a screen similar to the one below.
Clicking on a specific URL will filter the traces for that URL and display them in the trace list table.
Learn about filter expressions here
Notice the Trace Map that shows the services that particular request traversed through. Below the Trace Map is the Segment timeline which shows the different Segments and Subsegments the trace contains.
Notice that the subsegments that have error responses have a yellow error icon.
This will take you to the exceptions tab of the subsegment and show you the captured exception details.
Learn about Annotations - https://docs.aws.amazon.com/xray/latest/devguide/xray-concepts.html#xray-concepts-annotations
Return to the Service map by navigating to the X-Ray home page from the AWS Console.
Select the payforadoption
node, then click View traces
.
Click the URL that ends with /Payment/MakePayment
and select any one of the traces.
As shown on the screenshot below, if you click on a node on the trace, the associated segment will be selected.
payforadoption
as indicated above.This will show you the Subsegment details.
Annotations
tab. You will be able to see the custom Annotations that were instrumented through the X-Ray SDK. You can see PetId
and PetType
being annotated in the trace.Annotations are indexed and can be used to filter traces.
For example, a Filter expression like annotation.PetType="bunny"
will show me the traces that have the annotations of PetType=bunny.
Metadata
tab to see the metadata information that was added to the trace.
X-Ray can also show details about the AWS service being used in the trace.
adoptions
. For this subsgement, you will see a new tab called SQL with the prepared SQL query and various informations about the target database.Wrapping SQL queries with X-Ray will automatically create subsgemnts with additional structured metadata. https://docs.aws.amazon.com/xray/latest/devguide/xray-sdk-go-sqlclients.html
DynamoDB
subsegment.Resources
tab and check the details such as the table name, the operation that was performed on the table, and the region.You can check this out for other services as well to see service specific information.
This concludes this section. You may continue on to the next section.