Java tracing walkthrough

This section will explain how we implemented tracing using OpenTelemetry for the PetSearch Java service. Multiple getting started examples can be found on the official AWS Distro for OpenTelemetry documentation for Go, Java, Javascript and Python, as well as other platform configuration examples.

Learn more about the OpenTelemetry specification which guides languages implementations here.

X-Ray Setup

All traces generated by the application are collected with the OpenTelemetry collector setup on ECS as sidecar.

ADOT setup

Tracing with AWS auto instrumentation agent

To collect tracing information from a Java application using the auto instrumentation agent, we needed to download the Java agent and use the -javaagent flag to the JVM. You can see this setup in the project’s Dockerfile:

ADD https://github.com/aws-observability/aws-otel-java-instrumentation/releases/download/v0.18.0-aws.1/aws-opentelemetry-agent.jar /app/aws-opentelemetry-agent.jar
ENV JAVA_TOOL_OPTIONS "-javaagent:/app/aws-opentelemetry-agent.jar"

By default, the OpenTelemetry Java agent uses the OTLP exporter to exports the traces in OTLP format. There are several settings that can be customized for our specific needs using either system properties or environment variables.

In the snippet below, the value of the OTEL_RESOURCE_ATTRIBUTES environment variable we are configuring is the service name as PetSearch. This effect of this can be seen on the Service Map.

The value of the OTEL_IMR_EXPORT_INTERVAL environment variable indicates the frequency at which the OTLP metrics will be exported to the OTEL Collector.

The value of the OTEL_EXPORTER_OTLP_ENDPOINT environment variable indicates the endpoint at which the OTEL Collector is listening on. The default configuration for the AWS OTel Collector can be seen here

ENV OTEL_RESOURCE_ATTRIBUTES "service.name=PetSearch"
ENV OTEL_IMR_EXPORT_INTERVAL "10000"
ENV OTEL_EXPORTER_OTLP_ENDPOINT "http://localhost:55680"

PetSearch service is built using Spring framework, and we use the AWS SDK to to make calls to several AWS services. The AWS SDK is supported by OpenTelemetry for auto instrumentation, so any request handled by Spring MVC and the resulting calls to AWS services are automatically traced and published.

The screenshot below shows the metadata captured using auto-instrumentation.

Trace overview

Custom tracing attributes and Spans

When we used the auto instrumentation agent and one of the supported libraries, all the tracing information was already available. However, if you are using a library that doesn’t support auto-instrumentation or if you want to enrich the information provided by default, you can add custom tracing attributes:

Span.current().setAttribute(e.getKey(), e.getValue());

The code below shows how you can add a span. You can see this in the search method in the SearchController.java file.

Span span = metricEmitter.spanBuilder("Scanning DynamoDB Table").startSpan();

try(Scope scope = span.makeCurrent()) {

    List<Pet> result = ddbClient.scan(
        buildScanRequest(petType, petColor, petId))
            .getItems().stream().map(this::mapToPet)
            .collect(Collectors.toList());
    return result;

} catch (Exception e) {
    span.recordException(e);
    logger.error("Error while searching, building the resulting body", e);
    throw e;
} finally {
    span.end();
}

The effect of this span can be seen when viewing a trace as shown below. Show span

This concludes this section. You may continue on to the next section.