OpenTelemetry Java agent for Dynatrace: instrument an application and send its data

OpenTelemetry Java agent for Dynatrace: download it, configure OTLP export, deploy on a VM or Kubernetes and check traces, metrics and logs arrive.
Agent OpenTelemetry Java : instrumenter une application et l'envoyer à Dynatrace

Your team wants standard, portable instrumentation for a Spring Boot application without changing its code. It has no OneAgent on these containers, or it has chosen OpenTelemetry as the platform standard. The OpenTelemetry Java agent for Dynatrace meets that need: one JAR file added at JVM startup, and traces, metrics and logs flow over OTLP to Dynatrace.

Goal of this tutorial: download the agent, configure it with environment variables to send all three signals to Dynatrace, deploy it on a VM and then on Kubernetes, and check that the data arrives. The content applies to both Dynatrace SaaS and Dynatrace Managed; only the endpoint URL changes.

If OneAgent already monitors the same process, first read OneAgent and OpenTelemetry: avoiding duplicate spans. Two auto-instrumentations in one JVM produce duplicate spans.

Prerequisites

  • Java 8 or later (check the compatibility matrix of the agent version you choose).
  • A Dynatrace API token with the openTelemetryTrace.ingest, metrics.ingest and logs.ingest scopes.
  • The OTLP endpoint URL:
    • SaaS: https://<env-id>.live.dynatrace.com/api/v2/otlp;
    • Managed: https://<your-domain>/e/<env-id>/api/v2/otlp;
    • through an ActiveGate: https://<activegate>:9999/e/<env-id>/api/v2/otlp.
  • Outbound HTTPS access from the application, or an intermediate Collector.

Steps to set up the OpenTelemetry Java agent for Dynatrace

Step 1 — Download the agent

mkdir -p /opt/otel
curl -fsSL -o /opt/otel/opentelemetry-javaagent.jar \
  https://github.com/open-telemetry/opentelemetry-java-instrumentation/releases/latest/download/opentelemetry-javaagent.jar

In production, replace latest with a pinned version (for example download/v2.x.y/) so every build is reproducible, and test upgrades like any other dependency.

Step 2 — Configure export to Dynatrace

Create an environment file /etc/checkout/otel.env:

OTEL_SERVICE_NAME=checkout
OTEL_RESOURCE_ATTRIBUTES=service.version=1.4.2,deployment.environment.name=production
OTEL_EXPORTER_OTLP_ENDPOINT=https://abc12345.live.dynatrace.com/api/v2/otlp
OTEL_EXPORTER_OTLP_HEADERS=Authorization=Api-Token%20dt0c01.XXXX.YYYY
OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
OTEL_TRACES_EXPORTER=otlp
OTEL_METRICS_EXPORTER=otlp
OTEL_LOGS_EXPORTER=otlp
OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE=delta
OTEL_PROPAGATORS=tracecontext,baggage

Critical lines:

  • OTEL_EXPORTER_OTLP_ENDPOINT: the base URL, without /v1/traces. The agent appends /v1/traces, /v1/metrics and /v1/logs itself.
  • OTEL_EXPORTER_OTLP_HEADERS: the space between Api-Token and the token is encoded as %20, as the OpenTelemetry environment variable specification requires.
  • http/protobuf: the Dynatrace OTLP API expects HTTP with protobuf. For gRPC, go through a Collector.
  • delta: Dynatrace expects metrics with delta temporality. Without this line, counters may be rejected: the full story is in OTLP metrics temporality.

Never commit this file with the token: inject it from your secrets manager.

Step 3 — Start the JVM with the agent (VM or bare metal)

A minimal systemd unit:

[Unit]
Description=Checkout service
After=network-online.target

[Service]
EnvironmentFile=/etc/checkout/otel.env
ExecStart=/usr/bin/java -javaagent:/opt/otel/opentelemetry-javaagent.jar -jar /opt/checkout/checkout.jar
Restart=on-failure

[Install]
WantedBy=multi-user.target
sudo systemctl daemon-reload
sudo systemctl restart checkout

The -javaagent option must come before -jar, otherwise it is passed to the application as an argument and ignored.

Step 4 — The Kubernetes variant

Copy the agent into a shared volume with an init container, then load it with JAVA_TOOL_OPTIONS:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: checkout
spec:
  replicas: 2
  selector:
    matchLabels: { app: checkout }
  template:
    metadata:
      labels: { app: checkout }
    spec:
      volumes:
        - name: otel
          emptyDir: {}
      initContainers:
        - name: otel-agent
          image: curlimages/curl:8.10.1
          command: ["curl", "-fsSL", "-o", "/otel/opentelemetry-javaagent.jar",
                    "https://github.com/open-telemetry/opentelemetry-java-instrumentation/releases/download/v2.10.0/opentelemetry-javaagent.jar"]
          volumeMounts: [{ name: otel, mountPath: /otel }]
      containers:
        - name: app
          image: registry.example.com/checkout:1.4.2
          envFrom:
            - secretRef: { name: checkout-otel }
          env:
            - name: JAVA_TOOL_OPTIONS
              value: "-javaagent:/otel/opentelemetry-javaagent.jar"
          volumeMounts: [{ name: otel, mountPath: /otel }]

The checkout-otel secret holds the variables from step 2. The agent version and curl image are examples: use a released version you have validated. With the OpenTelemetry Operator, the Instrumentation resource automates this injection. For centralised export, point the endpoint at an OpenTelemetry Collector that holds the token instead.

How to check it works

  1. Startup logs: the JVM prints a line such as opentelemetry-javaagent - version: 2.x.y. If it is missing, the agent is not loaded.
  2. Export errors: search the logs for Failed to export. A 401 or 403 means a wrong token or scope; a 404 means a wrong endpoint URL.
  3. One-off debug mode: OTEL_JAVAAGENT_DEBUG=true prints spans to the console. Remove it afterwards, it is very verbose.
  4. In Dynatrace SaaS, run this in a Notebook:
fetch spans, from: now() - 15m
| filter service.name == "checkout"
| summarize requetes = count(), by: { span.name }
| sort requetes desc
| limit 20

You should see the application’s HTTP routes. For logs: fetch logs | filter service.name == "checkout" | limit 20. On Managed, the service appears in the services list and the traces in the distributed traces view; metrics are visible in the data explorer. The exact screens depend on your version: see the Dynatrace OpenTelemetry documentation.

Troubleshooting

  • No data, no error: the image does not read JAVA_TOOL_OPTIONS (some base images redefine it in their startup script). Check with kubectl exec that the JVM prints “Picked up JAVA_TOOL_OPTIONS”.
  • Traces present, metrics missing: temporality stayed cumulative, or the token lacks the metrics.ingest scope.
  • Duplicated logs: the application already writes to standard output, collected by another agent. Choose a single path for logs.
  • Slower startup: the agent adds cost to JVM startup. Adjust readiness probes if they fail after adding the agent.
  • TLS errors: behind a corporate proxy, the JVM must trust the proxy certificate (Java truststore), otherwise export fails silently from the application’s point of view.

Best practices

  • One stable service.name per service: it carries SLOs and alerts, see SLO burn rate alerts.
  • Set service.version on every deployment: comparing two versions becomes instant.
  • Disable unneeded instrumentations rather than filtering afterwards (OTEL_INSTRUMENTATION_<NAME>_ENABLED=false).
  • Sample with care: overly aggressive client-side sampling distorts SLIs computed from spans.
  • Never run the OpenTelemetry agent in a JVM already instrumented by OneAgent.

Frequently asked questions

Do I need to change the application code?

Not for basic instrumentation: the agent automatically instruments common frameworks (HTTP, JDBC, messaging). Add the OpenTelemetry API only for business spans or attributes.

Direct export or through a Collector?

Direct export suits a test or a handful of services. Beyond that, a Collector centralises the token, filtering and retries.

Are application logs captured?

The agent hooks into the appenders of common logging libraries (Logback, Log4j) when log export is on, with the related trace ID. Check your library in the list of supported instrumentations.

Conclusion

With one JAR and about ten variables, the OpenTelemetry Java agent sends traces, metrics and logs to Dynatrace without touching the code. Pin the version, enforce delta temporality, keep one instrumentation owner per process and verify in Grail. Your SLOs then rest on standard, portable data.

span.observer is an independent community blog, not affiliated with Dynatrace LLC. Dynatrace is a trademark of Dynatrace LLC. Features evolve quickly: check their availability in your environment and in the official documentation.

Total
0
Shares
Leave a Reply

Your email address will not be published. Required fields are marked *

Previous Post
OneAgent OpenTelemetry doublons : éviter les spans en double et le contexte cassé

OneAgent OpenTelemetry duplicate spans: how to avoid them and keep context intact

Next Post
Temporalité métriques OTLP vers Dynatrace : delta ou cumulative, l'erreur qui fait disparaître vos compteurs

OTLP metrics delta temporality in Dynatrace: the cumulative mistake that makes counters vanish

Related Posts