A FastAPI service that slows down without leaving a trail is an on-call nightmare: the logs say “200 OK”, the users say “it’s slow”. Instrumenting OpenTelemetry Python FastAPI gives you traces, metrics and shared context, without installing OneAgent on the host, which is often impossible on managed containers or serverless platforms.
Goal: by the end of this tutorial, your FastAPI service sends its spans and metrics over OTLP to Dynatrace, directly or through a Collector, and you know how to verify ingestion. It applies to Dynatrace SaaS (Grail storage, DQL queries) and to Dynatrace Managed (the environment’s OTLP endpoint, analysis in the classic screens). Differences are flagged at each step.
If your spans do not show up at the end, keep the troubleshooting guide OpenTelemetry traces not showing in Dynatrace close at hand.
The problem: an opaque Python API
Without instrumentation, a slow request looks like any other. You see neither the 800 ms outbound HTTP call nor the unindexed SQL query. MTTR grows because the on-call engineer rebuilds the path by hand. With traces, the duration of every step is one click away, and you can build a reliable latency SLI.
Prerequisites
- Python 3.10 or later (minimum required by current OpenTelemetry SDK versions), FastAPI and Uvicorn.
- A Dynatrace SaaS or Managed environment with OTLP ingestion available.
- An API access token with the
openTelemetryTrace.ingest,metrics.ingestandlogs.ingestscopes (confirm the exact names in your token creation screen). - Optional: an OpenTelemetry Collector, recommended in production (see the OpenTelemetry Collector on Kubernetes).
Instrumenting OpenTelemetry Python FastAPI step by step
Step 1: a minimal application
# main.py
from fastapi import FastAPI
import httpx
app = FastAPI()
@app.get("/health")
async def health():
return {"status": "ok"}
@app.get("/orders/{order_id}")
async def get_order(order_id: int):
async with httpx.AsyncClient() as client:
r = await client.get("https://httpbin.org/delay/1")
return {"order_id": order_id, "upstream": r.status_code}
Step 2: install the OpenTelemetry packages
pip install fastapi uvicorn httpx \
opentelemetry-distro \
opentelemetry-exporter-otlp-proto-http
# installs the instrumentations matching the libraries it detects
# (fastapi, httpx, requests, sqlalchemy...)
opentelemetry-bootstrap -a install
Critical point: use the HTTP/protobuf exporter. The Dynatrace OTLP endpoint accepts OTLP over HTTP; gRPC goes through a Collector.
Step 3: configure export to Dynatrace
export OTEL_SERVICE_NAME="orders-api"
export OTEL_RESOURCE_ATTRIBUTES="service.version=1.4.2,deployment.environment.name=prod"
# SaaS
export OTEL_EXPORTER_OTLP_ENDPOINT="https://<environment-id>.live.dynatrace.com/api/v2/otlp"
# Managed (instead of the line above)
# export OTEL_EXPORTER_OTLP_ENDPOINT="https://<your-domain>/e/<environment-id>/api/v2/otlp"
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Api-Token%20<your-token>"
export OTEL_EXPORTER_OTLP_PROTOCOL="http/protobuf"
export OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE="delta"
export OTEL_TRACES_EXPORTER="otlp"
export OTEL_METRICS_EXPORTER="otlp"
export OTEL_LOGS_EXPORTER="otlp"
# logs: the opentelemetry-instrumentation-logging package (installed by bootstrap)
# attaches the handler by default in recent versions
Three lines deserve attention:
OTEL_EXPORTER_OTLP_HEADERS: the space betweenApi-Tokenand the token is encoded as%20, otherwise the header is misread.OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE=delta: Dynatrace expects delta temporality; cumulative metrics are rejected. Details in OTLP metrics temporality.OTEL_SERVICE_NAME: this is the service name in Dynatrace. Without it you getunknown_service.
The exporter appends /v1/traces, /v1/metrics and /v1/logs to the base endpoint itself. Do not write them into OTEL_EXPORTER_OTLP_ENDPOINT.
For logs, the old OTEL_PYTHON_LOGGING_AUTO_INSTRUMENTATION_ENABLED=true variable enables the SDK LoggingHandler, deprecated since opentelemetry-sdk 1.40. With recent versions, leave it unset: the logging instrumentation installs its own handler (can be disabled with OTEL_PYTHON_LOG_AUTO_INSTRUMENTATION=false). On older versions, the old variable is still required.
Step 4: start the instrumented application
opentelemetry-instrument uvicorn main:app --host 0.0.0.0 --port 8000
Do not use --reload with auto-instrumentation: the restarted child process does not always inherit the instrumentation. With several workers (Gunicorn), initialise the SDK after the fork, in a post_fork hook, otherwise the background exporters misbehave.
Step 5 (alternative): explicit instrumentation in code
If you prefer to control initialisation, for example to add business spans, configure the SDK in code:
# telemetry.py
from opentelemetry import trace
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.instrumentation.fastapi import FastAPIInstrumentor
from opentelemetry.instrumentation.httpx import HTTPXClientInstrumentor
def setup_tracing(app):
resource = Resource.create({
"service.name": "orders-api",
"service.version": "1.4.2",
})
provider = TracerProvider(resource=resource)
# endpoint and headers are read from the OTEL_EXPORTER_OTLP_* variables
provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter()))
trace.set_tracer_provider(provider)
FastAPIInstrumentor.instrument_app(app, excluded_urls="health")
HTTPXClientInstrumentor().instrument()
# in main.py
from opentelemetry import trace
from telemetry import setup_tracing
setup_tracing(app)
tracer = trace.get_tracer("orders")
@app.post("/orders")
async def create_order(payload: dict):
with tracer.start_as_current_span("validate-order") as span:
span.set_attribute("order.items", len(payload.get("items", [])))
return {"created": True}
Critical line: excluded_urls="health" removes health probes from traces. They skew latency percentiles and cost volume for nothing.
Step 6: go through a Collector in production
Point the application at the local Collector (http://otel-collector:4318) and let the Collector hold the token. You gain batching, retries and a single configuration point. The Collector then exports with otlphttp:
exporters:
otlphttp/dynatrace:
endpoint: "https://<environment-id>.live.dynatrace.com/api/v2/otlp"
headers:
Authorization: "Api-Token ${env:DT_API_TOKEN}"
How to check that it works
- Application side: for a local test, set
OTEL_TRACES_EXPORTER=otlp,console. Spans are printed to standard output; if they appear there but not in Dynatrace, the issue is the export (URL, token, proxy). - Generate traffic:
for i in $(seq 1 20); do curl -s localhost:8000/orders/$i; done. - Dynatrace SaaS: open the Distributed Tracing app, filter on the
orders-apiservice, or run in a notebook:fetch spans | filter service.name == "orders-api" | summarize count(), avg(duration), by: {span.name} - Dynatrace Managed: open the distributed traces view (exact name depends on your version) and filter on the service. The OTel service may take a few minutes to appear.
- HTTP errors: a 401 or 403 in the exporter logs points to a wrong token or scope; a 404 to a wrong endpoint URL.
Best practices
- Do not mix OneAgent and the OTel SDK on the same process without thinking: duplicates are likely (see OneAgent and OpenTelemetry duplicate spans).
- Set
service.name,service.versionand the environment: they make deployments comparable and speed up diagnosis. - Keep the token out of the code: environment variable or Kubernetes secret.
- Link your logs to traces to jump from an error to its request: see OpenTelemetry logs with trace_id.
- Build a latency SLI on server spans, then a burn rate alert.
Frequently asked questions
Do I need OneAgent to see a FastAPI application in Dynatrace?
No. OTLP export is enough for traces, metrics and logs. OneAgent additionally brings automatic host and process discovery; without it, enrichment relies on the resource attributes you provide.
Why do my metrics not arrive when traces do?
Usually the temporality is still cumulative or the token lacks the metrics ingest scope. Check the temporality preference variable and the scopes.
Does auto-instrumentation work with Gunicorn and several workers?
Yes, provided the SDK is initialised in each worker after the fork. The OpenTelemetry Python documentation covers this case; test it before production.
Conclusion
A few environment variables and one launch command are enough to make a FastAPI API observable in Dynatrace, on SaaS and Managed alike. Start with auto-instrumentation, verify ingestion, then add business spans where on-call needs them. For endpoint details, refer to the Dynatrace OpenTelemetry documentation and the OpenTelemetry Python documentation.
span.observer is an independent community blog, not affiliated with Dynatrace LLC. Dynatrace is a trademark of its owner.