How to choose SLIs: a practical method, from DQL queries to objectives

How to choose SLIs that matter: start from user journeys, define good events, measure 14 days of history in DQL and set objectives your team can keep.
Choisir ses SLI : la méthode pratique, des requêtes DQL aux objectifs

A team creates twenty SLOs in one afternoon, one per available metric. Three months later nobody looks at them: they are all green while users complain. The problem is not the tool, it is knowing how to choose SLIs.

Goal of this tutorial: apply a five-step method to choose SLIs from the user journey, write them in DQL, measure their history and derive realistic objectives. The queries target Dynatrace SaaS (Grail); the approach also applies to Dynatrace Managed, with metric expressions instead of DQL.

For the general SRE context, see AI and reliability: what SREs need to know.

What makes a good SLI

A good SLI ticks three boxes:

  • it reflects what the user experiences (a page that does not load), not the state of a machine (a CPU at 90%);
  • it is expressed as a ratio: good events / valid events, between 0 and 100%;
  • it moves when the service degrades, and only then.

CPU, memory or pod count remain useful for diagnosis. They are not SLIs.

Prerequisites

  • SaaS: access to Notebooks and read access to spans and service metrics in Grail.
  • Managed: access to Data Explorer and the builtin:service.* metrics.
  • At least 14 days of data on the service, to cover two weekly cycles.
  • One hour with the product owner: the method starts with a conversation, not a query.

How to choose SLIs in 5 steps

Step 1 — List the critical journeys

Ask: “If this action fails, who calls support?” Keep three to five journeys at most. Example for an online shop:

Journey Entry service What the user expects
Search for a product search Fast results
Add to basket cart An action that succeeds
Pay checkout A confirmed payment, without error

Step 2 — Match an SLI type to each journey

  • Availability: requests without error / valid requests.
  • Latency: requests under a threshold / valid requests.
  • Quality: complete responses (no degraded result) / responses.
  • Freshness (asynchronous processing): data updated less than N minutes ago / data.

For search: latency. For the basket: availability. For payment: availability and latency.

Step 3 — Define valid and good events precisely

This is the step most often rushed. Write it down:

  • valid: server request on the payment endpoint, excluding health checks and synthetic traffic;
  • good: response without a server error (a 4xx error caused by the client remains “good” on the service side).

This definition decides what consumes the error budget. Have it validated by the product team.

Step 4 — Measure the history in DQL

Availability per day over 14 days, from spans:

fetch spans, from: now() - 14d
| filter service.name == "checkout" and span.kind == "server"
| filterOut contains(url.path, "/health")
| makeTimeseries {
    total = count(),
    errors = countIf(span.status_code == "error")
  }, interval: 1d
| fieldsAdd availability = 100 * (total[] - errors[]) / total[]

Latency distribution, to choose the threshold:

fetch spans, from: now() - 14d
| filter service.name == "search" and span.kind == "server"
| summarize p50 = percentile(duration, 50),
            p90 = percentile(duration, 90),
            p99 = percentile(duration, 99)

Critical lines: filterOut removes health checks from the denominator. The url.path field depends on your instrumentation; adapt it. If your services are instrumented by OneAgent, the dt.service.request.count metric and its counterpart dt.service.request.failure_count give the same result faster (names to check in the Dynatrace documentation). Over 14 days, a query on spans can be costly: run it once, not in a loop.

On Managed, open Data Explorer with builtin:service.errors.server.rate and builtin:service.response.time (percentiles), split by day.

Step 5 — Derive realistic objectives

Simple rule: start from what the service already does, slightly below, then tighten if users demand it.

  • Measured availability between 99.82% and 99.97% per day: a 99.5% objective over 28 days is achievable and leaves a usable budget.
  • Latency p90 at 320 ms, p99 at 1.1 s: a 500 ms threshold with a 95% objective reflects everyday experience without alerting on the tail of the distribution.

A 100% objective forbids any change. An objective above current performance is in permanent alert. Both are useless.

The infrastructure SLI trap

A Kubernetes node can fail without any user noticing, thanks to replicas. Conversely, a saturated database can slow down every payment while all pods are “healthy”. So keep infrastructure signals in your dashboards and diagnostic alerts, and reserve SLIs for user-facing signals. This separation reduces on-call noise and makes the error budget credible to product.

How to check that it works

  1. Replay a past incident: on that day, the SLI must drop clearly. If it stays flat, the SLI does not measure what users experience.
  2. Check that a quiet day stays above the objective: otherwise, the objective is too high.
  3. Create the SLO and look at the remaining error budget over 28 days: it must be positive without sitting at 100% all the time.

Creating the SLOs themselves is covered in creating an OpenTelemetry SLO in DQL.

Best practices

  • Three to five SLIs per service at most: beyond that, nobody reads them.
  • Measure as close to the user as possible (load balancer, entry service, RUM) rather than at the database.
  • Document each SLI definition next to its query, with the review date.
  • Review objectives every quarter with product, blamelessly: a missed SLO is data, not a fault.
  • Then wire up SLO burn rate alerts to be warned before the budget runs out.

Frequently asked questions

Do I need a different SLI per endpoint?

No. Group the endpoints of the same journey. Only isolate a critical endpoint whose behaviour differs strongly (payment, for example).

Percentile or ratio for latency?

The “requests under the threshold” ratio accumulates cleanly over a window and gives an error budget. The percentile remains useful to choose the threshold, in step 4.

What if I do not have 14 days of history?

Start with a cautious objective, mark it “provisional” and revise it after two weeks of data.

Conclusion

Choosing SLIs means starting from the user, defining good events precisely, measuring the history, then setting an achievable objective. An afternoon of method beats twenty SLOs that nobody reads.

Recommended next step: book the hour with your product owner this week, list the three most critical journeys, and run the step 4 queries on them. You will have defensible SLIs and objectives before the next planning session.

Keep each SLI definition, its query and its review date in the same place, so that the next person on call understands what the number means without asking.

span.observer is an independent community blog, not affiliated with Dynatrace LLC. Dynatrace is a trademark of Dynatrace LLC. SaaS features evolve quickly: check their availability in your environment and in the official documentation.

Total
0
Shares
1 comment
Leave a Reply

Your email address will not be published. Required fields are marked *

Previous Post
SLO OpenTelemetry dans Dynatrace : disponibilité et latence en DQL

OpenTelemetry SLO in Dynatrace: availability and latency in DQL

Next Post
SLO Dynatrace Terraform : un SLO pour chaque service, généré en masse

Dynatrace SLO Terraform: one SLO per service, generated at scale

Related Posts