Module 5 — Dynatrace Davis alerting in Managed: anomalies, problems and alerting profiles

Dynatrace Davis alerting in Managed: understand anomaly detection, read a problem and build alerting profiles that notify the right team at the right time.
Module 5 — Davis : alerting en Managed, anomalies, problèmes et profils d'alerte

Module 5 of the Dynatrace Managed course covers Dynatrace Davis alerting, from anomaly detection to notifying the right team.

Course: Dynatrace Managed · Module 5/8 · Duration: ~60 min · Level: Intermediate

Objectives:

  • Understand how Davis detects an anomaly and turns it into a problem.
  • Distinguish automatic thresholds (baselines) from static thresholds, and know when to adjust one or the other.
  • Read a problem: severity, impacted entities, root cause, timeline.
  • Build an alerting profile and link it to a notification integration.

Prerequisites: modules 1 to 4 of the course, a demo or test Managed environment with a few hosts and services monitored by OneAgent, and an account with permission to change the environment settings.

Monitoring lots of things is pointless if nobody is told at the right time. Module 5 of the Dynatrace Managed course is dedicated to Dynatrace Davis alerting in Managed: how the analysis engine spots abnormal behaviour and groups it into problems with a root cause, and then how you decide who is notified, when and through which channel.

We stay in the classic interface of a Managed environment here. If you have skipped steps, the course overview sets out the recommended order, and module 4 taught you how to navigate the data that Davis will now watch.

How does Dynatrace Davis alerting work in Managed?

The whole chain has four links. Remember it, because it structures the rest of the module:

Link Role Where to configure it
Anomaly detection Spot a deviation (response time, error rate, saturation, unavailability) The environment’s anomaly detection settings
Event Record the deviation on a specific entity (host, process, service, application) Generated automatically or by a custom event
Problem Group related events, identify the root cause and the impact Calculated by Davis from the Smartscape topology
Notification Alert the right team through the right channel Alerting profiles + notification integrations

The key point: you do not alert on events, you alert on problems. A saturated database can generate a dozen cascading events; Davis attaches them to a single problem, which avoids a storm of messages.

Understanding anomaly detection

Automatic baselines

For services and applications, Davis learns the usual behaviour of each entity: response time, failure rate, request volume, taking variations by hour and day into account. A degradation alert fires when the deviation from this baseline is significant and sustained, not at the first isolated spike.

Static thresholds

For some metrics, a fixed value makes more sense: a disk above 90%, a contractual response time SLA, a maximum number of connections. You can then replace the automatic mode with a fixed threshold, at environment level or for a specific entity. Rule of thumb: start with the default settings, observe for two to three weeks, then only adjust the entities that generate noise or that have explicit business requirements.

Custom events for alerting

When no built-in detection covers your need (a business metric, a metric from an extension, an application counter), you can define a custom event on a metric: a condition, an observation period, a severity. The event then feeds Davis like any other and can open a problem.

Maintenance windows

A planned release or a server restart produces expected anomalies. Maintenance windows let you suspend alerts, or even detection, on a given scope and for a given period. Always declare them.

Reading a problem

Open the environment’s problem list. Each problem shows a severity (availability, error, slowdown, resource, custom alert), an open or closed status, the impacted entities and, when Davis has identified it, the root cause. The detailed view shows the timeline and the topological path from cause to impact.

Three reading habits:

  1. Start from the impact: how many users or requests are affected? That is what sets the priority.
  2. Go back to the proposed root cause, then check in the metrics of the entity concerned (module 4) that the explanation holds.
  3. Look at what changed: deployment, restart, configuration change.

Configuring alerting profiles

An alerting profile is a filter: it decides which problems trigger a notification. It usually combines:

  • a scope, most often a management zone (covered in module 6);
  • rules per severity, each with a delay: the problem must stay open for a certain number of minutes before alerting;
  • filters on the tags of the entities concerned, for example to target only the front-end tier.

A reasonable starting point for a critical application:

Profile: prod-payment-oncall
  Scope         : management zone prod-payment
  Availability  : alert immediately
  Error         : alert after 5 min
  Slowdown      : alert after 15 min
  Resource      : alert after 30 min
  Custom alert  : alert after 5 min

Notification integrations (email, webhook, ITSM or on-call management tools) are then attached to a profile. On the Managed side, remember that it is the cluster that sends the notifications: it must be able to reach your SMTP server and the webhook URLs. The article Alerting profiles and notification integrations in Managed details these network prerequisites and the SMTP configuration in the CMC.

Hands-on exercise

Step 1 — List recent problems

Open the problem list of your test environment and filter on the last 7 days. For three of them, note the severity, the proposed root cause and the duration.

Expected result: a small table showing which types of problem dominate in your environment.

Step 2 — Trigger a controlled anomaly

On a test host, generate CPU load for about fifteen minutes, for example:

timeout 900 sh -c 'while :; do :; done' &
timeout 900 sh -c 'while :; do :; done' &

Expected result: a CPU saturation event on the host, then a resource problem if the condition persists.

Step 3 — Adjust a threshold

In the anomaly detection settings for this host only, lower the CPU saturation threshold, then rerun the load.

Expected result: the problem opens earlier. Then restore the default value.

Step 4 — Create a profile and a notification

Create an alerting profile limited to the management zone or tag of your test host, with a “resource” rule and no delay. Attach an email or webhook integration to it, pointing to a test mailbox or receiver.

Expected result: when you rerun the load, you receive a notification when the problem opens, then a second one when it closes if the integration is configured that way.

Step 5 — Test a maintenance window

Declare a 30-minute maintenance window on the host and rerun the load.

Expected result: no notification is sent during the window.

Quiz

  1. What does an alerting profile trigger a notification on?
    A. Every raw event · B. The problems calculated by Davis · C. Every metric that exceeds its average
  2. What is the main benefit of automatic baselines?
    A. They take normal variations by hour and day into account · B. They disable alerting at night · C. They replace management zones
  3. What is the delay of a severity rule in an alerting profile for?
    A. Delaying anomaly detection · B. Only alerting if the problem stays open long enough · C. Limiting problem retention
  4. A webhook notification never arrives in Managed. What do you check first?
    A. The OneAgent licence · B. Network connectivity from the cluster to the target URL · C. The hosts’ monitoring mode

Answers

  1. B. Davis groups events into problems, and it is these problems that alerting profiles filter.
  2. A. The learned baseline includes seasonality, which avoids alerting on a usual spike.
  3. B. The delay filters out short problems that resolve themselves and reduces noise.
  4. B. In Managed, notifications are sent from the cluster: it must be able to reach the target.

Further reading

← Module 4: Exploring the data · Course overview · Module 6: Tags, management zones and access →

Frequently asked questions

Should baselines be disabled to avoid false alerts?

Rarely. It is better to lengthen the delays in alerting profiles and adjust the sensitivity of the noisy entities only. Disabling detection also removes the real alerts.

Can a problem concern several teams?

Yes. If the impacted entities belong to several management zones, the problem is visible in each of them and can trigger several alerting profiles, and therefore several notifications.

Can profiles and notifications be managed as code?

Yes, through the environment’s configuration APIs or configuration-as-code tools. That is the subject of module 8, which shows how to version these settings.

With Davis properly tuned and alerting profiles aligned with your teams, every notification should call for an action. The next module lays the foundations that make this targeting possible: tags, management zones and access rights.

span.observer is an independent community blog, not affiliated with Dynatrace LLC. This course does not replace the official Dynatrace University training and certifications. Check features in the official documentation for your version.

Total
0
Shares
3 comments
Leave a Reply

Your email address will not be published. Required fields are marked *

Previous Post
Module 4 — Explorer : données Managed, hôtes, services, PurePath Dynatrace et problèmes

Module 4 — Exploring Managed data: hosts, services, Dynatrace PurePath and problems

Next Post
Module 6 — Tags, zones, accès en Managed : tagging automatique, management zones et permissions

Module 6 — Dynatrace management zones, tags and access in Managed: automatic tagging and permissions

Related Posts