Module 7 — Dynatrace Managed cluster operations: health, backups, updates and storage

Dynatrace Managed cluster operations: monitor node health, test your backups, plan updates in the right order and keep storage and retention under control.
Module 7 — Exploitation : cluster Managed, santé, sauvegardes, mises à jour et stockage

Module 7 of the course covers Dynatrace Managed cluster operations: the health, backup, update and storage routine of the platform team.

Course: Dynatrace Managed · Module 7/8 · Duration: ~75 min · Level: Advanced

Objectives:

  • Read the health status of a Managed cluster and its nodes.
  • Set up and check the cluster backup.
  • Plan updates in the right order: cluster, ActiveGate, OneAgent.
  • Track storage usage and act on retention.
  • Formalise a weekly and monthly operations routine.

Prerequisites: modules 1 to 6, in particular module 2 on architecture; a test Managed cluster, a Cluster Management Console (CMC) administrator account and SSH access to the nodes.

Once the environments are in place and the teams are on board, the cluster has to run without surprises for years. Module 7 of the Dynatrace Managed course covers Dynatrace Managed cluster operations: monitoring its health, making sure a usable backup exists, applying updates without interrupting monitoring and keeping storage under control. This is the daily work of the platform team, and it is what sets Managed apart from a SaaS offering where the vendor takes care of it.

Module 2 introduced the nodes and their components; we build on that vocabulary here. The course overview lists all the steps.

What do Dynatrace Managed cluster operations involve?

The work can be summarised in four areas, each with its own frequency:

Area Question Recommended frequency
Health Are all nodes and services responding? Continuously (external monitoring) + a daily glance
Backup Is the last backup complete and restorable? Weekly check, periodic restore test
Updates Are the cluster and the agents on a supported version? Regular cycle, for example monthly
Storage How many days of headroom are left before saturation? Weekly

Monitoring cluster health

The CMC home page gives the overview: list of nodes, the status of each, resource usage, alerts on the cluster. On each node, several services work together (Dynatrace Server, storage databases for metrics and events, transaction storage, web interface); a node can be reachable while one of its components is degraded.

Three good practices:

  • Do not rely only on the cluster to monitor itself: add simple external monitoring (HTTPS availability of the interface, disk space, status of system services).
  • Monitor the certificates exposed to users and agents, whose expiry causes abrupt outages.
  • Keep an odd number of nodes (three at minimum) when high availability is expected, and check that no node stays offline for long.

On a node, a few system commands give a first reading:

df -h /var/opt/dynatrace-managed
free -h
uptime
systemctl list-units --type=service | grep -i dynatrace

The storage path above is the default one; adapt it if you chose another location during installation. The article Monitoring the health of your Managed cluster details a complete routine.

Backing up and knowing how to restore

The cluster backup is enabled in the CMC and writes to shared storage, typically an NFS share mounted on all nodes at the same path. It covers the configuration (naming rules, tags, management zones, alerting profiles, and so on) and, depending on the options chosen, metrics and user sessions. Transaction storage, however, is never backed up: after a restore, expect gaps in PurePath traces.

A backup is only worth something if the restore has been tested. Plan at least one dry-run restore a year on test infrastructure, with the same installer and the same version as the original cluster. Also keep a copy outside the main datacenter. The step-by-step procedure is described in Backing up and restoring a Dynatrace Managed cluster.

Planning updates

The ordering rule is simple: cluster first, then ActiveGate, OneAgent last. A component must not be newer than the cluster it connects to. If the cluster is connected to Mission Control, the CMC can fetch new versions and apply them in a maintenance window; in offline mode, the packages are transferred and applied manually.

For OneAgents, proceed in waves: test, then a sample of production, then the rest. Automatic updates can be set per environment or per host group, with a maintenance window. See Planning Managed cluster and OneAgent updates.

Keeping storage under control

Storage grows with the number of hosts, the volume of captured transactions and the retention period. The levers are, in order:

  1. Measure usage per node and identify the most demanding environments.
  2. Adjust retention per environment and per data type, within the permitted limits.
  3. Reduce at the source: switch some hosts to infrastructure-only mode, limit the capture of data of little use.
  4. Extend the storage or add a node when growth is sustained.

A cluster whose disks are filling up ends up protecting itself by reducing what it keeps: better to anticipate. The article Data retention and storage management in Managed goes deeper into the subject.

Hands-on exercise

Step 1 — Establish the baseline status

In the CMC of your test cluster, record each node’s status, version and disk usage. Complete this with the df -h command on each node.

Expected result: a dated status sheet that will serve as a point of comparison.

Step 2 — Check the backup

Check in the CMC that backup is enabled, then on a node list the contents of the share:

ls -lh /mnt/dynatrace-backup
du -sh /mnt/dynatrace-backup/*

Adapt the path to your mount point.

Expected result: a recent backup, consistent in size with the previous ones.

Step 3 — Prepare an update

Find the next available version, read its release notes and check the required free disk space. Write the plan: window, order (cluster, then ActiveGate, then OneAgent), rollback criteria.

Expected result: a one-page update plan that your manager can sign off.

Step 4 — Estimate the storage headroom

Compare the usage from step 1 with the usage one to two weeks ago (if you have it) to estimate the growth per day, then the number of days left before reaching 80% of the disk.

Expected result: a headroom figure in days, to track every week.

Step 5 — Write the routine

Put together a weekly and monthly checklist from the previous steps.

Expected result: an operations procedure shared with the platform team.

Quiz

  1. In which order do you update a Managed deployment?
    A. OneAgent, ActiveGate, cluster · B. Cluster, ActiveGate, OneAgent · C. The order does not matter
  2. Where is the cluster backup usually written?
    A. To shared storage mounted at the same path on all nodes · B. In a OneAgent · C. Only in the memory of the master node
  3. Why add external monitoring to the cluster?
    A. Because the CMC shows no information · B. To be alerted even if the cluster itself is unavailable · C. To replace the backup
  4. What is the first lever when storage grows?
    A. Remove a node · B. Measure usage and identify the demanding environments · C. Disable backup

Answers

  1. B. A component must never be newer than the cluster it connects to.
  2. A. The backup uses shared storage, typically NFS, accessible from every node.
  3. B. A system cannot report its own total failure; an external check can.
  4. B. You act on retention or the source only after measuring where the space is going.

Further reading

← Module 6: Tags, management zones and access · Course overview · Module 8: Automating with the APIs →

Frequently asked questions

Can I skip several versions when updating the cluster?

The possible update paths depend on the versions involved. Check the release notes: a cluster that is far behind often has to catch up in several steps, which is why a regular rhythm pays off.

Does high availability remove the need for backups?

No. Several nodes protect against losing a server, not against a configuration error, corruption or an incident affecting the whole site.

Who should have access to the CMC?

A small group of cluster administrators. Application teams do not need it: they work in their environments, with the rights defined in module 6.

A well-run cluster is recognisable by how uneventful it is: no full disk, no outdated version, a tested backup. To free up time on these repetitive tasks, the last module shows how to automate them with the APIs.

span.observer is an independent community blog, not affiliated with Dynatrace LLC. This course does not replace the official Dynatrace University training and certifications. Check features in the official documentation for your version.

Total
0
Shares
Leave a Reply

Your email address will not be published. Required fields are marked *

Previous Post
Module 6 — Tags, zones, accès en Managed : tagging automatique, management zones et permissions

Module 6 — Dynatrace management zones, tags and access in Managed: automatic tagging and permissions

Next Post
Module 8 — Automatisation par API en Managed : API d'environnement, API Cluster et configuration as code

Module 8 — Automating with the Dynatrace Managed API: environment API, Cluster API and configuration as code

Related Posts