Manage thousands of hosts with the new OneAgent on a host REST API (Preview)

Update: the Preview program for this feature has been closed.

Dynatrace helps you monitor hyper-complex environments where tribal knowledge about entities and their relationships isn’t sufficient. To achieve this, Dynatrace provides important components such as OneAgent and ActiveGate. Although we strive for full automation during rollout and maintenance, customers with complex environments often request precise configurations that can work at scale.

Dynatrace OneAgent life-cycle management allows you to control your OneAgent instance operations and core settings and answers questions like:

• What are the statuses of my Dynatrace OneAgents?
• Are my OneAgents and ActiveGates performing well?
• Do I have all the latest versions installed?

So you can easily manage your OneAgents at scale, we’ve introduced support for OneAgent life-cycle management via the REST API. This is in addition to the UI and CLI (command-line interface) tools we already provide.

Why this is important

Dynatrace is built for scale. Our largest customers are global companies with absolutely massive application stacks, spreading across cloud and enterprise platforms, multiple physical geographies, tens of thousands of virtual and physical hosts and hundreds of thousands of containers. Thanks to the ease of deployment, the time-to-value for monitoring of such complex environments is very short.

But the deployment of OneAgents is not where the lifecycle of OneAgents ends. Like any other software, OneAgents need to be maintained, updated, and monitored.

Among other APM solutions, Dynatrace offers a unique advantage: our deployments provide centralized configuration that can be managed from a single place. You don’t need to build Ansible, Chef, Puppet, or SaltStack scripts to control or update your OneAgents. All this can be done centrally from your Dynatrace cluster, regardless if you’re monitoring physical hosts, AWS EC2 server instances, services running in Kubernetes Pods, virtual machines under VMware, or any supported operating system or technology that can be monitored using Dynatrace.

Central configuration comes with two important benefits:

• Ease of integration: Just connect to your Dynatrace cluster API—we’ll take care of the REST (pun intended)
• Speed of management: with a single command you can manage hundreds or thousands of Dynatrace OneAgents almost instantaneously, wherever they are and whatever they are configured to do.

This is why customers who have already participated in the aforementioned preview of this solution are so excited for the new capabilities. Let’s take a closer look at how these capabilities are used.

A few customer use cases

Over 50 of the largest Dynatrace customers—from finance, retail, and government verticals—have participated in the Preview. Their feedback has been very positive.

“We have over 7000 hosts in our environment. The OneAgent on a host REST API is critical for us to easily automate the switch between full stack and infrastructure monitoring mode.” – a Dynatrace customer

Further use cases include:

• keeping track of OneAgent update progress via API queries
• turning monitoring on and off based on server status in a CMDB
• automatic enablement and disablement of monitoring for specific hosts
• setting passive servers into infrastructure-only monitoring mode and the active datacenter servers into full-stack mode during failover

More use cases to come

Our investment into REST API for OneAgent management is part of a larger set of planned improvements. In the future, you’ll also be able to:

• automate software management using your CMDB and scripting solution of choice
• increase control over deployment and configuration of Dynatrace modules via REST APIs and CLI (command-line interface)
• easily identify Dynatrace modules that are in a particular operation or configuration state
• get answers to questions about your OneAgents and easily turn those answers into actions

How to access the OneAgent on a host REST API

All the new and updated REST API methods are available in the REST API Explorer, which you can easily access from Dynatrace. The API Explorer allows you to interactively explore and use the Dynatrace REST API right from your web browser.

From the User menu at the top right, select Environment API. Scroll down and expand the OneAgent on a host endpoint.

Additional methods are also available in the Configuration API—select the User menu > Configuration API to open the API Explorer. Expand the OneAgent on a host endpoints to view and try out the available methods.

Alternatively, you can read about all the REST API methods by accessing our online help:

• OneAgent on a host Environment API
• OneAgent on a host Configuration API

Simplify OneAgent life-cycle management with the new REST API

Enhancements added in the first Preview (February, 2020)

Read and update OneAgent status and configuration

With the OneAgent on a host Environment API endpoint, you can read various aspects of OneAgents status and configuration.

• Check OneAgent availability status (check the availabilityState field).
• Check OneAgent monitoring mode (full-stack or infrastructure-only).
• Check the auto-update setting of your OneAgent instances.
• Check the versions of your OneAgent instances.
• Check whether the specific OneAgent version was labeled as “faulty.” In such cases, you can also take corrective actions (upgrade to a fixed version preferably, or disable if needed).
• Check which monitoring modules are running. You can list the existing deep-code monitoring modules this way, but the same functionality also pertains to other OneAgent modules.
• Check the version of the monitoring module being used.
• Check the versions of the monitoring modules labeled as faulty.
• Check all the names and versions of the OneAgent extensions (plugins).
• Check the operational state of OneAgents extensions (plugins).
• Check all the possible OneAgent versions that are deployable to a particular OneAgent.

Search for OneAgents by their operational and configuration status

The following query parameters of the request allow you to search for OneAgent instances by various aspects of status and configuration.

• Search by monitoring status (OneAgent enabled or disabled—availabilityState).
• Search by OneAgent operation mode (full-stack or infrastructure-only—MonitoringType).
• Search by version number (the version number can be equal to, greater, or smaller than a given reference numer—agentVersionIs in conjunction with agentVersionNumber).
• Search for OneAgent instances labeled as “faulty” (faultyVersion).
• Search for OneAgent instances where a specific monitoring module is present and active (technologyModuleType).
• Search for OneAgent instances with a module of a specific version (or greater or smaller than—technologyModuleType in conjunction with technologyModuleVersionIs and technologyModuleVersionNumber).
• Search for OneAgent instances where a specific module was labeled as “faulty” (technologyModuleType in conjunction with technologyModuleFaultyVersion).
• Search for OneAgent instances where a specific extension, or plugin (specified by name), is deployed (pluginName).
• Search for OneAgents with an extension (plugin) of a specific version (or greater or smaller than—pluginName in conjunction with pluginVersionIs and pluginVersionNumber).
• Search for OneAgent instances where the extension (plugin) is in a specific operational state (pluginName in conjunction with pluginState).
• Search for OneAgent instances with specific auto-update settings (autoUpdateSetting).

Other read and search options

The Dynatrace API comes with many other ways of reading the properties and settings of OneAgents and searching for them. In order to get the best out of what we offer, it’s important to explore the API via the API Explorer and to adjust the search results and techniques to your particular needs.

Specifically, here are some of the most useful read and search attributes that can be found in the ../api/v1/oneagents family:

• tag—Filters the resulting set of hosts by the specified tag. You can specify several tags in the following format: tag=tag1&tag=tag2. The host must match all the specified tags, and in case of key-value tags such as imported AWS or CloudFoundry tags, use following format: [context]key:value.
• managementZoneId (ID) or its alternative managementZone (name)—Only returns hosts that are part of the specified management zone.
• hostGroupId and hostGroupName—Filters the resulting set of hosts by the specified host group by ID or name.
• osType—Filters the resulting set of hosts by their OS type: AIX, DARWIN, HPUX, LINUX, SOLARIS, WINDOWS, ZOS.

Note: To check the host units effectively consumed by the OneAgent deployed on a particular host you can use this previously released REST API endpoint:

../api/v1/entity/infrastructure/hosts/{entity (consumedHostUnits: string)

Paginated search results

In some cases, the result of search operations can amount to multiple pages. While processing the results, take note of the “next page” identifiers and resend the search query for follow-up portions of the result set.

Update OneAgent configuration

There are several write operations available as part of the OneAgent on a host Configuration API in this release. Here’s a description of the full functionality of the endpoints.

With the Configuration API endpoint for OneAgent auto-update you can:

• Check the OneAgent auto-update setting.
• Update the OneAgent auto-update setting.

With the Configuration API endpoint for OneAgent monitoring configuration you can:

• Check if OneAgent is enabled or disabled on a particular host.
• Enable or disable OneAgent on a particular host.
• Check OneAgent monitoring mode (full-stack or infrastructure-only).
• Change the OneAgent monitoring mode.

With the Configuration API endpoint for OneAgent technology monitoring configuration you can:

• Check how monitoring modules are configured (to validate whether the configuration actually matches the operational status).

Enhancements added in the second Preview (June, 2020)

Easily create large-scale lifecycle patterns across the largest environments

Thanks to API-based controls for auto- and manual updates, it’s now possible to create fast, large-scale lifecycle patterns across numerous sections of your environment: staging, testing, production, etc, and to integrate those patterns directly into your CMDB solution of choice. Combined with the previously released ability to control the operational status of OneAgents, you can now build a complete and comprehensive end-to-end solution.

Enable or disable auto-update mechanism on host, host group, and tenant level

Please note that this setting is inherited via parent-child relationships, so host-level settings override the host-group and tenant-level settings. It’s also possible now to read the settings that were applied earlier on all the three levels of abstraction.

Invoke an update to a specified version

This setting is provided only for those hosts and host groups where auto-update is disabled. It allows for triggering of manual updates. Note that this request effectively means that the update will be scheduled for execution at the next time OneAgent sends its heartbeat to the Dynatrace cluster (typically every minute) and that the cluster may decide to distribute the update load into smaller batches. In essence, as a consequence of these mechanisms being in place, the actual update may be delayed by a few minutes.

Extended read and search mechanism

In order to provide better support for host groups, we’ve also extended the read and search mechanism. This means that now you can:

• read what host group a given host belongs to
• search for hosts within a specific host group

Important

• This is only the first noteworthy release of this project—much, much more is coming.
• This release focuses on the REST API, but we’re also going to implement UI changes to assure full parity.
• Next, we’ll focus more on write operations and make it possible to increasingly control deployment via Dynatrace APIs.

In addition to the REST API, also note that we’re continuously pushing our changes to the OneAgent CLI (command-line interface). You can read more about the current release in this blog post.

Feedback?

As always, we welcome your feedback and comments. Please share your thoughts via Dynatrace Community, directly within the product through Dynatrace ONE chat, or your Dynatrace Account Manager.