Using Dynatrace AI based Monitoring to Ensure Blockchain Performance

The explosion of Bitcoin and cryptocurrency has been a proving ground for Blockchain transactions.  Blockchain is a constantly growing list of records called blocks, which are connected and secured using cryptography (1). Blockchains provide permanent verifiable means of recording transactions between two parties. Serious actors in Finance, eCommerce and IT are massively investing in and building out Blockchain frameworks. Blockchain by its very nature requires tremendous processing power and network conversations with actors exchanging blocks. There has not been a lot of research into the performance impact adding Blockchain transactions into ecommerce applications will have. Blockchain adoption raises several operational concerns:

1. It is unclear how poor performance will impact end users or applications relying on a Blockchain transaction.
2. As Blockchain becomes part of the fabric of cloud based, new stack ecommerce/financial applications the onus on how Blockchain behaves and performs will not only fall on to those providing the core Blockchain processing, but any actor integrating it into their applications or exchanges.
3. The explosion of complexity around cloud and container based approaches these Blockchain applications will be based on will require new discipline and insight to ensure that they are adequately monitored from a performance perspective.

That last item is where most organizations will struggle.  Blockchain use at web scale will happen in ridiculously complex environments which will include elastically scaling compute resources which may only exist temporarily.  The scale of these applications will require a new monitoring paradigm as it will be impractical and in many cases impossible for humans to monitor the health of these Blockchain components.

Monitoring Blockchains

Most literature which can be found on monitoring Blockchain focuses on very specific metrics regarding the health of the chain. Here is a view from an IBM Bluemix UI.

Above you can see metrics like Total Blocks, Recent Instantiations, Invocations, etc. (2)

Below is an example of some Blockchain activity from https://Blockchain.info/charts

Above you can see that in this example Blockchain data is being aggregated into Block Details, Mining Information, Network Activity and Wallet Activity.  This is clearly pertinent information about the Blockchain, however:

1. What happens if there is an infrastructure issue with your Blockchain processing?
2. What happens if there is a network issue between Blockchain actors?
3. How do the above metrics allow an organization to diagnose that a performance issue is occurring with their Blockchain?
4. How do the above metrics allow an organization to understand the impact to end users or service calls depending on a Blockchain?

Monitoring Blockchains requires visibility into the entire technology stack.  In addition, monitoring Blockchains requires monitoring every transaction. You can’t skip requests or sample/throttle data when monitoring Blockchains and blindly trust that the application, services, process, network or infrastructure layers are always providing 100% availability and optimal performance.

Automating Performance Monitoring for Blockchains

Here is an example of how to monitor a simple Blockchain based on Ethereum (3).  Ethereum (https://www.ethereum.org) is an open-source Blockchain technology.  In this case it is deployed as GOlang (a Google programming language popular in new stack deployments).

In this example, we are using Dynatrace to automatically discover the GO processes which were created when we spun up Ethereum in the Amazon Cloud using EC2 (AWS).

The Blockchain is being computed in the geth process.

As Blockchains get deployed into containers and micro-services in cloud environments, Dynatrace automatically discovers these processes. Dynatrace will automatically pull in the log files from the geth process (more on this in a bit).

Blockchains are about communicating between two parties, below is how Dynatrace automatically discovers requests being made to potential externally held Blockchains. Calls to Blockchains can be executed in a variety of ways, one of which is via JSON based REST calls.

The screenshot above shows how external request to Blockchains can be monitored.  Below is an example of how network traffic coming in to and out of our cloud instance hosting the Blockchain is discovered and monitored.

By including Dynatrace as part of your container/cloud deployments you can automate the discovery of your Blockchain services. This is as simple as including Dynatrace in your buildpacks, as your Blockchain compute resources are created they will be automatically instrumented and monitored.

Deterministic AI to the Rescue

The sample Blockchain I showed above is an overly simple example. Blockchains will exist in highly complex environments executing thousands of transactions per minute.

These environments will challenge even the most mature IT organization to monitor these business-critical transactions.  This is where deterministic Artificial Intelligence monitoring, using specific machine learning algorithms will be able to auto-discover, auto-baseline 100% of your traffic in a full stack fashion (from End User to Application to Blockchain to Infrastructure) and be able to determine the impact of performance events as well as the root cause.

Deterministic AI is the only approach to effectively monitor these environments. Cognitive AI will take too long to understand all of the entities involved, and by the time it could have understood the relationships the environment will have changed.  Monitoring Blockchain containers running in elastic cloud environments which will be constantly changing and evolving requires a deterministic AI approach.  This is what Dynatrace provides, an inherent understanding of all the compute entities involved in operating a Blockchain based application.  Understanding the entities, relationships and dependencies is only part of what needs to be done.  Dynatrace employs a variety of machine learning algorithms to baseline, compare time frames, create multi-dimensional views of data to then determine and rank issues which inevitably occur in the complex environments.

What this provides organizations is the ability to build self-healing Blockchain applications.  For example, if a Blockchain process is having a memory issue, Dynatrace can detect that state and be used to trigger a Chef, Puppet or Ansible script to adjust the memory settings automatically to maintain that Blockchain processing.

Tailor Your Blockchain Monitoring

While the Dynatrace AI will automatically discover and baseline your Blockchain processes, there may be situations where there are details locked away in your Blockchain logs that you want to be aware of.

Below is an example of how Dynatrace is able to monitor Blockchain log files.  You can create rules to look for important strings within your Blockchain log files and automatically trigger external events based on those events/strings.

This works automatically and doesn’t require expensive, complicated third party log analysis tools pushing data into third party data aggregation tools which would then have to push out to external automation scripts.

In addition to monitoring Blockchain log files, Dynatrace can be used to insert additional metadata, what we call tags, over your Blockchain compute resources. These tags could be used to pull things like the Chaincode ID, Chaincode Version or Peer ID.  This will allow your teams to create separate visualizations, reports, alerts, etc… for only those Blockchains they are working with.

Blockchain is Too Important Not to Monitor, Try it for Yourself!

The example we showed above is a highly-simplified example, your Blockchain applications will exist in much more complicated environments.  We invite organizations to try Dynatrace in your Blockchain enabled environments. Here is a Free Trial of Dynatrace which you can download and install into your environments where you are running your Blockchain processes.  Dynatrace can easily install on to Hosts directly or you can include Dynatrace in your container/cloud build-packs with nominal effort.

Go ahead and give this a try with your Blockchains, please post your experience below, we would love to get your feedback.

1. https://en.wikipedia.org/wiki/Blockchain
2. https://console.bluemix.net/docs/services/blockchain/howto/monitor_network.html
3. https://blockgeeks.com/two-node-setup-of-a-private-ethereum/