Wall Street & Use of Artificial Intelligence Leads the Way for Digital Performance Management Revolution

The evolution of APM (Application Performance Management)

APM (Application Performance Management) is on the precipice of dramatic change. The complexity of cloud native and new stack applications is making many of the traditional ways of monitoring applications irrelevant. With applications which can elastically scale using containers to meet unprecedented amounts of demand, the APM industry needs to reconsider the focus on simple problem identification to provide value for businesses. Understanding how application performance visibility can provide better data as to what should get scaled and what does not need to get scaled is just as important to the business. Managing complexity is the challenge of the next generation of applications.

A radically new approach is needed.

Are there industries which have been pioneering a different way of doing things? The answer is “yes” and, interestingly enough, it starts with Wall Street. For years the financial services industry has been using highly advanced analytics, predicative algorithms and automation to execute transactions and service customers. It’s time that a wider cross section of industries take advantage of these techniques in helping them manage complex applications. Call it Artificial Intelligence – Machine Learning – Algorithms, this automation shows that we can identify patterns in data and react to it.

Let‘s look at some real world examples.

Stock Trading

In 2001 IBM published a paper which highlighted how several algorithms (MGD and ZIP) were able to outperform actual human stock traders. These algorithms look for patterns in data in real time.  Using machine learning these algorithms can adapt and react to the way in which these patterns change. The decision to make (or not make) a trade is entirely automated. The use of these algorithms combined with machine learning is responsible for billions of dollars of financial transactions. This is not science fiction; it’s reality.

Are there other examples? Yes, there are.

Credit Card Protection – Did you really buy this?

Credit card fraud is an estimated $16-billion-dollar problem. Credit card companies monitor billions of transactional events. These companies use machine learning and algorithms to identify anomalies in credit card holders’ activities. When a transaction is occurring on their card and the user is in a location they are not usually associated with, the credit card companies use these algorithms and machine learning to flag those transactions as being suspect. Many of us have received these calls from the credit card companies asking us if we’ve been aware of odd transactions. This has now become a common industry use of machine learning.

These examples show that Artificial Intelligence – Machine Learning – Algorithms are a proven technology that can be used to analyze (and react to) massive amounts of data in real time. Exactly the sort of thing needed in addressing the complexity management challenge coming with the next generation of applications.

Digital Performance Management: the evolution of APM

Industry leaders like Jason Bloomberg have described the need for an Artificial Intelligence-Driven Vision for Digital Performance Management. He describes how machine learning can be applied to the complex data sets generated by Digital Performance Management assets.

Application Performance Management has been becoming highly diversified over the past decade. Businesses have been driving this, requiring a variety of services encompassing the needs of different parts of their organizations; operations, development, LOB, etc. Recently, a new understanding of how businesses consume APM data has brought about the emergence of Digital Performance Management. Understanding that the entire business relies on APM data to better serve end users, more and more emphasis has revolved around capturing the entire (full stack) transaction (in a gap free fashion) for users actions. Be it a touch or swipe or API request, regardless of how the end user interacts with an application, there is a complex chain of meta data captured responding to an end user’s request. The amount of data generated is immense. Some APM vendors approach this by looking at slices of data when things go wrong. The problem with this approach is that businesses are looking to understand everything about every transaction as opposed to a slice of it. Algorithms work better with more data. With more data, machine learning learns better. Accuracy increases. Hence, capturing data slices isn’t ideal for complete Digital Performance Management. Being able to see everything allows businesses to better understand what end users are doing, how performance impacts conversion, how to better protect the brand, and determine where to make technology investments. This is why business are turning to Digital Performance Management.

Artificial Intelligence – Machine Learning – Algorithms are particularly well suited for managing dynamic environments. For the algorithms to work effectively, they need to take into account the auto discovery of compute resources as they are being created. In a world where containerized new stack capabilities can be elastically generated on demand, understanding what is being created and deprecated is vital in order for businesses to know what is touching the end user. The algorithms also need to understand the data being generated by end users and automatically baseline not only the amount of traffic but the performance of the traffic to identify anomalies impacting end users. Read more here.

While the Artificial Intelligence — Machine Learning – Algorithms can automatically manage massive amounts of data, the benefits for businesses to deploy these technologies along with their Digital Performance Management assets is that they help reduce costs associated with problem identification and root cause analysis. Businesses no longer have to divert their most costly human capital to ad hoc diagnostics from the innovation they should be working on.

Digital Performance Management is an example of a strategic set of highly complex real time data that businesses depend on. Artificial Intelligence – Machine Learning – Algorithms, whatever you want to call it, is an industry-proven solution that Wall Street has been using for years to help manage highly-complex dynamic data sets. It’s time for businesses to leverage the advances in machine learning to help them manage the complexity involved in servicing their end users.