Sign up for the ITPro Today newsletter
Stay on top of the IT universe with commentary, news analysis, how-to's, and tips delivered to your inbox daily.
Explosion of Data in the Cloud Era Leading to Observability Complexity
There's just too many applications and too much data for IT professionals to easily monitor, but there are solutions that might help.
4 Min Read
Dynatrace
As organizations continue to adopt cloud-native technologies, the complexity of their technology stacks has increased significantly, but what is the actual impact? A new report from Dynatrace aims to provide that answer.
The report titled, "The state of observability 2024," is based on a survey of 1,300 CIOs and technology leaders and reveals that while multicloud environments and cloud-native architectures enable rapid transformation and innovation, they also create an explosion of data that is becoming increasingly difficult for IT teams to manage.
The report found that 88% of organizations have experienced an increase in complexity over the past 12 months, with the average multicloud environment spanning 12 different platforms and services. This complexity not only makes it more challenging to deliver outstanding customer experiences but also makes applications more difficult to protect.
Key findings include:
88% of organizations report increased complexity in their technology stack over the past 12 months.
87% of technology leaders say multicloud complexity hinders the delivery of outstanding customer experiences.
86% of technology leaders say cloud-native technology stacks produce an overwhelming amount of data.
81% of technology leaders say manual approaches to log management and analytics cannot keep pace with the rate of change and data volumes.
"Over half of organizations — 51% — said that the complexity of their technology stack will continue to increase, which was surprising because it highlights the education that needs to be done around adopting a platform-centric approach," Jay Livens, senior director of product marketing at Dynatrace, told ITPro Today.
The Struggle to Keep Up with Data
One of the most significant challenges organizations face is the sheer volume of data generated by cloud-native technology stacks.
According to the report, an overwhelming 86% of technology leaders say that these stacks produce an explosion of data that is beyond humans' ability to manage effectively. To cope with this data deluge, organizations use an average of 10 different monitoring and observability tools to manage applications, infrastructure, and user experience. However, 85% of technology leaders say that the number of tools, platforms, dashboards, and applications they rely on only adds to the complexity of managing a multicloud environment.
As organizations grapple with the increasing complexity and data volumes, traditional approaches to log management and analytics are proving inadequate. The report found that 81% of technology leaders say manual approaches cannot keep up with the rate of change in their technology stack and the volumes of data it produces. This not only hinders their ability to gain valuable insights from their data but also takes time away from innovation, with 81% of technology leaders saying that the time their teams spend maintaining monitoring tools and preparing data for analysis steals time from innovation.
Best Practices to Improve Observability
There are several things that organizations can implement to manage the increasing complexity of their multicloud environments.
Livens noted that organizations are starting to unify monitoring tools and automate some tasks in order to adapt to the complexity of multicloud environments, but the research reveals there are still significant challenges in managing this complexity. To address these challenges and drive business value, advanced AI, analytics, and automation capabilities are essential.
To overcome the challenges posed by cloud-native technologies, organizations are turning to AIOps.
The report found that 72% of organizations have adopted AIOps to reduce the complexity of managing their multicloud environment. However, 97% of technology leaders say that probabilistic machine learning approaches have limited the value AIOps delivers due to the manual effort needed to gain reliable insights.
"While AIOps is a crucial practice, it relies on probabilistic methods, which can be imprecise and time-consuming to implement," Livens said. "I hope to see more organizations adopting mature AI, analytics, and automation strategies to overcome the complexity of modern technology stacks. By doing so, they can reduce manual efforts and streamline operations, empowering teams to spend more time on innovation and less time on routine maintenance tasks."
About the Author(s)
You May Also Like
