Home > Shaping an Observability Journey for the Future

Shaping an Observability Journey for the Future

Shaping an Observability Journey for the Future blog image
The world of IT has gone through a significant amount of change in the last five to 10 years, and the rate of digital transformation can be challenging for organizations of all sizes. Keeping up with new and evolving capabilities alongside an increasingly distributed talent pool can be tough. That’s why it has always been my goal to keep up with the latest trends and provide the best tools for professionals working in the IT trenches. One of those emerging trends is observability. And in early 2021, we embarked on our own digital transformation and evolution towards this new paradigm. For us here at SolarWinds, observability is more than a toolset. It’s an essential framework for organizing cross-departmental teams and processes around. And as the IT role has expanded and become more critical to our lives, new roles have emerged and become integral to ensuring superior customer experiences. There’s a balance of generality and specialization needing consideration amongst IT, DevOps, SecOps, and AIOps sectors. Finding the proper balance will vary based on your own cloud maturity, goals, and organizational objectives. We firmly believe a common data set is essential for meeting the demands and expectations of today’s consumers and must be orchestrated and implemented through an observability solution.

Simplifying Our Digital Transformation Journey With Observability

As a chief information officer, I experience the same problems as everyone else, where I need to support multiple teams, products, and locations. There’s nothing worse than being on a call at 2 a.m. with a group of people who are unhappy about being woken up in the middle of the night. Conversations get tense as people fall into defensive mode. When asked, “Where’s the problem?” each group—network, systems, database, apps—will go into their tool and tell you their piece is fine. In such scenarios, you need to diffuse tensions and refocus on the data. What’s the data telling us? Can we run a test? Can we run a benchmark? Can we segregate the problem? None of these answers fix the issue affecting the system—they merely diagnose the problem. It’s a huge win when we can remove these barriers and help everyone, remote teams included, immediately focus on the core area experiencing the issue. This realization left us wondering if we were missing the bigger picture buried under the layers of data. We all saw the data differently because of our siloed tools and mindsets. We quickly realized changing the way we all observed the data could improve our own collaboration and analysis efforts. This is where our need for an observability solution was born.

Creating a Framework Based on Our Own Experiences

Like many other companies dealing with the complexities of a hybrid world, we were deeply entrenched with multiple cloud providers while still having a significant on-premises deployment when we began our digital transformation journey. Moving to an observability solution would require not only a tools evaluation and migration but also a shift in mindset. As we looked inward, we knew from our deep customer feedback channels many enterprises that looked like us and were built like us could benefit from our own experience. This thinking framed a project I headed called our SolarWinds® Observability Center (or SWOC for short). The SWOC would be a physical place where we could demonstrate how an observability solution works for an enterprise of our global scale. When we started, we outlined four goals:
  1. Use our own solution on our own platform
    • As we began building out our SWOC, we started with our existing product suite to capture a baseline. We wanted to be experts in the transformation journey steps organizations with hybrid environments have to think through, such as migration, integrations, training, and improving other key components like systems or workflows.
  2. Provide unvarnished feedback between our product and engineering teams
    • When developing our observability solution, we needed to change our mentality from individual departments and move it towards one of organization-wide impact. We needed new ways for teams to communicate along with shared resources for them to communicate across.
  3. Document and share insight into our digital transformation journey
    • Our goal is to hand our customers a playbook with a flexible operational roadmap to follow. We currently have detailed documentation we’re using today to provide expert insight and color for a customer’s wide-ranging requirements and projects.
  4. Drive innovation through our own implementation
    • We’re currently building out a room in the Austin office to resemble a traditional network operations center (NOC) using SolarWinds® Observability. This way, we can show practitioners firsthand how we’re using our products and building these connections.
With these requirements in mind, we took a crucial step in our digital transformation journey and achieved many of these goals when we announced the release of our SolarWinds Platform, including our Hybrid Cloud Observability solution, in the first half of 2022.

How We Implemented Our SolarWinds Observability Center

I, like many others, don’t have an unlimited budget or resources, so I needed to establish our 24/7 NOC using SolarWinds products as cost-effectively as possible. We built, from the ground up, a dedicated team to provide round-the-clock support to help us address the following challenges:
  • How do we get our support teams trained up?
  • How do we transition from alerting response capabilities currently handled by individual teams and products within SolarWinds to this NOC function?
  • How do we ensure we have adequate run books?
  • How do we handle escalation?
At SolarWinds, we traditionally responded to abnormalities through monitoring and leveraging automation, which would frequently be reactionary. With the SWOC and the Hybrid Cloud Observability solution, we now have a centralized team watching our environment constantly, proactively seeing flags before they alert and before degradation triggers are hit.

What We Learned

We were ambitious in our execution, challenging ourselves to be fully stood up in 90 days. We didn’t quite do it in 90 days, but we learned much throughout the process and now know a great deal more about the needed personnel to pull off a transformation of this magnitude. For example, in a traditional NOC model, you might expect your NOC engineers to monitor, triage, and remediate a certain number of issues and then escalate anything more complex. With full-stack observability, you can see network, database, and application layers along with other critical components all in one view. So, we determined you can’t expect every individual in the SWOC to have the same depth of expertise or do the level of triage and troubleshooting across all areas, as the breadth of expertise is simply too great. We had to make conscious trade-offs regarding what alerts could be managed by the SWOC and which would be immediate escalations. It still requires cross-team collaboration, with the big difference now being we’re all using a common tool and looking at the same data.

Put SolarWinds Observability on Your Roadmap

We want to help you achieve the same successes we saw implementing observability. We’re excited to share our experiences with you since we gained many lessons from our initial deployment and continue to learn from our ongoing work, so anyone undergoing a similar transformation can benefit. Hybrid Cloud Observability enriches monitoring with single-pane-of-glass observability, automation, and intelligence, built on the SolarWinds Platform developed with our next-generation build system. I invite you to try this solution yourself or get in touch with one of our transformation specialists for further details.
Chris Day
Chris Day is CIO at SolarWinds, an IT infrastructure management company headquartered in Austin, Texas, with more than 30 offices and 3,200 employees worldwide. With…
Read more