A Sysadmin’s Guide to Spaghetti Cabling: Bridging the Gap between IT Operations and Business
No, it’s not the latest culinary invention from a famous Italian chef: spaghetti cabling (a nice wording for cabling inferno) is a sour dish we’d rather not eat. Beyond this unsavory term hides the complexity of many environments that have grown organically, where “quick fixes” have crystallized into permanent solutions, and where data center racks are entangled in cables, as if they had become a modern version of Shelob’s Lair.
These cabling horrors are not an act of art. Instead, they prosaically connect systems together to form the backbone of infrastructures that support many organizations. Having had experience in the past with spaghetti cabling, I can very vividly remember the endless back-and-forth discussions with my colleagues. This usually happened when one was trying to identify the switch port to patch panel connectivity while the other was checking if the system network interface is up or down. That then resulted in trying to figure out if patch panel ports were correctly mapped with wall outlet plug identification. All of this to troubleshoot a problem that would be very trivial if it wasn’t for careless and unprofessional cabling.
The analogy with other infrastructure assets is very similar: it can be very difficult for administrators to find a needle in the haystack, especially when the asset is not physical and the infrastructure is large. Multi-tiered architectures, or daisy-chained business processes relying on multiple sources of data, increase potential failure points in the data processing stream. This sometimes makes troubleshooting a far more complex endeavor than it used to be due to upstream or downstream dependencies.
One would expect that upstream dependencies would impact a system in such a way that it is no longer able to process data, and thus come to a halt without impact to downstream systems. While this can be a safe assumption, there are also cases where the issue isn’t a hard stop. Instead, the issue becomes data corruption. Either by handing over incorrect data or by handing over only fragments of usable data. In such occurrences, it is also necessary to identify the downstream systems and stop them to avoid further damage until the core issue has been investigated and fixed.
Thus, there is a real need for mapping the upstream and downstream dependencies of an application. There are cases in which it’s preferable to bring an entire system to a halt rather than risk financial losses (and eventually litigation, not to mention sanctions), if incorrect data makes its way into production systems. In that case, it would ultimately impact the quality of a manufactured product (think critical products, such as medicines, food, etc.) or a data batch meant for further consumption by a third party (financial reconciliation data, credit ratings, etc.).
Beyond troubleshooting, it’s crucial for organizations to have an end-to-end vision of their systems and assets, preferably into a System of Record. This could be for inventory purposes or for management processes, whether based on ITIL or not. The IT view is not always the same as the business view, however both are bound by the same common goal: to help the organization deliver on its business objectives. The service owner will focus on the business and process outcomes, while the IT organization will usually focus on uptime and quality of service. Understanding how assets are grouped and interact together is key in maintaining fast reaction capabilities, if not acting proactively to avoid outages.
There is no magical recipe to untangle the webs of spaghetti cabling, however advanced detection/mapping capabilities. Existing information in the organization should help IT and business obtain a precise map of existing systems, and understand how data flows in and out of the system with a little detective work.
In our view, the following activities are key enablers to obtain full-view clarity on the infrastructure:
- Business service view: the business service view is essential in understanding the dependencies between assets, systems, and processes. Existing service maps and documentation, such as business impact assessments, should ideally contain enough information to capture the process view and system dependencies.
- Infrastructure view: it is advisable to rely on infrastructure monitoring tools with advanced mapping / relationship / traffic-flow analysis capabilities. These can be used to complement/validate existing business service views listed above (for lucky administrators / IT departments), or as a starting point to map traffic flows first, then reach out to business stakeholders to formalize the views and system relationships.
- Impact conditions and parent-child relationships: these usually would be captured in a System of Record, such as a CMDB, but might eventually be also available on a monitoring system. An event impacting a parent asset would usually cascade down to child assets.
- Finally, regular service mapping review sessions between IT and business stakeholders are advised to assert any changes.
Taken in its tightest interpretation, the inner circle of handling “spaghetti cabling” problems should remain within the sphere of IT Operations Management. However, professional and conscious system administrators will always be looking at how to improve things, and will likely expand to the other activities described above.
In our view, it is an excellent way to further develop one’s skills. First, by going above and beyond one’s scope of activities, it can help us build a track record of dependability and reliability. Second, engaging with the business can help us foster our communication skills and move from a sometimes tense and frail relationship to building bridges of trust. And finally, the ability to understand how IT can contribute to the resolution of business challenges can help us move our vision from a purely IT-centric view to a more holistic understanding of how organizations work, and how our prioritization of certain actions can help better grease the wheels.