Monitoring Isn't Observability

Observability is all the rage, an emerging term that’s trending up very quickly in certain circles even while it remains unknown in others. As such, there isn’t a single widely understood meaning for the term, and much confusion is inevitably following. What is observability? What does it mean? Perhaps just as importantly, what is observability NOT? TL:DR; Monitoring tells you whether a system is working, observability lets you ask why it isn't working.

Observability Is a High-Entropy Concept

Despite some great articles, the concept of observability is in a state of confusion, high energy, heterogeneity, and amorphousness. Broadly speaking, that is. People understand and use observability in a variety of ways, and there’s often not a lot of intersection among them. Observability actually has a long history. It has a formal definition, albeit only a partially useful one IMO: In control theory, observability is a measure of how well internal states of a system can be inferred from knowledge of its external outputs. The observability and controllability of a system are mathematical duals. (Wikipedia) If you’re like me, saying observability is the mathematical dual of something else doesn’t help much. I suspect the people who care about a mathematical definition are vanishingly few compared to the rest of us. But the rest of us are increasingly using and engaging in conversations about observability, and the disagreements and objections are contributing to the escalation of the conversations. (Mea culpa.) For example, monitoring tools vendors are jumping on the observability bandwagon as fast as they jumped on the DevOps bandwagon a few years ago. Monitoring vendors’ homepages are being updated en masse to feature observability prominently. Simultaneously, some folks are reacting with dismissal and even anger, calling it a disingenuous rebranding of monitoring. Well, yes and no. Like any term that emerges into people’s consciousness in this way, it’s bound to be controversial as people assign it disjoint meanings, even though I think that’s OK and we’ll converge eventually. For the moment, observability in popular culture seems to mostly mean “monitoring, but old wine in a new skin.” Personally, I’ve been using the term for a long time, and it’s kind of surprised me to see it emerge so suddenly into wide usage. For example, I mentioned observability a lot in my book Best Practices for Architecting Highly Monitorable Applications, which I first drafted more than two years ago.

Observability Is a System Attribute

The first part of the Wikipedia definition is legit. Observability is a property of a system. Here’s how I think about the intersection of the monitoring-related buzzwords:

• Observability
• Instrumentation
• Telemetry
• Analytics
• Monitoring

A helpful analogy that I think places observability into context: observability is not the microscope. It’s the clarity of the slide under the microscope. You might respond, as Theo did, “DTrace and now eBPF challenge that.” But I draw a distinction: in my view, those aren’t microscopes. They’re Windex for the slide. They’re better-designed slides. Another way to think about observability as an attribute of a system is to consider some other helpful system properties:

• Functionality
• Performance
• Testability
• Operability
• Maintainability
• Efficiency
• Teachability
• Usability

Those are all adjunct nouns (an adjective that's been noun-ified). Observability, too, is a noun, whereas monitoring is a verb.

Observability Is a Cultural Trait

Observability as can also modify other nouns, as in “observability team.” In this sense, observability as culture is the degree to which a team or company values the ability to inspect and understand systems, their workload, and their behavior. Companies who have a strong observability culture often have observability teams, although many of them don’t call it that. An observability team is dedicated to making systems observable and (usually) building monitoring, to carry observability to a conclusion in knowledge, decisions, and actions. In other words, the observability team is usually tasked with making sure the OODA loop is actually a loop, instead of having a break in it. (I never thought I’d invoke the OODA loop!) Culturally speaking, observability is highly preferable to tooling. If observability is a cultural value, the results will come, even if there are frustrations such as poor tooling. But tools alone will achieve nothing. As Theo also said, “You can't really have effective monitoring without a culture of observability. Tools are not enough.”

What Observability Is Not

In my opinion, observability isn’t a lot of things that seem to be implied by the context in which the term is used. Maybe I shouldn’t judge too quickly, but I feel like a lot of people are kind of buzzword-dropping when they say “observability” in a context where monitoring or instrumentation would fit much better and is more grammatically correct. It seems (seems!) they believe they’ll sound edgier, more forward-thinking, or something. I think this is what I’ve also seen other react to negatively. I don’t view this negatively myself; I just think it’s a phase we’re going through. So I offer this in an attempt to contribute to the discussion. I don’t think it’s the best idea to use “observability” per se, when you really mean any of:

• Tools. See the microscope/slide analogy again.
• Monitoring. Monitoring is a verb, something you do. Observability isn’t a verb, it's a thing you (hopefully) have.
• Debugging. Observability isn’t just a deeper or richer kind of monitoring. Observability

When someone says “X is a tool that provides observability…” I mentally substitute the words so I hear “X is a tool that provides instrumentation” or “X is a tool that provides analytics,” and it makes a lot more sense to me.

Parting Thoughts

I’ll leave you with a few thoughts. I think it’s legitimate and productive to talk about the observability of systems, and observability teams, and so on. But I don’t think it’s useful to use the word in a way that makes no logical sense and is bad grammar. Clarity of communication should be the goal; it’s how mature engineers behave. Using words to add sizzle at the cost of clarity runs counter to what’s best for all of us. Secondly, if people squint at you when you say “observability,” it might be a sign you’re confusing them and should use a term that’s just as good or better, and which they already understand. A term like monitoring, or debugging, or instrumentation. Third, I have no complaint with most uses of observability, such as someone talking about an “observability tool.” I’m not going to repeat the mistakes of those who argued about the One True Meaning of DevOps in a new domain, for example. I don’t think it’s helpful to try to trademark and claim to be the arbitrator of a word’s meaning. Let’s not reject or shout down, let’s not exclude, let’s be inclusive; let’s speak clearly and listen thoughtfully. And concisely:

Monitoring tells you whether the system works. Observability lets you ask why it's not working.

— Baron Schwartz (@xaprb) October 19, 2017

Finally, I encourage you to read Cindy Sridharan’s article on observability and how it’s different from monitoring. I don’t agree with all of it, but I think she significantly advanced the conversation. Plus she has a hell of a lot of great insights about monitoring and many other topics, and links to further reading. (Also: this). PS: has observability been abbreviated o11y yet? Let’s do it.