The 5 Principles that a Performance Management Service Should Follow
August 17, 2017 |
I’ve recently taken some time to think about what it means for our organization to operate and identify as a "performance management service." It's not as simple as placing ourselves in a specific industry category or engineering a certain kind of app. There are other, complex, less tangible elements too: how we interact with the systems we monitor, our responsibilities as experts in our field, the obligations we have to our customers, and the standards we hold ourselves to.
As a result of these thoughts and considerations, I've written this list of five principles that I think communicate the goals that any monitoring service should accomplish (or, at the very least, aim to accomplish). Hopefully none of these are surprising. Along with each principle, I've added some details and explanation, to show how these ideas actually look in the real world, when a performance management service is interacting with users. I’d be interested in hearing what other people think about this list and the values it communicates—don't hesitate to let me know in the comments section below.
Leave no trace (aka do no harm).
- Monitoring shouldn’t affect the system that's being monitored.
- "We must never negatively impact the systems we observe!"
- You must have a reasonable standard for how to define a "negative impact." Monitoring will always have some overhead, which makes it even more vital that such a standard be clearly defined and understood—and never violated thereafter.
- Medical doctors take the oath to do no harm in the course of their work ("primum non nocere"). A monitoring service should do the same.
Just because you can collect certain types of data doesn’t mean you should.
- When you collect data, you're implicitly saying it is important.
- Avoid unrelated data or vanity metrics at all cost.
- Users are communicating that they trust you when they grant you access to their systems; don’t violate that trust by collecting something you shouldn’t.
- And, of course, avoid collecting information that you aren’t allowed to have (e.g. don’t violate security/compliance laws).
Present data clearly and transparently.
- A confusing chart/metric can be just as harmful as an incorrect measurement.
- Users have a right to know what computations your monitoring platform has performed on their raw data prior to their viewing it.
- Your service should do the heavy lifting of isolating and presenting the data that's relevant to the user; if the user has to sift through data and determine what is and isn't valuable, the service should be more deliberate and clear in its presentation.
- Avoid superfluous information; focus your user’s attention on what is most important to them. A monitoring service and the people who design it should have strong opinions on what users should be paying attention to and on what is and is not important to measure. By using a monitoring product, the customers of the service implicitly endorse and invest in the practices the service has established. The monitoring service should, then, ideally, communicate and clarify these principles as part of the product's UX. An especially adept service should work to educate its user base on its opinions and positions in its marketing efforts and community presence.
- It’s important to understand the distinction between focusing your data and censoring the record. If you determine that something is of lesser importance but still potentially significant, the data should be available, just not emphasized.
Don’t interrupt your users' focus.
- Alerts demand attention, and superfluous alerts can quickly desensitize users to their importance. John Allspaw, former CTO at Etsy, has written extensively and very well on this subject, with articles such as his Open Letter to monitoring and alerting companies and his piece on understanding your effect on "users' attention," both worth reading closely.
- When you create interruptions, there must be no tolerance for false-positives/unactionable alerts.
- Users can choose to be alerted by something, but you should strive to guide them so that any alerts they customize still contain the most possible value.
Don’t lose data.
Taken altogether, these cardinal rules for monitoring services should guide you in not only doing effective, useful work, but in doing ethical work too. Many of these ideas might seem obvious or self-evident, but there are occasions when fulfilling them can be more complicated than you might think. As we've hinted at several times above, there's an implicit, delicate contract made between a monitoring service and any of its users — fulfilling your half of the agreement involves following through on each of the items listed above. If you can, your users will benefit from a data-rich, transparent, actionable, non-intrusive service—and they'll thank you for it.
- Users rely on you for a record of what has happened inside their system. You need to preserve that record.
- Lost information is lost trust; without trust, a monitoring service is useless