In August, a ransomware attack hit another company. Unfortunately, it hit a regional cloud provider in Europe this time, and we can call this a “critical hit.” So far, we know a virtual server got compromised and used as a jump host; from there, the attacker started to encrypt all volumes in the same domain. Based on pure luck or some profound reconnaissance, the same server migrated into a different data center and continued its unplanned job from there. The result was not only the production, but also the backups were encrypted.
I would rather not blame anyone. Things can happen. But I would like to raise awareness about a particular security topic:
Cloud Backups
Many organizations use clouds in one form or another, but let’s focus on PaaS and SaaS for simplicity.
For a SaaS solution, we mostly have no choice but to trust whatever promise the provider made or, more precisely, what was agreed to in the contract. Typically, we’ll hear, “We take care of backing up your data,” but as usual, the devil is in the details. The fine print might say, “Archival backups to support customer-initiated data restores for 60 days” or something similar.
Another vendor, and sorry for the caps, but it’s copy and paste, says, “(vendor) DOES NOT GUARANTEE OR WARRANT THAT ANY CONTENT YOU MAY STORE OR ACCESS THROUGH THE SERVICE WILL NOT BE SUBJECT TO INADVERTENT DAMAGE, CORRUPTION, LOSS, OR REMOVAL IN ACCORDANCE WITH THE TERMS OF THIS AGREEMENT, AND APPLE SHALL NOT BE RESPONSIBLE SHOULD SUCH DAMAGE, CORRUPTION, LOSS, OR REMOVAL OCCUR. “
At least they’re honest and say, “It is your responsibility to maintain appropriate alternate backups of your information and data.”
The problem is most people need to be made aware. Both businesses and individuals choose SaaS solutions for their simplicity; they would rather not manage them themselves and expect backups to be included. That’s not happening.
And now for the platforms.
For PaaS, the situation is different. It is, or should be, obvious that backups are a shared responsibility, similar to security. The provider takes care of physical security and availability, but you must protect your data.
When you set up instances or data stores on any hyperscalers’ infrastructure, you will find a checkbox for backups. You can set up frequency, retention, and location and will be charged for it. It’s comfortable, doesn’t require any complex setup, and works more or less automatically. What could possibly go wrong? Do you remember what I explained at the beginning?
So, here we are: You need to take care of your backups, and you’re better off not using the same provider.
It’s simple, and you have options.
First, a few companies specialize in precisely this task, and guess what? They’re called BaaS, Backup as a Service. They have integrations available for all major providers and more, and you could link SaaS or on-premises resources to their services. It would be my first choice to check a few of them out, see which one comes with the necessary integrations, and check the boxes.
If budgeting is a concern, look into alternatives. While different technologies exist, Amazon S3™ is the de-facto standard used by various businesses thanks to its API.
I’m picking this example for no particular reason; I happened to have the app already open in another browser window:
No black magic is involved; add basic data for authentication, and you’re good. It will look similar to any other application or service. Yes, it’s a few more boxes than “click here,” and accounting might ask you why there’s another bill from another service. But it’s far, far easier than trying to explain why everything is gone. Like, absolutely everything.
TL;DR
Use a different cloud provider for production and backup. Options exist; use them. It’s no big deal and helps to prevent the ultimate disaster.
The worst part of the initial story is this particular cloud provider is mainly done. The situation is a worst-case scenario; it will be challenging for them to stay in business, and they might even take some of their customers with them. Let’s hope for the best.
Double down on your backups, and ensure the CIO and the CISO are involved.
Oh, and also make sure they are tested, but that’s another story.
