Announcer: This is SolarWinds TechPod, The Brown Report. In this episode, host Stefanie Hammond…
Stefanie: “Over 85 percent of successful breaches stem from small and medium-sized businesses really having poor cyberhygiene.”
Announcer: speaks to Tim Brown, SolarWinds VP of Security Architecture…
Tim: “How much security risk are you willing to accept? What will you do to reduce your risk?’
Announcer: …about what managed service providers need to do to keep themselves, and their customers, safe in an increasingly hostile security environment.
Stefanie: Hello there. My name is Stephanie Hammond. I’m a senior channel sales specialist for SolarWinds MSP and today I’m joined by Tim Brown, our VP of security. In today’s session we’re going to be talking about some of the fundamentals that MSPs need to think about as they establish and evolve their security offerings. So welcome, Tim.
Tim: Thanks, Stephanie. I’m really excited to be here. It’s a great opportunity to talk about how MSPs should focus on cybersecurity and talk about the basics and some of the advanced concepts that you need to know.
Stefanie: Excellent. All right—well, let’s dive right in. The question that I need to ask is: you didn’t rise to the position of VP of Security at SolarWinds without having spent a significant time in the industry. In your opinion, how has the state of cybersecurity changed over the last few years and how are we seeing the industry respond?
Tim: Yeah, security is always changing, you know, that’s why it’s an industry that I’ve always loved, right? I’ve been doing it for 20 years. And when you look at what has evolved over the years, you see the threat actors are continually changing what they’re doing and the methods that they’re using. They tend to focus on where the money is, where the control is, where the they can make the most impact. And you see that through the changes. So in the last few years, right, we’ve seen a huge ride in ransomware—and ransomware works, right? And it’s been the model that people have been utilizing to take advantage of people. One of the main reasons why is they no longer need to exfiltrate data; they don’t need to take data out of a system when they compromise it. That takes a huge piece out of their program that they would need to develop. So not taking data out allows them to do things more effectively and more efficiently. Remember, they’re running businesses, and if you look at the rise in cryptomining—the reason why cryptomining has actually risen so much and so quickly is because of a lot of our bad guys are not so bad. Truthfully, they looking to not cause harm. They’re looking to make money, right? Cryptomining allows them to take over a machine, take over a processor for that machine, but not actually do harm to that machine. So you see shifts in the adversary and what their focus is just over time, and the places they can make money or gain control.
Stefanie: Excellent. Alright, so what advice would you give to an MSP? They’re a novice when it comes to providing security services. They’re looking to branch out into this space. What’s your advice on how to begin those security discussions with their customers if they’re not having those discussions already today with them?
Tim: So the important thing is to think about security not as a binary. Don’t think of it as “I am secure, I am not secure.” That’s a kind of naïve approach to it, and it’s not an effective approach. Most of our MSPs are already managing business risk for their clients and they understand what their business is. They understand what the impact of their business would be if something occurred. So start that conversation on the same idea of, what would happen if you had a security incident? How much risk are you willing to accept? What will you do to reduce your risk? So when you think of it as a risk conversation it’s a very different conversation than an “Am I secure?” It’s a conversation of, “Okay, how much risk can you face and what can you do to reduce that risk?”
Stefanie: Alright, so when they go out and are starting to think about protecting their customers from a security perspective, what’s the first area of focus that the MSP should look at and consider?
Tim: You know, what they need to do is look at the company’s business, look at their customer’s business, and take a look and say, “Okay, so in this business, what’s the most important things that you have? What would happen if those most important things get compromised?” So if it’s a hospital, what would happen if patient care gets compromised? If it’s a small retail business, what happens if they can’t take credit cards for the day? What happens if the information they collect gets lost? If it’s a retailer, what happens if something occurs? So understand what I like to call their “crown jewels” and then start thinking about how you protect that, how you eliminate risk on those items.
Stefanie: So again, this concept of using security as a differentiator—as VP of security, you have to have some great examples of businesses that have used security as a differentiator, used it well. Are there ways that an MSP maybe can help companies recognize, when they’re having their conversations, that security could be a differentiator and a selling point for them?
Tim: Yeah, absolutely. I always liked to use security as a differentiator for the clients, rather than insurance. You should look for opportunities there. I’ve talked to thousands of customers over my years and every once in a while somebody surprises me. I had this company coming in to talk to me, and it was a manufacturer or a pipe—you know, physical pipe, right? So I figured I’m going to talk to them about good hygiene. I figured I’m going to talk to them about the basics. Instead, I ended up talking to them about advanced analytics. I ended up talking to them about insider threats. I ended up talking to them about all these advanced things and I’m like, these guys have the best security I’ve seen from anyone. It was just a really, really big surprise. So I’m like, why? You’re an old-school manufacturer of pipe! How come your security program is so good? And it was because of their customers. Their customers where BP, Exxon Mobile, Shell. All of their customers needed them to reach a high level of security to allow them access to their internal systems. Since they had access to their internal systems, they streamlined their whole order process, so that they could efficiently get pipe to where it needed to be when it needed to be there. And changed their whole business model. And so it was all about their business, in order to use security to get into the door of the more mature security people—and it set the tone for their entire company. So that’s what you need to look at for small businesses: who are their partners? If you know, Target—how was Target infiltrated? It was infiltrated by the HVAC vendor. The HVAC vendor did not have an appropriate level of security and they had too much access to the Target back-end. That HVAC vendor was the size that would very much be an MSP customer. Another one we’ll talk about quickly with somebody that didn’t do that and another small business and that business was a custom manufacturer of pumps. They used pumps in a lot of different things. And I talked to them. It’s like, “Oh, well, yeah, we don’t have much for security, we don’t have security guards, we just manufacture pumps.” And I said, “Oh, are they connected?” They said, “Oh yeah, we just released, we’re now WiFi connected, we connect everything and we can assess everything. We can access anything from anywhere in the world. It’s making everything so much more efficient.” I was like, “Well, where are your pumps used?” “Well, they’re used in nuclear power plants.” So after that conversation, they have absolutely taken a stronger look at their security. And MSPs need to think about, “Okay, what business is this and how does that business affect it? How interconnected is that to the environment? What would bad guys do if they got hold of that information?” And then help their clients realize that they could be more secure, but also use security as a differentiator for them. Now when they go in to sell their pumps, they can talk about how secure they are and what they can do for the environments that they’re selling into.
Stefanie: So Tim, we have a security white paper called 10 Steps to Proactive Security. And you speak about this concept of practicing good cyberhygiene. Love that term. Because over 85 percent of successful breaches stem from small- and medium-sized business really having poor cyberhygiene. Can you elaborate on what this means, to practice good cyberhygiene—and how MSPs can use this concept to protect their customers and maybe uncover some new revenue opportunities for themselves?
Tim: Yeah, absolutely, and it’s not just small/medium businesses that suffer from good cyberhygiene issues. If you look at a lot of the major breaches, 85 percent of them are because of poor cyberhygiene. When we look at hygiene, what does that mean? That means you’re taking care of the basics. That means that you’re patching systems. That means that you’re managing them well. You know a well-managed environment is a secure environment. You need to be managing it well. That means you need to be patching well. You need to have antivirus there. You need to have backup in place. You need to be able to watch your logs. You need to be looking out for threats that are very apparent in the environment, if you just are watching tightly. And our MSPs today do this very well, it’s one of the key things that they do, they manage their environment, they manage them well, and they monitor them well, they understand that system went down, this space is going to too high, there’s a processor that is coming out of control on a certain device. So they are practicing the basics of good cyberhygiene every day. What they need to do is just take it up a little bit more and you know, ensure that the patches are there, help measure people and measure the level of risks that people are under and let them know how important that practice is. Because if they’re doing their job well on the management side, it’s reducing risk an incredible amount. So lots of opportunities for them to think about that and talk to their clients about that. One of my other jobs here at SolarWinds is I run the operations teams for SolarWinds. SolarWinds has, I think, 56 countries and huge environments. So one of the things that we’ve focused on, that my team focuses on, is on heavy-duty hygiene, right? We look at every log, we understand when somebody logs in, we understand when somebody is trying to gain access to an administrative account, we understand which systems have updated antivirus on them. We do that every day and it is one of the hardest and unsexiest parts of security that there is. It really is, but it’s what is so important to do well.
Stefanie: On that thought: How can the MSP measure themselves to determine if they’re doing a good job at protecting and managing their own customers’ risks?
Tim: Yeah, and I think it’s important that they are looking at the set of functions that they do for the customer. I think it’s important that they inform the customer and measure their customers and let the customer know what they have done. So providing appropriate reports, providing appropriate proof of what they’re doing. Outlining what they do for their clients and outlining it on a monthly basis so the customer understands that, “Hey, this is what I’ve done for you. You had these machines that were out of scope, that needed to be patched. I’ve patched that. You have these machines that didn’t have antivirus on them or didn’t have up-to-date signatures. We fixed that. We didn’t have this.” So continually doing the good hygiene, measuring themselves and reporting to the customer because that will give the customers a good sense that you’re doing work for me, you’re keeping up for me, you’re helping me to make sure that, you know, I’ve taken care of the things that can cause the most damage.
Stefanie: So any other advice on how MSPs can turn security into that key differentiator?
Tim: Yeah, there’s a lot of different things, right? So being in the security industry for a while, one of the things that I would rather sell is the advantages people have of being secure as opposed to the insurance side of being secure. Right? And that starts with business knowledge, that starts with doing appropriate security for their business. So if an MSP can go into their client and talk about, “Okay, here’s what your critical devices are, here’s what your critical environment is. We don’t need to protect everything at the same level, but these are the things that would cause the most harm if they were compromised.” So they can go in and have that conversation and that conversation leads to more business for them. That conversation leads for them to be more of a business partner to them. That conversation leads to additional high price consulting services for them.
Stefanie: Absolutely. And switching over to the regulatory environment. It’s getting more and more complex. That four letter word, GDPR, of course comes to mind. How does all of that play into security for the MSPs?
Tim: So regulation has always been a security guy’s friend. It’s been a bane to many customers, right? And trouble for many customers. But from a security perspective, it’s a reason to do the right thing. From a security perspective, it’s a reason for customers to spend money. Any customer that you talk to in a regulated industry will say: “What’s my first priority? Beating my regulation so that I can still survive and make money.” First priority every year for every customer. So from an MSP perspective, they have to realize that—regulated industries, this is where the money is going to be spent. So 90 percent of the regulations out there have some form of, “You must secure your data, you must secure the environment. You must make sure that the environment has appropriate controls in place so that you’re not negligent.” So that goes for every regulation. GDPR specifically, it is about protecting personal data and it’s about making sure that you have appropriate rights and rules to be able to inform your customers that you have collected data on them and the data that you collect on them must be appropriately protected. So the MSP has a lot of opportunities to be that business partner to their clients for consulting: “What should I do for GDPR?” And then has an ongoing responsibility to protect that personal data that they have collected.
Stefanie: So we all know the saying, knowledge is power. Our MSPs are always looking for advice on how to have those conversations with customers, especially as they want to get more into selling security services. For those MSPs that don’t have a large team, they don’t have anybody on staff with a specific security background currently. You know, one particular customer of mine comes to mind where they only deal in the education space, but they want to get into more of the security services selling. But again, they don’t have a large team, they don’t have anybody on staff with specific security background or knowledge. What steps would you recommend that MSP takes first?
Tim: You know, I think we go back to the top 10 things that they should focus on and think about the business outside of what they already do. So they’re already a business partner to their educational facilities, right? So they already are saying, “Okay, how do I protect students? How do I manage you? How do I help make sure your systems are up and running?” And then they start looking at, “Alright, so where’s the most risk that you face and how do I protect that risk for you?” So it is a more practical conversation than security-specialized knowledge, to start with. When you think about education, one of the biggest threats we have is pre-credit youth. Pre-credit youth are worth a lot of money on the black market because I can establish credit for them, they don’t know they have credit, and I can utilize that on the back-end. So it’s very easy to sell a full record for a pre-credit youth. So that’s one of the things that I really need to protect. Well, when I look at that, okay, so what does that mean? How do I protect it? Well, again, a practical approach, right? So who has access to the database of everybody? All right, I’m going to make sure that I only limit that level of access to the right people. What systems can access it? Let me make sure those are protected in a better way. So look at it from a step-wise perspective. This is one of your “crown jewels” again, right, that you’re going to protect better. So be practical. And now after you go through the practical steps, start learning some more about what your adversary would do, what methods that they would use to infiltrate an environment like that, and then put protections in place that are appropriate for the level of security that you need. So it’s not a go buy the silver bullet, it’s not a go buy the next product, it’s do the hard work and have a good focus on how am I going to protect those assets that need the most protection.
Stefanie: So talking about security, you know, from that standpoint, I really want to include the people behind all of this. So maybe let’s expand on that point.
Tim: People are always one of your weakest links, right? And so the more education that you can help as an MSP, help your clients have and help them understand both education of the what ifs. What happens if Stephanie’s machine gets compromised? What’s the result? What gets stolen? What gets lost? What access could somebody have? Is it a great level of access that could be given or is it a small level of access? And the more you can limit that level of access or the level of impact something has, the better you are.
Stefanie: Right. Okay. Leading into that, when I talk with a lot of my customers having conversations with them, security is still this beast that they’re trying to kind of wrap their arms around. And I’ve had some partners, some customers present me with their security service offerings. Convoluted. Can you apply the K-I-S-S principle to putting together a security package that MSPs can offer to small/medium size businesses?
Tim: Yeah, absolutely. If you really do keep it simple, if you really do start with basics and then go to more advanced services for certain clients, that really works well. Right? So have certain packages that fit, you know, always have the minimal central core, always have the minimal things that you say “I’m going to take you on as a customer only if you do these things.”
Stefanie: So set a baseline.
Tim: Set a baseline and then set advanced services beyond that baseline, do the baseline really well because that’s where a lot of the attacks will come from. But then other customers really do need some more advanced services. The other part of it is, what other types of consulting services the MSP can offer within there. One of the great things about security services is that they are usually higher-priced, right? So, you know, you can be a virtual CSO for someone, you can do PEN testing for someone, you can do vulnerability assessment. And from an MSP perspective, that means they need to have a couple of skilled people to be able to produce those services. The next tier of services is when you truly go into a 7×24 monitoring and management. And that’s one you shouldn’t go into lightly, right? You should really think about what it will cost. And the effort that would take you to get to that.
Stefanie: Or possibly outsource.
Tim: Or possibly outsource. The outsourcing model is getting more and more popular and it works pretty well as long as the business works well and the margins work well. It’s absolutely a good option. Going into 7×24 monitoring just ends up with a lot of costs and a lot of complexity, a lot of difficulty. So that’s when we just want to make sure that people who do it realize what they’re getting into.
Stefanie: So Tim, you were just talking about the baseline activities that an MSP should be looking at when speaking to a customer about providing security services. Can you maybe remind us or go through kind of a short list of what that baseline would look like to make sure that MSPs are properly covering themselves when they’re talking to their customer?
Tim: Yeah, absolutely. So the baseline really starts with good management, right? So I start with that. Can they manage the environment well? It may sound cliché, but a well-managed environment is a secure environment. So start there. Get antivirus on all your boxes, make sure those boxes are patched, make sure you have good backups in place, make sure that you have a good sense of who has access to what and control the identities, especially. So those are some of the key ones that you need to do. You also want to monitor and measure, right? If you’re doing those few things, then you have set up a lot of the good hygiene that’s necessary to keep people basically secure.
Stefanie: Right? And there’s a lot of different pieces of technology and software out there that MSPs are kind of getting pummeled with seems like on a weekly basis. Recommendations on what that technology tool stack would look like when you’re trying to put together baseline security services?
Tim: Yeah. The security industry has just gone through a big influx of capital in the last five or six years. And with that there has been everybody promising every solution that’s going to fix everything. Right? And if you look at what that’s caused—it’s caused a lot of fud in the environment. It really has. Some solutions are really good, but people are often buying the new solutions, using the new solutions and forgetting about the hard work around good hygiene. If we’re getting the patch, they’re not doing the work around monitoring; they’re not doing the work because they bought product XYZ, right? Right. So it’s important to look at what you need, but also look at what you need to do the basics. And then after that is complete, then you start investing in more advanced components.
Stefanie: Layering them in.
Tim: Layering them in. Exactly. Make sure you do the first and the basics well, then start layering in additional services on top of that. Add protection. So there’s a lot of good ones, right? So what you need to think about is next generation antivirus. Good for you, beyond regular antivirus. For certain environments, very important, is true identity services, insider threat monitoring. Is advanced network controls necessary for you? Could be. Are basic network controls enough? So each one of those kind of adds onto what you have started to do and they add value for those environments that need that extra value. Don’t need to get it there for everyone. Probably don’t need to get it there for all of the environments, but in some environments you really do want to do more advanced protection.
Stefanie: So it definitely sounds like MSPs should be using security, you know, as a means of getting in the door, having those conversations, trying to win that new business. Again, coming back to advice that you can give to help MSPs really build up their confidence so that they can start having those security and business risk conversations with their clients. And I’m thinking more specifically around the objections that they’re going to hear. And I hear them from my customers all the time, saying “We don’t need that. We’re not that big. We’re just tiny. No one’s going to be targeting us, we don’t have the budget for something like that. And I thought you already were protecting us. So now you’re saying you’re not, we need more?” So: advice on how MSPs can get in the door and kind of overcome some of those objections they may hear?
Tim: So one thing about security in the last few years, it’s always in the news, right? So everybody has heard of some level of an attack. So they realized they are out there now. So the customers, large and small, know that they have some level of threat. What they don’t have is that linkage between how much I’m at risk versus not at risk. And they often believe as you said, that they’re not at risk. So the first step is always identifying opportunities to have them not be a victim of opportunity. Right? So they are not simply what I like to term as a drive-by, right? If I’m just scanning the network for stuff and I find their systems, and they’re completely vulnerable, I will compromise them. Right? Plain and simple. Because it’s just my automated process that’s going through. So every customer should meet that kind of minimal essential core of security. And if you can’t get the customer to there, you often have to walk away from that customer.
Stefanie: It’s a hard message.
Tim: It is. But as your IT provider, which is what they are, they have to say, “You need to take some responsibility for your own security and allow me to do my job to secure you. If you can’t do that, then I’m sorry. I can’t be your provider.” And it’s a hard thing to do. But when you look at the implications of saying that “I am your IT provider, but I’m not securing you whatsoever. Your machines aren’t patched. You’re not running antivirus on the systems. Your risk is too high and you’re not willing to pay that money to get there.” That’s not an acceptable customer.
Stefanie: Tough message, but a good message.
Tim: It’s an appropriate message, I believe, and it is hard. It’s hard to walk away from business, but when you think about what is going to happen if you’re on a fixed fee pricing model and you have offered them a service for a fixed fee and you have a high likelihood that they will get compromised, that fixed fee will cost you a lot of money.
Stefanie: Not just money; reputation and everything else. So in all of this, is there such a thing as an MSP being able to provide absolute security to your customers? And I know this is a rhetorical question.
Tim: Absolutely not, absolutely not, and they should make sure that in their contracts and other things that they expressly say that. That they will do the best they can to be able to protect the environment, but nothing is perfect. Right? Nothing anywhere is perfect. Even at large corporations, we adjust our security for different areas of the business, right? We give up land. We say that this is not as secure as this simply because we can’t afford to be having the same level of high security across the environment. So even at a small business you’re going to say, “Okay, here’s where you are very, very, very secure and we’re going to do our best to make sure that those crown jewels do not get impacted, but down on these other systems, yup, somebody can lose a laptop. Okay, so you’ve lost data. Was the data encrypted? Okay, well, our harm is not so much.” You know, but those things can happen. Somebody can get infected at home and come back in. So those things can happen. But what’s the impact that would occur? So there is no silver bullet, there is no 100% secure, but our MSPs can play a great role in really helping to protect the small business that doesn’t understand and, you know, pushing them to have good hygiene and pushing them to take appropriate level of risk in their environments.
Stefanie: So Tim, what would you say would be, the most devastating hacking security breach incident that you’ve been involved with in your career and, you know, really what was the root cause? What was the outcome? I’m sure, again, you didn’t get to this level without some stories.
Tim: Yeah, so I can’t go into too many specifics about some of them. But in some of the larger organizations I’ve been in, we had a lot of nation-state activity, and what that means is that we’ve got a very sophisticated threat actor coming at us and we can eliminate them from an environment and we can get them out of an environment, but they continually come back with a new model. So hence the idea of nothing is 100% secure. And this was for a large enterprise and you know, what we had to do is we knew who they were. We went through and fully diagnosed the methods and models that they were utilizing and we could determine that they were in our environment. We could determine what they were after in the environment. We were able to stop the exfiltration of what they were able to get at. But one of the most important parts of it was how you respond to incidents and what you did in response to an attack like that. And it’s not just any attack, right? So from an SMB or MSP perspective, you know, one of the things that they need to consider is what they’re going to do when they have an incident, what are they going to do for their client, what is a client going to do when they have an incident? And that’s one of the most important things. It’s about being prepared. So in order to be prepared, they need to have a plan. They need to test that plan, they need to test that plan often. They need to have appropriate people in the chain of command on that plan and have external people ready to be called if necessary. So let’s take a second and go through a real example, right? So a small company gets breached and knows a threat actor is there because a machine is discovered that has some advanced malware on it that’s exfiltrating data. What do we do next, right? You call the MSP, and what should the MSP do? Now, if the MSP has thought, “This isn’t okay, we’re going to invoke our incident response plan and that incident response plan will have a number of things that we’re going to do. First, we’re going to try to put safeguards in place and contain it. Next thing we’re going to is figure out, has it spread. We’re going to hunt for it in other places. Next thing we’re going to do is figure out what our communications plan is. Do we have to tell our clients, do we have to communicate appropriately? Next thing is having legal involved in that communications plan. Then you decide, okay, where am I going to respond? What am I going to do?” So just the blocking and tackling is not enough. You have to have both communications plan, response plan, and then remediation plan and then a follow-up plan to decide, how am I going to not make that happen again?
Stefanie: One of the other questions that I received from one of my customers when he heard I was having the opportunity to interview you today—his question was, you know, obviously we’ve seen a lot of high profile breaches where the cause of the breach was related to more human error versus more of a technical flaw. The example he provides is an IT administrator turns off multifactor authentication as they’re working through a technical issue and forgets to turn it back on. As an industry the MSPs tend to focus on going back to the technology tools and blending the tools to solve the problems only to get burned by workflow processes or lack thereof. So that really ties into what you were just saying. So any additional advice, guidance on how to manage the human element?
Tim: Yeah. The human element is always one of our weakest links, right. And being able to adjust behavior for people is not necessarily the skillset for a technologist. That’s one of the things that we see in a lot of industries. One industry that’s really done it well is the manufacturing industry. So the manufacturing industry, if you look at anybody that’s doing large manufacturing—so Bechtel, for example, right, they build nuclear plants and power plants and dams and manufacturing all over the world, right? Huge business, multibillion dollar business. What they were facing was a 4% fine from OSHA because of safety violations. So they put a complete safety program in place that is now kind of a standard for the industry. What that safety program consists of is every meeting starts with a safety message. From the CEO down. The CEO walks into the room, says, “Okay, we’ll start with our safety message. The exits are here and here, please watch the cords on the floor and be careful if we have an alarm going off. Now let’s start our meeting.” They take it extremely seriously. They have messages and signs all over the building about safety. That has reduced their overall OSHA violations, I believe from 4.5% to something under 1%, saving them billions of dollars in the mix. And how did they do that? They modified human behavior to be able to think about safety at a different level. Same thing can happen with security, right? And a few companies have worked well at creating a culture of security and the MSP needs to realize that people are one of the weakest links and helping them, helping their customers with a security culture. Start at the basics. Start giving them notices, give them information, give them knowledge, let them know what’s appropriate and what’s not. So there’s some basic things to start leading to that culture of security.
Stefanie: Right. And then making sure we’re documenting things and we have processes and procedures all in place and they’re being checked.
Tim: Absolutely. Absolutely. And just being prepared for an incident if it occurs and understand, “Okay, how can you use this as a training event for people? Okay, you made the mistake, okay, let’s figure out how we can use that to help that not happen for more people again.”
Stefanie: So, thank you, Tim. That was fantastic advice. A wonderful discussion. I’m sure our MSP listeners have gained some extremely valuable insights from your deep knowledge of the cybersecurity landscape.
Tim: Thanks, Stefanie. It was great to talk to you and I hope people got some good value out of it.
Announcer: Thanks for visiting. We’ll catch you on the next episode of The Brown Report from SolarWinds TechPod.
Announcer: Remember, subscribe, rate, and review the podcast in iTunes, Google Play, Spotify, and SoundCloud.
