Announcer: This episode of TechPod is brought to you by the SolarWinds IT Trends Report 2020, the Universal Language of IT. Explore the priority areas tech pros manage in a world where roles have converged, and the reality of how it’s affecting skill sets across IT departments and nontechnical areas. Visit IT-trends.solarwinds.com to learn more about key findings.
Leon: For most of our listeners, and most of the world for that matter, February 2020 marks the point in time when everything changed. As stay-at-home and shelter-in-place orders locked into place, schools, stores, religious institutions and businesses began to shutter their physical locations.
Leon: Seemingly overnight, organizations cobbled together remote work solutions even as employees cleared away dining room tables and picked over electronic store shelves for Chromebooks, webcams, and noise-canceling headphones to turn their homes into multi-use office, gym, school, house, and pet centers.
Leon: But that was then and this is what day is it again? Who knows? Who cares? The topic businesses are now facing is the reverse, people are coming back to the office. So, how are IT departments helping their organizations craft a successful return to office or RTO plan?
Leon: With me today to discuss all of this is the Director o f IT Infrastructure, Physical and Cyber Security for Texas, RE, J.W. Richards. Welcome.
J.W.: Howdy, y’all. Thank you for having me.
Leon: Thank you for being here. And also, we have Rani Johnson, who is the SolarWinds CIO. Thank you for joining us.
Rani: My pleasure. Thank you.
Leon: All right, so before we dive into this really interesting topic that I think a lot of people are wrestling with right now, I want to give everyone a chance for shameless self-promotion.
Leon: That means you can talk about basically anything you want, about you, your organization, groups that you’re associated with, where people can find out more about you, or any of that stuff. So, J.W., SolarWinds family etiquette states that you get to go first.
J.W.: Well, thank you very much. I am @JWRichIV on the Twittersphere. I’m the only J.W. Richards the Fourth with an IV on LinkedIn, it turns out. You’ll find that those are ragingly inactive because it when I ever can get to the point where I’m not plugged into a computer, I don’t plug into a computer. I work for Texas RE as Leon mentioned. We ensure the reliability and security of the electric grid for Texas. In other words, we make sure that power providers do the right things and keep your lights and AC running.
Leon: Nice. All right. So, Rani, tell us a little bit about you.
Rani: While I am shameless, I have no self-promotion and no social media to share out. I’ve been doing engineering and IT for 25 years. I absolutely love it. I have a sign on my desk that says Rani, Queen of the Nerds. Absolutely, it’s my joy to do this work. And so, I look forward to this conversation.
Leon: Wonderful. Alright. On the off chance that people listening are wondering about who I am, I’m Leon Adato. I am a Head Geek. Yes, that is actually my job title. I took the job just for that. At SolarWinds, you can find me on the Twitters, which I say just to send my children into spasms of horror. You can find me there @LeonAdato and also you can find me THWACK.com which is the SolarWinds user forum and community. And that’s @adatole.
Leon: And if you’re scribbling this down, stop, put your hands back on the wheel and pay attention to where you’re… No, we’re going to have show notes and we’re going to have everything we mentioned, any link, any URL, any document that we refer to is going to be in there. So, you don’t have to write this stuff down.
Leon: All right, with all of that out of the way, I want to start with a little bit of a catch-up. Let’s go back to the beginning and how we got to our current situation. What stands out in your mind as you think about those first few whiplash days when the stay-at-home orders locked into place. What was that like?
J.W.: I mean, for us, the transition was pretty easy. We had a very nice work from home schedule in place. We have tested our business continuity plan annually over the past three years even to up to a week that we forced everyone to work remotely under some guise of zombie apocalypse or… Actually the transition to it went a lot better than a lot of folks had it because of not just the planning that my team was able to do over the past several years, but the allowance that our leadership had given us to be a work remote or work from home type of organization.
Leon: Nice. So, in programming terms, you kept on running the chaos monkey on staffing?
J.W.: You got it.
Rani: For us, Leon, it was a little bit more complicated. We very much for a work-from-office shop in that we have a lot of our support and sales folks that have use our call centers. And so, the biggest change for us was very quickly deploying soft phones. And so, we had to cut over our entire kind of desk phone infrastructure to soft phones and in a short three days and that work needed, we had to test to make sure that that work met the needs of our business. We did have business continuity plans. And so, for most of our infrastructure and contingency plans, we were able to transition pretty smoothly. But it was a big switch from moving from desk phones to soft phones for our staff.
Leon: Right. Right. Exactly. I mean, just the physical move-out, I think, was a challenge for a lot of organizations, right? Like people needed to… realized they needed to get stuff, they couldn’t get back in the office, some were under quarantine orders. Did you have to deal with any of that?
Rani: Absolutely. So, some of the offices closed before we’d actually finished some of that cut-over. And so, we found in some scenarios, our IT teams getting special permission, in some cases from the government, in other cases from our building property managers to get back into the buildings and basically collect IT equipment and in some cases, personal property so that we could make sure that we were able to have our teams work remotely successfully. I like to tell the story of the time we went back in and we rescued somebody’s fish. Someone had an aquarium with a pretty decent-sized betta fish in there. And so, we rescued the fish and then watched the fish on fish cam for the next few weeks.
Leon: Right, it’s become a mascot. That’s wonderful. J.W., you had mentioned when we were talking earlier about curbside delivery that you started to do. You got to know a lot more of the city than you did before.
J.W.: No, that’s a fact. I mean, one of the big wins that we were able to bring to our organization was this we call it the take-home or tech package. We were able to get in against a lot of odds that we were able to get IT equipment shipped to us and we were able to deploy extra-wide or ultra-wide monitors to our end users with a keyboard, mouse, noise-canceling headset, and that has become their primary workstation through all of this. And that was a big win for us for sure.
Leon: So, those were some of the successes that you hit and I don’t want to air dirty laundry. But no plan ever goes off flawlessly. What were some of the challenges or missteps that you ran into and how did you recover from them?
Rani: Well, Leon, one of the challenges that we uncovered is that we ended up getting several of our computers BitLockered. And we had no contingency plan where we couldn’t touch a computer where we needed to help people recover from a pretty significant hardware failure. We quickly figured out what we were going to do when we had to make an agile proxy so that people could recover themselves, but we hadn’t anticipated that.
Rani: Additionally, our work-from-home transition happened at the same time as kind of end of quarter. And we were dealing with a lot of people who hadn’t taken the time to reset their passwords. We had traditionally used our VPN for folks to either connect on-network or use the VPN to actually reset their passwords and people didn’t remember because they’re usually in the office when it’s time to reset their passwords. And so, we had to do a lot of white glove treatment on making sure people weren’t getting locked out at the worst possible time kind of end of quarter for our sales and support cycles.
Leon: Right. Right. And I think also patching was kind of a challenge?
Rani: It was. So, this happened a little bit after we went to work from home. We intentionally delayed our patching cycles because we felt like, okay, we’ll be back in the office in maybe three to four weeks. Well, we were wrong.
Leon: We were so naïve back then.
Rani: We were wrong. Yeah. Two months into that and we realized that this was going to be a longer duration. We had to accelerate some of our plans to actually patch online only that didn’t call require us to kind of phone back home through our networks.
Leon: Okay. J.W., anything you want to add? Again, not trying to embarrass the company, but anything that you ran into like, “oh, we weren’t expecting that.”
J.W.: Oh, no. I mean, I feel like we all trip every now and then. So, if we can point out the root we’re going to trip over let’s point it out.
J.W.: Early on in our business continuity plan testing over the past couple of years, we identified some things that we didn’t anticipate, obviously, but some are like ragingly big things like our voice over, our current VPN, didn’t allow for cellphone connection. So, people weren’t able to make phone calls during those work from home tests. So, with that being in the place that actually was the instigator for us actually to move to a voice over IP solution so that people can have that soft phone like you guys have moved to at SolarWinds and then it gets apps on their phones, any way that you can think of, they can get to their phone now.
J.W.: One of the more recent ones that came about during this transition was that we realized that our VPN server wasn’t closing connections properly. It wasn’t leaving them open from a security aspect, but it just wasn’t ending the session and it had a finite number of sessions. So, after 36, 48 hours, people logging in, logging off, it filled up and just stopped accepting connections. So, we did the old turn it off and on again sequence for the VPN server and that seemed to solve that. So, now that’s on a scheduled task. So, it ranges from super big not being able to answer your phone to something as simple as a scheduled task to clean out the bugs.
Leon: Right. Okay. And that actually brings up another thing that if you’ve been in it for more than, I don’t know, 15 minutes, you’ve run into having to invent new processes completely on the fly. Rani’s laughing really hard right now, she was like, yes. So, what were some of the processes that you had to just develop right then and there? Like, okay, here we go. We’re going to just make this happen.
J.W.: So, we got notified on a Wednesday. And it was the Wednesday that I left for vacation for spring break, actually, that we were going to a voluntary work from home because maybe this pandemic might have some impact on us. We’re not sure. It goes back to your statement of how naïve we were at the time. So, on the road, I asked my wife to drive. I logged in to my computer and I start sending out emails and I’m working with the team that was still onsite there about these projects that we had for a month, three months out, end-of-the-summer kind of thing. We’re going to roll out Teams. We’re going to roll Webex single sign-on to everyone. We’re going to make sure that there are voice over IP solution that has a meeting partition added to it that everybody can sign into that. We are going to shove every tool in our tool belt at everyone and then we’ll see where the fallout is on the backside.
J.W.: Prior to this, we had always planned for and trained ahead of time. And Teams is a pretty hefty paradigm shift for most people that have been on Lync or Skype for Business where we had come from, there’s a lot of difference there. And prior to this, we would have trained ahead of time and then handed people the solution. But what we found for this is just piling technology on top of them, they were actually able to go out there and get familiar with it. So, then we started having these Tech Tuesdays meetings after I got back from vacation, thankfully. And we trained on the backside of it. And what we found out was people were able to ask more intelligent and more telling questions because they had actually touched the technology. And we were able to use that as a way to move forward and then I got tapped for hosting virtual happy hours on Thursdays and—
Leon: Because you’re a giver.
J.W.: I totally am. I am a man of the people. That is what I—
Leon: Especially when beer is involved.
J.W.: It is. Any type of alcohol, I’m your person. But we were able to not only push more technology on top of people, we were able to do it in a more safe environment where people felt—and I say people, where I, as the IT director, felt better about possibly tripping and falling on my face with this technology than if I were doing it in an all-hands meeting or God forbid a board meeting or something where if it went wrong, it was real wrong, real bad.
J.W.: And we’ve been able to transition a lot of people over to Teams, into Webex. And it was a way that to get technology out there that I never would have anticipated or thought of using, but that I will definitely use going forward.
Leon: Rani, how about you?
Rani: So, there are several things. One of the things that I was excited about, that was an infrastructure that was in place, a process infrastructure that was in place, is that we did have a business continuity team. What we needed to immediately go to was a daily cadence meeting where we could assemble the executives and let them know what our plans were because some of them were not, we couldn’t anticipate a global pandemic. And so, once we stood up this daily meeting, we actually were able to, I’ll call it negotiate with the leadership team around when we actually did go to work from home. And we wanted to make sure that we had the phone infrastructure in place prior to us sending people home.
Rani: Some things we ended up having to do or things that we ended up deciding as part of that daily cadence meeting is that we needed to actually change our internet allowance for some of our regions where we’d never asked them to use their home internet to do work services that would require them to VPN back in. And so, those are some processes that we had to change. J.W. was thoughtful in having a ready-to-go work-from-home bundle. We didn’t do that. And we hadn’t anticipated that, but we realized we needed to go and negotiate with our vendors to secure monitors and additional peripherals so that we can send people out with the same kind of two-monitor system that they had in the offices to be as productive.
Rani: And so, we had to deploy a few new services. One of the things that I began to do with the desk was daily stand ups as well, so that we could look at how our employees were being impacted and where there were some potential productivity obstructions.
Rani: And so, we started to see some challenges with Webex. So, we started to monitor Webex regularly. Then we realized we just need to monitor every collaboration platform that we have. And we began doing that. And where we saw there were either obstacles in instruction or obstacles in actually the technology, we quickly presented workarounds. And so, we built a SharePoint site for the company to reference on how to do things better. And it became a platform that was used by not just kind of IT, it became the continuity platform, the HR platform, the communications platform, for us to communicate back to the business on where they can get more information around work-from-home tools and working from home. So, we didn’t get to happy hours quickly, but we’re there now.
J.W.: Glad to hear that. That’s where we all want to be at the end of the day, right?
Rani: Exactly.
Leon: So, I want to take a minute and talk about burnout. In IT, I think we’re all familiar with the experience and just the occurrence of it. But I think that the pandemic and working from home has really changed our perspective on what that means and how to address it. So, what were some of your experiences, both with yourself and also with staff as far as how you’re managing that?
Rani: That’s been a tough one. We actually monitor I’ll call it the delivery, the number of tickets. We’ve actually been monitoring the number of calls that our staff have been making and even the number of meetings people have been on. We’ve seen at least a consistent – for the last few months – 30% increase in the overall, I’ll call it, productivity of the staff. And it’s frightening because we also are seeing people not taking vacations. And I feel like people are a little bit in a wheel, kind of the hamster wheel, that just keeps going faster and they don’t quite know how to get off. So, we’re trying to encourage people taking time off, even if they can’t go away or travel on vacation, but that is really a tough one.
J.W.: And for us, it’s trying to shift the window of when IT support is available because that’s the primary function of what we do is, we support the internal customer, as I call them. And we’ve had people reach out to us and they’re trying to work with kids at home, they’re trying to work with pets at home, they’re trying to do all of that. So, we’ve had people reach out and say, “Hey, just to let y’all know. I’m working from 4:30 in the morning until my kids wake up, and then I’m going to be offline until they go down for nap and then I’m going to be back online after 8:00.” And we as the IT organization, we’re glad to do that. I mean, I love that Rani has the “Queen Geek” sign on hers, because I tell everybody when we’re onboarding the people in the organization that we wouldn’t be doing this if we didn’t love it. And we love fixing problems and we love… throw us an issue that you don’t think can be solved. And that’s when we get most excited. And that geek really rears up and starts trying to do stuff.
J.W.: But it doesn’t matter what time they’re working. We talked about in a previous conversation that IT problems don’t really care if there’s a pandemic going on or not. Things are going to fail and you’re going to have to figure out how to fix them. The biggest hurdle, I think, is the people that are used to that 8:00 to 5:00 job driving in in the morning, driving home at night and that commute is their onboarding to work for the day and their disconnect at night. And some of the people that I interface with are having a really hard time making that separation when it’s really easy to just walk back into that office at the end of the day and check a few more emails and then, dang, now it’s 11:00 at night. I should have been in bed a long time ago.
Leon: Right. SolarWinds put out a work-from-home guide, and one of the things that was suggested by one of our product managers, Joe Reves is that if you’re used to that, then take a minute in the morning and step outside the house, go for a walk, maybe around the block, and then come into “work.” And then at the end of the day, do the same thing. Step outside of your office, step outside of where that work space is, go out and then come “home.” And just mentally create those bumpers between it so that you still have that, because otherwise you end up with what one colleague of mine called work-life blending, not work-life balance, but work-life blending where there actually is no differential between 11:00 p.m. and 11:00 a.m. And I might go to a doctor’s appointment at 10:00 in the morning and I might be working at 1:00 in the morning. Just everything’s happening all the time. And most of us are not built for that kind of blur between those lines.
Rani: Yeah, that’s a great suggestion. One of the things we’ve most recently implemented and it’s kind of in a pilot phase only in North America is we’ve asked our desk to work, basically four tens and giving them a whole day away, trying to encourage them to getting some level of separation. But also as we’re preparing for return to office, making sure that they have limited exposure, and we want them only in the office, maybe one of those four days. And so, we’re kind of limiting the number of folks who would have to interface when it’s time to return to office for the rest of the staff, and then just giving them that break because we needed to do something to get them off that kind of that hamster wheel.
J.W.: Yeah, for sure.
Leon: That’s fantastic. Alright. So, that’s a lot of how we got here and sort of what the current state is. And now we’re looking at future state, right? We’re talking about RTO, return to office or RFH, return from home, or God help us, the new normal, or whatever we’re going to call it now. And so, the first thing I want to know is how did you both prioritize the issues that you foresee? How are you looking at those and looking at those variables and problems and sort of putting them into a framework where you can begin to tackle them?
Rani: So, I mentioned before that we have a business continuity team. And that’s not just an IT function. That’s IT, that’s facilities, that’s the business leaders, and obviously, our people team, our HR. And we basically have kind of decided we’re going to take a look at all of our offices. We’re going to make sure that there’s someone responsible as an HR leader to look at the governing mandates around return to office and making sure that we’re able to satisfy at least that minimum requirement and also raise it to the requirement that we want to make sure that our team members feel safe as they return.
Rani: And so, we’ve looked at what we have to do minimally to allow return. There are some offices that we basically said, look, if you’re not what we call, what’s that word, “essential,” if you’re not an essential worker, then we’re going to ask you to stay home because we don’t think we can make a safe enough environment for you. But for most of the offices, we’ve actually allowed people to return on a voluntary basis. So, it’s absolutely return-to-office optional. We’re monitoring capacity of each one of those sites to make sure that we’re aligning with those government mandates. And then additionally, in each one of those offices, we’ve set aside some protocols and making sure that we actually handed out cool face masks, nose-and-mouth masks, that have cool SolarWinds logos on them even.
Leon: Of course, they do.
J.W.: I want one of those.
Rani: And we also put up great signage so people know, they’re reminded to social distance and even there’s a recommended some pattern or workflow. And then in offices that we can’t achieve the appropriate amount of separation as requested or in some cases mandated by the government, we are going to restrict the kind of influx if that were to happen. We’ve obviously, we’re not mandating that any of our staff actually work in the office, at this point, but as we get closer to what will be kind of the return to office, we’ll have to make sure that we’ve got the appropriate distancing. But we monitor every office and every governing requirement, and sometimes you end up with a state and local that are somewhat in contradiction. We have to manage those to that higher standard. And then also make sure that we have a standard that makes our employees feel safe.
J.W.: Ours is a lot simpler. I mean, we have the different layers like Rani was talking about with varying between the city, the county, the state, and the federal, but luckily, we don’t have to cross country borders. A lot of ours, we’re doing the same thing that Rani was describing of making sure we have proper cleaning agents, proper PPE, and hand sanitizer. And then myself and the HR manager are working very closely together to develop those return plans that allow for proper social distancing and maintaining, we’re going to shoot for 50% occupancy when we finally do go back to work even for a month to go through that reacclimatization process of because there’s not just going to be the shift of people coming back, there’s a mental thing that has to happen.
J.W.: Because we’ve been doing this for already three solid months and there’s going to be a couple of more months before we’re ever back in the office like that. So, there’s a rolling period where we stage people from a 25 and a 50, and then a full 100% occupancy and then even after that, allotting for a lot of flexibility. People have a lot of personal stories and personal experiences that have dictated the way that they feel about this pandemic that is not the same for everyone. And we’re trying to keep that in mind and make sure that the employees know that we’re here to support them and that we value their safety and security. And we’ll try to go as far as we can to make sure everybody’s taken well care of.
Leon: Right. Yeah. I think the psychological piece is probably it’s an entirely different podcast, like its own conversation. But you mentioned something. I mean, you started to talk about like just the physicality of moving in and again, we’ve worked in IT for a while. There’s a little bit more gray in my beard than the two of you, which is fine.
Rani: Well, I don’t have a beard, Leon.
Leon: Okay, well, then, I mean, I’ll send you one? You grow it or you faux it, right? Or vBeards is a whole group. Anyway. I think that just the sheer mechanics of moving people around, some of us who are listening to this and participating have been involved in building moves, and facility things, and those are not simple activities or endeavors to undertake. So, what are some thoughts about how people are just going to physically get back into the office after you’ve ripped desks apart and distributed 300 monitors over the course of a weekend or whatever it is. What are some of the things that you’re looking at to get people back to their desk and actually operational?
Rani: One of the things that we deployed, I mentioned that business continuity website that became a resource for the company. And on that, we have been sharing with people the work that we’ve done to prepare each one of the offices. And in there, we’re inviting people to return to the office if they feel comfortable. And so, we’re seeing about 5% of our staff taking advantage of maybe having some separation and getting that kind of that work-life balance we talked about by coming back in the office. And so, we’re really doing that kind of on a voluntary basis. And that’s allowing us to kind of perfect our practice some of that return-to-office procedures.
Rani: Our desk, as I mentioned, the North American group has moved to four tens, but we’ve already gotten them back in the office preparing for how they’ll handle the return of equipment, because we had so much distributed. And also, how they’ll handle break-fixes as we start to handle basically get people new—there’s so much spilling on computers that are happening. And so, we’re figuring out on the desk how we’re going to handle interfacing with people as they come back in the office. And we’ve provided them with instructions on how to return computers back into use after storing and quarantining the computers for seven days, and then also how to clean that equipment so that we can safely make sure that they’re not being potentially put at risk as they’re handling it.
Leon: I hadn’t even thought about that. Yeah, the equipment itself has to be, like you said, quarantined and just make sure that that’s good. And just acquiring the PPE, just getting your hands on it, right? I mean, that you can’t just go down to Costco and get some.
Rani: No, unfortunately, we’re working with our procurement team and facilities team so that we’d be negotiating probably for a few months and finally have been able to secure what we need around the gloves and the cleaning equipment. And then even kind of the safety kind of shields so that we can actually create some separation between the desk and people, if they come in. At some point, we’ll see more than one person at the desk at a time and so, we want to make sure that we’ve got proper areas of separation for them too.
J.W.: Yeah, and the refit of the old work spaces that everybody has been building, the stacking them deep selling them cheap kind of work areas, where cubes are on top of one another. And there’s no partition, or a very short partition between those. There’s going to be a lot of people that won’t be able to refit those on the fly like we’re going to be asked to do.
J.W.: So, one of our contingencies is just the A and B teams or the A, B, C and D teams that we’re going to assign people are going to have to build in that social distancing to just where you sit.
J.W.: If Leon and I were cube mates and we sit right next to each other in our cube for the past eight years or wherever we sit, we’re not going to be coming into work on the same day and that’s been one of the big challenges that the HR manager and I have been trying to assign people days that they’re going to come in based off of the location they sit in. And just building in that 25, 50, 100% roll in that we’ve been talking about.
Leon: And that speaks to I think the ebb and flow that RTO is going to present because people are going to come in, they’re going to whether it’s vacation or people are going to get sick, not pandemic-sick, but just sick-sick, and things like that. So, I mean, that becomes a logistical nightmare, right? I mean, again, J.W. to your point, like you’ve got Monday, Wednesday, Friday, I’ve got Tuesday, Thursday so that we’re not in the same space, but then all of a sudden, I get sick and that throws my schedule off or there’s a holiday coming up or whatever it is. So, how are you going to handle as people’s needs change throughout that RTO phase-in?
J.W.: It’s really everybody, you had to stay nimble. I mean, we are good at IT because we can take on change and we can be nimble through all the processes that we have to wade through. And this is one of those new processes that we have to work through. We’ve proven that we can do the work-from-home thing. We proved it with our test over the past few years. Everybody’s proved the real-world test that we’re all in right now if you’re able to do this, and if you need to do this phase-in, phase-out kind of thing because of the ebb and flow of the pandemic or the flu or a strain of pneumonic fever that comes in that’s going to end out 2020 because 2020 keeps throwing new and exciting things at us, that we’ve figured out the next stage of normal, whatever that might look like.
J.W.: And the second thing, and to your point of you brought up a really good thing, not everybody’s going to get the current sickness that causes the pandemic. They might get a normal cold, or a flu, or something like that. But one thing that we’ve been talking about is being a little more cognizant from a managerial standpoint of saying, if somebody’s sick, call them out on it and make them go home. Because there’s been maybe a little bit of lax treatment of people that come to work sick. Well, we’ve proven that we can work from home. You’ve proven as an individual that you can work from home successfully. Go work from home and be sick there and don’t bring it in.
Leon: Which is nice. I mean, I think that previously, a lot of organizations weren’t quite sure what to do with it. So, at least now we know, not only we know, but also the organization knows that it can still be done like business will in fact continue.
Leon: So, I’m curious what processes you both are planning and implementing before return to office begins. What are some things that you don’t have in place now but you know you’ve got to have set to go before you really pull the trigger on this?
Rani: So, there’s a few things now. Some of them are in place now, but absolutely weren’t when we went to work from home. We were doing patching generally through our being on network or through VPN. We’ve had to move away from that, so that we can maintain the security necessary. We’re looking at different deploying kind of security at the agent level for things like our data loss prevention software. We’re absolutely going to be looking at a new, what I call kind of remote provisioning, so that we don’t have people who have to actually physically hand computers to our new staff that are being onboarded.
Rani: And I tell the story sometimes around our Manila teams just really did went above and beyond. We were literally meeting couriers in the streets to move between streets that were under quarantine and those that weren’t, ,and we were sometimes going courier to courier to make sure that our new hires who were being onboarded could get their laptops. I loved our service level. But I don’t like the idea that people had to kind of do those unnatural acts for us at onboard. And so, I want to get to a place where we don’t have to actually physically hold possession of a computer to onboard people. And so, several of these things were things that we were talking about beginning to pilot, but we’re pushing the kind of the acceleration button on that. And then during this period, to make sure that our leaders had confidence in the fact that the teams were doing the work that we expected, and also monitoring to make sure our teams weren’t doing too much, because they certainly got to that point, we developed new monitoring reports that showed call volumes, that showed meeting volumes and then also started to prepare different standard reports around other productivity measures. And so, we will absolutely continue doing that kind of work.
Leon: That’s great. I mean, as a monitoring company, it’s good. And that’s been a conversation that I’ve seen a lot out in a lot of different spheres about the measurements for productivity, which in some cases, were yelling over cube walls, right? “Hey, I got another sale,” or whatever it is. Now, we really do need to look at the sales motion, the data. But also, I love the idea that we are monitoring people’s stress levels in the sense of, you’ve closed too many tickets, which is something that help desk person never expected to hear, maybe you need to slow it down, it’s okay. Because we know that this is, in fact, it’s not even a marathon. This is life and you need to pace yourself for life. So, I think that’s fantastic. And leveraging the monitoring to help facilitate that is fantastic.
Leon: So, at this point, I think I want to synthesize where we started,, and where we are today and where we expect to go tomorrow, and talk about some lessons learned. So, the first thing I’d like to know from both of you, since you’re in positions of IT leadership, is did you talk to any of your peers, and maybe have a chance to learn some things vicariously rather than having to bang your shins against the table in the dark on your own.
Rani: Well, I certainly reached out to my peers, but they were in the same situation that we were. So, we commiserated more than kind of advised each other. But certainly when we figure out some processes, and the things that were working well, we absolutely shared. And in fact, I’m excited to learn from J.W., I love the idea these kind of take-home packs and I’m going to borrow that.
J.W.: Please do. I think everybody should go that route. So, for us, we’re one of seven sister organizations across North America, and at the beginning of this, we were having two meetings a week with the business continuity, disaster recovery type teams. And we shared all of our findings, and to Rani’s point, we commiserated about the failings that we were all experiencing. But we were able to bring some findings back in, and unfortunately at this point, memories are short, but I couldn’t even tell you the ones that we brought in as Texas RE versus the ones that the sister organizations presented.
J.W.: But, it was just that never be afraid to say that you’re having problems, because I felt in those groups where we were able to say, “Hey, man, I don’t know how to fix this.” And somebody was like, “Hey, I’m having the same problem. But we tried this, what did you think?” And just having that level of vulnerability and saying that something was wrong. And fortunately, all of the people on that phone call I’ve worked with in a couple of other groups for the past six or seven years, and we have a good rapport. We’ve met each other in person back when people did that pre-March. And we have a good working relationship. And we’ve been able to tackle some stuff in the past and it just really fell into this. So, one of the big lessons learned for me is make sure you have a support group that understands where you’re coming from, and build those relationships early on.
Rani: One thing I will say, fortunately, I have the fortune of having DevOps under my purview, and what I actually did is reach into my DevOps teams to help to see if we could solve some of these problems, particularly when we were having, reaching inventory and shortages and areas where we weren’t able to get the equipment that we thought we needed for scale over this extended period of time. And so, they introduced us to some plans and a lot of equipment manufacturers have around fast equipment delivery services where you don’t actually have to take possession until you absolutely need them. And so, we leverage some of what was a kind of a contingency practice within DevOps.
Rani: They also talked us through, hey, you know what, we’re finding some of our data centers saying that we are going to not allow access during this period of time. And so, we went in early and kind of pre-negotiated access to some of our facilities and data centers. And so, we borrowed that actually some from our DevOps peers versus some from some of our IT peers just because they’ve seen those problems occasionally, and separately, never pandemic-style, but it was really nice to be able to learn from our DevOps teams in this particular area.
Leon: Nice. So, we touched on a little bit, but I want to maybe explicitly say that some of the things that you implemented during this work-from-home period that you absolutely plan to continue when you come back into the office. What are some things that you just are so delighted by? We talked about the remote work packs, and some of the patching processes, but what are some of the things that you know right now we’re absolutely doing that forever?
J.W.: The business continuity documentation has to catch up with where we are. One of the documentation failings that we learned early on was it was an all-or-nothing thing. The business continuity plans that I’ve developed and I’ve seen and I’ve researched, they’re all-or-nothing. They’re due to catastrophic failures. You can’t reach your building for it because of fire or flood. I’ve never built a scenario, in the mind at least, that is a little bit of a failure because we didn’t have a catastrophic failure. Our data center is intact. Our people can still access the network. All I needed them to do was take their laptop home and everything was hunky-dory. But there were some aspects of it that should have triggered our business continuity plan. But when it’s just this big, giant lever that you can turn on or off, that it’s not really conducive to that. So, I’m going to work on building in kind of a more metered business continuity plan engagement that we can do and continue to test it and hope that we can go back to actually testing it and not real-worlding it because turns out real-worlding it, it’s not that fun.
Leon: No testing and production is never…
Rani: Say it again, Leon. Say it again.
J.W.: I have not thought about the pandemic being a test in production, but my gosh, is that not the right way to say this.
Leon: It’s absolutely. Yeah. Roll out on Friday, test in production. Rani, what else you got?
Rani: Yeah. J.W. Mentioned it. Business continuity as a thing and I think that our leaders understand it differently now. Before, we would contingency plan and then go and request them to pull the lever when the risk was realized. Now, we’re able to make different levels of investments to ensure that we have the contingency that’s necessary.
Rani: The other thing for us is that we are approaching 90% all cloud for our corporate infrastructure and systems and services. I want to finish that out. We need to be 100% cloud for anything that is reasonable with maybe some exceptions with a data warehouse, so that we don’t have the kind of potential bottlenecks for routing traffic back on-prem anywhere.
Leon: That’s good. And I would remiss if I didn’t mention that, I think that monitoring some things that we didn’t expect to monitor before, you had talked before about instrumenting some monitoring for behaviors for different ways to view success or productivity, but I also think there’s just service monitoring. We didn’t worry about the Webex performance before, but we kind of care about it now.
Rani: Absolutely. And so, one of the things and it was more out of concern initially. But it turned out to be kind of a proof point that our teams really, really, when moving to work from home were more committed than we ever expected. And so, we could see that our call volume increased. We saw their meeting volume increase. We could saw our team engagement increase even kind of internal, in internal meetings. And so, the monitoring of our SaaS services that we didn’t consider to be that kind of critical really with regard to what that meant for our productivity metric, we looked at differently and we’ll continue that.
Leon: Nice. Alright, so I’m going to ask the opposite. What are some things that we’ve done during work from home that we 100% know that we are never doing again? Is there anything that you’re just like, I can’t wait to stop doing this thing?
Rani: I want off the hamster wheel.
J.W.: Preach. That’s the one thing I wanted to nail home. I want to stop working in the evenings. I want to stop getting called on the weekend, only to have somebody say, “Oh, I didn’t even realize it was Saturday evening. I’m sorry.”
Leon: Right. Right. Exactly. Yeah, that work-life blending versus work-life balance.
Rani: Absolutely. I think everything else we learned in this are great practices or tests of agility, and we need to continue to implement those things. But we definitely need to bring some more work-life balance to our lives.
Leon: For folks who are listening to this podcast, what advice do you have for them if they’re listening and realizing that they need to get on the ball with their RTO plan? What do they need to start with or get moving with?
J.W.: So, I mean, just like everything else, business continuity, return to office, RTO, BCP, the only wrong time to start planning for that is tomorrow, that you can’t beat yourself up that, “Oh, I didn’t plan for this pandemic coming into this. I didn’t have a business continuity plan. I didn’t know how to work from home.” Okay, that’s fine. But you’ve got to plan now. It’s obvious that there needs to be something in place. You’ve got to stay nimble. We don’t know what the next weeks and months are going to bring, but for the RTO, we’ve got to stay flexible because that’s going to be the biggest factor to success that we see going forward. And stay diligent, stay secure. And don’t slack on your security mindset. We entered this pandemic with kind of a firefighting mentality. And I’m proud to say that my team has never missed a beat from a pushing the ball forward and making sure that we were safe and secure from a network and cyber aspect. But we’ve also done a lot of work as a company to make sure that our people were taken care of and that we’re taking care of ourselves at the same time.
Rani: I couldn’t said it better than J.W. I think we end on that.
Leon: Okay. Well, then, we’re going to end on that note, but I want to give you one more chance. If people want to find out more about you, your organization, where can they find you?
Rani: So, SolarWinds has a fantastic COVID response on our website. And it tells you the work that we’re doing to make sure that our customers and our staff are safe.
Rani: So, you can find out more there, and then I’m rani.johnson@linkedin. I welcome your invitations for friendship and also, we’re hiring. So, we’re looking for awesome leaders in IT right now.
J.W.: I am @jwrichiv on the Twittersphere. Like I said, I’m the only J.W. Richards, IV on LinkedIn, so it should be easy to track down.
J.W.: Our company website is texasre.org. Come over and see what great things the crew over here is doing to keep the Texas bulk electric system safe and secure and that we have AC and Texas in a hot summer, so.
Leon: Right, which is kind of a requirement, like that’s a necessary—
Rani: Essential.
J.W.: We like to think so. Essential, yes, it’s essential.
Leon: Yeah. It is essential. That was it. It’s an essential service. All right, Rani, J.W., thank you so much for sharing your expertise today on TechPod.
Rani: My pleasure.
J.W.: Thank you.
Leon: And I want to thank everyone who took time out of their day to listen to this podcast. For more information on SolarWinds TechPod, visit orangematter.solarwinds.com
