Not only will we reveal our newest Head Geek™, but we’re finally pulling the curtain back on our all-new new Lab set! In this episode, the Head Geeks dive deep into the now-converged database schemas for NPM-NCM installations, and offer some tips for keeping your Orion® Platform install running smoothly. Also, they show you how to use the new IP address collision detection and remediation tool in IPAM/UDT.
Episode Transcript
Hello, I'm Lawrence Garvin.
And I'm Patrick Hubbard, and welcome to another episode of SolarWinds Lab.
Hey, so how was your holiday break?
Holiday break was great. I think one of the things that made it really awesome was how many of you all actually came out to THWACKcamp this year.
THWACKcamp, yes.
Or I guess now last year, because now we're into 2015, and it was amazing, we had over a thousand folks show up for the live chat, lots and lots of great feedback. I think at one point we were getting a maximum of seven or eight chat messages per second.
Wow. And all of that content...
Twitter eat your heart out. Yeah, and all of that content was just really great. It was even better than a Twitter live chat I think.
Yeah.
But you know our loving hands at home, chat client didn't fall down, which is really good. And the best thing was the feedback that you guys had on the show. And the questions that you were asking during THWACKcamp were just really great, especially since we made some changes. Right? Like we moved it to December, with the feedback we were getting was, "I'm either going to be stuck at work," or, "I'm happily at work because my manager's gone." So you could actually start thinking about what you were going to do going into this year. And the other thing about THWACKcamp is that it's a little more thought leadership, and best practices, where our show is a little bit more how-to. So we were able to really focus that content, on what you guys were looking for at the end of the year, and again, that was all based on your feedback to the THWACKcamp page, so thanks again for making that a great show.
Good stuff for sure.
Yeah.
So what's new?
Let's see what else is new... I got a new SSD. I'm very very happy about that. I've been upset and complaining about this laptop for a really long time.
I recall that.
Yes, so much so that I think IT was sick and tired of me at the help desk, and I don't complain anymore, it's just amazing, and this is also the one I use for demo capture, so I think we'll get little bit better...
And the show's will go faster now.
I hope so, yeah. And we won't lose anything when we do screen caps off of that.
Cool, so what else is new?
Oh, we're going to do a couple of cool topics today. So the first thing we're going to do is we're going to talk about the NCM database integration into the Orion Platform. We had some questions about whether that's efficient or not, or anything that they can do to tweak it and make it run better. The reality is, it's a whole lot better as long as you know what you're doing, so we're going to walk through that, we're going to bring a guru to kind of talk about the database specific side of that.
Okay.
Yeah. And the other thing we're going to show, a couple weeks were going to roll the new integration between IPAM and UDT, especially around IP address collision and then that automated IP address for mediation tool that's built into it. There's a little bit of a how-to on making that work, so we'll actually show you the screens that allow you to do that, and walk you through a couple other features in there as well.
Okay, cool, and so anything else?
Oh, I don't know, we have our new set.
We do, and I'll bet the video team has a package for that.
They do, or else this is going to seem really awkward. [Techno music]
Yeeha! I love that package. There's really good stuff in it. And I love this set. It's just really great. Feels good being here but I'm thinking maybe we could get the stools back like we had in 2012...
Yeah...
And then we could sit.
And we may actually need those, because they renewed us, not just for 2015, but they actually ordered 24 episodes. So that means, we are doubling the rate of episodes for the show to 2 a month, and the reason for that of course is, you have asked so many questions, and want us to cover so many topics, that rather than go to more than 30 minutes per show, we're going to double up on the number of shows.
Yeah, cause we don't want to do that to you, and we don't want to do that to us.
No, absolutely. So the other thing is, more than ever, make sure you swing by our homepage, which is, lab.solarwinds.com. You want to follow along and try that at home?
Everybody.
Three, two one... [in unison] lab.solarwinds.com Make sure you sign up for show reminders and especially send us feedback and questions about our products or how-tos or anything you can think of related to SolarWinds and we're going to cover it on future episodes.
Yep, for sure.
Hey, we're still here too, you know.
Hi.
Oh, how ya doing? You guys just can't ignore us like it's our prom night all over again.
It was that bad? [Laughing]
Yeah, I just don't want you guys getting too comfortable with that two shot of yours.
Oh, Tom, it's actually cooler than that, watch this. Camera one. Camera two. Camera three.
Oh, that's cool.
Yeah, that's right; we finally have more cameras too. And you can even do it faster than that, you can say, camera one, camera two, camera three. Camera one, camera three, camera two. Camera one, camera one, two three.
So, new set and all these new cameras.
And an updated demo laptop.
That's true, but there is just one more thing.
And what would that be?
Take a look at this. [Dramatic music]
Only brave individuals, woven in courage, and forged in the fires of geekdom may seek the grail. It has become a symbol, a symbol of hope, courage, and network unity. A brotherhood of geeks. Many will dream, only the chosen few will earn the right to don the lab coat, for they are no longer mere mortals, but SolarWinds Head Geeks.
That's right, we have another new Head Geek.
How many are there going to be ultimately?
Just you wait and see.
Eventually there can only be one. [Laughing]
So with no further latency, please welcome our newest virtualization Head Geek, Kong Yang.
Yay ha.
Yay. [Techno music] [Applause]
Me?
Welcome, welcome.
Thank you.
Hey, how ya doing?
Right here.
Oh, oh, ah.
Uh uh.
I guess that won't work.
All right, first this sir.
Oh, thank you.
Very nice.
You almost got it. [In unison] There ya go.
Great.
First job skill.
Yes.
Hi geeks.
Yes, so you're the first person to use the new do we call it a door? I don't know. Why don't you guys in chat put the name for what you think we ought to call that. But yeah, so Kong, you're an interesting addition to the team, I mean you've got a deep background in virtualization, but you actually started as an application engineer and architect in the gaming industry in Las Vegas.
That's true, but my start with virtualization started with GSX in the lab, and even thought the CLI was terrible, I knew that there was opportunity.
Yeah, and it was real interesting because I think that was one of those that we installed as our first server-based virtualization hypervisor after, maybe it started with Workstation,
That's right.
And it was a little rough.
That's right, and in terms of virtualization, the potential was there, regardless of the technology or the vendor.
That's true, and Hyper-V and Citrix both came up to play as well too, they're doing real well.
Absolutely.
That's right.
Looking back at the overhead of the app from the application perspective, with GSX it was horrible, 55 to 65 percent of the overhead, but you could see the opportunity for consolidation and HA.
Yeah, you could get two or three different OS's all in one machine.
Exactly. So I waited patiently for the technology to mature. Right? And that took me on a different path into consulting. From there, I was able to have the opportunity to design and architect Dell's first sizing and capacity tool.
Yeah, that's great, and I think we've all spent a lot of time in that configurator tool over the years, building out racks of gear, and especially as it's gotten more and more dense over time, we've watched the power efficiency increase, and we've gotten more than two VM's on a hypervisor...
Oooo.
Yeah, [laughing] You know it's funny cause, you know VM's are really cool, but how we've come to that is different for each person. You've been, since the early days, been a big proponent, driving it into organizations, done architecture around it. I think for me, and maybe some other people, we looked at it and said, we had that first brush with GSX, you know, he who will not be named, and then said, aww this is going to be cool, maybe eventually, but not right now. Then all of a sudden, at the last minute it's like, oh, but we're going to virtualize everything, you know, you need to get caught up.
That happens a lot in data centers.
Probably way too many.
Okay, so we all know that I like bacon, just a bit. [In unison] Just a bit. So I heard your geek question should be more about, what's your favorite burger?
The Awful-Awful from the Little Nugget Diner, of course. It's just plain awesome sauce.
And that's in Reno, right?
Yes.
So, wait, we've got bacon, burger, is this like a new geek theme we have going?
Maybe, I'm all about the breakfast tacos.
So, let's do this, since we don't need the whole crew here for this next segment, for the how-to section, I'll stick around, so we talk about the NCM database, and Lawrence and Kong, you guys can escape, or es-cap-ay.
Es-cap-ay.
Hey, works for me. Let's go party. [Laughing] [Electronic hum]
All right, so, let's talk about some databases, some NCM databases, and then Patrick, you can talk about IPAM, UDT, IP address collision tool. Address collisions; is that like crossing the streams?
Yeah, it's a lot like crossing the streams, it happens more often than you might think. Especially if you want to empower folks to set up their own IP address ranges, right?
Right.
Sometimes delegated to administration. You will still have to help them out sometimes, so this is a way of kind of making that a little bit easier.
Oh, wonderful.
Right, okay, so as far as NCM goes, with the latest release, we've completed something that started a while ago, which is the complete integration of NCM into the Orion Platform.
Right, and many of our users have been using NCM for five years or more. But for the new users, NCM used to be called, a little thing named Cirrus, and that ran as a stand-alone GUI. But then it got its own little stand-alone web console, and then that web console got pulled right up into the other Orion Platform modules.
That's right. And along the way, it kept its own database for a really long time. And so it was a completely independent product. Then it was flying in formation, and then it was sort of integrated with common discovery. And now, it's fully integrated into a single server, but there were differences between upgrades and installs, so we can kind of talk about that.
Right, now we even did a show, awhile back, about how to use the installer to get your databases onto the same server, but now it's all in one database, so it is much easier to manage.
Right, it is, but there are a couple of things, tips and tricks that you're going to want to know about in order to migrate, and maintain the data, make it super silky smooth.
Yes, so if they don't watch this episode, then they'll just blow up their server.
Oh yeah. [Imitates explosion]
That's always a possibility, but no, the manual’s great, and we're just going to give you a cheat sheet to make it look super easy.
Awesome! Well let's take a look.
All right.
Okay, so we're looking at NCM here. We've been looking at NCM for a really long time, it has been from a GUI standpoint, really well integrated for about a year and a half now. Remember before, the controls are kind of wrapped up in the tab, at any event, it's been a part of the Orion Platform core for a really long time. Scroll down to the bottom, see the core number, it's been in there a long time. So, what we're really talking about here is the database. And if you look at the polling details, if you look at all the information that it's using, the way that it's creating an Orion node with a node ID that maps to the device that you're doing the download, none of that has changed. So, the only things really changed here, in addition to some, a few enhancements in the GUI, is lots and lots of things in the database, and that's really what we're going to be talking about. And that's the main reason that we want to have Tom here, because he's going to be able to help explain why you should not be nervous that this is less efficient than having two separate databases. And I get it, because we had two separate databases for a long time, needed to feel good about that, letting go is kind of difficult, so that's really what we're going to talk through here. So, the first thing to note, really the first question is, and I had someone ask me this at a trade show, not too long ago, is isn't inherently one database going to be less efficient than having two separate databases?
Oh, I hear that a lot as well. It's hard to understand why. Sometimes people just think in sheer size, size means less efficient, because more data has to go through your buffer pool and all that. But other times it's more about the complexity of the data trying to merge two databases into one. So, one easy way to do that with SQL Server is simply create two schemas, for one database and the other database. Now you have, guess what, you have two databases inside of one database.
That's true.
It's still just kind of a trick. But now you do have a large database, so now what does that really mean? That's when you dive in, you say, well, what's the data we're really pulling through the buffer pool anyway? What are the queries we're really going after? What are the queries that need to touch both of those databases or schemas? How can we design maybe a little bit better? Maybe we have stuff that maybe needs to be de-normalized. Maybe we have something that needs to be a little bit more normalized.
Right.
What you have is, you have--basically, you have two cars, and you're taking the parts off of those two cars, and you're trying to make one brand new car, and you can't just expect it to be perfect without a little bit of redesign. A lot of times though, people, they'll say, "Oh, one database "will be awful," because they don't want to deal with the redesign, they just want it to work.
But in this case, some of the redesigns were things like a lot of those cross database joins that we had before…
Yeah.
…are gone now.
Gone.
So they're a little bit more efficient. And they're certainly easier to support, and to debug, and to know what's going on.
Definitely, definitely easier to support, is a huge factor.
The other thing is, we had a Band-Aid on here, once upon a time, it was a feature of SQL Server that's kind of handy if you're looking for words that are more than three characters long, depending on how you set up, and that's full-text search. So we've replaced that actually with an indexing service that's actually doing much better tokenization, storing it in a more accessible way.
Good.
And you actually had an interesting point about full-text search. You were talking about it was designed to solve a problem, a few years ago.
Yeah, if you think about the way databases have evolved over ten to fifteen years, full-text search is one of those things that came about as a result of... So most databases start as something kind of small, and get bigger…
Right.
…for lots of reasons. Perhaps this is one reason, and you find that you need to query from it in a certain way, and you say, “Hey wouldn't it be faster if we could just build something to make that happen?” And that's what you get a tool like a full-text search. What ends up happening is full-text search can only scale so far, right? Its scalability and its functionality can only go so far. So people turn to different solutions, like in those SQL solution, or a key-value pair, and now you get things like a columnstore index.
Right.
And so you have different ways to get the job done. A product like SQL 2014 is much more robust in all of the options you have, beyond just using something like a full-text search.
There's another about the installer that's a lot easier to use now. And that is, it doesn't ask you for the database information for the connection, right?
Okay.
So, when you install it, it's going to go ahead and start migrating data for you the first time that you connect to the page. Now that's one thing, about half of you, and actually that's a thing to ask in chat. Tell us in chat how many of you have actually upgraded to 7.3 or beyond, and how many of you haven't, because it'd be interesting to see if you haven't, why haven't you? If this is one of the issues that's maybe holding you back, you're just kind of concerned about how that works. We don't blame them, because there's been a lot of changes about the way that that NCM database has worked over the years. So getting to the state that's super easy to support, and super easy for you guys to manage is really important, that's why we're taking time talking about it here. But once you run that installer, the very first time, it doesn't start the migration immediately, and the first time you go to an NCM page, then it starts the migration. And for most of you that only have a thousand devices, or a couple thousand devices, it just takes a couple of minutes to migrate down from the first. One of the early Beta builds where I think it took four or five hours. And for customers that have tens of thousands of units that they’re backing up every day, and there's lots and lots [mumbles] and they want to keep all of their history, and you actually have the option of not keeping all of your history in that install wizard, it may take at most an hour. But for most people it doesn't. What's really cool is while you're doing it, you'll get this page. And we'll pull up a screenshot of what that page looks like here for you so you can see it, that page will tell you, "Hey, NCM is not available to you right now, I'm doing a migration," and enjoy that. Take a screenshot of that, because you'll only see it once. The other thing that's nice is everything else in NPM, or SAM, continues working normally. So you don't have to worry about that. But yeah, that migration is actually pretty cool. The other thing too, there's been a lot of performance tweaks as a part of this. For example, and I know this is something that we were actually talking about at lunch, GUID. There are a lot of GUID's in the data for NCM. Like, for example, you'll always notice, and you can see up here where I'm looking at this config ID in the URL, right?
Yup.
Config ID equals…
It's right there.
It's a GUID, and there it is with the curly brackets, telling us that it's been serialized out to a string. Well in the olden days, [clears throat] that was actually stored as a char value, in the database, and then it would be the index, and the primary key on this config. That is not necessarily the most efficient way to do indexes and joins on data, especially if you have lots.
I think my ears are bleeding.
Yes. So, there's a huge performance improvement there. We went through the database, and actually correctly data-typed a lot of these columns. So for example, if it's a GUID, it's a GUID type table. If it's an integer, it's now an integer of the right size. And this actually goes across a number of different tables, including inventory, and a lot of the other data. So, there are a lot of performance improvements as a part of that, because if you're going to go in and make changes to the database, you might as well take advantage of that.
That's one of the things I mentioned earlier. A lot of times people don't want to get into those design details, and make those types of changes, and they look for different means. But yeah, it's wonderful to hear the steps that they took to clean up some of those antiquated ways of thinking. Yeah.
And the first time that we talked about it, and I told you about it, you cringed, he actually cringed not quite as big as he did the first time I told him, but I just love watching him do that [laughing] when you talk about it, talk about the data type. One thing that is a little bit different is your NCM will take, the database portion for NCM, will take a little bit more room, in that we have switched to an nvarchar datatype for the configs. Before it was just a regular ASCII data, basically. So we do support double byte characters in now, because we're finding that more and more configs now actually have localized characters, and it's pretty nice to have your backup not die, or the image doesn't actually match what's out on your device. So, then there was only one other thing I wanted to mention, is if you really want to set yourself at ease, that this is not causing additional impact on your Orion Platform server, whether it's NPM, SAM or you have all the modules, and hopefully you've upgraded your Netflow, so that you're now using Flow Storage which should have bought you a lot of overhead, but if you're all nervous about this, if you're not already running...
Database Performance Analyzer.
That's right. Then just go ahead and download it and play with it. I know it sounds like it's a pitch to download it, you don't have to buy it, just download it and play with it, and what you'll find is, do a couple of tests, right? When would you recommend that they run it?
I'd tell you to start running it now to get the baseline of what your performance is like, right now, before you do your upgrade.
So this is on the Orion database?
Right, do it right now. Point it at Orion right now and just collect the metrics just to get your baseline. So that way, after the upgrade is done, then you use DPA again, you're looking at it, and then see the difference in performance.
That's right, and actually you said earlier something else that was really cool, which was about during the download period, go ahead and compare that data as well.
Yup.
Because most of you are doing your downloads and backups in the middle of the night, it's pretty handy to be able to know what the real demand on the database server is...
That's right.
As a result of adjusting all of those backups, so you can actually compare the history charts of when it's just sort of sitting there in the middle of the day, versus two o'clock in the morning, or whenever you're doing your backups. And the nice thing about that that's going to give you an opportunity to really fine tune the performance of NCM. So, let's say you totally peak the database, which you probably won't, you're not going to have that many devices, but let's say you have thousands, and thousands of devices, well one of the things you can do then, is start to explore things like, go into your polar settings, and actually set the polar throttling for maximum simultaneous downloads. I mean that's one of the first things that you do to kind of throttle that down. Now, if you have that many configs that you're backing up every night, that many devices, that may not be an option for you, because you may never get done, so another option there instead, is to chunk them up into groups, and then set up multiple backup jobs, and then you can also set that through the console, which is really really handy. So then break them up into sections, and chances are if you're that big, you're probably geo-distributed anyway, so midnight right here, is not midnight everywhere on your network, so then you can actually time zone adjust those anyway, and load balance those to make sure that those work well. But, yeah, just using DPA, even for a few days just to kind of check that out and see how it works, fine tune that, is a great way to get additional performance in your downloads and backups without having to buy anything new at all, just scale out what you already have.
Right.
I think that's really about it. Was there anything else that you wanted to talk about for the NCM performance improvement with this update?
Yeah, there's one more thing to think about. So, who likes automation? Who likes to automate all the things? [Laughing] So, for anybody big into automation, are they afraid that we might break something?
You talking about SWIS?
I am talking about SWIS.
You're talking about SWQL, you're talking about...
Of course you are.
I am.
Because you're on camera, so we're talking about SWIS and SWQL.
And it's especially important to talk about SWIS, when we're talking about NCM, because that's actually, what I'm finding, one of the biggest drivers for you guys that are doing automation, because if you want to actually programmatically upload, make config changes, basically interactive CLI on those devices, they're already set up, and it really does turn NCM into an automation platform for that. And so in the episode where I actually talked about how to use the automation, there's a THWACKcamp episode from...
Last year.
Last year, from 2013, go check that out. One of the things that I mentioned then, and this is a great example of why that's important, is separating the schema and the objects that you work with through SWIS. And the physical schema behind it is really important, because here we have dropped--didn't drop a whole database-- [laughing] I'm sorry, that means a whole nother thing to you doesn't it?
It means something different.
So we migrated all the data into the database.
That's right.
We added tables. We modified data types. We made a lot of changes in the schema to accomplish that. But you're automation doesn't change, because you're working with high-level objects that don't need to know anything about what's going on through the information service. Remember that's the service that actually maps the information service queries at port 17-777, or interprocess at port 17-778 to the physical schema of the database underneath it. So the great thing is all of your stuff will continue to run, because SWIS is unaffected, because the information services were automatically updated to reflect those changes.
Right.
Yeah.
And I love the fact that we've gone out of our way here to make it as seamless as possible, and to make sure people are still up and running, and their existing code won't break. It's going to migrate. It's going to work.
Because I know that every time I automate something it's always on the weekend, and that's when I have to go fix it.
Right.
That's pretty easy, and the part I like best about that is after this last element, NCM's going to be like any other Orion module, in terms of upgrades.
Exactly.
Well, and it really is now. I mean, this has been out for a while. Probably half, a little bit more than half, of you guys have already upgraded this version. You should go ahead and complete it, especially now that we've walked you through it. And Tom, thanks again for talking to us about why we did it, and the reason behind it, and the real benefits that they actually get. Again, do not be afraid, you're not going to mess up your system. The automated migration is really smooth and easy. And it will definitely perform better on your system. Should perform better on your system, unless you have now doubled the rate of your backups, which you might want to do, that would probably be a good thing. Okay, so then the other thing we wanted to cover in this episode was, I just real quickly wanted to walk you through the new IP address collision feature, when you're using IPAM and UDT integrated together. There's a couple of tricks to make sure that you're set up right, so that you can get the remediation working, and how to use that, cause it's in a couple of different screens.
Right, and we've used a few new protocols, some port IP address mapping that we're going to show you how it works. But once we do that, it'll be ready to go.
Yeah, and I might even show you a little IPv6.
You're going to be the only one.
I keep trying to do IPv6 and nobody cares.
I care.
And they really don't care. Do you still want to see more about IPv6? If you do...
Let him know.
Throw it over here in the chat, and let us know. But yes, there's a couple enhancements there, and if we have time, we'll talk about that too.
Okay, so walk us through this.
Okay, so, IPAM, we know it, we love it, it's really really cool, and let's just get right to it here. One of the things that you get in this version is this IP conflict resource.
Oooo, shiny.
There were ways to tickle the data before to get this information, now we're pulling it up for you in a view, and the really important thing here is to make sure that you also have your UDT configured, and I'm going to show you why in a second, with read/write permissions and set. So as long as that community string has set and read/write, you're going to be able to see the magic here in just a second. So the first thing that happens, as a part of that, is you can actually see the list of conflicts that you have. And under each one, you're getting the physical port that it's connected to, and device and user information that you otherwise wouldn't get. And how many times when you're trying to resolve an issue involving IP conflict, it'd be really nice to have log on information, physical port information, is it wireless, is it whatever else.
Well that's exactly the point. I mean, IPAM has always been able to tell you that this Mac address over here is conflicting with this Mac address over there. But it doesn't tell me whom to call. You know in certain cases, all right, I have my server Mac addresses in a file someplace, or I can get it off of a RRB table on a router. But really, I see these too. I see an Apple and I see a Samsung, like you can see in that first line. Well, what is going on there? So this is really a lot of fun, in terms of the information you get.
And this is the classic IP address drill-down detail table, but it's got some new stuff in it. Like first of all it's saying, "Hey, you have a conflict."
Right. We've made some improvements so that on this page, if you're just tooling around on this page, you know that you have an IP address conflict. You also can see whom it's with, okay? Once again, you can see there's the Mac addresses, and we give little vendor icons. You can see the one on the left is a Samsung, and the one on the right is an Apple device. But whose phone am I calling? Well, you can see there it happens to be Nico Rosberg. Both times. So, what happened here? Well, I can make up a story, which is that Nico came in with his phone, and his phone got an IP address, and then he came and sat down with his Mac laptop, and he plugged it in, and that one got another IP address. He didn't hard-code anything, and I know that just from looking at that screen. How do I know that?
Because they're both DHCP scopes.
Okay, they're both DHCP scopes, and it'll actually tell me how to fix it.
Yeah, that's really cool too, because based on the type of conflict, go down here and say, "Show remediation." And it'll actually give you some steps to fix that.
Right.
Now of course the handiest one, right off the bat, is to kill the port that one of them is sitting on. But you need to figure out which one you're going to kill, so it's really handy to do things like look at your history events, and then figure out what happened there.
Now, I'm going to say a couple things about the story I just invented. First of all, we know it can't be his phone. We know it can't be his phone because it's connected to a port.
Yeah, well maybe USB connection. [Laughing]
Maybe, but probably not. So we know that that Samsung thing is actually hard plugged, and this was one of my fears when I first saw this screen was, oh, you can't shut down the port for that one because that's the port that's connected to the access point. Well it's not, and I'm going to show you how in just a minute. But here, with just a click of a button, you can shut down one port, resolve the conflict so he can, again, assuming that the Samsung is a tablet, or another secondary device, because you can call him to find out. Then you shut down that port, because the fix here, is that you have to un-overlap these two scopes. And that's not something that you're necessarily going to do [snap] on the fly. You're going to be careful about that.
Right, or you could just click shut down, and lean and see if you hear screaming. [Laughing] And then turn it right back on again, and then walk down and say, hey, did anyone have any problems.
Right, oh, I don't know why.
Yeah, so definitely there's that. The other one that was...a couple of other examples here of things that you would see, going back to your point about wireless, here's one right here. A little bit more typical wireless presentation, right? Where we've got a Huawei, and I think this is a RBD device, right?
Broadcom.
Broadcom, yeah. Okay, so it's telling us, that this is actually sitting on an SSID, and it tells us that the SSID is lab. So, that is really, really handy, to be able to start with what is the SSID. Not only is it on an access point, am I going to upset a lot of people if I kill the access point, port...
Notice, there's no clicky.
Yeah, you can't click it. It's telling you it's on the SSID. Now it may be an extra step to go fix that, but if you look at it here, the remediation recommendation is different. It's actually giving you some suggestions about how to do that. Like here it's actually, what, 'ackle' out the Mac, on the WAP itself, or then you can actually go shut down the port if that's an option. But I think option A there sounds the best to me. Let's just 'ackle' that out and take it off. And chances are, if this is running on any sort of halfway modern--Oh, that's right, if I click on that it's going to take me directly out to that SSID, and I can look at everybody else sitting on that. But if it's a halfway modern access point, it's going to be thin, and it's going to have a controller. I may be able to just go into the GUI and take it out of service that way.
There's another situation for conflicts that I wanted to go, so go back and go to the next row down.
Oh yes, this is a good one.
Okay, so this is another situation…
Yeah, this one never happens.
No, no, no, no. So once again, just looking at the screen we can tell a couple of things that are going on here. First of all, on the left hand side we have an IBM device. Also, notice there is no scope listed underneath that, which means that it is hard-coded. And on the right hand side, it's a VM, and it's getting its IP address from a DHCP scope. So, that means that, and this never happens, somebody configured the DHCP scope to include my static address pool, that I'm giving to hard code my IBM, my AIX box that's been running for a million years, now has a conflict, no, nobody's happy with that.
So I'm probably going to want to go see who assigned, who's managing DHCP NW Pool 1 and have a chat with them. And this is actually a great hint for delegation. So we've talked a couple of times about giving people, empowering people to manage their own sub-nets. So you can decide whether they can, not only manage IP addresses in that sub-net, or make IP address changes to a DHCP scope, but you can also decide whether or not they can manage those scopes. And so this would be an example of maybe we don't want that admin managing this scope. Maybe all the addresses in the scope, but maybe they shouldn't be allowed to create new scopes, willy nilly, across my static IP address server range. And especially if that IBM box is an AIX. We'll make the finance guys really upset.
Probably.
Yeah, so that's just really cool. So again, the take-aways there are: it will help you definitely expand the more details for remediation options, details under the violation, because these are dynamically generated based on the conflict type, and it'll really help you with troubleshooting. They're also in descending order of ease of use--and what was it I'm trying to say--least likely to upset users on your network. Right, so you definitely can learn a lot by checking those out. The other thing is use your history events table to tell you, to give you some information about who's on first, or whether it maps to your policy. That'll help you figure out which one is the real conflicting device. And then the other thing is, to enable this integration with UDT, and most of all being able to activate the ability to shut down ports directly from the interface here-- just make sure that you've given it a community string with read/write permissions and set. And then you'll be able to do it right from here. That's really cool, right?
Amazing.
Okay, so definitely that's going to be out in about a week. Those of you who are using IPAM, and most folks actually have IPAM and UDT already, so that's how to set that up and you'll be off and running with that. There is one thing, and I know you guys groan when I talk about it, but I do want to show one little cool thing about IPv6.
I am all about IP version 6. I'm okay with that. Lawrence isn't here, and he's the one...
That's true.
He talked about it in one of his articles about 'we don't think we need it yet,' and everything. I think that's sort of the Luddite way to go. I am A-okay for this. I'm ready for it.
I will tell you that once upon a time, I was highly resistant to learning about IPv6. So the point now that there's a little bit of a 'la la la la la, I am not listening to you. . .' [Laughing] And when we...
Too many numbers. Just too many numbers.
When IPAM added IPv6, I started playing with it, and I learned more about how to set up IPv6 from this wizard than all of the boring reading that I had fallen asleep at bedtime trying to get through. And one of the nice things about doing IPv6 is that you do have the whole concept of IPv6--the subnet really starts with a prefix, right? And so the prefix runs down and then you have your local addresses inside that space. So the wizard will go ahead and take you through the process of defining those subnets. And once the subnets are defined, now the near infinite scope of potential addresses is now way more limited down, and you could do some things with it. So, for example, if I am sitting here looking at my North America group, those are all the addresses in Austin for example, right? So--and those are all prefixes underneath that. So here, if I wanted to do IPv6 Discovery, before you could actually hand it a CSV list of addresses, and it would automatically add them. But then you had another tool to go get that, right. We do not let you, we still do not let you scan...
We are not going to let you scan all the IPv6...no.
IPv6 no? We're not going to do that?
The address phase, no, because you will hate us [laughing] and it will seem like something is broken. And the reality is you should not count grains of sand. But, you should definitely have a more automated way of doing that, and so I am showing you this because it's kind of hidden, and it's not all that obvious. And they're not making that much noise about it in the release notes, so I'm going to show it to you. If you are messing with IPv6, click on Discover IPs.
Don't be afraid.
Don't be afraid, you will automatically pull in the routers that are connected to that. If not, you can add a CSV list here of the routers that you want to scan, and it will go out, pull the tables, look for IPv6 addresses that are transiting traffic across that, or at least in the table, right?
Right.
Okay.
And then it will go ahead and pull those in for you, and you can start to see them. So, it is a much cleaner way of getting them than you having to go use some other tool, or a spreadsheet, and then import a CSV and getting that way. So definitely go play with that. It's really interesting when you discover how many dual-stack devices that you have on your network. It's kind of frightening. Wow, there can't be that many.
Yes, there can. It's enabled on everything now. Nobody looks there because they're afraid of it.
That's right, and so then the next step after that would be, go watch the THWACKcamp 2014 segment where we were talking about the dangers of dual-stack-- making sure that that's one of the things that you're actually scanning for. But yeah, so in short, that's it. That's IPv6 address scanning and discovery. And the main thing is automated conflict; IP address conflict detection, and remediation with expert guidance built in.
And one of the things I just want to re-emphasize is, again, that the integration UDT means that you can find out who to call, and how long, and how many times they've logged in, with this address, or that address. You can see all that history because we're pulling it from sources like Active Directory, and so on. So the UDT piece tells you the 'who' along with the 'what' that IPAM is giving you.
That's right, and the cost of this upgrade?
You know with your maintenance, right there, it's free.
That's right, it's free as long as you're on maintenance. Go get it. It'll be on the portal in about a week. A bunch of you are already Beta testing. I think actually it's out of Beta now. It's into RC. So, some of you are still doing RC testing. Thanks again for that. I hope you go and get THWACK points. Make sure you get your badge for that. But yeah, so this will be out shortly. Go pull it out of the portal and you'll be all set.
One quick commercial about the THWACK points. I'd just like to say that because that was one of the reasons why I'm here today, is sort of a THWACK point number total is that...
Oh brag, brag, brag.
RC and Beta's pay in significant amounts of THWACK points, a thousand THWACK points, so...
Couple of them are two.
Yeah, so it really helps us if you Beta and RC. And it helps you if you Beta and RC because you have a chance to play with them. And Betas and RCs are unlimited, in the sense that you can put as many devices on them as you want to.
That's true.
You put it on a VM sandbox, and you play with it. And it's not like you only get five or ten. So in some cases you can see where the benefit of increasing your license count is. You can find that out all for the price of just kicking the tires on a version.
Cool.
That's true, and then you also can get points for doing UX design, which is really cool, because they will show you mock-ups, even before there's alpha code, and say, "Do you like how this works?" "Do you think this button ought to be there?" So you get points for that, and you don't actually have to set anything up, which is really nice. So, then the question, since you brought it up is, well how do you get invited to participate in a Beta?
Good question.
You watch on THWACK, and they have, they just say, we're looking for Beta users, and you fill out a form.
I would say troll the PMs. So go to the products that you know and love and use, look at the folks that you see over and over again. You'll see Jeremy out there. You'll see Rob out there. All the folks that you already know. And just post into those forums, say, "How do I participate in a Beta?" And they will respond to you and that's one of the quickest ways to do that.
So that's pretty cool.
Yeah, it's really handy, especially if you're using IPAM to delegate administration or remote teams, whether it's remote campus, or maybe you're letting the VM team manage their own subnets. It's really great because then you don't have to worry about it. But if they do actually blow it, and have two IP addresses step on it, then you've got a way to step in and fix it for them.
Or, you can take the time and you can teach them how to use the tool. And then they can dig themselves out of their own hole.
Yeah, that would probably be a good thing for them to do.
Yeah.
Gentlemen, welcome back.
Welcome back.
Hello fellow geeks. Well that was a great show, guys.
It really was. I especially like the NCM history lesson. I think that was very helpful. Now, you know, when I had NCM environments in the past, I have been able to move them though the migration process okay, but I have to tell you, it's going to be really nice not to have to do that anymore.
And Kong, again, welcome to the team. It's going to be really great having you a part of SolarWinds Lab, both for future episodes, and also to participate and hang out with the chat. And your expertise in virtualization is going to make this really, really interesting-- especially when we think about some of the upcoming episodes that we've got, talking about virtualization and how that integrates with SAM.
I'm already testing it in the lab.
On a VM, I'll bet.
Of course. [Laughing]
All right. So before we wrap up, just one more reminder to be sure and visit us at our homepage, which is lab.solarwinds.com. And with the new schedule, you're definitely going to want to sign up for reminders so that you can be here to chat live with us in this chat box, which should be over to the side. Of course if it's not, then you definitely want to be here for the live show. The other thing is that we have so many people on, especially we saw it with THWACKcamp, and we've seen it with episodes of SolarWinds Lab, you get to hang out with a lot of the characters from THWACK in real time. And it's just really fun to watch you guys talk amongst yourselves as well as speak with us on the chat. So be sure you let us know also what you want to see on future episodes. Remember that with our new schedule we're going to burn down the backlog of requests that you guys have for content to cover. So let us know on our homepage lab.solarwinds.com what you'd like to see on the show. I think that might pretty much take care of us today.
Think so, yeah.
All right. Well Leon, you want to take us out?
I'd love to. I'm Leon Adato.
I'm Thomas LaRock.
I'm Lawrence Garvin.
I'm Patrick Hubbard.
And I'm Kong Yang. Thanks for watching SolarWinds Lab. SolarWinds Lab Episode 22: New Geek, New Set, NCM DB Tips, and IP Address Conflict Detection
May 12, 2018 |
Video
Not only will we reveal our newest Head Geek™, but we’re finally pulling the curtain back on our all-new new Lab set! In this episode, the Head Geeks dive deep into the now-converged database schemas for NPM-NCM installations, and offer some tips for keeping your Orion® Platform install running smoothly. Also, they show you how to use the new IP address collision detection and remediation tool in IPAM/UDT.
Episode Transcript
Hello, I'm Lawrence Garvin.
And I'm Patrick Hubbard, and welcome to another episode of SolarWinds Lab.
Hey, so how was your holiday break?
Holiday break was great. I think one of the things that made it really awesome was how many of you all actually came out to THWACKcamp this year.
THWACKcamp, yes.
Or I guess now last year, because now we're into 2015, and it was amazing, we had over a thousand folks show up for the live chat, lots and lots of great feedback. I think at one point we were getting a maximum of seven or eight chat messages per second.
Wow. And all of that content...
Twitter eat your heart out. Yeah, and all of that content was just really great. It was even better than a Twitter live chat I think.
Yeah.
But you know our loving hands at home, chat client didn't fall down, which is really good. And the best thing was the feedback that you guys had on the show. And the questions that you were asking during THWACKcamp were just really great, especially since we made some changes. Right? Like we moved it to December, with the feedback we were getting was, "I'm either going to be stuck at work," or, "I'm happily at work because my manager's gone." So you could actually start thinking about what you were going to do going into this year. And the other thing about THWACKcamp is that it's a little more thought leadership, and best practices, where our show is a little bit more how-to. So we were able to really focus that content, on what you guys were looking for at the end of the year, and again, that was all based on your feedback to the THWACKcamp page, so thanks again for making that a great show.
Good stuff for sure.
Yeah.
So what's new?
Let's see what else is new... I got a new SSD. I'm very very happy about that. I've been upset and complaining about this laptop for a really long time.
I recall that.
Yes, so much so that I think IT was sick and tired of me at the help desk, and I don't complain anymore, it's just amazing, and this is also the one I use for demo capture, so I think we'll get little bit better...
And the show's will go faster now.
I hope so, yeah. And we won't lose anything when we do screen caps off of that.
Cool, so what else is new?
Oh, we're going to do a couple of cool topics today. So the first thing we're going to do is we're going to talk about the NCM database integration into the Orion Platform. We had some questions about whether that's efficient or not, or anything that they can do to tweak it and make it run better. The reality is, it's a whole lot better as long as you know what you're doing, so we're going to walk through that, we're going to bring a guru to kind of talk about the database specific side of that.
Okay.
Yeah. And the other thing we're going to show, a couple weeks were going to roll the new integration between IPAM and UDT, especially around IP address collision and then that automated IP address for mediation tool that's built into it. There's a little bit of a how-to on making that work, so we'll actually show you the screens that allow you to do that, and walk you through a couple other features in there as well.
Okay, cool, and so anything else?
Oh, I don't know, we have our new set.
We do, and I'll bet the video team has a package for that.
They do, or else this is going to seem really awkward. [Techno music]
Yeeha! I love that package. There's really good stuff in it. And I love this set. It's just really great. Feels good being here but I'm thinking maybe we could get the stools back like we had in 2012...
Yeah...
And then we could sit.
And we may actually need those, because they renewed us, not just for 2015, but they actually ordered 24 episodes. So that means, we are doubling the rate of episodes for the show to 2 a month, and the reason for that of course is, you have asked so many questions, and want us to cover so many topics, that rather than go to more than 30 minutes per show, we're going to double up on the number of shows.
Yeah, cause we don't want to do that to you, and we don't want to do that to us.
No, absolutely. So the other thing is, more than ever, make sure you swing by our homepage, which is, lab.solarwinds.com. You want to follow along and try that at home?
Everybody.
Three, two one... [in unison] lab.solarwinds.com Make sure you sign up for show reminders and especially send us feedback and questions about our products or how-tos or anything you can think of related to SolarWinds and we're going to cover it on future episodes.
Yep, for sure.
Hey, we're still here too, you know.
Hi.
Oh, how ya doing? You guys just can't ignore us like it's our prom night all over again.
It was that bad? [Laughing]
Yeah, I just don't want you guys getting too comfortable with that two shot of yours.
Oh, Tom, it's actually cooler than that, watch this. Camera one. Camera two. Camera three.
Oh, that's cool.
Yeah, that's right; we finally have more cameras too. And you can even do it faster than that, you can say, camera one, camera two, camera three. Camera one, camera three, camera two. Camera one, camera one, two three.
So, new set and all these new cameras.
And an updated demo laptop.
That's true, but there is just one more thing.
And what would that be?
Take a look at this. [Dramatic music]
Only brave individuals, woven in courage, and forged in the fires of geekdom may seek the grail. It has become a symbol, a symbol of hope, courage, and network unity. A brotherhood of geeks. Many will dream, only the chosen few will earn the right to don the lab coat, for they are no longer mere mortals, but SolarWinds Head Geeks.
That's right, we have another new Head Geek.
How many are there going to be ultimately?
Just you wait and see.
Eventually there can only be one. [Laughing]
So with no further latency, please welcome our newest virtualization Head Geek, Kong Yang.
Yay ha.
Yay. [Techno music] [Applause]
Me?
Welcome, welcome.
Thank you.
Hey, how ya doing?
Right here.
Oh, oh, ah.
Uh uh.
I guess that won't work.
All right, first this sir.
Oh, thank you.
Very nice.
You almost got it. [In unison] There ya go.
Great.
First job skill.
Yes.
Hi geeks.
Yes, so you're the first person to use the new do we call it a door? I don't know. Why don't you guys in chat put the name for what you think we ought to call that. But yeah, so Kong, you're an interesting addition to the team, I mean you've got a deep background in virtualization, but you actually started as an application engineer and architect in the gaming industry in Las Vegas.
That's true, but my start with virtualization started with GSX in the lab, and even thought the CLI was terrible, I knew that there was opportunity.
Yeah, and it was real interesting because I think that was one of those that we installed as our first server-based virtualization hypervisor after, maybe it started with Workstation,
That's right.
And it was a little rough.
That's right, and in terms of virtualization, the potential was there, regardless of the technology or the vendor.
That's true, and Hyper-V and Citrix both came up to play as well too, they're doing real well.
Absolutely.
That's right.
Looking back at the overhead of the app from the application perspective, with GSX it was horrible, 55 to 65 percent of the overhead, but you could see the opportunity for consolidation and HA.
Yeah, you could get two or three different OS's all in one machine.
Exactly. So I waited patiently for the technology to mature. Right? And that took me on a different path into consulting. From there, I was able to have the opportunity to design and architect Dell's first sizing and capacity tool.
Yeah, that's great, and I think we've all spent a lot of time in that configurator tool over the years, building out racks of gear, and especially as it's gotten more and more dense over time, we've watched the power efficiency increase, and we've gotten more than two VM's on a hypervisor...
Oooo.
Yeah, [laughing] You know it's funny cause, you know VM's are really cool, but how we've come to that is different for each person. You've been, since the early days, been a big proponent, driving it into organizations, done architecture around it. I think for me, and maybe some other people, we looked at it and said, we had that first brush with GSX, you know, he who will not be named, and then said, aww this is going to be cool, maybe eventually, but not right now. Then all of a sudden, at the last minute it's like, oh, but we're going to virtualize everything, you know, you need to get caught up.
That happens a lot in data centers.
Probably way too many.
Okay, so we all know that I like bacon, just a bit. [In unison] Just a bit. So I heard your geek question should be more about, what's your favorite burger?
The Awful-Awful from the Little Nugget Diner, of course. It's just plain awesome sauce.
And that's in Reno, right?
Yes.
So, wait, we've got bacon, burger, is this like a new geek theme we have going?
Maybe, I'm all about the breakfast tacos.
So, let's do this, since we don't need the whole crew here for this next segment, for the how-to section, I'll stick around, so we talk about the NCM database, and Lawrence and Kong, you guys can escape, or es-cap-ay.
Es-cap-ay.
Hey, works for me. Let's go party. [Laughing] [Electronic hum]
All right, so, let's talk about some databases, some NCM databases, and then Patrick, you can talk about IPAM, UDT, IP address collision tool. Address collisions; is that like crossing the streams?
Yeah, it's a lot like crossing the streams, it happens more often than you might think. Especially if you want to empower folks to set up their own IP address ranges, right?
Right.
Sometimes delegated to administration. You will still have to help them out sometimes, so this is a way of kind of making that a little bit easier.
Oh, wonderful.
Right, okay, so as far as NCM goes, with the latest release, we've completed something that started a while ago, which is the complete integration of NCM into the Orion Platform.
Right, and many of our users have been using NCM for five years or more. But for the new users, NCM used to be called, a little thing named Cirrus, and that ran as a stand-alone GUI. But then it got its own little stand-alone web console, and then that web console got pulled right up into the other Orion Platform modules.
That's right. And along the way, it kept its own database for a really long time. And so it was a completely independent product. Then it was flying in formation, and then it was sort of integrated with common discovery. And now, it's fully integrated into a single server, but there were differences between upgrades and installs, so we can kind of talk about that.
Right, now we even did a show, awhile back, about how to use the installer to get your databases onto the same server, but now it's all in one database, so it is much easier to manage.
Right, it is, but there are a couple of things, tips and tricks that you're going to want to know about in order to migrate, and maintain the data, make it super silky smooth.
Yes, so if they don't watch this episode, then they'll just blow up their server.
Oh yeah. [Imitates explosion]
That's always a possibility, but no, the manual’s great, and we're just going to give you a cheat sheet to make it look super easy.
Awesome! Well let's take a look.
All right.
Okay, so we're looking at NCM here. We've been looking at NCM for a really long time, it has been from a GUI standpoint, really well integrated for about a year and a half now. Remember before, the controls are kind of wrapped up in the tab, at any event, it's been a part of the Orion Platform core for a really long time. Scroll down to the bottom, see the core number, it's been in there a long time. So, what we're really talking about here is the database. And if you look at the polling details, if you look at all the information that it's using, the way that it's creating an Orion node with a node ID that maps to the device that you're doing the download, none of that has changed. So, the only things really changed here, in addition to some, a few enhancements in the GUI, is lots and lots of things in the database, and that's really what we're going to be talking about. And that's the main reason that we want to have Tom here, because he's going to be able to help explain why you should not be nervous that this is less efficient than having two separate databases. And I get it, because we had two separate databases for a long time, needed to feel good about that, letting go is kind of difficult, so that's really what we're going to talk through here. So, the first thing to note, really the first question is, and I had someone ask me this at a trade show, not too long ago, is isn't inherently one database going to be less efficient than having two separate databases?
Oh, I hear that a lot as well. It's hard to understand why. Sometimes people just think in sheer size, size means less efficient, because more data has to go through your buffer pool and all that. But other times it's more about the complexity of the data trying to merge two databases into one. So, one easy way to do that with SQL Server is simply create two schemas, for one database and the other database. Now you have, guess what, you have two databases inside of one database.
That's true.
It's still just kind of a trick. But now you do have a large database, so now what does that really mean? That's when you dive in, you say, well, what's the data we're really pulling through the buffer pool anyway? What are the queries we're really going after? What are the queries that need to touch both of those databases or schemas? How can we design maybe a little bit better? Maybe we have stuff that maybe needs to be de-normalized. Maybe we have something that needs to be a little bit more normalized.
Right.
What you have is, you have--basically, you have two cars, and you're taking the parts off of those two cars, and you're trying to make one brand new car, and you can't just expect it to be perfect without a little bit of redesign. A lot of times though, people, they'll say, "Oh, one database "will be awful," because they don't want to deal with the redesign, they just want it to work.
But in this case, some of the redesigns were things like a lot of those cross database joins that we had before…
Yeah.
…are gone now.
Gone.
So they're a little bit more efficient. And they're certainly easier to support, and to debug, and to know what's going on.
Definitely, definitely easier to support, is a huge factor.
The other thing is, we had a Band-Aid on here, once upon a time, it was a feature of SQL Server that's kind of handy if you're looking for words that are more than three characters long, depending on how you set up, and that's full-text search. So we've replaced that actually with an indexing service that's actually doing much better tokenization, storing it in a more accessible way.
Good.
And you actually had an interesting point about full-text search. You were talking about it was designed to solve a problem, a few years ago.
Yeah, if you think about the way databases have evolved over ten to fifteen years, full-text search is one of those things that came about as a result of... So most databases start as something kind of small, and get bigger…
Right.
…for lots of reasons. Perhaps this is one reason, and you find that you need to query from it in a certain way, and you say, “Hey wouldn't it be faster if we could just build something to make that happen?” And that's what you get a tool like a full-text search. What ends up happening is full-text search can only scale so far, right? Its scalability and its functionality can only go so far. So people turn to different solutions, like in those SQL solution, or a key-value pair, and now you get things like a columnstore index.
Right.
And so you have different ways to get the job done. A product like SQL 2014 is much more robust in all of the options you have, beyond just using something like a full-text search.
There's another about the installer that's a lot easier to use now. And that is, it doesn't ask you for the database information for the connection, right?
Okay.
So, when you install it, it's going to go ahead and start migrating data for you the first time that you connect to the page. Now that's one thing, about half of you, and actually that's a thing to ask in chat. Tell us in chat how many of you have actually upgraded to 7.3 or beyond, and how many of you haven't, because it'd be interesting to see if you haven't, why haven't you? If this is one of the issues that's maybe holding you back, you're just kind of concerned about how that works. We don't blame them, because there's been a lot of changes about the way that that NCM database has worked over the years. So getting to the state that's super easy to support, and super easy for you guys to manage is really important, that's why we're taking time talking about it here. But once you run that installer, the very first time, it doesn't start the migration immediately, and the first time you go to an NCM page, then it starts the migration. And for most of you that only have a thousand devices, or a couple thousand devices, it just takes a couple of minutes to migrate down from the first. One of the early Beta builds where I think it took four or five hours. And for customers that have tens of thousands of units that they’re backing up every day, and there's lots and lots [mumbles] and they want to keep all of their history, and you actually have the option of not keeping all of your history in that install wizard, it may take at most an hour. But for most people it doesn't. What's really cool is while you're doing it, you'll get this page. And we'll pull up a screenshot of what that page looks like here for you so you can see it, that page will tell you, "Hey, NCM is not available to you right now, I'm doing a migration," and enjoy that. Take a screenshot of that, because you'll only see it once. The other thing that's nice is everything else in NPM, or SAM, continues working normally. So you don't have to worry about that. But yeah, that migration is actually pretty cool. The other thing too, there's been a lot of performance tweaks as a part of this. For example, and I know this is something that we were actually talking about at lunch, GUID. There are a lot of GUID's in the data for NCM. Like, for example, you'll always notice, and you can see up here where I'm looking at this config ID in the URL, right?
Yup.
Config ID equals…
It's right there.
It's a GUID, and there it is with the curly brackets, telling us that it's been serialized out to a string. Well in the olden days, [clears throat] that was actually stored as a char value, in the database, and then it would be the index, and the primary key on this config. That is not necessarily the most efficient way to do indexes and joins on data, especially if you have lots.
I think my ears are bleeding.
Yes. So, there's a huge performance improvement there. We went through the database, and actually correctly data-typed a lot of these columns. So for example, if it's a GUID, it's a GUID type table. If it's an integer, it's now an integer of the right size. And this actually goes across a number of different tables, including inventory, and a lot of the other data. So, there are a lot of performance improvements as a part of that, because if you're going to go in and make changes to the database, you might as well take advantage of that.
That's one of the things I mentioned earlier. A lot of times people don't want to get into those design details, and make those types of changes, and they look for different means. But yeah, it's wonderful to hear the steps that they took to clean up some of those antiquated ways of thinking. Yeah.
And the first time that we talked about it, and I told you about it, you cringed, he actually cringed not quite as big as he did the first time I told him, but I just love watching him do that [laughing] when you talk about it, talk about the data type. One thing that is a little bit different is your NCM will take, the database portion for NCM, will take a little bit more room, in that we have switched to an nvarchar datatype for the configs. Before it was just a regular ASCII data, basically. So we do support double byte characters in now, because we're finding that more and more configs now actually have localized characters, and it's pretty nice to have your backup not die, or the image doesn't actually match what's out on your device. So, then there was only one other thing I wanted to mention, is if you really want to set yourself at ease, that this is not causing additional impact on your Orion Platform server, whether it's NPM, SAM or you have all the modules, and hopefully you've upgraded your Netflow, so that you're now using Flow Storage which should have bought you a lot of overhead, but if you're all nervous about this, if you're not already running...
Database Performance Analyzer.
That's right. Then just go ahead and download it and play with it. I know it sounds like it's a pitch to download it, you don't have to buy it, just download it and play with it, and what you'll find is, do a couple of tests, right? When would you recommend that they run it?
I'd tell you to start running it now to get the baseline of what your performance is like, right now, before you do your upgrade.
So this is on the Orion database?
Right, do it right now. Point it at Orion right now and just collect the metrics just to get your baseline. So that way, after the upgrade is done, then you use DPA again, you're looking at it, and then see the difference in performance.
That's right, and actually you said earlier something else that was really cool, which was about during the download period, go ahead and compare that data as well.
Yup.
Because most of you are doing your downloads and backups in the middle of the night, it's pretty handy to be able to know what the real demand on the database server is...
That's right.
As a result of adjusting all of those backups, so you can actually compare the history charts of when it's just sort of sitting there in the middle of the day, versus two o'clock in the morning, or whenever you're doing your backups. And the nice thing about that that's going to give you an opportunity to really fine tune the performance of NCM. So, let's say you totally peak the database, which you probably won't, you're not going to have that many devices, but let's say you have thousands, and thousands of devices, well one of the things you can do then, is start to explore things like, go into your polar settings, and actually set the polar throttling for maximum simultaneous downloads. I mean that's one of the first things that you do to kind of throttle that down. Now, if you have that many configs that you're backing up every night, that many devices, that may not be an option for you, because you may never get done, so another option there instead, is to chunk them up into groups, and then set up multiple backup jobs, and then you can also set that through the console, which is really really handy. So then break them up into sections, and chances are if you're that big, you're probably geo-distributed anyway, so midnight right here, is not midnight everywhere on your network, so then you can actually time zone adjust those anyway, and load balance those to make sure that those work well. But, yeah, just using DPA, even for a few days just to kind of check that out and see how it works, fine tune that, is a great way to get additional performance in your downloads and backups without having to buy anything new at all, just scale out what you already have.
Right.
I think that's really about it. Was there anything else that you wanted to talk about for the NCM performance improvement with this update?
Yeah, there's one more thing to think about. So, who likes automation? Who likes to automate all the things? [Laughing] So, for anybody big into automation, are they afraid that we might break something?
You talking about SWIS?
I am talking about SWIS.
You're talking about SWQL, you're talking about...
Of course you are.
I am.
Because you're on camera, so we're talking about SWIS and SWQL.
And it's especially important to talk about SWIS, when we're talking about NCM, because that's actually, what I'm finding, one of the biggest drivers for you guys that are doing automation, because if you want to actually programmatically upload, make config changes, basically interactive CLI on those devices, they're already set up, and it really does turn NCM into an automation platform for that. And so in the episode where I actually talked about how to use the automation, there's a THWACKcamp episode from...
Last year.
Last year, from 2013, go check that out. One of the things that I mentioned then, and this is a great example of why that's important, is separating the schema and the objects that you work with through SWIS. And the physical schema behind it is really important, because here we have dropped--didn't drop a whole database-- [laughing] I'm sorry, that means a whole nother thing to you doesn't it?
It means something different.
So we migrated all the data into the database.
That's right.
We added tables. We modified data types. We made a lot of changes in the schema to accomplish that. But you're automation doesn't change, because you're working with high-level objects that don't need to know anything about what's going on through the information service. Remember that's the service that actually maps the information service queries at port 17-777, or interprocess at port 17-778 to the physical schema of the database underneath it. So the great thing is all of your stuff will continue to run, because SWIS is unaffected, because the information services were automatically updated to reflect those changes.
Right.
Yeah.
And I love the fact that we've gone out of our way here to make it as seamless as possible, and to make sure people are still up and running, and their existing code won't break. It's going to migrate. It's going to work.
Because I know that every time I automate something it's always on the weekend, and that's when I have to go fix it.
Right.
That's pretty easy, and the part I like best about that is after this last element, NCM's going to be like any other Orion module, in terms of upgrades.
Exactly.
Well, and it really is now. I mean, this has been out for a while. Probably half, a little bit more than half, of you guys have already upgraded this version. You should go ahead and complete it, especially now that we've walked you through it. And Tom, thanks again for talking to us about why we did it, and the reason behind it, and the real benefits that they actually get. Again, do not be afraid, you're not going to mess up your system. The automated migration is really smooth and easy. And it will definitely perform better on your system. Should perform better on your system, unless you have now doubled the rate of your backups, which you might want to do, that would probably be a good thing. Okay, so then the other thing we wanted to cover in this episode was, I just real quickly wanted to walk you through the new IP address collision feature, when you're using IPAM and UDT integrated together. There's a couple of tricks to make sure that you're set up right, so that you can get the remediation working, and how to use that, cause it's in a couple of different screens.
Right, and we've used a few new protocols, some port IP address mapping that we're going to show you how it works. But once we do that, it'll be ready to go.
Yeah, and I might even show you a little IPv6.
You're going to be the only one.
I keep trying to do IPv6 and nobody cares.
I care.
And they really don't care. Do you still want to see more about IPv6? If you do...
Let him know.
Throw it over here in the chat, and let us know. But yes, there's a couple enhancements there, and if we have time, we'll talk about that too.
Okay, so walk us through this.
Okay, so, IPAM, we know it, we love it, it's really really cool, and let's just get right to it here. One of the things that you get in this version is this IP conflict resource.
Oooo, shiny.
There were ways to tickle the data before to get this information, now we're pulling it up for you in a view, and the really important thing here is to make sure that you also have your UDT configured, and I'm going to show you why in a second, with read/write permissions and set. So as long as that community string has set and read/write, you're going to be able to see the magic here in just a second. So the first thing that happens, as a part of that, is you can actually see the list of conflicts that you have. And under each one, you're getting the physical port that it's connected to, and device and user information that you otherwise wouldn't get. And how many times when you're trying to resolve an issue involving IP conflict, it'd be really nice to have log on information, physical port information, is it wireless, is it whatever else.
Well that's exactly the point. I mean, IPAM has always been able to tell you that this Mac address over here is conflicting with this Mac address over there. But it doesn't tell me whom to call. You know in certain cases, all right, I have my server Mac addresses in a file someplace, or I can get it off of a RRB table on a router. But really, I see these too. I see an Apple and I see a Samsung, like you can see in that first line. Well, what is going on there? So this is really a lot of fun, in terms of the information you get.
And this is the classic IP address drill-down detail table, but it's got some new stuff in it. Like first of all it's saying, "Hey, you have a conflict."
Right. We've made some improvements so that on this page, if you're just tooling around on this page, you know that you have an IP address conflict. You also can see whom it's with, okay? Once again, you can see there's the Mac addresses, and we give little vendor icons. You can see the one on the left is a Samsung, and the one on the right is an Apple device. But whose phone am I calling? Well, you can see there it happens to be Nico Rosberg. Both times. So, what happened here? Well, I can make up a story, which is that Nico came in with his phone, and his phone got an IP address, and then he came and sat down with his Mac laptop, and he plugged it in, and that one got another IP address. He didn't hard-code anything, and I know that just from looking at that screen. How do I know that?
Because they're both DHCP scopes.
Okay, they're both DHCP scopes, and it'll actually tell me how to fix it.
Yeah, that's really cool too, because based on the type of conflict, go down here and say, "Show remediation." And it'll actually give you some steps to fix that.
Right.
Now of course the handiest one, right off the bat, is to kill the port that one of them is sitting on. But you need to figure out which one you're going to kill, so it's really handy to do things like look at your history events, and then figure out what happened there.
Now, I'm going to say a couple things about the story I just invented. First of all, we know it can't be his phone. We know it can't be his phone because it's connected to a port.
Yeah, well maybe USB connection. [Laughing]
Maybe, but probably not. So we know that that Samsung thing is actually hard plugged, and this was one of my fears when I first saw this screen was, oh, you can't shut down the port for that one because that's the port that's connected to the access point. Well it's not, and I'm going to show you how in just a minute. But here, with just a click of a button, you can shut down one port, resolve the conflict so he can, again, assuming that the Samsung is a tablet, or another secondary device, because you can call him to find out. Then you shut down that port, because the fix here, is that you have to un-overlap these two scopes. And that's not something that you're necessarily going to do [snap] on the fly. You're going to be careful about that.
Right, or you could just click shut down, and lean and see if you hear screaming. [Laughing] And then turn it right back on again, and then walk down and say, hey, did anyone have any problems.
Right, oh, I don't know why.
Yeah, so definitely there's that. The other one that was...a couple of other examples here of things that you would see, going back to your point about wireless, here's one right here. A little bit more typical wireless presentation, right? Where we've got a Huawei, and I think this is a RBD device, right?
Broadcom.
Broadcom, yeah. Okay, so it's telling us, that this is actually sitting on an SSID, and it tells us that the SSID is lab. So, that is really, really handy, to be able to start with what is the SSID. Not only is it on an access point, am I going to upset a lot of people if I kill the access point, port...
Notice, there's no clicky.
Yeah, you can't click it. It's telling you it's on the SSID. Now it may be an extra step to go fix that, but if you look at it here, the remediation recommendation is different. It's actually giving you some suggestions about how to do that. Like here it's actually, what, 'ackle' out the Mac, on the WAP itself, or then you can actually go shut down the port if that's an option. But I think option A there sounds the best to me. Let's just 'ackle' that out and take it off. And chances are, if this is running on any sort of halfway modern--Oh, that's right, if I click on that it's going to take me directly out to that SSID, and I can look at everybody else sitting on that. But if it's a halfway modern access point, it's going to be thin, and it's going to have a controller. I may be able to just go into the GUI and take it out of service that way.
There's another situation for conflicts that I wanted to go, so go back and go to the next row down.
Oh yes, this is a good one.
Okay, so this is another situation…
Yeah, this one never happens.
No, no, no, no. So once again, just looking at the screen we can tell a couple of things that are going on here. First of all, on the left hand side we have an IBM device. Also, notice there is no scope listed underneath that, which means that it is hard-coded. And on the right hand side, it's a VM, and it's getting its IP address from a DHCP scope. So, that means that, and this never happens, somebody configured the DHCP scope to include my static address pool, that I'm giving to hard code my IBM, my AIX box that's been running for a million years, now has a conflict, no, nobody's happy with that.
So I'm probably going to want to go see who assigned, who's managing DHCP NW Pool 1 and have a chat with them. And this is actually a great hint for delegation. So we've talked a couple of times about giving people, empowering people to manage their own sub-nets. So you can decide whether they can, not only manage IP addresses in that sub-net, or make IP address changes to a DHCP scope, but you can also decide whether or not they can manage those scopes. And so this would be an example of maybe we don't want that admin managing this scope. Maybe all the addresses in the scope, but maybe they shouldn't be allowed to create new scopes, willy nilly, across my static IP address server range. And especially if that IBM box is an AIX. We'll make the finance guys really upset.
Probably.
Yeah, so that's just really cool. So again, the take-aways there are: it will help you definitely expand the more details for remediation options, details under the violation, because these are dynamically generated based on the conflict type, and it'll really help you with troubleshooting. They're also in descending order of ease of use--and what was it I'm trying to say--least likely to upset users on your network. Right, so you definitely can learn a lot by checking those out. The other thing is use your history events table to tell you, to give you some information about who's on first, or whether it maps to your policy. That'll help you figure out which one is the real conflicting device. And then the other thing is, to enable this integration with UDT, and most of all being able to activate the ability to shut down ports directly from the interface here-- just make sure that you've given it a community string with read/write permissions and set. And then you'll be able to do it right from here. That's really cool, right?
Amazing.
Okay, so definitely that's going to be out in about a week. Those of you who are using IPAM, and most folks actually have IPAM and UDT already, so that's how to set that up and you'll be off and running with that. There is one thing, and I know you guys groan when I talk about it, but I do want to show one little cool thing about IPv6.
I am all about IP version 6. I'm okay with that. Lawrence isn't here, and he's the one...
That's true.
He talked about it in one of his articles about 'we don't think we need it yet,' and everything. I think that's sort of the Luddite way to go. I am A-okay for this. I'm ready for it.
I will tell you that once upon a time, I was highly resistant to learning about IPv6. So the point now that there's a little bit of a 'la la la la la, I am not listening to you. . .' [Laughing] And when we...
Too many numbers. Just too many numbers.
When IPAM added IPv6, I started playing with it, and I learned more about how to set up IPv6 from this wizard than all of the boring reading that I had fallen asleep at bedtime trying to get through. And one of the nice things about doing IPv6 is that you do have the whole concept of IPv6--the subnet really starts with a prefix, right? And so the prefix runs down and then you have your local addresses inside that space. So the wizard will go ahead and take you through the process of defining those subnets. And once the subnets are defined, now the near infinite scope of potential addresses is now way more limited down, and you could do some things with it. So, for example, if I am sitting here looking at my North America group, those are all the addresses in Austin for example, right? So--and those are all prefixes underneath that. So here, if I wanted to do IPv6 Discovery, before you could actually hand it a CSV list of addresses, and it would automatically add them. But then you had another tool to go get that, right. We do not let you, we still do not let you scan...
We are not going to let you scan all the IPv6...no.
IPv6 no? We're not going to do that?
The address phase, no, because you will hate us [laughing] and it will seem like something is broken. And the reality is you should not count grains of sand. But, you should definitely have a more automated way of doing that, and so I am showing you this because it's kind of hidden, and it's not all that obvious. And they're not making that much noise about it in the release notes, so I'm going to show it to you. If you are messing with IPv6, click on Discover IPs.
Don't be afraid.
Don't be afraid, you will automatically pull in the routers that are connected to that. If not, you can add a CSV list here of the routers that you want to scan, and it will go out, pull the tables, look for IPv6 addresses that are transiting traffic across that, or at least in the table, right?
Right.
Okay.
And then it will go ahead and pull those in for you, and you can start to see them. So, it is a much cleaner way of getting them than you having to go use some other tool, or a spreadsheet, and then import a CSV and getting that way. So definitely go play with that. It's really interesting when you discover how many dual-stack devices that you have on your network. It's kind of frightening. Wow, there can't be that many.
Yes, there can. It's enabled on everything now. Nobody looks there because they're afraid of it.
That's right, and so then the next step after that would be, go watch the THWACKcamp 2014 segment where we were talking about the dangers of dual-stack-- making sure that that's one of the things that you're actually scanning for. But yeah, so in short, that's it. That's IPv6 address scanning and discovery. And the main thing is automated conflict; IP address conflict detection, and remediation with expert guidance built in.
And one of the things I just want to re-emphasize is, again, that the integration UDT means that you can find out who to call, and how long, and how many times they've logged in, with this address, or that address. You can see all that history because we're pulling it from sources like Active Directory, and so on. So the UDT piece tells you the 'who' along with the 'what' that IPAM is giving you.
That's right, and the cost of this upgrade?
You know with your maintenance, right there, it's free.
That's right, it's free as long as you're on maintenance. Go get it. It'll be on the portal in about a week. A bunch of you are already Beta testing. I think actually it's out of Beta now. It's into RC. So, some of you are still doing RC testing. Thanks again for that. I hope you go and get THWACK points. Make sure you get your badge for that. But yeah, so this will be out shortly. Go pull it out of the portal and you'll be all set.
One quick commercial about the THWACK points. I'd just like to say that because that was one of the reasons why I'm here today, is sort of a THWACK point number total is that...
Oh brag, brag, brag.
RC and Beta's pay in significant amounts of THWACK points, a thousand THWACK points, so...
Couple of them are two.
Yeah, so it really helps us if you Beta and RC. And it helps you if you Beta and RC because you have a chance to play with them. And Betas and RCs are unlimited, in the sense that you can put as many devices on them as you want to.
That's true.
You put it on a VM sandbox, and you play with it. And it's not like you only get five or ten. So in some cases you can see where the benefit of increasing your license count is. You can find that out all for the price of just kicking the tires on a version.
Cool.
That's true, and then you also can get points for doing UX design, which is really cool, because they will show you mock-ups, even before there's alpha code, and say, "Do you like how this works?" "Do you think this button ought to be there?" So you get points for that, and you don't actually have to set anything up, which is really nice. So, then the question, since you brought it up is, well how do you get invited to participate in a Beta?
Good question.
You watch on THWACK, and they have, they just say, we're looking for Beta users, and you fill out a form.
I would say troll the PMs. So go to the products that you know and love and use, look at the folks that you see over and over again. You'll see Jeremy out there. You'll see Rob out there. All the folks that you already know. And just post into those forums, say, "How do I participate in a Beta?" And they will respond to you and that's one of the quickest ways to do that.
So that's pretty cool.
Yeah, it's really handy, especially if you're using IPAM to delegate administration or remote teams, whether it's remote campus, or maybe you're letting the VM team manage their own subnets. It's really great because then you don't have to worry about it. But if they do actually blow it, and have two IP addresses step on it, then you've got a way to step in and fix it for them.
Or, you can take the time and you can teach them how to use the tool. And then they can dig themselves out of their own hole.
Yeah, that would probably be a good thing for them to do.
Yeah.
Gentlemen, welcome back.
Welcome back.
Hello fellow geeks. Well that was a great show, guys.
It really was. I especially like the NCM history lesson. I think that was very helpful. Now, you know, when I had NCM environments in the past, I have been able to move them though the migration process okay, but I have to tell you, it's going to be really nice not to have to do that anymore.
And Kong, again, welcome to the team. It's going to be really great having you a part of SolarWinds Lab, both for future episodes, and also to participate and hang out with the chat. And your expertise in virtualization is going to make this really, really interesting-- especially when we think about some of the upcoming episodes that we've got, talking about virtualization and how that integrates with SAM.
I'm already testing it in the lab.
On a VM, I'll bet.
Of course. [Laughing]
All right. So before we wrap up, just one more reminder to be sure and visit us at our homepage, which is lab.solarwinds.com. And with the new schedule, you're definitely going to want to sign up for reminders so that you can be here to chat live with us in this chat box, which should be over to the side. Of course if it's not, then you definitely want to be here for the live show. The other thing is that we have so many people on, especially we saw it with THWACKcamp, and we've seen it with episodes of SolarWinds Lab, you get to hang out with a lot of the characters from THWACK in real time. And it's just really fun to watch you guys talk amongst yourselves as well as speak with us on the chat. So be sure you let us know also what you want to see on future episodes. Remember that with our new schedule we're going to burn down the backlog of requests that you guys have for content to cover. So let us know on our homepage lab.solarwinds.com what you'd like to see on the show. I think that might pretty much take care of us today.
Think so, yeah.
All right. Well Leon, you want to take us out?
I'd love to. I'm Leon Adato.
I'm Thomas LaRock.
I'm Lawrence Garvin.
I'm Patrick Hubbard.
And I'm Kong Yang. Thanks for watching SolarWinds Lab. 
