Hey! Mitch here, back with another weekly tech tip. Today I'm gonna be talking about keeping your IPMI secure. So I've talked about the IPMI a little bit in previous videos, but this week I'm gonna take you over to my desk, and I'm going to show you ways to keep your IPMI secure.
So as I touched on in a previous video, the IPMI is a great interface for system administrators to manage their server. And it does this by way of the BMC, or the “Baseboard Management Controller”. So, when I try to explain to people what the BMC is, I essentially just tell them it's kind of like a Raspberry Pi that's attached to their motherboard itself. And so, the BMC can effectively both allow you to manage your motherboard, as well as-it has a bunch of sensors that are attached to it that can read out different component stats for your motherboard as well.
So some of you may have seen in the news lately that there's a security flaw found on the BMC of Super Micro motherboards. Thankfully this has been patched on the X10 and the X11 motherboards, with X9 coming soon. Unfortunately, however, this doesn't come with an automatic fix for you. You are gonna have to sit down and patch it yourself, and for those reasons I thought we can come over to my desk and I would show you not only how to patch your IPMI, but also give you some tips to secure your motherboard for, maybe future problems that may occur.
Alright, so the first thing we're gonna do right away is to ensure our IPMI’s are patched and no longer vulnerable to the security flaws that Eclipsiam found. So the first thing we're going to do is log into our server’s IPMI. This is done simply by going onto your web browser and entering the IP address you assigned in the motherboard BIOS to your IPMI. If you didn't change your username and password, the default is going to be in all capitals (ADMIN/ADMIN), and later on the guide I'm going to be showing you how to change this because this is definitely something you want to get rid of; you never want to keep anything at their default settings.
Okay, so the first thing we're gonna do now that we're in here, is we're gonna check and ensure that we know exactly what model our motherboard is. So you're gonna come up to the system, just like I did, and then click on “FRU Reading”. So if you come on down here, you see “Product Board”, “Product Name”; so it's “X11DPL-i”. Okay, so from there, you want to go over to Super Micro’s website, where they give the new BIOS and new IPMI firmware updates, and so you're gonna see at the-this website right here, you're gonna see all the different possible updates. So you're gonna come over and just type in your board name.
Alright, so you can see there's two different choices here. So obviously one is a BIOS and one is an IPMI. So that's what we want to do today, is update the IPMI. So you would come in here, click on this (PDF Link), and accept, and it would download a ZIP file. Now I've already done this, just in the interest of time, so you go into your folder and unzip it, and once you unzip it you're gonna see these different files.
Now the important one we're gonna look at here is the BIN file. So that's the file you're gonna need to update your IPMI. Alright, so let's go back into our IPMI, and we're gonna go over here to “Maintenance”, and when we go into “Maintenance”, we're gonna click on “Firmware Update”. Now, here's the one thing to note; it says here if it puts it in a special mode, so essentially once you enter “Update Mode”, your system has to reboot whether you update or not. So anyway, we're gonna enter our “Update Mode”. Now, “Yes”, (just asking us if we are sure we know what we're doing).
Okay, so, here we're going to choose the file. So, we're gonna go in and find our file that we just created, or that we just downloaded. So I got mine in IPMI, and there we go, there's our .BIN file. From here, we're gonna upload our firmware. And this might take a little bit of time depending on how fast your connection is to your server.
Alright, so now it's complete. So there's a few things here that you might want to check before hitting “Upgrade”. So one thing-definitely you want to preserve your configuration, so this just makes sure that it's not going to default back to factory settings, and then another big one here is make sure to preserve your SSL so in case you created a certificate for your IPMI interface, so that's something you may want to leave checked as well. Most likely you're just going to leave everything checked here and then you're going to hit “Start Upgrade”. Now you can see that the device is now in upgrade mode, so we're just going to wait until this finishes and then once it's done, it's going to automatically reboot the system.
So as you can see, it has now kicked us back out to the login screen, so this is while it's doing its rebooting, and so obviously you can't be signed in while it's rebooting. Now, I could try to sign in right now but most likely it's not finished rebooting. If you wanted to ensure you know if it's rebooted or not, you could always come into your start menu and open up a command prompt, and then just come over and try to ping the address itself. Okay, so as you can see, we've got a ping response, so that means the server looks like it's back up, so let's go back in and let's continue on.
Okay, so we're back in, and our IPMI flaws have been completely wiped away with the new patch and the new update, so we're no longer susceptible to those flaws.
And actually, I thought I would actually just talk a little bit about the flaws that were found. So essentially there was four main flaws that they found and they include things like plaintext authentication, unencrypted network traffic, weak encryption methods, and then finally there was an authentication bypass. Now I think it was essentially if someone logged in with admin privileges and then logged out, someone could come in behind them and reuse those authentication. Now, this is only a massive problem if you're a company that actually forwards their IPMI to the internet, which we absolutely never recommend. You never want to have your IPMI accessible to the Internet. Now that being said, if you are a company or someone that needs to have remote access to your server from outside of your network, there is ways to do that properly, but forwarding your IPMI is just, that's absolutely not the way to go about it.
Actually, I wrote a blog there recently on our 45 Drives website and it explains all of this and goes through some ways to harden your IPMI and also different network configurations you can use; it goes through the base configuration that you wouldn't want to use all the way up until the best possible scenario that you could have. And so I'll link that in the description below, or you can also just go to 45drives.com, and then click on “Blog” and then it would be the top one.
So continuing on, as I talked about before, one thing you definitely want to get rid of right away once you set up your IPMI is your default username and password. Now on this server, obviously, it's not a big deal because this is just a test environment just to show you guys how to do things, so it doesn't really matter here. But essentially what you would want to do right away when you get your server and if you setup IPMI is modify this administration user. If you can change the password, that's great, but if you could remove the name as well, because anyone trying to access your IPMI-they're probably going to know the default username is “ADMIN” as well.
So you would just come in-“Modify (User)”, and then you would come in and change the name to say whatever; even “ADMINISTRATOR5” or something like that, just anything to get it out of the realm of default, and then you would click “Change Password”, and then put a new password. And then also, you would set what type of privileges you want to give them. So that would be the main one. Also, just to let you know, there is better ways to manage users that are allowed to have access to your IPMI, and that is, you can use your “Active Directory”, as well as LDAP and “RADIUS” Servers. So there is a few ways to give privileges to your IPMI that aren't just default username/password kind of ID's that IPMI allows for.
So the next thing I'm going to talk about is… let's say maybe you see this and you are very worried about the flaws that are in the the IPMI currently, but you can't take down your server for an update, because maybe it's performing mission-critical activities right now and you just can't restart your server currently, but you still want to be protected.
So, there is a way to do that right now just to be safe, and what you would do is you come into “Configuration”, you click on “Port”, and because most of the flaws really came from the virtual media port, and allowing people/outside attackers to have access to this port, and allow them to mount either USB drives, or ISO’s, or any type of software on your IPMI. So, one way to just kind of take that out of the equation completely until you can get your IPMI up and running again after the patch, is just unclick “Virtual media port” and save it. And so from then on out, you also won't be able to install any software through the IPMI, but that means attackers won't be able to either. And then, once you get your IPMI patched, you can come back in and you can recheck it, and save, and then you'd be back to normal.
Okay, so the last thing I'm going to talk about today is IP access control. So, in an ideal network environment, what you would have is you would have a VLAN, so a dedicated land for your management ports, or your servers, anything that's sensitive, and you want kept off the main network in your office or wherever your server may be. So that's the ideal way to have things, and then you would have a specific designated computer or set of computers that can login to the IPMI through this VLAN. Now, many places just can't do that, whether it's, they just don't have the money for the extra switching, or the know-how, or things like that, especially with a small business environment, you may not have an IT guy on hand. So if you're not able to do that, the next best thing that I would recommend is have what's called an “IP access control list”, and what this essentially will do for you is allow only certain IP addresses, so certain workstations on your network to have access to your IPMI.
Now, what that does is, one, it allows maybe employees or workers or whatever to not be able to even stumble in and even get to the login part of your server’s IPMI, which most likely they wouldn't have any reason to, so why not just take that out of the equation altogether. And also you can think if an attacker does get access to your network, it may be through one certain vector, like a client that opened up an email. Now if you have proper security in place (network security), you should be able to stop that spread to be just for that user and they shouldn't be able to get out throughout the rest of your network.
And now if that's the case, what you would like to have is this in place so that attacker couldn't then go and get into your IPMI and start poking around and trying to brute force their way in. So what you would do here is you would enable IP access control and click “OK”, and so from here once you do this though, the only IP’s or the only workstations that will be allowed or will be able to access your IPMI will be the addresses on this list. So if we want to start adding them, essentially, it's as easy as putting the IP address in of the workstation, and either give a “ACCEPT” or a “DROP”. So we can allow them or not allow them or just drop it, so when it's a drop, it just won't even acknowledge that that's a website or that's a location that they can access.
So one last thing I want to talk about is just the ability for IPMI to log any attempts to log in, pretty much anything you can think of, you can actually have that forwarded to a Syslog server, and it's very simple to set that up if you have your Syslog server already running; you just come down to “Configuration”, go to “Syslog”, and obviously it says it's not activated, so we need to activate it, and it's as simple as putting in the IP of your Syslog server and what port it listens on, and then you click “Save”, and then from there you have a pretty good record of the comings and goings of people in your IPMI, and then you know for sure who's coming in, who's logging in, who's doing what; so I would recommend if you have the resources to set something like that up for sure.
Alright guys, well thanks for coming along for the ride here, showing some of the things you can secure in your IPMI and best practices. Now as I said before, I'll link the article that I wrote in the description below that really goes into more detail on what happened with the security flaw, and it actually gives a full, detailed guide on what I did previously with the patching of the IPMI, and it also shows some best practices in networking and things like that. So yeah, if you want to check that out it'll be down there for sure, and thanks for watching.
Alright guys, so hopefully you can take some of the things I showed you in this video and apply it to your infrastructure to help keep you secure in the future. So, if there's any questions or comments that you might have about this video or any ideas for future videos, be sure to leave it down in the comment section. And until then, we'll see you in the next video.
Contact us to discuss your storage needs and to find out why the Storinator is right for your business.Contact 45Drives