Supermicro teething troubles and voltage warnings
During my recent build of the vHydra server I found myself rather frustrated with Supermicro for a couple of reasons.
Firstly their UK distribution doesn’t seem to be working particularly well as there’s a two week wait for most parts which are apparently shipped from the US on demand. There are UK based resellers (I tried www.boston.co.uk) but even then some parts still have a long lead time (around a week) and I found them to be expensive compared to alternative web based vendors.
Secondly their technical support was somewhat lacking. Once I’d built the server I found I was getting an overvoltage warning on the second, empty, CPU socket. As I was planning on populating this socket (once the second CPU and heatsink arrived, another three weeks wait :-() I was keen to know if this was a false positive or whether the board should be returned as faulty.
I emailed Supermicro technical support who went through the usual information gathering – firmware, BIOS, motherboard details etc. They identified that the IPMI firmware was out of date (I had v1.48) and suggested I upgrade that first. They provided the location of the latest firmware (1.76 as of October 2012) and I used the IPMI web GUI to update the firmware. The upgrade procedure appeared to have worked although I had to reconfigure the IP address etc as during the firmware upgrade it was recommended to reset to factory defaults, which I did. Luckily my server is in the basement rather than at a remote datacentre so this was trivial. However when I brought up the IPMI web console and tried to login, the default credentials (ADMIN/ADMIN for those who don’t know) didn’t work. I also found that VMware vCentre, which until now had also reported the voltage warning, was now not seeing any CIM data. While the server was still working fine it meant I had no way of knowing if the overvoltage warning was still an issue (and therefore didn’t want to fit the second CPU which might get toasted).
At this point Supermicro technical support advised I try reflashing the IPMI firmware using either Linux or Windows (as I couldn’t use the web GUI anymore) and said if that didn’t work I’d have to RMA the motherboard and return it to them as there was no way to reset the default credentials. Knowing that would mean losing my server for a protracted period of time I did a quick Google search and found it IS possible to do, in fact it’s pretty easy. That’s a big FAIL for technical support!
Resetting the IPMI configuration/credentials
You need the Supermicro IPMICFG utility which you can download from here (I downloaded v1.02). You’d think their own technical support team would know and advise about this.
NOTE: I used IPMICFG not IPMITOOL which was referred to in most of the articles I found. IPMICFG is newer (it was released at the end of May 2012 around the same time as my motherboard) but your mileage may vary.
To reset the IPMI you need an OS installed on the server containing the IPMI card. I wasn’t sure it would run on ESXi given that it’s a proprietery kernel but I copied the files across and to my relief it worked fine and was able to communicate with the IPMI controller. While checking the configuration and running a few diagnostic tests I found the root cause of my problem – the default credentials had been removed during the firmware upgrade;
I briefly tried adding another user but couldn’t find the syntax required so I simply issued another factory reset;
This returned a ‘Factory reset completed’ message and after a couple of minutes I could see that the default ADMIN user had been restored. Furthermore I could now see that the IPMI firmware update had fixed the voltage warning – the CPU2 vCore was now correctly shown as N/A rather than the 1.8V warning I had previously;
My server and IPMI are now working correctly although for some reason vCenter’s hardware status only shows a single entry (compared to about thirty on the old firmware). Overall it’s left me with a feeling that Supermicro support wasn’t up to scratch. There are no release notes on the website Supermicro saying what’s new in BIOS updates or IPMI firmware updates, so no way of checking if your issue is known or catered for. I’m a believer in updating only when required, especially when any firmware upgrade has the potential to turn a motherboard into a brick. They’re also the only company on the planet with no support forums for end users. I know Supermicro don’t target the average consumer but given how easy it is to implement a forum (even if you don’t moderate or contribute much) it’s far from ideal.
UPDATE 30th October 2012 – There are firmware release notes bundles with the firmware update, and this voltage warning was resolved in v1.70 onwards. Thanks to Steve (in the comments) who let me know that firmware 1.81 (in beta) is available via Supermicro support, whereas 1.76 is the latest on their website. 1.81 is expected to be released within 30 days.
I guess you get what you pay for. The Supermicro motherboards are feature rich and available in a wide range which is very useful but I wonder if the cheaper price is reflected in the support you’ll receive (tier 1 providers like HP are by no means perfect either however!).