Home labs are always a hotly contested topic. Do you need one, do you not? Can you host it or should it be physical? Can you make do with hand-me down gear or should you buy new? Will it stay powered on or only powered on for lab work? And this goes on and on until you’re absolutely tired of thinking about it.
In my case, I’ve had a home lab for around 4 or 5 years now. I went the new physical route. I found a Dell tower (T110) with a Xeon processor and 16GB RAM for pretty cheap and made it work. It was good enough for multiple VCAPs, a bunch of blog posts, and enough learning to earn a couple new social designations.
However, slowly over the years, my home lab was being shutdown less and less. There were things now running in it which had an uptime requirement (mostly self-imposed) and that 16GB of RAM was almost always over-subscribed.
This all means that it was time for a home lab upgrade! There were lots of characteristics to take into account such as the lab’s footprint, cooling, power, connectivity, noise, and so forth. After a couple months battling back and forth between the new Intel NUCs and SuperMicro Xeon-D systems, I ended up going the SuperMicro route. I also decided to go with the rackmount option. The rackmount cases have slightly less depth than most closet shelves and plenty of space to adding in a bunch of extra drives or PCIe devices.
Next was choosing the board. This offers its own complexity because the X10SDV boards have gained popularity over the years and there’s a bunch of different combinations. I ended up landing on the board with the Intel Xeon D-1528 processor. It has 6 cores (12 threads), goes up to 128GB RAM, and only requires 35W of power. I also skipped the sticker shock of going straight to 128GB right away and went with 64GB.
As the links indicate, I used WiredZone for the purchase. They’re an authorized reseller and had everything in stock. I would definitely order from them again.
Couple days later the parts started arriving and it was time for assembly! This was a little more frustrating than I anticipated. Pain points were the power connection and the fans.
The power problem was due to the case PSU including a 20-pin connection while the motherboard had a 24-pin power connection. I had read a couple places that these boards could be powered multiple ways, one through the 24-pin connection and the other through a 4-pin connection. However, they should NOT be used at the same time. After some discussion on the OpenHomeLab slack group with Mark Brookfield, we got it sorted how the power should be on pins 1 through 10 and 13 through 22. Big thanks to the OpenHomeLab folks and Mark especially. I was also able to confirm this configuration through WiredZone, whom responded back in less than 24 hours.
The other issue was with the fans. I ordered some fans since the server would be sitting on a shelf somewhere in my house. Basically, the more fans to move the hot air out the better. The fun with the fans was because there’s not really any instructions on what to do with them or the fan holders. The fan holders did end up connecting together and then they basically sit on top of these pins. From that point, the fans literally just sit in the holders and are connected to the fan accessory ports on the motherboard. The case top is what helps hold everything together. Certainly not what I expected, but it’s moderately better than zip tying the fans to the case.
Everything else was extremely straight forward. Only other wish, the case could use some extra room for better cable management. You’ll see why in the assembly images below.
It’s all cabled up and I can finally hit the power button and watch it spring to life, which it did with no fuss.
First thing I did afterwards was configure the BIOS settings as per Paul Braren‘s advice over at TinkerTry: Recommended BIOS Settings for Supermicro SuperServer SYS-5028D-TN4T
Next up, install ESXi to an attached USB thumb drive. I did this by way of the motherboard’s KVM and it was quite easy, just like it would be via an HP iLO or Dell DRAC.
Lastly, install the VIB to make the 10GbE ports work properly. Paul has a great write-up on that as well over at TinkerTry: How to download and install the Intel Xeon D 10GbE X552/X557 driver/VIB for VMware ESXi 6.x
I had the system running for 90 days straight before I shut it down to start the upgrade to vSphere 6.5. No hiccups or any issues whatsoever. I’ve been extremely happy with the purchase, almost to the point of wondering why I hadn’t done it sooner.
If you have any questions as you’re reading this, feel free to use the comments or find me on Twitter. Also check out the OpenHomeLab site and their slack channel, and definitely browse through Paul Braren‘s blog TinkerTry. Paul has done an immense amount of research and countless hours of hands-on work with these SuperMicro motherboards.
Now to start thinking through the next home lab upgrade whether I should do vSAN or upgrade my Synology…