New Home Lab

The Quest Begins

I trolled the homelab subreddit for a while, discussed options with friends and colleagues, and did a ton of research. I wanted something quiet, reasonably powerful, and expandable. While cost was a factor, it didn’t drive my decisions.

Many folks advocated purchasing older servers from eBay, where the Dell PowerEdge R710 is among the most popular options. It’s true there are really good deals out there for fairly powerful servers at very reasonable prices, but I chose a different route.

Unfortunately, commercial servers are generally pretty loud. I have to keep the server in my home office so I need something that won’t be distractingly loud. I also intend to migrate eight 3.5-inch drives and those older servers won’t accommodate that need.

Configuration Finalized

After some back and forth, some minor changes, and one final pass to validate compatibility, I settled on a set of components I was reasonably sure would work well together, meet all my current needs, and allow for further expansion down the road.

Motherboard

I looked at a number of manufacturers as well as consumer, workstation, and server motherboards. Boards with the Intel X99 chipset were contenders for a while, but the Intel C612 chipset won out for its expandability and other higher-end features.

The short list included boards from Supermicro and ASRock Rack. I decided I didn’t need dual processors. I valued expansion slot configuration, memory slot count and total supported memory, presence of a BMC, as well as the type and number of NICs.

Because I plan on using Storage Spaces, boards with built-in HBAs provided little additional benefit, so I focused my efforts on boards that would allow me to run Hyper-V and expand memory and storage as needed.

That led me to the Supermicro X10SRi-F motherboard. I ordered it from Wiredzone.com.

Many folks stating it wouldn’t fit in standard ATX cases due to a non-standard mounting hole layout. That turned out to be partially true, but completely non-problematic. I purchased some plastic standoffs from Best Byte Computers to help.

Processor

This one was actually pretty easy. I knew I wanted an Intel Xeon processor with more than four cores. I landed on the E5-2600 v4 series and looked at which processor gave me the highest number of cores for my money.

I decided to go with the Intel Xeon E5-2620 v4. I ordered it from B&H.

CPU Cooler

I almost went with a Noctua cooler, but ultimately decided they were going to make it more difficult for me to mount and get at the memory. I wanted a decent cooler and a quiet fan, so I went with a hybrid approach.

I ended up choosing the Supermicro SNK-P0050AP4. I ordered it from Amazon.

I decided to replace its fan with a Noctua NF-A9 PWM. I also ordered that from Amazon.

Memory

I limited my search to memory on Supermicro’s compatibility list. I originally intended on using 32GB DIMMs, but I ended up finding a very good deal on 16GB DIMMs. While this will limit me to a maximum of 128GB of memory, it should meet all my near term needs.

The price difference between 2133MHz and 2400MHz memory was negligible or non-existent, so I went with the 2400MHz memory just in case.

I chose Micron memory from Supermicro. I ordered it from the Supermicro eStore.

System and Virtual Machine Storage

I must confess I only looked at Samsung SSDs. I’ve used SSDs from a number of manufacturers in the past and I feel Samsung is the best. I almost went with PROs instead of EVOs, but just couldn’t justify the additional cost.

I went with the Samsung 250GB 850 EVO for my system drive. I ordered it from B&H.

I went with the Samsung 1TB 960 EVO for virtual machine storage. I ordered it from B&H.

Since the motherboard doesn’t have an M.2 slot, I needed a PCIe adapter to install it. There are a few options on the market, and I almost went with the Angelbird Wings PX1, but I ultimately decided against the luxury option.

I chose the Lycom DT-120. I ordered it from Amazon.

Power Supply

I’m a big fan of Corsair power supplies for a number of reasons. For this application I won’t need a lot of wattage, but I did want modular cables and silent operation. I looked at the AXi, HXi, and RMx series, which are all great options.

I decided on the Corsair RM750x. I ordered it from Newegg.

Though there isn’t anything wrong with the Corsair cables, I’m thinking about replacing them with custom cables from CableMod. What can I say? Individually sleeved power supply cables are pretty awesome!

Case

This one was difficult. I really went back and forth between a rack enclosure and a tower enclosure. Then back and forth between a Supermicro enclosure or some other brand. Then back to the rack versus tower debate. Seriously, it was pretty tough.

In the end, I valued low noise, ease of installation, aesthetics, and expandability most. I’ve used Corsair cases for some time now. My main desktop PC lives within a Graphite Series 760T, which I love. I particularly like the default 2-in/1-out fan configuration, which gives me a bit of positive pressure to help keep dust from settling on the components.

I chose the much loved Corsair Graphite Series 760T. I ordered it from Newegg.

Since the included fans do not support PWM, I again turned to Noctua for replacements.

I went with three Noctua NF-A14 PWM fans. I ordered them from Amazon.

Let’s Build This Thing

The built went very well. I only ran into a couple tiny speed bumps, which I will describe here. I’m extremely pleased with the performance of the system as well as how quiet it is. I’m actually quite amazed at how quiet it is, even when under load.

Here’s everything ready for installation.

img_1862

Processor and Memory Installation

There’s nothing exciting to see here. it’s all relatively painless and exactly as you might expect if you have previous build experience. I went ahead and prepped everything on the bench as I find that easier to manage.

While not seen in the photos, replacing the fan on the cooler requires popping out and reusing four friction-fit connectors. I also had to remove the silicone anti-vibration pads from the cooler side of the Noctua fan. Finally, the j-shaped clips had to be adjusted to accommodate the thicker walls of the Noctua fan.

It’s all pretty self-explanatory once you see it. I’m sorry I don’t have any photos to share. I received the replacement fans after everything else was put together and was so excited to get them installed, that I forgot to take pictures of the process.

img_1872

Motherboard Installation

It’s true that some the mounting holes in the motherboard aren’t where you might expect them to be on a normal consumer motherboard, but it certainly doesn’t present any sort of issues or challenges.

First, I installed a flat-bottomed plastic standoff in the mounting hole next to the memory slots. This mounting hole does not align with any available holes in the case’s motherboard tray, so the plastic standoff is a perfect solution.

img_1873

Next, I removed the round-nosed standoff as seen in the following picture. There is no mounting hole in the motherboard that corresponds with this standoff, so it’s got to go.

Finally, I moved the standoff as seen in the following picture. I think this new location is typically used for microATX rather than full ATX motherboards.

img_1875b

That’s all that was needed to get the motherboard installed. Everything else, from installing the I/O shield to running the power cables was exactly the same as every other build I’ve done.

Putting the Rest Together

After getting the motherboard situated, I installed the power supply and drives, and cabled everything. The first picture was taken before I replaced the stock fans. The second is how it looks now that everything has been installed.

img_1879

img_1882

Fan Surge Correction

Once the Noctua fans were installed, I powered the server on and found that the fans were going from slow to fast and back again. After some research I found the fan sensor defaults in the BMC need to be adjusted because the normal Noctua fan speed is below the default threshold. The BMC freaks out a bit and causes the fan surge.

Since I already had Hyper-V setup at this point I deployed a new virtual machine running Ubuntu Server 16.04.2 LTS to make the adjustment.

Once Ubuntu was installed, I logged in and installed the IPMI tool using the following command.

$ sudo apt-get install ipmitool

Next, I noted that the Noctua NF-A9 PWM was connected to the FAN1 header and executed the following command to adjust its sensor thresholds. I chose these particular values because the A9 runs at 400 RPM.

$ ipmitool -I lanplus -H 192.168.1.112 -U ADMIN -P ADMIN sensor thresh FAN1 lower 200 250 300

Next, I noted that the three Noctua NF-A14 PWMs were connected to the FAN3, FAN4, and FAN5 headers and executed the following commands to adjust their sensor thresholds. I chose these particular values because the A14 runs at 300 RPM.

$ ipmitool -I lanplus -H 192.168.1.112 -U ADMIN -P ADMIN sensor thresh FAN3 lower 100 150 200
$ ipmitool -I lanplus -H 192.168.1.112 -U ADMIN -P ADMIN sensor thresh FAN4 lower 100 150 200
$ ipmitool -I lanplus -H 192.168.1.112 -U ADMIN -P ADMIN sensor thresh FAN5 lower 100 150 200

Finally, I rebooted the BMC by executing the following command.

$ ipmitool -I lanplus -H 192.168.1.112 -U ADMIN -P ADMIN mc reset cold

Note that you will need to supply your BMC’s IP address, username, and password in place of the values I have included here.

Cost, Noise, Heat, and Performance

Cost

Alright, so this thing wasn’t cheap. It cost me $2,056.44 to build. Here’s a breakdown of the costs, which include tax and shipping where applicable.

  • Motherboard: $265.20
  • Processor: $414.99
  • CPU Cooler: $46.08
  • Memory: $357.10
  • System Drive: $98.00
  • Virtual Machine Drive: $479.99
  • M.2 Adapter: $20.70
  • Case: $178.07
  • Power Supply: $109.24
  • Plastic Standoffs: $4.27
  • Cooling Fans: $82.80

But it was totally worth it!

Noise

I used the NIOSH Sound Level Meter app on my iPhone and here are the results.

Before I started on this adventure, my home office was reading 40.5db(A) on average, which isn’t terribly loud. I was hoping to decrease that a bit, and was excited to find that I decreased it by a lot more than I expected!

When the new server is idle, my home office is reading 28.3db(A) on average. When I place the server under load by running Prime95, the average reading increases by 2.8db(A) to 31.1db(A).

The fans run at 400 RPM (for the A9) and 300 RPM (for the A14s) at idle. They ramp up to 1000 RPM (for the A9) and 800 RPM (for the A14s) when at load. Even under load, these fans are extremely quiet.

Mission accomplished!

Heat

I’m using an AcuRite Indoor Temperature and Humidity Monitor and I’m reluctant to conclude that this server has made any appreciable difference in the temperature of my home office. The temperature has been pretty rock solid before and after I introduced the new server to the environment.

Performance

I’m completely happy with the performance of the system. I’m enjoying having more processor cores and memory to work with, and the drives are awesome!

Just because I’m so excited about these drives, here are a couple screenshots. The first screenshot is from the 850 EVO and the second is from the 960 EVO.

850evo-960evo

Conclusion

I spent a bit over $2,000 on a custom white box home lab server to run Hyper-V and I couldn’t be happier. I spent a lot of time researching components, locating the best dealers, and tracking down the best prices. I knew where I needed to look out for complications (like the motherboard standoff situation), but didn’t know about others (like the fan sensor settings). In the end, Everything worked out and I now have a fantastic little home lab server that’s as quiet as a mouse.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s