HomeLab Part 1: Design Process – Low Power Designs

This post or rather this series was in my queue now for about 5-6 months. Sometimes I didn’t have time and sometimes I didn’t have passion to finalize it. During this months I created my new lab, thought about other possibilities to enhance it and tried to get these series about my new Homelab published. The first Part of this series will include my thoughts around the process of evaluating my new equipment.
After reading some great posts from Erik Bussink and Frank Dennenman about their home lab upgrades, I was a little bit jealous and decided to upgrade my current home lab which can be found here. It was not because of the posts but rather because my current lab is running out of resources, especially memory. I have this plan since January 2014 and it took me 2-4 months (couldn’t remember how long it really took) to evaluate all the possibilities and I’m little bit like Dagobert Duck when it comes to throwing money down the drain.

My two main concerns for the new lab are costs and power consumption. Therefor I have done some research and created 4 designs for my new home lab with all advantages and disadvantages. As of writing this post the designs went up to 6/7 because between January and now (December) lot of things happened in the hardware space. But first of all, what do I want to accomplish with my new lab:

• Testing new VMware Software (Beta, NSX, Horizon6 etc.)
• Testing new Software (Unitrends, Veeam, Nakivo etc.)
• Testing VSAN physical -> as of writing (December), this will be moved to 2015 because of money contraints
• Testing VSAN virtual

There are some features my new lab should also consist of:

• Remote Management (IPMI, Remote KVM)
• Possibility to use more than 32GB memory
• Expandable
• lasts for the next 1-2 years

One of my current problems in the lab is that I only have 1 ESX host that can run 24/7 without wasting too much power and money. When I need more memory for testing e.g. VMware Betas I have to power on the x3650 M2 which will drain about 170 Watts which is too much.
This part of the series will cover the Low Power Designs.
My first design is based on the Intel NUC. This box is not quite expensive and has also a really low power consumption because of the processor.

Low Power Design 1

Component	Type	Cost
Server	Intel NUC Kit D54250WYKH	310 EUR
Mainboard	incl.
CPU	incl.
CPU Cooler	incl.
Memory	Corsair Value Select CMSO16GX3M2C1600C11	143 EUR
SSD	Samsung SSD 840 EVO 250GB, mSATA 6Gb/s	129 EUR
HDD	Western Digital WD Black Mobile 750GB WD7500BPKX	59 EUR
Power Supply	incl.
Case	incl.
Misc.	Power supply cord C5	9 EUR
Price per Server		650 EUR

Advantage

• Price
• Formfactor
• Power consumption
• Hyper Threading
• VT-x/VT-d

Disadvantage

• Limited to 16GB memory
• only one Gbit NIC
• only 2 Core CPU (4 threads /w HT)
• CPU performance
• only mini-HDMI or mini-Displayport, no VGA/DVI
• no IPMI (Remote KVM)
• limited to 2x 2.5” HDD/SSD

Low power Design 2

Component	Type	Cost
Server	HP ProLiant MicroServer G8, Celeron G1610T, 2.3GHz, 2GB RAM	242 EUR
Mainboard	incl.
CPU	incl.
CPU Cooler	incl.
Memory	Kingston ValueRAM KVR1333D3E9SK2/16G	178 EUR
SSD	Samsung SSD 840 EVO 250GB, 2.5″, SATA 6Gb/s	111 EUR
HDD	Western Digital WD Black 1TB WD1003FZEX	72 EUR
Power Supply	incl.
Case	incl.
Misc.	-
Price per Server		603 EUR

Advantage

• Price
• Form factor
• Power consumption
• two NC332i Gbit NICs (HCL supported)
• IPMI (iLO4)
• expandable (HDD)
• VT-x

Disadvantage

• Limited to 16GB memory
• Form factor bigger than Low Power Design 1
• Only 2 Core CPU (2 threads)
• CPU performance
• no Remote KVM, only with iLO4 Advanced pack (additional cost)
• no Hyper Threading
• no VT-d
• SATA Controller B120i (not on HCL)

As you can see both low power designs provide a lot of features at reasonable prices, but both design lacks the amount of memory I can put into them. When going for one of the low power designs I would choose the MicroServer because of the following features:

• 2 onboard NICs
• IPMI (KVM with additional license)
• expandable
• 7-8% cheaper than Intel NUC
• 250GB more space

Some of you may ask why there is only 750GB in the Intel NUC and 1TB in the MicroServer. First of all the disk are different form factors (2,5” for 750GB and 3,5” for 1TB). The WD disk was the only disk with 7200RPM and 5 years warranty. I could also have choosen a 1TB 2,5” disk but then warranty would only be 3 years. This decision depends on what requirement you have for your homelab.
Because this post was in my queue since July, in the meantime new potentially lab servers were released.
For the third low power design I used the HP ML10. I don’t exactly know where HP is positioning this server but from a performance standpoint it’s between the MicroServer Gen8 and the ML310e Gen8.
Low Power Design 3

Component	Type	Cost
Server	HP ProLiant ML10, Pentium G2130 3.2GHz, 2GB RAM	221 EUR
Mainboard	incl.
CPU	incl.
CPU Cooler	incl.
Memory	Kingston ValueRAM KVR1333D3E9SK2/16G	178 EUR
SSD	Samsung SSD 840 EVO 250GB, 2.5″, SATA 6Gb/s	111 EUR
HDD	Western Digital WD Black 1TB WD1003FZEX	72 EUR
Power Supply	incl.
Case	incl.
Misc.	HP 360T Dual-port NIC (eBay incl. shipping)	30 EUR
Price per Server		612 EUR (16GB) 790 EUR (32GB)

Advantage

• Clock speed
• 32GB memory
• IPMI (iLO4)
• expandable (HDD)
• VT-x

Disadvantage

• Price
• Power consumption higher than Low Power Design 2
• Form factor bigger than Low Power Design 2
• Only 2 Core CPU (2 threads)
• no Remote KVM, only with iLO4 Advanced pack (additional cost)
• no Hyper Threading
• no VT-d
• SATA Controller B110i (not on HCL)
• Only one NIC Controller NC112i (not on HCL, but should be the same as NC112T which is on the HCL)

Compared to the Microserver the ML10 is newer but using the old B110i SATA Controller and has only 1 NIC so you need an additional NIC. You can take a HP NC332T NIC (111€) which makes the whole build more expensive or you’re searching through eBay to find a HP NC360T for 30€ incl. shipping. The biggest advantage over all low power designs is the clock speed of 3.2 GHz and the possibility to put 32GB memory in each server. When putting 32GB of memory into the ML10 than the price is a disadvantage because it makes the server 31% more expensive than the Low Power Design 2.
The next designs are both based on the Intel Avoton SoC which comes in either Quad-Core (C2550) or Octa-Core (C2750). The main different between these two are the vendors. For design 4 I choose AsRock and for design 5 SuperMicro.

Low Power Design 4

Component	Type	Cost
Server	Whitebox
Mainboard	AsRock C2750D4I	370 EUR
CPU	incl.
CPU Cooler	incl.
Memory	Kingston ValueRAM KVR1333D3E9SK2/16G (2x 178 EUR) Corsair ValueSelect DIMM Kit 16GB (2x 133 EUR)	356 EUR (Kingston, ECC) 266 EUR (Corsair, non-ECC)
SSD	Samsung SSD 840 EVO 250GB, 2.5", SATA 6Gb/s	111 EUR
HDD	Western Digital WD Black 1TB WD1003FZEX	72 EUR
Power Supply	Enermax Revolution X't 530W ATX 2.4	78 EUR
Case	Fractal Design Node 304	70 EUR
Misc.	-
Price per Server		1057 EUR (Octa, Kingston) 989 EUR (Quad, Kingston) 967 EUR (Octa, Corsair) 899 EUR (Quad, Corsair)

Advantage

• Clock speed
• 8 Cores
• Support for 64GB UDIMM non-ECC/ECC
• Power consumption
• Form factor
• two Intel i210 Gbit NICs (HCL supported
• IPMI (Remote KVM)
• expandable (HDD)
• VT-x

Disadvantage

• Price
• no Hyper Threading
• no VT-d
• SATA Controller C2000 (not on HCL, but should work)
• SATA Controller Marvell SE9172/9230 (not on HCL, no ESXi driver)

Low Power Design 5

Component	Type	Cost
Server	Whitebox
Mainboard	Supermicro A1SAi-2750F	389 EUR
CPU	incl.
CPU Cooler	incl.
Memory	Kingston ValueRAM KVR16LSE11/8 (4x 89 EUR)	356 EUR
SSD	Samsung SSD 840 EVO 250GB, 2.5", SATA 6Gb/s	111 EUR
HDD	Western Digital WD Black 1TB WD1003FZEX	72 EUR
Power Supply	Enermax Revolution X't 530W ATX 2.4	78 EUR
Case	Fractal Design Node 304	70 EUR
Misc.	-
Price per Server		1076 EUR (Octa) 968 EUR (Quad)

Advantage

• Clock speed
• 8 Cores
• Support for 64GB UDIMM
• Power consumption
• Form factor
• four Intel i354 Gbit NICs (HCL supported, should work)
• IPMI (Remote KVM)
• expandable (HDD)
• VT-x

Disadvantage

• Price
• no Hyper Threading
• no VT-d
• SATA Controller C2000 (not on HCL, but should work)
• Intel i354 Gbit NICs need customized ESXi drivers

For design 4 and 5 there are only a few difference like price, SO-DIMM/DIMM and 4 Gbit NIC with the SuperMicro board. As you can see design 1-3 ranges from 600-650€ whereas design 4-5 ranges from 900-1100€ which is not so cheap for a low power build. But why did I choose the Avoton for a build? I saw this boards for my first time at a presentation from a TAM colleague, Marco Baaijen, who created a really cool datacenter-in-a-box (incl. VSAN) with the AsRock boards. When using 3 boards you need only one power supply because the boards don’t need much power. Here are some pictures of this build:

This whole build is only using approx. 130 Watts. The only drawback is that you can only use UDIMMs and 16GB UDIMMs ECC are not very common. These boards will definitely make a decent VSAN Management Cluster.
Update:
Thanks to my colleague Marco Baaijen who pointed me to the fact that you can also use non-ECC memory with the AsRock Avoton mainboard which makes the overall price approx. 80-100€ cheaper. I also updated the low power design 4 with Corsair non-ECC DIMMs.
I’m note quite sure if this will also count for the SuperMicro board. I try to get this info too. So stay tuned.
If you have any question drop me a comment or send me a tweet. Feedback is very appreciable.