Proxmox Server Part I: The Hardware

Before I begin, I would like to extend my warmest holiday wishes to all. This Christmas was very special for me and my family as this was my daughter's first real Christmas (she was two months old this time last year). :-)

Since 2013, I have been primarily using ESXi as my hypervisor of choice for my lab and home network needs (Domain Controller, DNS, DHCP, RADIUS, etc.) Back in September, my ESXi server began randomly crashing, seemingly with no logs indicating what may be occurring. While troubleshooting this issue, I recognized the need for some sort of backup. Rather than going the traditional route of adding a secondary ESXi server, I decided to purchase a refurbished server and give Proxmox a try. 

The goal of this build was to build a cost effective backup and avoid incurring the costs of building a brand new server. Build preferences included:

1. 2x Xeon CPUs 
2. 64GB RAM (Minimum)
3. 4x Hot Swap Bays (Minimum)
4. 2x Integrated Network Interfaces (Minimum)
5. ESXi supported Hardware RAID controller (In case Proxmox is abandoned)
6. 2 Units / Rack (Maximum)
7. Out-of-Band Management

I was able to find an excellent deal on a Dell R710 server on eBay that met and exceeded all of my specifications (see below).

1. Dual Xeon X5570 2.93GHz
2. 64 GB ECC DD3 
3. 8x 2.5'' SATA Hot swap Bays
4. 2x 500GB 7.2K Dell Enterprise HDDs
5. 4x Integrated NetXtreme II Gigabit Network Interfaces
6. PERC H700 RAID Controller
7. 2U 
8. iDRAC6 Support (Out-of-Band)
9. Dual PSUs 
10. Rail Kit Included
11. 90 Day Warranty

Server Price:  $285.00

Shipping:        $47.14

The only items that i needed to purchase separately were the additional SSDs (my preference) for boot, the actual iDRAC6 interface (not included with server), and additional drive caddies.

1. 2x CISNO 2.5" SAS HDD Tray/Caddy for Dell R710  - $25.80
2. 5pk Dell Expansion Slot Cover - $5.25
3. Dell K869T iDRAC6 Interface -  $9.99
4. Assurant Protection Plan - $11.99
5. 2x Black Diamond BDSSDS128G-MU 128GB SSD - $62.40 

Subtotal: $115.43

Shipping: $34.03

GRAND TOTAL: $481.60

Though refurbished, that's a lot of horsepower for less than $500. This is especially pleasing considering my 2013 ESXi build, consisting of all new hardware, was more than $2500. See the hardware below. 

Dell R710 - 2U/Dual Xeon X5570/64GB DDR3/8 Hot Swap Bays

SSDs installed in Bays 0 and 1, HDDs in 2 and 3

Dual Xeon X5570 - 16/19 DIMM slots populated 

Quad-Port Integrated NIC / Dual-Port Add-In NIC (Spare) / OOB Interface

Tucked tightly between my ESXi server and lab-only Cisco Catalyst 3750

 The only real issue I ran into was the SSDs that i purchased (mentioned above), though listed as enterprise grade on NewEgg, were not welcomed by the Perc H700 RAID controller. The controller continued throwing errors about drive failures on slots 0 and 1 (the slots the SSDs populated). The drives worked perfectly and passed all tests and were even successfully configured for RAID-1 by the controller but errors continued to fill the server's logs and the indicator lights continued to show amber LEDs. After doing some research, I found that Dell (and likely other OEMs) RAID controllers do not play nice with non-Dell, non-Enterprise storage devices. One Dell forum user mentioned that enterprise grade Intel SSDs can be used without throwing errors on the controllers. 

Luckily, I had two enterprise 80GB Intel S3510 SSDs  being used as boot drives (mirrored) in my FreeNAS build. As many FreeNAS users can agree, the Intels might have been overkill for the FreeNAS box because FreeNAS is generally not extremely picky about the boot drives that it uses. As a matter of fact, many FreeNAS builders recommend using USB thumb drives as the boot disk. So I simply removed the Black DIamond SSDs from the server, connected them to the FreeNAS box, mirrored the boot from the Intels, and verified functionality. Afterwards, I installed the Intel S3510 SSDs in the server and, voila; all disk errors were cleared. 

As far as storage is concerned, I ran my typical hypervisor setup of four drives; two SSDs housing the operating system in RAID-1 and two local storage (data) HDDs in RAID-1. 

iDRAC6 worked well with minimum configuration as with most out-of-band management interfaces. Typical features such as remote power control, console access, virtual media mounting, and logging are all available from the iDRAC6 interface. Of course i managed all RAID configuration and OS deployment from the iDRAC6 embedded web server. 

Overall, the Dell R710 is an excellent piece of hardware to lab with for virtualization practice. In subsequent posts, I will detail my experiences with Proxmox and my overhaul of my virtual infrastructure. I hope everyone has a happy and safe New Year and I look forward to blogging in 2017.