{"id":1516,"date":"2011-07-19T16:43:41","date_gmt":"2011-07-19T21:43:41","guid":{"rendered":"http:\/\/www.jasemccarty.com\/blog\/?p=1516"},"modified":"2012-08-20T07:38:56","modified_gmt":"2012-08-20T13:38:56","slug":"home-lab-hosts-just-in-time-for-vsphere-5","status":"publish","type":"post","link":"https:\/\/www.jasemccarty.com\/blog\/home-lab-hosts-just-in-time-for-vsphere-5\/","title":{"rendered":"Home Lab Hosts &#8211; Just in time for vSphere 5"},"content":{"rendered":"<p><a href=\"http:\/\/www.jasemccarty.com\/blog\/wp-content\/uploads\/2011\/07\/labparts.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"alignright size-thumbnail wp-image-1549\" title=\"labparts\" src=\"http:\/\/www.jasemccarty.com\/blog\/wp-content\/uploads\/2011\/07\/labparts-150x150.jpg\" alt=\"\" width=\"150\" height=\"150\" \/><\/a>I&#8217;ve been looking at what type of hardware for a home lab for quite a while.<\/p>\n<p>Several people have setup some pretty decent\/usable home labs&#8230; Here are a few to point out:<\/p>\n<ul>\n<li>Kendrick Coleman &#8211; <strong><a title=\"vSphere Home Lab - &quot;The Green Machines&quot;\" href=\"http:\/\/www.kendrickcoleman.com\/index.php?\/Tech-Blog\/vmware-vsphere-home-lab-qthe-green-machinesq.html\" target=\"_blank\">vSphere Home Lab &#8211; &#8220;The Green Machines&#8221;<\/a><\/strong><\/li>\n<li>Jeramiah Dooley &#8211; <strong><a title=\"VMware vSphere Home Lab: Keeping Up with the Joneses\" href=\"http:\/\/vmforsp.typepad.com\/vm-for-service-providers\/2011\/04\/vmware-vsphere-home-lab-keeping-up-with-the-joneses.html\" target=\"_blank\">VMware vSphere Home Lab: Keeping up with the Joneses<\/a><br \/>\n<\/strong><\/li>\n<li>Tommy Trogden &#8211; <strong><a title=\"The VMware Home Lab\" href=\"http:\/\/vtexan.com\/2010\/12\/07\/the-vmware-home-lab\/\" target=\"_blank\">The VMware Home Lab<\/a><\/strong><\/li>\n<li>Duncan Epping &#8211; <a title=\"My Homelab\" href=\"http:\/\/www.yellow-bricks.com\/2010\/01\/19\/my-homelab\/\" target=\"_blank\"><strong>My Homelab<\/strong><\/a><\/li>\n<li>Simon Seagrave &#8211; <strong><a title=\"VMware ESX(i) Home Lab; Why, What and How? Considerations when building your own home lab.\" href=\"http:\/\/www.techhead.co.uk\/vmware-esxi-home-lab-why-what-and-how-considerations-when-building-your-own-home-lab\" target=\"_blank\">VMware ESX(i) Home Lab; Why, What and How? Considerations when building your own home lab.<\/a><\/strong><\/li>\n<\/ul>\n<p>As I mentioned in a <strong><a title=\"vSphere Home Lab \u2013 What type of hosts?\" href=\"http:\/\/www.jasemccarty.com\/blog\/?p=1452\">previous post<\/a><\/strong>, picking hosts can be a daunting task.\u00a0 The process of picking hosts includes taking a few things into account.\u00a0 I looked at desktop configurations that would support vSphere and server configurations that vSphere supports.<\/p>\n<p><!--more-->Two of the things that I found to be very common in the comparison&#8230; (Also keep in mind that Micro ATX is the form factor I chose for space)<\/p>\n<ol>\n<li>If I configured a desktop board that &#8220;<strong><a title=\"ESX \/ ESXi 4.0 Whitebox HCL\" href=\"http:\/\/www.vm-help.com\/esx40i\/esx40_whitebox_HCL.php\" target=\"_blank\">has been known<\/a><\/strong>&#8221; to work,\n<ul>\n<li>I could get a motherboard what was reasonably priced.<\/li>\n<li>RAM that is compatible is very reasonable priced.<\/li>\n<li>There is only a single onboard nic.<\/li>\n<li>The onboard nic was more than likely not supported, and additional nics are required.<\/li>\n<li>The available expansion slots ranged from 1-2 PCI-X slots, 1-4 PCI express slots.<\/li>\n<li>AMD gave better options for Hex Core CPUs.<\/li>\n<\/ul>\n<\/li>\n<li>If I configured a server board that &#8220;<strong><a title=\"VMware Compatibility Guide\" href=\"http:\/\/www.vmware.com\/go\/hcl\" target=\"_blank\">is verified<\/a><\/strong>&#8221; to work,\n<ul>\n<li>More often than not, ECC RAM was required.\u00a0 And as I mentioned in my previous post, ECC RAM is not cheap.<\/li>\n<li>One or more onboard nics are available, and often times, are supported by ESXi without requiring additional drivers.<\/li>\n<li>The motherboard is more expensive than a desktop motherboard.<\/li>\n<\/ul>\n<\/li>\n<\/ol>\n<p>What to I sacrifice to get the best of both worlds?\u00a0 RAM at a better price, while having to purchase more nics? How many cores do I really need&#8230;<\/p>\n<p><strong>CPU<\/strong><br \/>\nDo I need more cores like Jeramiah chose? Did he give anything up by choosing AMD over Intel?\u00a0 Is VMDirectPath supported on the AMD desktop based boards? Not sure.<\/p>\n<p>Also, because many SandyBridge server boards support the Intel Pentium G620(T)\/G840(T)\/G850 and Core i3-2100(T)\/2105\/2120, I had some options.\u00a0 The Pentium GXXX processors have dual cores, but no Hyperthreading, while the Core i3-2100 series procs have dual cores and Hyperthreading.<\/p>\n<p>Also, because I chose to buy 2 hosts, and not 5 like Jeramiah, I wanted to be sure I could run nested ESXi boxes and have enough cores for the nested ESXi hosts as well.\u00a0 So I had to choose hosts that had at least 4 cores.\u00a0 That knocks the Pentium G6xx\/8xx processors out of the running.\u00a0 The Core i3-2100 series procs had the right number of logical cores, but not the number of physical cores.\u00a0 I almost picked the Core i3-2100T, a low power version of the Core i3-2100.\u00a0 But I&#8217;m pretty certain that I&#8217;m going to need at least 4 cores running nested ESXi hosts from time to time.<\/p>\n<p>So I decided to go with the Xeon E3-1200 series processors.\u00a0 The E3-1220 offers 4 cores, but unfortunately no Hyperthreading.\u00a0 I picked the Xeon E3-1230 processor as the processor of choice, because it has 4 cores with Hyperthreading.\u00a0 Another deciding factor, was that it supports VMDirectPath.\u00a0 This is another featureset that the Core i3-2100 series doesn&#8217;t support.\u00a0 In the Core i3\/5\/7 series processors, VMDirectPath is not supported (along the line) until the Core i5-2400.\u00a0 The Core i5 processors unfortunately aren&#8217;t supported on the server boards I looked at.\u00a0 The processors were $235 each.<\/p>\n<p><strong>Case+Power Supply<\/strong><br \/>\nI wanted a small footprint for the hosts, so I made sure the motherboard I picked was a Micro ATX form factor.\u00a0 I picked a Micro ATX slim case for the hosts, in the event I ever decide to keep them at the house.\u00a0 Also, 1U server cases are pretty much about double the price of the slim case, and I&#8217;m trying to keep things reasonable.\u00a0 Right now I&#8217;m planning on putting them in a datacenter I have remote access to.\u00a0 I&#8217;ll cover that a little more when I talk about my boards. The cases ended up being about $50 each. One thing to keep in mind, is that because I&#8217;m using a server board, the power supply should be able to support it.<\/p>\n<p><strong>RAM<\/strong><br \/>\nPicking RAM was a little aggravating.\u00a0 What kind of RAM does my board support?\u00a0 I mean REALLY support?\u00a0 After picking a board, I went to the vendor&#8217;s website and looked at supported RAM.\u00a0 Whoa&#8230; $120 for 8GB of ECC RAM (2x4GB).\u00a0 That&#8217;s kind of high in comparison to desktop DDR3 memory, which averages around $60-$90 for 8GB (2x4GB) of RAM.<\/p>\n<p>The board I selected will take up to 32GB of RAM, but unfortunately no 16GB (2x8GB) ECC memory is listed as supported.\u00a0 To be honest, I found that to be the case across most of the boards I looked at.\u00a0 Only the boards that were out of my price range supported 8GB memory sticks.\u00a0 And 16GB (2x8GB) 0f ECC memory seemed to hover around $180-$350.\u00a0 Maybe later, I&#8217;ll upgrade to 32GB when more memory SKUs are supported on my boards.<\/p>\n<p>I did find some supported memory for $48 for 4GB of ECC RAM (Kingston KVR1333D3E9S\/4G).\u00a0 That comes out to $96 per 8GB.\u00a0 Much more attractive than $120, especially when buying 32GB across 2 hosts.\u00a0 That&#8217;s a savings of $24&#215;4 or $96.\u00a0 That puts it in line with desktop DDR3 memory.\u00a0 With memory not significantly more expensive, I was alright with going the ECC route.\u00a0\u00a0 Total memory cost was about $384.<\/p>\n<p><strong>Boards<\/strong><br \/>\nNow on to the boards.\u00a0 I wanted a board that was on the VMware Compatibility Guide, but wasn&#8217;t too expensive.\u00a0 I really wanted to stay with a SandyBridge board\/processor combination.\u00a0 That limitation wasn&#8217;t really helpful.\u00a0 If you look at the HCL, there aren&#8217;t many SandyBridge systems listed.\u00a0 In fact, only 2 motherboards that support the Xeon E3-1200 series are on the HCL.\u00a0 They are the Intel S1200BTL and Intel S1200BTS.<\/p>\n<p>What is the difference between these 2 systems? The BTL has a C204 chipset and the BTS has a C202 chipset. What&#8217;s the difference between the two? Suffice to say that the C204 is a little better grade than the C202, with the C204 supporting SATA III, while the C202 doesn&#8217;t.\u00a0 Remember that I wanted a Micro ATX footprint, and the BTS is the only one that is Micro ATX. Another thing I would like to add, is the ability to use IPMI and iKVM capabilities.\u00a0 The S1200BTL supports iKVM with an add-on card.\u00a0 Again, the BTL is not a Micro ATX form factor.\u00a0 A little bit of a debacle here.\u00a0 The features I want, are on the board I don&#8217;t want (size-wise).<\/p>\n<p>Because there were a couple C202 and C204 based systems, along with the Intel S1200BT boards, on the VMware HCL, I figured that other C20x boards would also work.\u00a0 It has been discussed several times on the VMTN forums how the Compatibility Guide is validated, and just because a system\/board isn&#8217;t on the list, it doesn&#8217;t mean it won&#8217;t work.\u00a0 It just means it hasn&#8217;t been tested.<\/p>\n<p>I started looking at some of the C20x boards available.\u00a0 I looked at ASUS, SuperMicro, and TYAN.\u00a0 Several of these had multiple nics onboard, some with only one being recognized by ESXi 4.1, some with both.\u00a0 I tried to narrow my results to those where both nics were supported.\u00a0 As I mentioned earlier, I wanted a board that supported iKVM, given that I&#8217;ll be putting these in a remote datacenter.\u00a0 Some boards had native support, while others required a $50+ add-on card.\u00a0 I also wanted to keep my cost down as well.<\/p>\n<p>I ended up going with a TYAN S5510GM3NR.\u00a0 I picked this board, because all 3 onboard nics are recognized (as mentioned in the product reviews on NewEgg), and one could be also used for <strong><a title=\"Tyan IPMI 2.0 Remote Management WebGUI Tour\" href=\"http:\/\/www.servethehome.com\/tyan-ipmi-20-remote-management-webgui-tour\/\" target=\"_blank\">iKVM<\/a><\/strong>, without an add-on card.\u00a0 I also liked the fact that the S5510GM3NR also has a C204 chipset, supporting SATA III, if I decide to put a SATA III drive in it for local VM storage.\u00a0 The board retailed for about $180.<\/p>\n<p>Because the Core i3\/i5\/i7 SandyBridge processors include onboard graphics, I had to take that into consideration. This is because not all Xeon E3-1200 processors have onboard graphics support.\u00a0 The Tyan board includes onboard graphics, mitigating the need for a video card.<\/p>\n<p>One caveat to this board, is that it is an EPS12V and not an ATX12V board.\u00a0 Standard power supplies for desktop systems (like the one I selected) typically are ATX12V and not EPS12V.\u00a0 I&#8217;m not going to go into the details of the differences, but suffice to say, I initially thought I was going to have to select a different case, to accommodate an EPS12V power supply.\u00a0 ATX12V boards have an additional 4 pin power connector (other than the standard 20+4), while EPS12V boards have an additional 8 pin power connector.\u00a0 A 4 to 8 pin converter did the trick for me.<\/p>\n<p>Also, with the board being unsupported, I didn&#8217;t expect the Manufacturer\/Model to be listed as &#8220;empty&#8221; in the ESXi interface.\u00a0 Not really a big deal, given that all the components display properly in the Hardware Status tab.<\/p>\n<p><strong>NICs<br \/>\n<\/strong>The boards have 3 onboard nics that are supported, but I wanted to add additional nics to be able to separate my management\/VM\/vMotion network from my storage network.\u00a0 Again, I wanted to keep the build price as cost effective as possible, so I picked 2 Intel EXPI9301CTBLK nics that others have seen success with.\u00a0 I would have liked a single dual gigabit nic like Jeramiah chose, but I really didn&#8217;t want to pay $150 a nic, where the EXPI9301CTBLK was only $25, that totals out at about $100 for the additional nics.<\/p>\n<p>Another thing I didn&#8217;t realize, is that the iKVM nic, is also used as an ESXi nic.\u00a0 So a single nic has a dual use for ESXi and for iKVM.\u00a0<strong> <\/strong>All 5 nics are reported as Intel 82574L.<\/p>\n<p><strong>SSD<\/strong><br \/>\nI was working in Atlanta a couple weeks ago, and went by Micro Center.\u00a0 I noticed they had a 60GB SSD SATA II drive for $100.\u00a0 The drive is a rebranded A-DATA S599 drive with some good reviews.\u00a0 I wanted to get an SSD drive for each host, so I could leverage vSphere 5&#8217;s host cache feature.\u00a0 The drives retailed for $99.99 each ($89.99 today), plus $3.50 for some 2.5 to 3.5 rail kits.<\/p>\n<p><strong>The Cost<\/strong><br \/>\nBelow is the cost of the 2 systems.<\/p>\n<table width=\"100%\" border=\"1\">\n<tbody>\n<tr>\n<td>Item<\/td>\n<td>Model<\/td>\n<td>Cost<\/td>\n<td>Quantity<\/td>\n<td>Total<\/td>\n<\/tr>\n<tr>\n<td>Motherboard<\/td>\n<td><a title=\"Tyan S5510GM3NR\" href=\"http:\/\/www.newegg.com\/Product\/Product.aspx?Item=N82E16813151243\" target=\"_blank\">Tyan S5510GM3NR<\/a><\/td>\n<td>$189.99<\/td>\n<td>2<\/td>\n<td>$379.98<\/td>\n<\/tr>\n<tr>\n<td>CPU<\/td>\n<td><a title=\"Intel Xeon E3-1230 Sandy Bridge 3.2GHz 4 x 256KB L2 Cache 8MB L3 Cache LGA 1155 80W Quad-Core Server Processor BX80623E31230\" href=\"http:\/\/www.newegg.com\/Product\/Product.aspx?Item=N82E16819115083\" target=\"_blank\">Intel E3-1230<\/a><\/td>\n<td>$234.99<\/td>\n<td>2<\/td>\n<td>$469.98<\/td>\n<\/tr>\n<tr>\n<td>Memory<\/td>\n<td><a title=\"Kingston KVR1333D3E9S\/4G\" href=\"http:\/\/www.newegg.com\/Product\/Product.aspx?Item=N82E16820139077\" target=\"_blank\">Kingston 4GB Unbuffered ECC RAM<\/a><\/td>\n<td>$47.99<\/td>\n<td>8<\/td>\n<td>$383.92<\/td>\n<\/tr>\n<tr>\n<td>Nic<\/td>\n<td><a title=\"Intel EXPI9301CTBLK Network Adapter 10\/ 100\/ 1000Mbps PCI-Express 1 x RJ45\" href=\"http:\/\/www.newegg.com\/Product\/Product.aspx?Item=N82E16833106033\" target=\"_blank\">Intel EXPI9301CTBLK Single Port Network Adapter<\/a><\/td>\n<td>$24.99<\/td>\n<td>4<\/td>\n<td>$99.96<\/td>\n<\/tr>\n<tr>\n<td>Case<\/td>\n<td><a title=\"Rosewill R379-M Black\/ Silver 0.8mm SGCC Steel Slim MicroATX Computer Case with ATX12V Flex 300W Power Supply\" href=\"http:\/\/www.newegg.com\/Product\/Product.aspx?Item=N82E16811147098\" target=\"_blank\">Rosewill R379-M Slim MicroATX Case w\/Power Supply<br \/>\n<\/a><\/td>\n<td>$49.99<\/td>\n<td>2<\/td>\n<td>$99.98<\/td>\n<\/tr>\n<tr>\n<td>SSD<\/td>\n<td><a title=\"64GB SATA II 3.0Gb\/s 2.5\" href=\"http:\/\/www.microcenter.com\/single_product_results.phtml?product_id=0351760\" target=\"_blank\">64GB Microcenter G2 SSD<\/a><\/td>\n<td>$99.99<\/td>\n<td>2<\/td>\n<td>$199.98<\/td>\n<\/tr>\n<tr>\n<td>Adapter<\/td>\n<td><a title=\"2.5\" href=\"http:\/\/www.microcenter.com\/single_product_results.phtml?product_id=0316761\" target=\"_blank\">2.5&#8243; Hard Drive Mounting Bracket<\/a><\/td>\n<td>$3.49<\/td>\n<td>2<\/td>\n<td>$6.98<\/td>\n<\/tr>\n<tr>\n<td>4 to 8 pin power adapter<\/td>\n<td><a title=\"2.5\" href=\"http:\/\/www.newegg.com\/Product\/Product.aspx?Item=N82E16812198024\" target=\"_blank\">Athena Power 8&#8243; Extension &amp; Conversion Four-In-One Cable<\/a><\/td>\n<td>$4.99<\/td>\n<td>2<\/td>\n<td>$9.98<\/td>\n<\/tr>\n<tr>\n<td colspan=\"3\"><\/td>\n<td>Total<\/td>\n<td>$1,650.76<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>$1,650.76.<\/p>\n<p>That&#8217;s not what I wanted to spend&#8230; I could have gotten away for something like $1,200 for both of the hosts had I gone with a desktop board, desktop DDR3 memory, not being able to use the onboard nic <strong>(Update<\/strong>: Kendrick Coleman blogged about the <strong><a title=\"Realtek 8111E Works With VMware vSphere 5\" href=\"http:\/\/kendrickcoleman.com\/index.php?\/Tech-Blog\/realtek-8111e-works-with-vmware-vsphere-5.html\" target=\"_blank\">RealTek 8111e working<\/a>)<\/strong>, no SSDs, and no iKVM.\u00a0 That would be about $600 per host.\u00a0 If I divide the cost by 2, each host costs $825.38.\u00a0 Given that the SSDs were about $100 each, had I not even added them (and used USB sticks instead) each host would have been about $725. I think having 5 supported nics, iKVM, a quad core (Hyperthreaded) processor, and local SSD for $825 each is a pretty good deal. Also, keep in mind I have 8 logical cores and VMDirectPath support.<\/p>\n<p>Now I&#8217;m off to find a decent switch that will accommodate both hosts and a dual port storage system.<\/p>\n<p>BTW I&#8217;m not picking on Jeramiah&#8217;s choices, they are just as valid.\u00a0 Thanks Jeramiah for being such a good sport. I&#8217;m looking forward to your Part 3 post.<\/p>\n<p>Update 07\/30\/11: I have a <a title=\"Home Lab Hosts \u2013 Could it have been leaner? ($$$)\" href=\"http:\/\/www.jasemccarty.com\/blog\/?p=1571\"><strong>newer post<\/strong><\/a> that details how I could have done it for a little less money.<\/p>\n<p>Update 08\/02\/12: I have an <strong><a title=\"vSphere 5 Lab Hosts \u2013 A Year Later\" href=\"http:\/\/www.jasemccarty.com\/blog\/?p=2263\">even newer post<\/a><\/strong> that compares 2011 cost vs. 2012 cost, including working 8GB UDIMMs.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>I&#8217;ve been looking at what type of hardware for a home lab for quite a while. Several people have setup some pretty decent\/usable home labs&#8230; &hellip; <\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[12],"tags":[53,129,105,112],"class_list":["post-1516","post","type-post","status-publish","format-standard","hentry","category-virtualization","tag-lab","tag-s5510","tag-vmware","tag-vsphere5"],"_links":{"self":[{"href":"https:\/\/www.jasemccarty.com\/blog\/wp-json\/wp\/v2\/posts\/1516","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.jasemccarty.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.jasemccarty.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.jasemccarty.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.jasemccarty.com\/blog\/wp-json\/wp\/v2\/comments?post=1516"}],"version-history":[{"count":4,"href":"https:\/\/www.jasemccarty.com\/blog\/wp-json\/wp\/v2\/posts\/1516\/revisions"}],"predecessor-version":[{"id":2323,"href":"https:\/\/www.jasemccarty.com\/blog\/wp-json\/wp\/v2\/posts\/1516\/revisions\/2323"}],"wp:attachment":[{"href":"https:\/\/www.jasemccarty.com\/blog\/wp-json\/wp\/v2\/media?parent=1516"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.jasemccarty.com\/blog\/wp-json\/wp\/v2\/categories?post=1516"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.jasemccarty.com\/blog\/wp-json\/wp\/v2\/tags?post=1516"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}