{"id":3996,"date":"2018-09-17T17:01:50","date_gmt":"2018-09-17T22:01:50","guid":{"rendered":"http:\/\/www.jasemccarty.com\/blog\/?p=3996"},"modified":"2020-02-16T17:02:28","modified_gmt":"2020-02-16T23:02:28","slug":"vsan-2n-witness-consolidation","status":"publish","type":"post","link":"https:\/\/www.jasemccarty.com\/blog\/vsan-2n-witness-consolidation\/","title":{"rendered":"Understanding vSAN Witness Consolidation &#038; 2 Node vSAN"},"content":{"rendered":"<p>The vSAN Witness Appliance was introduced as a free alternative to using a physical ESXi host as a vSAN Witness Host. This appliance is only used for housing vSAN Object Witness Components, and is not allowed to run any virtual machine workloads.<\/p>\n<p>When deploying the vSAN Witness Appliance, there are 3 potential deployment options: Tiny, Normal, &amp; Large. These deployment profiles all have 2 vCPUs, 1 vmdk for the ESXi installation, 1 vmdk for the vSAN cache device, and at least 1 vmdk for the vSAN capacity.<\/p>\n<h3>Typical Witness Appliance Requirements for 2 Node vSAN<\/h3>\n<p>When deploying 2 Node vSAN, a vSAN Witness Appliance is typically deployed with the &#8220;Tiny&#8221; profile.<\/p>\n<p>A closer look at the <strong>Tiny profile<\/strong> shows the following requirements:<\/p>\n<ul>\n<li>vCPUs: 2<\/li>\n<li>vRAM: 8GB<\/li>\n<li>ESXi installation vmdk: 12GB (as of 6.5, previous were 8GB)<\/li>\n<li>vSAN Cache vmdk: 10GB (this is common across profiles)\n<ul>\n<li>Does not ever need to be larger than 10GB<\/li>\n<li>Does not have to be backed by a flash device, but must be tagged as a flash device<\/li>\n<\/ul>\n<\/li>\n<li>vSAN Capacity vmdk: 15GB\n<ul>\n<li>Can be tagged as flash, but not required<\/li>\n<\/ul>\n<\/li>\n<li>Management VMkernel Interface (vmk0)\n<ul>\n<li>Must be reachable by vCenter for Management<\/li>\n<li>Can be tagged for vSAN Traffic if the WitnessPg is not.<\/li>\n<\/ul>\n<\/li>\n<li>WitnessPg VMkernel Interface (vmk1)\n<ul>\n<li>Cannot be on the same network at the Management Network (<a href=\"https:\/\/kb.vmware.com\/kb\/2010877\" target=\"_blank\" rel=\"noopener noreferrer\">KB 2010877<\/a>)<\/li>\n<li>If vSAN Traffic is tagged, must be able to communicate with vSAN Data Nodes using vSAN VMkernel Interface or alternate VMkernel Interface when using Witness Traffic Separation<\/li>\n<li>Not required if vSAN Traffic is tagged on the Management VMkernel Interface<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>Also, the vSAN Witness Appliance needs to run on a *supported vSphere 5.5 or higher environment.<br \/>\n<em>*vSphere 5.5 End of General Support is September 19, 2018.<\/em><\/p>\n<h3>What are the requirements for multiple Witness Appliances?<\/h3>\n<p>When deploying multiple vSAN Witness Appliances, what are the requirements? This is common question that comes up when sizing for multiple 2 Node vSAN clusters.<\/p>\n<p>Recent testing has been done specifically on the vSAN Witness Appliance when used in conjunction with 2 Node vSAN Clusters, and we have some updated underlying hardware guidance.<\/p>\n<h3>Supported Consolidation Ratios when using the vSAN Witness Appliance &amp; 2 Node vSAN<\/h3>\n<p>Officially supported consolidation ratios for the vSAN Witness Appliance when used with 2 Node vSAN Clusters is as follows:<\/p>\n<ul>\n<li>CPU Consolidation &#8211; A vCPU to pCPU consolidation of 8:1 is supported<\/li>\n<li>Memory Consolidation\/Overcommitment &#8211; A vRAM to pRAM consolidation of 8:1 is supported<\/li>\n<li>Capacity Utilization &#8211; Capacity utilization remains to be 1:1, but additional considerations should to be taken into account\n<ul>\n<li>vSAN Witness Components are 16MB each<\/li>\n<li>A fully utilized &#8220;Tiny&#8221; vSAN Witness Appliance&#8217;s capacity disk can contain a maximum of 750 components<\/li>\n<li>Capacity utilization is going to be entirely dependent on the number of vSAN Witness Components.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>The following illustration shows a single vSphere Host with multiple workloads (DNS, File Services, vCenter, etc) as well as 8 vSAN Witness Appliances.<\/p>\n<p><a href=\"http:\/\/blogs.vmware.com\/virtualblocks\/files\/2018\/09\/VSWA-CONSOLIDATION.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-13552 size-large\" src=\"http:\/\/blogs.vmware.com\/virtualblocks\/files\/2018\/09\/VSWA-CONSOLIDATION-1024x817.png\" alt=\"Witness Consolidation\" width=\"580\" height=\"463\" srcset=\"https:\/\/blogs.vmware.com\/virtualblocks\/files\/2018\/09\/VSWA-CONSOLIDATION-1024x817.png 1024w, https:\/\/blogs.vmware.com\/virtualblocks\/files\/2018\/09\/VSWA-CONSOLIDATION-213x170.png 213w, https:\/\/blogs.vmware.com\/virtualblocks\/files\/2018\/09\/VSWA-CONSOLIDATION-768x613.png 768w, https:\/\/blogs.vmware.com\/virtualblocks\/files\/2018\/09\/VSWA-CONSOLIDATION-238x190.png 238w, https:\/\/blogs.vmware.com\/virtualblocks\/files\/2018\/09\/VSWA-CONSOLIDATION.png 1505w\" sizes=\"auto, (max-width: 580px) 100vw, 580px\" \/><\/a><\/p>\n<p>While the host has 16GB of physical RAM, the 12 VMs\/Appliances have been allocated 82GB of RAM. Also consider that vSphere itself requires some of the physical RAM.<\/p>\n<h3>How many resources does a vSAN Witness Appliance use?<\/h3>\n<p>Consider that the vSAN Witness Appliance is essentially an ESXi installation inside a VM. ESXi itself with vSAN does require a minimal amount of RAM.<\/p>\n<p>Using the vSAN Memory Sizing KB (<a href=\"https:\/\/kb.vmware.com\/kb\/2113954\" target=\"_blank\" rel=\"noopener noreferrer\">https:\/\/kb.vmware.com\/kb\/2113954<\/a>), it can be seen that a &#8220;Tiny&#8221; vSAN Witness Appliance requires just over 6.2GB of RAM:<br \/>\n5426 MB + (1 * (636 MB + (8MB * 10))) + (1 * 70 MB) = 6212MB<\/p>\n<p>The &#8220;Tiny&#8221; vSAN Witness Appliance is deployed with 8GB, which is sufficient for normal operation for vSAN 2 Node Clusters with up to 750 components.<\/p>\n<p>The vSAN Witness provides quorum for vSAN 2 Node Clusters but has a small amount of utilization that is not directly associated with data reads or writes to vSAN objects.<\/p>\n<p>Careful inspection of a vSAN Witness Appliance reveals that only a fraction of the provisioned resources are used during normal operations. Because the vSAN Witness only deals with vSAN object metadata, much of it&#8217;s resources remain largely idle. It isn&#8217;t until a failover occurs that the vSAN Witness requires additional resources.<\/p>\n<h3>Real world example of sizing a dedicated cluster for vSAN Witness Appliances<\/h3>\n<p><strong>Pre-Consolidation Guidance<\/strong><br \/>\nUsing a 1:1 ratio for CPU, RAM, &amp; Capacity for &#8220;Tiny&#8221; vSAN Witness Appliances, an example customer with 200 locations would require:<\/p>\n<ul>\n<li>400 vCPUs (200 x 2 vCPU)<\/li>\n<li>1.6TB of RAM (200 x 8GB RAM)<\/li>\n<li>9TB Capacity (200 x 45GB)* if fully utilized<br \/>\n*45GB = 12GB (OS), 10GB (cache), 15GB (capacity), &amp; 8GB (vSwap)<\/li>\n<li>~24TB RAW storage<\/li>\n<\/ul>\n<p>That could equate to a cluster with:<\/p>\n<ul>\n<li>10 Hosts\n<ul>\n<li>Dual 20 core CPUs<\/li>\n<li>192GB RAM<br \/>\n(Standard host configuration supporting 160GB for Appliances + ESXi\/vSAN Memory requirements)<\/li>\n<li>2.4TB RAW Capacity<\/li>\n<\/ul>\n<\/li>\n<li>Totals\n<ul>\n<li>400 cores<\/li>\n<li>1.92TB RAM<\/li>\n<li>24TB RAW Capacity (Mirroring + Slack)<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>As can be seen, these hosts are heavy on compute\/memory and light on capacity. Remember that there is no requirement for the vSAN Witness Appliance to be backed by flash devices. With the cost of NL-SAS devices being relatively cheap, it is much more cost effective to have fewer hosts with more capacity.<\/p>\n<p><strong>Using the Supported Consolidation Guidance<\/strong><br \/>\nUsing the newly supported guidance of 8:1 for compute\/memory, a smaller configuration would be appropriate.<\/p>\n<ul>\n<li>400 vCPUs\/8 = 50 vCPUs<\/li>\n<li>1.6TB RAM\/8 = 200GB RAM<\/li>\n<li>2.4TB Capacity (Mirroring+Slack)<\/li>\n<li>24TB RAW storage<\/li>\n<\/ul>\n<p>This equates to a much smaller cluster:<\/p>\n<ul>\n<li>3 Hosts\n<ul>\n<li>Dual 10 core CPUs<\/li>\n<li>96GB of RAM<br \/>\n(Standard host configuration supporting 67GB for Appliances + ESXi\/vSAN Memory requirements)<\/li>\n<li>8TB RAW Capacity<\/li>\n<\/ul>\n<\/li>\n<li>Totals\n<ul>\n<li>60 cores<\/li>\n<li>288GB RAM<\/li>\n<li>24TB RAW Capacity (Mirroring + Slack)<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>The above example, while utilizing the same amount of capacity, results in 1\/3 fewer hosts and roughly 1\/6th the amount of RAM when using the updated vSAN Witness Appliance consolidation guidance when used with 2 Node vSAN.<\/p>\n<p>This directly translates to less infrastructure, at a lower overall cost, required to run multiple vSAN Witness Appliances.<\/p>\n<h3><strong>Summary<\/strong><\/h3>\n<p>The updated vSAN Witness Appliance support consolidation ratio of 8:1 for CPU and Memory provides better guidance for customers using 2 Node vSAN Clusters. This testing and validation was performed only for 2 Node vSAN Clusters using the &#8220;Tiny&#8221; profile. The 8:1 consolidation ratio is has not been tested or recommended for &#8220;Normal&#8221; or &#8220;Large&#8221; vSAN Witness Appliances when used with 2 Node or Stretched vSAN Clusters.<\/p>\n<p>Look for more in-depth content around vSAN Witness Sizing with 2 Node vSAN in the coming weeks on <a href=\"https:\/\/storagehub.vmware.com\/\" target=\"_blank\" rel=\"noopener noreferrer\">StorageHub<\/a>.<\/p>\n<p>This was originally posted on the VMware Virtual Blocks site: https:\/\/blogs.vmware.com\/virtualblocks\/2018\/09\/17\/vsan-2n-witness-consolidation\/<\/p>\n","protected":false},"excerpt":{"rendered":"<p>The vSAN Witness Appliance was introduced as a free alternative to using a physical ESXi host as a vSAN Witness Host. This appliance is only &hellip; <\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-3996","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/www.jasemccarty.com\/blog\/wp-json\/wp\/v2\/posts\/3996","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.jasemccarty.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.jasemccarty.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.jasemccarty.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.jasemccarty.com\/blog\/wp-json\/wp\/v2\/comments?post=3996"}],"version-history":[{"count":2,"href":"https:\/\/www.jasemccarty.com\/blog\/wp-json\/wp\/v2\/posts\/3996\/revisions"}],"predecessor-version":[{"id":3998,"href":"https:\/\/www.jasemccarty.com\/blog\/wp-json\/wp\/v2\/posts\/3996\/revisions\/3998"}],"wp:attachment":[{"href":"https:\/\/www.jasemccarty.com\/blog\/wp-json\/wp\/v2\/media?parent=3996"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.jasemccarty.com\/blog\/wp-json\/wp\/v2\/categories?post=3996"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.jasemccarty.com\/blog\/wp-json\/wp\/v2\/tags?post=3996"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}