{"id":2014,"date":"2012-04-09T12:00:52","date_gmt":"2012-04-09T18:00:52","guid":{"rendered":"http:\/\/www.jasemccarty.com\/blog\/?p=2014"},"modified":"2012-12-30T14:50:51","modified_gmt":"2012-12-30T20:50:51","slug":"vmware-vsphere-and-emc-isilon-one-datastore-or-many","status":"publish","type":"post","link":"https:\/\/www.jasemccarty.com\/blog\/vmware-vsphere-and-emc-isilon-one-datastore-or-many\/","title":{"rendered":"VMware vSphere and EMC Isilon &#8211; One datastore or many?"},"content":{"rendered":"<p>As a vSpecialist, I am a member of a group that focuses on Big Data solutions. I primarily focus on Big Data storage, notably Isilon. I was introduced to Isilon early last year and really liked it.<\/p>\n<p>I got my first deep-dive with Isilon before EMC World 2011, where I was given the Isilon vLab, and asked to finish it up. I took the content that the Isilon team had provided, made some subtle changes, and polished it up a bit. The one thing that I found, was that Isilon was EASY to use. I was fortunate enough to use Isilon more throughout the year in 2011, as well as adding Isilon to the VMware Partner Labs at VMworld 2011.<\/p>\n<p>The first time I configured Isilon in the lab for use by vSphere (4.1 then), I didn&#8217;t really know what the best practices were.\u00a0 Fortunately, in January (2012), Isilon published a white paper on using Isilon with VMware vSphere 5. That paper can be found here:\u00a0<strong><a href=\"http:\/\/simple.isilon.com\/doc-viewer\/1739\/best-practices-guide-for-vmware-vsphere.pdf\" rel=\"nofollow\">http:\/\/simple.isilon.com\/doc-viewer\/1739\/best-practices-guide-for-vmware-vsphere.pdf<\/a><\/strong>. Information on configuring NFS starts at page 15.<\/p>\n<p>This post is a high level overview when using vSphere 5.0 and NFS storage from an Isilon cluster.<\/p>\n<p><strong><!--more-->Mounting cluster datastores on a single node<\/strong><br \/>\n<a href=\"http:\/\/www.jasemccarty.com\/blog\/wp-content\/uploads\/2012\/04\/ISI-POST01.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignright\" title=\"vSphere datastore on an Isilon cluster\" src=\"http:\/\/www.jasemccarty.com\/blog\/wp-content\/uploads\/2012\/04\/ISI-POST01-300x218.png\" alt=\"vSphere datastore on an Isilon cluster\" width=\"300\" height=\"218\" \/><\/a>As nodes are added to an Isilon cluster the shared resources can be accessed from any of the nodes.\u00a0 The default root (\/ifs) is available to all nodes in the cluster.\u00a0 From a basic perspective, a single node&#8217;s IP address could be used to access one or more NFS exports.<\/p>\n<p>In a situation where a node were to be taken offline for maintenance, or were to fail, datastores connected to a dedicated IP address would fail, unless SmartConnect Advanced was being used.<\/p>\n<p>Using SmartConnect Advanced in this manner, only accommodates for a node going offline, but does not handle any additional throughput.\u00a0 With all NFS datastores being mounted to a single node, throughput is limited to that with the specified node can provide.<\/p>\n<p><strong>SmartConnect for accessing NFS shares<\/strong><br \/>\nBefore going into the different methods of connecting to NFS shares for vSphere datastores, it is important to know how SmartConnect works.\u00a0 SmartConnect is the Isilon feature that handles connection availability and load balancing for Isilon clusters.<\/p>\n<p>I used to think that the coolest thing about Isilon, was that nodes can be added &#8220;on the fly&#8221;:<iframe loading=\"lazy\" src=\"http:\/\/www.youtube.com\/embed\/JnzJ7QQMgBk\" frameborder=\"0\" width=\"420\" height=\"315\"><\/iframe><\/p>\n<p>In the video, as nodes are added the amount of space is automatically expanded, but so are the number of front-end ports.<\/p>\n<p>With a traditional NAS target, as storage is expanded, the front-end ports are not.\u00a0 A typical NAS device\/array will have a finite number of ports, that can only be expanded upon, given the capabilities of the NAS controller\/head\/etc.\u00a0 With Isilon however, more nodes are added, more front-end ports are too. SmartConnect provides the ability to better manage the cluster availability and throughput across many nodes\/interfaces.\u00a0 I think SmartConnect is really the coolest part of the mix, once I got to know more about it.<\/p>\n<p>SmartConnect Basic handles Round Robin IP distribution as nodes are added to a cluster, while SmartConnect Advanced handles advanced IP distribution as nodes are added, removed, or are unavailable, to the cluster.<\/p>\n<p>In addition to managing IP addresses for a cluster, through DNS delegation, SmartConnect provides IP addresses for address resolution when connecting to a FQDN, rather than an IP address.<\/p>\n<p><strong>SmartConnect Basic<\/strong> has the following features:<\/p>\n<ul>\n<li>Static IP allocation to nodes<\/li>\n<li>Connection Policy Algorithms\n<ul>\n<li>Round Robin (which node is next)<\/li>\n<\/ul>\n<\/li>\n<li>No Rebalancing<\/li>\n<li>No IP Failover<\/li>\n<\/ul>\n<p><strong>SmartConnect Advanced<\/strong> has the following features:<\/p>\n<ul>\n<li>Dynamic IP allocation to nodes<\/li>\n<li>Connection Policy Algorithms\n<ul>\n<li>Round Robin (which node is next)<\/li>\n<li>Connection Count (which is next given connections each node has)<\/li>\n<li>Network Throughput (which is next given the amount of throughput each node has)<\/li>\n<li>CPU Usage (which is next given the amount of CPU usage each node has)<\/li>\n<\/ul>\n<\/li>\n<li>Automatic or Manual Rebalancing<\/li>\n<li>IP Failover Policy\n<ul>\n<li>Round Robin (which node is next)<\/li>\n<li>Connection Count (which is next given connections each node has)<\/li>\n<li>Network Throughput (which is next given the amount of throughput each node has)<\/li>\n<li>CPU Usage (which is next given the amount of CPU usage each node has)<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>SmartConnect Advanced, adds the ability to easily bring nodes offline for maintenance, as well as handle failure, while maintaining availability of IP based services such as NFS exports.<\/p>\n<p><strong>Mounting datastores on different nodes<\/strong><\/p>\n<p><a href=\"http:\/\/www.jasemccarty.com\/blog\/wp-content\/uploads\/2012\/04\/ISI-POST02.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignright size-medium wp-image-2046\" title=\"Multiple datastores on a single OneFS NFS export\" src=\"http:\/\/www.jasemccarty.com\/blog\/wp-content\/uploads\/2012\/04\/ISI-POST02-300x148.png\" alt=\"Multiple datastores on a single OneFS NFS export\" width=\"300\" height=\"148\" srcset=\"https:\/\/www.jasemccarty.com\/blog\/wp-content\/uploads\/2012\/04\/ISI-POST02-300x148.png 300w, https:\/\/www.jasemccarty.com\/blog\/wp-content\/uploads\/2012\/04\/ISI-POST02.png 537w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/a>One of the biggest benefits of Isilon, is the fact that a single namespace can scale to over 15PB.\u00a0 From a file system and management perspective, that&#8217;s pretty awesome, but from a connection perspective, a limited number of connections to a single node could easily be a bottleneck.<\/p>\n<p>Spreading the connections to each NFS export across nodes will increase the throughput in comparison to all connections on a single node.\u00a0 As can be seen in this graphic, there are 3 hosts connected to 1 Isilon cluster, with 3 datastores (NFS exports), on 3 different IP addresses. Each of the Isilon nodes has a <em>dynamic<\/em> IP address assigned to it.<\/p>\n<p>This configuration dedicates traffic for each NFS export to <em>a dedicated IP address<\/em>.\u00a0 This isn&#8217;t to be confused with a dedicated node.<\/p>\n<p><a href=\"http:\/\/www.jasemccarty.com\/blog\/wp-content\/uploads\/2012\/04\/ISI-POST03.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignright size-medium wp-image-2065\" title=\"Example of NFS failover process\" src=\"http:\/\/www.jasemccarty.com\/blog\/wp-content\/uploads\/2012\/04\/ISI-POST03-300x124.png\" alt=\"Example of NFS failover process\" width=\"300\" height=\"124\" srcset=\"https:\/\/www.jasemccarty.com\/blog\/wp-content\/uploads\/2012\/04\/ISI-POST03-300x124.png 300w, https:\/\/www.jasemccarty.com\/blog\/wp-content\/uploads\/2012\/04\/ISI-POST03.png 735w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/a>With SmartConnect Advanced, in the event of node going offline from maintenance or failure, one of the remaining nodes will service the request for the IP address that failed over.<\/p>\n<p>SmartConnect Advanced can appropriately move the failed IP address to another node based on any of the Connection Policies mentioned above.\u00a0 This ensures the best distributed load to the nodes are available.<\/p>\n<p><strong>Update:<\/strong> Here are a few screen shots from my Isilon lab&#8230;<\/p>\n<table align=\"center\">\n<tbody>\n<tr>\n<td style=\"text-align: center;\"><strong>6 Node Isilon Cluster<\/strong><br \/>\n<a href=\"http:\/\/www.jasemccarty.com\/blog\/wp-content\/uploads\/2012\/04\/ISI-POST04.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-medium wp-image-2098\" title=\"ISI-POST04\" src=\"http:\/\/www.jasemccarty.com\/blog\/wp-content\/uploads\/2012\/04\/ISI-POST04-300x143.png\" alt=\"\" width=\"300\" height=\"143\" srcset=\"https:\/\/www.jasemccarty.com\/blog\/wp-content\/uploads\/2012\/04\/ISI-POST04-300x143.png 300w, https:\/\/www.jasemccarty.com\/blog\/wp-content\/uploads\/2012\/04\/ISI-POST04.png 646w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/a><\/td>\n<\/tr>\n<tr>\n<td style=\"text-align: center;\"><strong>Failed node with only SmartConnect Basic<\/strong><br \/>\n<a href=\"http:\/\/www.jasemccarty.com\/blog\/wp-content\/uploads\/2012\/04\/ISI-POST05.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-medium wp-image-2099\" title=\"ISI-POST05\" src=\"http:\/\/www.jasemccarty.com\/blog\/wp-content\/uploads\/2012\/04\/ISI-POST05-300x154.png\" alt=\"\" width=\"300\" height=\"154\" srcset=\"https:\/\/www.jasemccarty.com\/blog\/wp-content\/uploads\/2012\/04\/ISI-POST05-300x154.png 300w, https:\/\/www.jasemccarty.com\/blog\/wp-content\/uploads\/2012\/04\/ISI-POST05.png 655w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/a>Notice Node 2&#8217;s IP address doesn&#8217;t fail over<\/td>\n<\/tr>\n<tr>\n<td style=\"text-align: center;\"><strong>Failed node with SmartConnect Advanced<\/strong><br \/>\n<a href=\"http:\/\/www.jasemccarty.com\/blog\/wp-content\/uploads\/2012\/04\/ISI-POST06.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-medium wp-image-2100\" title=\"ISI-POST06\" src=\"http:\/\/www.jasemccarty.com\/blog\/wp-content\/uploads\/2012\/04\/ISI-POST06-300x151.png\" alt=\"\" width=\"300\" height=\"151\" srcset=\"https:\/\/www.jasemccarty.com\/blog\/wp-content\/uploads\/2012\/04\/ISI-POST06-300x151.png 300w, https:\/\/www.jasemccarty.com\/blog\/wp-content\/uploads\/2012\/04\/ISI-POST06.png 654w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/a>Notice Node 6 takes 2 addresses<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>Notice that with SmartConnect Advanced the failed IP address moves to another node. In this case, node 6 is now answering for 2 different IP addresses.<\/p>\n<p><strong>More information<\/strong><br \/>\nThe white paper (<a title=\"Best Practices Guide for VMware vSphere\" href=\"http:\/\/simple.isilon.com\/doc-viewer\/1739\/best-practices-guide-for-vmware-vsphere.pdf\" target=\"_blank\"><strong>http:\/\/simple.isilon.com\/doc-viewer\/1739\/best-practices-guide-for-vmware-vsphere.pdf<\/strong><\/a>) has this information and more about the best practices when using Isilon (NFS\/iSCSI) and VMware vSphere 5.0.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>As a vSpecialist, I am a member of a group that focuses on Big Data solutions. I primarily focus on Big Data storage, notably Isilon. &hellip; <\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[10,12],"tags":[35,47,111],"class_list":["post-2014","post","type-post","status-publish","format-standard","hentry","category-storage-2","category-virtualization","tag-emc","tag-isilon","tag-vsphere"],"_links":{"self":[{"href":"https:\/\/www.jasemccarty.com\/blog\/wp-json\/wp\/v2\/posts\/2014","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.jasemccarty.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.jasemccarty.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.jasemccarty.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.jasemccarty.com\/blog\/wp-json\/wp\/v2\/comments?post=2014"}],"version-history":[{"count":81,"href":"https:\/\/www.jasemccarty.com\/blog\/wp-json\/wp\/v2\/posts\/2014\/revisions"}],"predecessor-version":[{"id":2490,"href":"https:\/\/www.jasemccarty.com\/blog\/wp-json\/wp\/v2\/posts\/2014\/revisions\/2490"}],"wp:attachment":[{"href":"https:\/\/www.jasemccarty.com\/blog\/wp-json\/wp\/v2\/media?parent=2014"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.jasemccarty.com\/blog\/wp-json\/wp\/v2\/categories?post=2014"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.jasemccarty.com\/blog\/wp-json\/wp\/v2\/tags?post=2014"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}