This blog will describe the Network Prioritization feature and how to configure it. Network Prioritization is used to designate which type of traffic should be directed through which network in a Failover Cluster running Windows Server 2008 R2. When designing a highly-available infrastructure, it is important to avoid any single points of failure, so there should be redundancy in all the hardware components, including the servers, storage and the networking. For this reason we recommend and assume that there are multiple networks in your cluster. If a network is unavailable Windows Server Failover Clustering will automatically direct traffic through another network to maintain service availability for applications or VMs. However some networks may be preferred for certain types of traffic based on the network’s speed, security or function, so it can be important to designate which network is used for which purpose.
Types of Cluster Network Traffic
Understanding the different categories of networking traffic can help you plan the best use for your networks.
Cluster & CSV Traffic
We define cluster traffic as the information needed for the nodes to communicate with each other to ensure that it functions correctly. This can include communication such as heartbeats for health-checking, updates to the cluster database, join requests from partitioned nodes, and much more. Additionally Cluster Shared Volumes (CSV) traffic will use this network for metadata updates, or when it is in redirected mode, meaning that a node hosting a VM cannot directly assess its disk, so it redirects the traffic via another node to that disk. This can be configured by right-clicking on the network in Failover Cluster Manager, selecting Properties, and selecting the radio button for “Allow cluster network communication on this network”.
Live Migration Traffic
If a cluster is taking advantage of the live migration feature to move running Virtual Machines (VMs) between cluster nodes, then live migration traffic is an important consideration. During a live migration, large chunks of memory are copied from one server to another as quickly as possible. This burts the network with heavy traffic, and can block other types of network communication from getting through. For this reason it is recommended to have a dedicated network for live migration traffic so that it does not interfere with other important network traffic. Live migration is commonly used when a physical host needs planned maintenance (such as patching and a reboot), so running VMs are live migrated from that host to avoid any downtime for the VM guests. The faster that this network is, the more traffic can pass through it, thus the host can be evacuated quicker, so it is very common to see the fastest network being dedicated towards live migration. This can be configured by right-clicking on the network in Failover Cluster Manager, selecting Properties, selecting the radio button for “Allow cluster network communication on this network”.
Public Traffic
Whether the cluster is providing high-availability for VMs, SQL databases, File Servers or anything else, a “client” needs to access that application, service or VM. A “client” is defined loosely as a user or application which needs to communicate with that workload running on the cluster. For the client to access that data, they need to make requests and have data sent back to them through those networks. These networks are generally less secure as they are more exposed and accessible, and they could be subject to network flooding, which could impact performance or throughput of traffic. For this reason it is recommended that you do not use this network for cluster traffic, live migration or any other use and to explicitly open it up to clients. This can be configured by right-clicking on the network in Failover Cluster Manager, selecting Properties, selecting the radio button for “Allow cluster network communication on this network” and selecting the checkbox “Allow clients to connect through this network”.
Storage Traffic
If your cluster uses iSCSI or Fibre Channel over Ethernet (FCoE) for the cluster’s shared storage, this traffic goes through an Ethernet network which the cluster will identify as a cluster network. To avoid storage I/O performance being affected with iSCSI or FCoE, it is recommended that you provide a dedicated network for storage traffic so that other network traffic does not interfere with this data. For this reason it is recommended that you do not use this network for cluster traffic, live migration or any other use. This can be configured by right-clicking on the network in Failover Cluster Manager, selecting Properties, and selecting the radio button for “Do not allow cluster network communication on this network”.
Network Prioritization
Since it is a best practice to use multiple networks in your cluster, there is likely the need to specify the function for the various cluster networks. This can be done on Windows Server 2008 R2 through the Network Prioritization (NP) feature. NP will list the order of cluster networks and give the ability for the networks to be “ranked”, where different ranks indicate different network roles. To rank a network, it is given a unique integer from 1 to 268,000,000+, which is called a “metric”.
To view the networks, their metric values, and if they were automatically or manually configured, run the clustering PowerShell cmdlet:
PS > Get-ClusterNetwork | ft Name, Metric, AutoMetric
PS > Get-ClusterNetwork | ft Name, Metric, AutoMetric
By default, all internal cluster network will have a metric value starting at 1000 and incrementing by 100. The first internal network which the cluster sees when it first comes online has a metric of 1000, the second has a metric of 1100, etc. We assume that a network is ‘internal’ if it does not have access to a default gateway. The initial list of internal networks is determined by the order which the network adapters were seen by the cluster when it was created.
By default all external cluster network will have a metric value starting at 10000 and incrementing by 100. The first external network which the cluster sees when it first comes online has a metric of 10000, the seconds has a metric of 10100, etc. We assume that a network is ‘external’ if it has access to a default gateway. The initial list of external networks is determined by the order which the network adapters were seen by the cluster when it was created.
The cluster will then use the order of the metrics as the order of networks. The lowest network will be used for “Cluster & CSV Traffic”. The second lowest network will be used for “Live Migration Traffic”. Additional networks with a metric below 10000 will be used as backup networks if the “Cluster & CSV Traffic” or “Live Migration Traffic” networks fail. The lowest network with a value of at least 10000 will be used for “Public Traffic”, and any additional networks with a metric above 10000 will be used as backup networks for “Public Traffic”. Give the highest possible values to any networks which you do not want any cluster or public traffic to go through, such as for “Storage Traffic”, so that they are never used, or only used when no other networks at all are available, depending on your settings.
So let’s say you get the following output from running
PS > Get-ClusterNetwork | ft Name, Metric, AutoMetric
PS > Get-ClusterNetwork | ft Name, Metric, AutoMetric
Name Metric AutoMetric
---- ------ ----------
Cluster Network 1 1000 True
Cluster Network 2 1100 True
Cluster Network 3 1200 True
Cluster Network 4 10000 True
Cluster Network 5 10100 True
---- ------ ----------
Cluster Network 1 1000 True
Cluster Network 2 1100 True
Cluster Network 3 1200 True
Cluster Network 4 10000 True
Cluster Network 5 10100 True
In this scenario, these networks would carry the following types of traffic:
· Cluster Network 1 (1000) – Cluster & CSV Traffic
· Cluster Network 2 (1100) – Live Migration Traffic
· Cluster Network 3 (1200) – Backup network for Cluster & CSV Traffic and Live Migration Traffic
· Cluster Network 4 (10000) – Public Traffic
· Cluster Network 5 (10100) – Backup network for Public Traffic
Configuring Network Prioritization
It is possible to customize NP if the cluster does not automatically assign networks to use the traffic pattern that you want, which will change the ranked order, and hence the function. For example, you may want Cluster Network 3 to be used for “Live Migration Traffic” as it is the fastest, so you would change its Metric to a value between 1000 and 1100, such as 1050, so that it is ranked second on the list. Once Cluster Network 3 has the second-lowest metric it will be used for Live Migration Traffic.
To change the value of a network metric, run:
PS > $n = Get-ClusterNetwork “Cluster Network 3”
PS > $n.Metric = 1050
PS > $n = Get-ClusterNetwork “Cluster Network 3”
PS > $n.Metric = 1050
This will change the metric of Cluster Network 3 to 1050.
Now you get the following output from running
PS > Get-ClusterNetwork | ft Name, Metric, AutoMetric
PS > Get-ClusterNetwork | ft Name, Metric, AutoMetric
Name Metric AutoMetric
---- ------ ----------
Cluster Network 1 1000 True
Cluster Network 3 1050 False
Cluster Network 2 1100 True
Cluster Network 4 10000 True
Cluster Network 5 10100 True
---- ------ ----------
Cluster Network 1 1000 True
Cluster Network 3 1050 False
Cluster Network 2 1100 True
Cluster Network 4 10000 True
Cluster Network 5 10100 True
You may have noticed that is a property associated with each network called AutoMetric. This indicates whether the Metric was set using the default values (True) or if it had been later adjusted by an admin (False). This gives insight into whether NP has been configured on the cluster. Using this flag, it is actually possible to change the value of a network back to its original and automatically assigned value, by running the cmdlet:
PS > $n = Get-ClusterNetwork “Cluster Network 3”
PS > $n.AutoMetric = $true
PS > $n = Get-ClusterNetwork “Cluster Network 3”
PS > $n.AutoMetric = $true
Overriding Network Prioritization Behavior
There are two ways to override the default behavior of NP. The first is by changing the network’s properties by right-clicking on the network in Failover Cluster Manager, selecting Properties, and changing the radio buttons or checkboxes. For example, if you select “Do not allow cluster network communication on this network”, then it will not be possible to send any “Cluster & CSV Traffic” or “Live Migration Traffic” through this network, even if the network has the lowest metric values. The cluster will honor this override and find the network with the next lowest value to send this type of traffic.
The second override is exclusively for “Live Migration Traffic”. The networks for live migration can be configured more granularly by right-clicking on any Virtual Machine resource, selecting Properties and clicking the Network for live migration tab. Here you have the ability to specify which networks can and cannot be used for “Live Migration Traffic” and in which order they should be used. Even though it appears that this setting may be unique to that specific VM, it is actually a global setting for live migration. This means that it will override the “Live Migration Traffic” network configured through NP and all VMs will perform a live migration through the network(s) specified here. If this setting is change multiple locations, the last change will be honored.
No comments:
Post a Comment