Failover Clustering with multiple NICs on the same IP subnet

During the design phase of an Exchange Server 2010 environment I was asked if servers forming a DAG can use a shared IP subnet for the MAPI network and the Replication network.

The answer is NO.

I remembered from previous Windows operating system versions that there are issues with multiple adapters on the same subnet. This is described in Microsoft Knowledgebase Article 175767.

The Planning for High Availability and Site Resilience chapter of the Exchange Server 2010 online documentation provides this section:

“Each network in each DAG member server must be on its own network subnet. Each server in the DAG can be on a different subnet, but the MAPI and Replication networks must be routable and provide connectivity, such that:

• Each network in each DAG member server is on its own network subnet that’s separate from the subnet used by each other network in the server.
• Each DAG member server’s MAPI network can communicate with each other DAG member’s MAPI network.
• Each DAG member server’s Replication network can communicate with each other DAG member’s Replication network.
• There is no direct routing that allows heartbeat traffic from the Replication network on one DAG member server to the MAPI network on another DAG member server, or vice versa, or between multiple Replication networks in the DAG.”

The technical background is explained in this article Windows Server 2008 Failover Clusters: Networking posted on the Ask the Core Team blog:

“It is important in any cluster that there are no NICs on the same node that are configured to be on the same subnet. This is because the cluster network driver uses the subnet to identify networks and will use the first one detected and ignore any other NICs configured on the same subnet on the same node. The cluster validation process will register a Warning if any network interfaces in a cluster node are configured to be on the same network. The only possible exception to this would be for iSCSI (Internet Small Computer System Interface) connections. If iSCSI is implemented in a cluster, and MPIO (Multi-Path Input/Output) is being used for fault-tolerant connections to iSCSI Storage, then it is possible that the network interfaces could be on the same network. In this configuration, the iSCSI network in the Failover Cluster Manager should be configured such that cluster would not use it for any cluster communications.”

This three part blog article provides very good information about the network configuration of a Windows Server 2008 Failover Cluster.