Comments on: Broadcom NIC Teaming and Hyper-V on Server 2008 R2 http://www.confusedamused.com/notebook/broadcom-nic-teaming-and-hyper-v-on-server-2008-r2/ Wed, 01 Sep 2010 17:12:42 +0000 http://wordpress.org/?v=2.9.1 hourly 1 By: Rogier http://www.confusedamused.com/notebook/broadcom-nic-teaming-and-hyper-v-on-server-2008-r2/comment-page-1/#comment-1601 Rogier Thu, 26 Aug 2010 12:47:56 +0000 http://www.confusedamused.com/?p=712#comment-1601 <p>Too bad no one mentions the driver version, that would really be usefull. My broadcom has version 14.1.5. Teaming has broken my network load balancing with virtual machines.</p> Too bad no one mentions the driver version, that would really be usefull. My broadcom has version 14.1.5. Teaming has broken my network load balancing with virtual machines.

]]> By: daniel bowbyes http://www.confusedamused.com/notebook/broadcom-nic-teaming-and-hyper-v-on-server-2008-r2/comment-page-1/#comment-1544 daniel bowbyes Thu, 17 Jun 2010 09:47:41 +0000 http://www.confusedamused.com/?p=712#comment-1544 <p>We have the same problem, if you live migrate a VM from R2 host A to R2 host B the Broadcom team on host A continues to send out spoofed ARP requests to hosts that the VM had TCP sessions opened to at the time of migration.</p> <p>Whats intresting is that with an Intel team the mac addresses of VM's are learnt by the switch, with the Broadcom teams the switch does not learn the MAC address of the VM's as the Broadcom Team over writes the source mac addresses of all packets with the team mac address.</p> We have the same problem, if you live migrate a VM from R2 host A to R2 host B the Broadcom team on host A continues to send out spoofed ARP requests to hosts that the VM had TCP sessions opened to at the time of migration.

Whats intresting is that with an Intel team the mac addresses of VM’s are learnt by the switch, with the Broadcom teams the switch does not learn the MAC address of the VM’s as the Broadcom Team over writes the source mac addresses of all packets with the team mac address.

]]> By: Tyler Gray http://www.confusedamused.com/notebook/broadcom-nic-teaming-and-hyper-v-on-server-2008-r2/comment-page-1/#comment-1542 Tyler Gray Wed, 16 Jun 2010 21:16:25 +0000 http://www.confusedamused.com/?p=712#comment-1542 <p>It appears that Broadcom may have finally fixed the issue in their BACS teaming software!!! </p> <p>I have only gotten a chance to do preliminary testing but our configuration is as follows: Fully Patched Windows Server 2008R2 with the latest NIC firmware (from IBM's site) and NIC Drivers and Broadcom Management Applications Installer (BACS software) from Broadcom's website. We are running IBM HS21 Blades with Broadcom 5708S NetXtreme II GigE NICs. Our NICs are teamed with No VLANs (testing that next) and in Smart Load Balancing with Failover mode (no standby) with LiveLink not configured. I upgraded the BACS software after first uninstalling the old (and rebooting) and did not bother with installation order of the Hyper-V Role. All Virtual Children have Dynamic MACs.</p> <p>The testing at this point has been pretty basic. All that I have done so far is basically migrate a Virtual Server Child from one Host to another. However, since this never worked before (exhibiting the mentioned behavior) it seems like things should work now! I am going to continue stress testing (rebooting children on different hosts, rebooting hosts, messing with the NICs in the team to verify connectivity happens on a down path, adding VLANing, etc.) to see if all of their bugs are gone. But as stated, I now no longer have my other Hosts that at any time had that server migrated to it attempting to claim that Child server's MAC/IP Addressing making pings go every which way.</p> <p>I just wanted to get this post up in order to see if other people have similar results and to let others know this may be working now!</p> It appears that Broadcom may have finally fixed the issue in their BACS teaming software!!!

I have only gotten a chance to do preliminary testing but our configuration is as follows: Fully Patched Windows Server 2008R2 with the latest NIC firmware (from IBM’s site) and NIC Drivers and Broadcom Management Applications Installer (BACS software) from Broadcom’s website. We are running IBM HS21 Blades with Broadcom 5708S NetXtreme II GigE NICs. Our NICs are teamed with No VLANs (testing that next) and in Smart Load Balancing with Failover mode (no standby) with LiveLink not configured. I upgraded the BACS software after first uninstalling the old (and rebooting) and did not bother with installation order of the Hyper-V Role. All Virtual Children have Dynamic MACs.

The testing at this point has been pretty basic. All that I have done so far is basically migrate a Virtual Server Child from one Host to another. However, since this never worked before (exhibiting the mentioned behavior) it seems like things should work now! I am going to continue stress testing (rebooting children on different hosts, rebooting hosts, messing with the NICs in the team to verify connectivity happens on a down path, adding VLANing, etc.) to see if all of their bugs are gone. But as stated, I now no longer have my other Hosts that at any time had that server migrated to it attempting to claim that Child server’s MAC/IP Addressing making pings go every which way.

I just wanted to get this post up in order to see if other people have similar results and to let others know this may be working now!

]]> By: Bruce http://www.confusedamused.com/notebook/broadcom-nic-teaming-and-hyper-v-on-server-2008-r2/comment-page-1/#comment-1512 Bruce Wed, 19 May 2010 17:54:54 +0000 http://www.confusedamused.com/?p=712#comment-1512 <p>As an update here this ran fine for a few weeks and now we're seeing that once a VM migrates the other virtual switch is still answering ARP replies for the VM. So what happens is half your pings go one direction and the other half the other way, making for a very unstable guest VM. We're disabling the teaming for now and I'm personally never, ever, ever buying Broadcom again.</p> <p>Anthony, the behavior you mentioned is correct - to an outside machine the MAC of all the VMs running on a single host will be the same. That's just how the Hyper-V virtual switch works with teaming so if you're having connectivity issues there's probably something else wrong.</p> As an update here this ran fine for a few weeks and now we’re seeing that once a VM migrates the other virtual switch is still answering ARP replies for the VM. So what happens is half your pings go one direction and the other half the other way, making for a very unstable guest VM. We’re disabling the teaming for now and I’m personally never, ever, ever buying Broadcom again.

Anthony, the behavior you mentioned is correct – to an outside machine the MAC of all the VMs running on a single host will be the same. That’s just how the Hyper-V virtual switch works with teaming so if you’re having connectivity issues there’s probably something else wrong.

]]> By: Doug Federwitz http://www.confusedamused.com/notebook/broadcom-nic-teaming-and-hyper-v-on-server-2008-r2/comment-page-1/#comment-1453 Doug Federwitz Tue, 23 Mar 2010 14:48:43 +0000 http://www.confusedamused.com/?p=712#comment-1453 <p>A side note on this BroadCom Teaming. For us failover clusters doing a large amounts of TCP/IP traffic would cause the active node to crash roughly every 4-6hours. MS and Dell solution was to turn off Chimney Offloads. This has fixed this problem. Dell stated they have a engineer call open with BroadCom to resolve the issue. Also BroadCom Teaming on our IBM Blades has also caused a couple of our new 2008 DC's to ARP out and kill our Cisco 6500's.</p> <p>So like stated save a headache and support calls get some Intel NIC's</p> A side note on this BroadCom Teaming. For us failover clusters doing a large amounts of TCP/IP traffic would cause the active node to crash roughly every 4-6hours. MS and Dell solution was to turn off Chimney Offloads. This has fixed this problem. Dell stated they have a engineer call open with BroadCom to resolve the issue. Also BroadCom Teaming on our IBM Blades has also caused a couple of our new 2008 DC’s to ARP out and kill our Cisco 6500’s.

So like stated save a headache and support calls get some Intel NIC’s

]]> By: Tom Pacyk http://www.confusedamused.com/notebook/broadcom-nic-teaming-and-hyper-v-on-server-2008-r2/comment-page-1/#comment-1450 Tom Pacyk Fri, 19 Mar 2010 22:02:41 +0000 http://www.confusedamused.com/?p=712#comment-1450 <p>As an update here this ran fine for a few weeks and now we're seeing that once a VM migrates the other virtual switch is still answering ARP replies for the VM. So what happens is half your pings go one direction and the other half the other way, making for a very unstable guest VM. We're disabling the teaming for now and I'm personally never, ever, ever buying Broadcom again.</p> <p>Anthony, the behavior you mentioned is correct - to an outside machine the MAC of all the VMs running on a single host will be the same. That's just how the Hyper-V virtual switch works with teaming so if you're having connectivity issues there's probably something else wrong.</p> As an update here this ran fine for a few weeks and now we’re seeing that once a VM migrates the other virtual switch is still answering ARP replies for the VM. So what happens is half your pings go one direction and the other half the other way, making for a very unstable guest VM. We’re disabling the teaming for now and I’m personally never, ever, ever buying Broadcom again.

Anthony, the behavior you mentioned is correct – to an outside machine the MAC of all the VMs running on a single host will be the same. That’s just how the Hyper-V virtual switch works with teaming so if you’re having connectivity issues there’s probably something else wrong.

]]> By: Berserker http://www.confusedamused.com/notebook/broadcom-nic-teaming-and-hyper-v-on-server-2008-r2/comment-page-1/#comment-1449 Berserker Fri, 19 Mar 2010 21:34:11 +0000 http://www.confusedamused.com/?p=712#comment-1449 <p>Would the option to enable MAC spoofing in HV R2 help with this issue?</p> <p>http://blogs.technet.com/jhoward/archive/2009/05/21/new-in-hyper-v-windows-server-2008-r2-part-2-mac-spoofing.aspx</p> <p>http://sbcengine.blogspot.com/2009/10/windows-nlb-on-hyper-v-r2.html</p> Would the option to enable MAC spoofing in HV R2 help with this issue?

http://blogs.technet.com/jhoward/archive/2009/05/21/new-in-hyper-v-windows-server-2008-r2-part-2-mac-spoofing.aspx

http://sbcengine.blogspot.com/2009/10/windows-nlb-on-hyper-v-r2.html

]]> By: Anthony Wharmby http://www.confusedamused.com/notebook/broadcom-nic-teaming-and-hyper-v-on-server-2008-r2/comment-page-1/#comment-1446 Anthony Wharmby Tue, 09 Mar 2010 14:58:57 +0000 http://www.confusedamused.com/?p=712#comment-1446 <p>I am having similiar issues, however even with static MAC's the problem still occurs.</p> <p>Using Broadcoms latest drivers whenever a team is assigned to the virtual switch, all of the VM's on that host acquire the MAC of the Team. It gets to the point where I can have 5-10 servers all with the same MAC and a lot of issues with packet loss.</p> <p>To get around this so far I have set the team to use FEC/GEC LAG even though each NIC in the team is connected to different switches.</p> I am having similiar issues, however even with static MAC’s the problem still occurs.

Using Broadcoms latest drivers whenever a team is assigned to the virtual switch, all of the VM’s on that host acquire the MAC of the Team. It gets to the point where I can have 5-10 servers all with the same MAC and a lot of issues with packet loss.

To get around this so far I have set the team to use FEC/GEC LAG even though each NIC in the team is connected to different switches.

]]> By: John Weber http://www.confusedamused.com/notebook/broadcom-nic-teaming-and-hyper-v-on-server-2008-r2/comment-page-1/#comment-1440 John Weber Mon, 22 Feb 2010 15:53:46 +0000 http://www.confusedamused.com/?p=712#comment-1440 <p>I have also noticed some Dell servers running Intel and Broadcom NICs together causes something on the Hyper-V host to simply shut off the server. Powers down for no obvious reason. Disabling EITHER NIC results in stable performance. No updating of NIC drivers helps this. Don't know if this is related or not.</p> I have also noticed some Dell servers running Intel and Broadcom NICs together causes something on the Hyper-V host to simply shut off the server. Powers down for no obvious reason. Disabling EITHER NIC results in stable performance. No updating of NIC drivers helps this. Don’t know if this is related or not.

]]>