WoodITWork.com

It looks like the continuing HP Blade / VMWare compatibility issues with the Broadcom Nic drivers are ongoing as ESX 4.1 now can’t use HPs Smart Link and DCC.

In my previous post I listed the Flex-10 ESX pre-requisites which was for ESX 4.0 and works with Smart Link and DCC.
http://www.wooditwork.com/2010/08/09/flex-10-esx-pre-requisites/

Let me explain how one of the components of Flex-10 was designed for network failover.

Flex-10 switches have a great sounding feature called Smart Link. When the uplink ports from your Flex-10 switches in your chassis show a down link Smart Link knows that the upstream network is unavailable and in turn marks the individual blade nic down. Your ESX hosts would see a down nic with a red cross and fail over to the other nic(s) in the ESX team.

Smart Link is enabled within the Virtual Connect Manager | Ethernet Networks

When this first came out HP didn’t really explain that for this to actually work as advertised every uplink for every Flex-Nic connected to one of the 10Gb Broadcom Nics would have to be down for the blade nic to see it as down. It couldn’t work just on a single Flex-Nic.

If you had any internal networks defined mapped to Flex-Nics through your server profiles these would never fail as they were only internal to your Virtual Connect domain and so your Flex-Nics could never show a down status.

Using Beacon Probing with ESX was the solution, in effect not using Smart Link at all. You can leave Smart Link enabled, it just won’t be able to use it.

Beacon probing sends out network packet probes over both nics to upstream switches outside the Flex-10 Virtual Connect network so can tell when an uplink is down.

Here’s VMWare’s take on Beacon Probing: http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1005577 although they do talk about having at least 3 Nics in a team which I feel is not required, 2 will do and works.

Beacon Probing is configured on the vSwitch under Nic Teaming | Network Failover Detection.

In order to work on an individual Flex-Nic it would need a driver component update called Device Control Channel (DCC) which would manage the up/down state of an individual Flex-Nic within a 10Gb nic.

This was all brought to life when the Broadcom 1.48 driver was released which enabled DCC which was the version I recommended be installed post your ESX 4.0 install. Smart Link would now work down to a Flex-Nic level.

You needed to have compatible HP Virtual Connect firmware (minimum 2.3) and each blade nic firmware had to be at least 5.0.11. Best though to go for the versions in my original post.

I think its probably best to keep Beacon Probing configured even after upgrading the nic driver. If Smart Link doesn’t t work as expected Beacon Probing will still kick in adding another level of failover.

ESX 4.1 has been released and now has a new Broadcom driver version 1.54.

HP has released an advisory saying this driver version now doesn’t work with Smart Link and DCC…great..one step forwards, 2 steps back!
http://h20000.www2.hp.com/bizsupport/TechSupport/Document.jsp?lang=en&cc=us&objectID=c02476622&jumpid=reg_R1002_USEN

The resolution they offer is a workaround to use Beacon Probing until a newer driver can be released.

So if you are intending to upgrade to or use ESX 4.1 you need to have Beacon Probing enabled otherwise your Flex-Nics won’t fail over and just have to wait until the driver is updated again to get Smart Link to work.