Ethernet Products
Determine ramifications of Intel® Ethernet products and technologies
4811 Discussions

ixgbe driver does not support MAC address change

gXu4
Beginner
3,462 Views

My ESXi 5.5 system has Intel X540-t2 installed, and using ixgbe driver version 3.21.

In the driver code, I can see the following lines, which shows it does not support MAC address change in ESXi 5.5, because if vfinfo[vf].pf_set_mac=true, when guest OS change its MAC, the ixgbe driver won't change it in vfinfo accordingly and will report error: VF attempted to set a new MAC address but it already has an administratively set MAC address

int ixgbe_ndo_set_vf_mac(struct net_device *netdev, int vf, u8 *mac)

{

s32 retval = 0;

struct ixgbe_adapter *adapter = netdev_priv(netdev);

if (!is_valid_ether_addr(mac) || (vf >= adapter->num_vfs))

return -EINVAL;

dev_info(pci_dev_to_dev(adapter->pdev), "setting MAC %pM on VF %d\n", mac, vf);

dev_info(pci_dev_to_dev(adapter->pdev), "Reload the VF driver to make this change effective.\n");

retval = ixgbe_set_vf_mac(adapter, vf, mac);

if (retval >= 0) {

/* pf_set_mac is used in ESX5.1 and base driver but not in ESX5.5 */

adapter->vfinfo[vf].pf_set_mac = true;

if (test_bit(__IXGBE_DOWN, &adapter->state)) {

dev_warn(pci_dev_to_dev(adapter->pdev), "The VF MAC address has been set, but the PF device is not up.\n");

dev_warn(pci_dev_to_dev(adapter->pdev), "Bring the PF device up before attempting to use the VF device.\n");

}

} else {

dev_warn(pci_dev_to_dev(adapter->pdev), "The VF MAC address was NOT set due to invalid or duplicate MAC address.\n");

}

return retval;

}

static int ixgbe_set_vf_mac_addr(struct ixgbe_adapter *adapter, u32 *msgbuf, u32 vf)

{

u8 *new_mac = ((u8 *)(&msgbuf[1]));

if (!is_valid_ether_addr(new_mac)) {

e_warn(drv, "VF %d attempted to set invalid mac\n", vf);

return -1;

}

if (adapter->vfinfo[vf].pf_set_mac && memcmp(adapter->vfinfo[vf].vf_mac_addresses, new_mac, ETH_ALEN)) {

u8 *pm = adapter->vfinfo[vf].vf_mac_addresses;

e_warn(drv, "VF %d attempted to set a new MAC address but it already has an administratively set MAC address %2.2X:%2.2X:%2.2X:%2.2X:%2.2X:%2.2X\n",

vf, pm[0], pm[1], pm[2], pm[3], pm[4], pm[5]);

e_warn(drv, "Check the VF driver and if it is not using the correct MAC address you may need to reload the VF driver\n");

return -1;

}

return ixgbe_set_vf_mac(adapter, vf, new_mac) < 0;

}

However, according to the https://pubs.vmware.com/vsphere-55/index.jsp?topic=/com.vmware.vsphere.networking.doc/GUID-DD13D453-98B9-4D26-85EA-A738293AEE00.html VMware document, it should be supported. Why this contradiction happens?

0 Kudos
12 Replies
st4
New Contributor III
1,699 Views

Hi Tobyfan,

Thank you the post. We need to check on this.

rgds,

wb

0 Kudos
gXu4
Beginner
1,699 Views

Actually, in the latest version of the driver, the field 'pf_set_mac' is for MACVLAN which does not apply to ESXi. However, the logic is the same. When Guest OS changes MAC, a message IXGBE_VF_SET_MAC_ADDR will be sent from VF to host driver. But according to the function ixgbe_set_vf_mac_addr(), it always fails for different MAC addressees.

For datapath, driver has register PFVFSPOOF.MACAS to manage the MAC anti-spoofing on all VFs, each bit in the register corresponds to a field spoofchk_enabled in vf structure. However this field of all VFs are set to true during initialization. System needs to call ixgbe_ndo_set_vf_spoofchk() disable it. But how do we know if ESXi call this function to when setting portgroup policy 'Change MAC address'?

0 Kudos
gXu4
Beginner
1,699 Views

After investigating the ixgbe driver code, and browsing on intel community. We come with following questions.

1) according to the function ixgbe_set_vf_mac_addr(), host driver always deny MAC change request from VF. What is the reason behind?

2) ixgbe driver provides an ndo function ixgbe_ndo_set_vf_spoofchk() for system to set VLAN anti-spoofing and MAC anti-spoofing of a VF. But they are set at this same function, which means if MAC anti-spoofing is turned off, VLAN anti-spoofing will be turned off as well. Why? On the other hand, we see in x550 driver, it provide a new function ixgbe_set_ethertype_anti_spoofing_X550() to only change the MAC anti-spoofing. What is the reason to bring up this additional function?

3) In another thread https://communities.intel.com/ https://communities.intel.com/thread/73179?start=15&tstart=0, some one got similar problem, but their experiment showed as long as VLAN is tagged on VF, the anti-spoofing is turned on. Cannot understand why.

4) In another thread /thread/43474?start=0&tstart=0 https://communities.intel.com/thread/43474?start=0&tstart=0, Some one from intel said, for bonding on top of SR-IOV, the fail_over_mac option of bonding has to be set active (which means Guest won't change the MAC of its interfaces in a bond, and packets will use slave's mac as source mac). We are not sure if this requirement is because of the same issue we are seeing.

 

Thanks,

Toby

 

0 Kudos
st4
New Contributor III
1,699 Views

Hi tobyfan,

 

Please re-test with the latest version of ixgbe driver 4.1.1.1. The driver is available for download at https://my.vmware.com/web/vmware/details?downloadGroup=DT-ESXI55-INTEL-IXGBE-4111&productId=353 https://my.vmware.com/web/vmware/details?downloadGroup=DT-ESXI55-INTEL-IXGBE-4111&productId=353

Just to double check where did you download the origin code of the driver you refer to?

Thanks,

 

wb
0 Kudos
gXu4
Beginner
1,699 Views

I have download the driver 4.1.1.1. The code is a little different, but the logic is the same. When Guest OS change the MAC of a NIC, a message is sent from VF, and the message handler calls function ixgbe_set_vf_mac_addr(), which always denies MAC modification. Look at the following code in driver 4.1.1.1

static int ixgbe_set_vf_mac_addr(struct ixgbe_adapter *adapter, u32 *msgbuf, u32 vf)

{

u8 *new_mac = ((u8 *)(&msgbuf[1]));

if (!is_valid_ether_addr(new_mac)) {

e_warn(drv, "VF %d attempted to set invalid mac\n", vf);

return -1;

}

if (memcmp(adapter->vfinfo[vf].vf_mac_addresses, new_mac, ETH_ALEN)) {

u8 *pm = adapter->vfinfo[vf].vf_mac_addresses;

e_warn(drv,

"VF %d attempted to set a new MAC address but it already has an administratively set MAC address %2.2X:%2.2X:%2.2X:%2.2X:%2.2X:%2.2X\n",

vf, pm[0], pm[1], pm[2], pm[3], pm[4], pm[5]);

e_warn(drv, "Check the VF driver and if it is not using the correct MAC address you may need to reload the VF driver\n");

return -1;

}

ixgbe_passthru_config(adapter, vf, VMK_CFG_MAC_CHANGED,

(void*)new_mac);

return 0;

}

0 Kudos
st4
New Contributor III
1,699 Views

Hi Tobyfan,

Thanks for the update. Please provide also where did you download the origin code of the driver you refer to in the initial post?

For your test result. we will further check.

rgds,

wb

0 Kudos
gXu4
Beginner
1,699 Views

I download the driver from source forge. https://sourceforge.net/projects/e1000/files/ixgbe%20stable/ Intel Ethernet Drivers and Utilities - Browse /ixgbe stable at SourceForge.net

thanks

0 Kudos
st4
New Contributor III
1,699 Views

Hi Tobyfan,

Thank you for the info. Let me check on this.

rgds,

wb

0 Kudos
st4
New Contributor III
1,699 Views

Hi Tobyfan,

Further checking VMware the OS vendor should be able to provide a solution since ESXi driver is provided by VMware. Can you help clarify also why this inquiry is address to Intel?

Thanks,

 

wb
0 Kudos
gXu4
Beginner
1,699 Views

I also contacted VMware for this issue.

The reason why this inquiry addresses to Intel is because I found the logic of the driver code is confusing (as I posted in this thread) and I cannot find answer in your http://www.intel.com/content/dam/www/public/us/en/documents/datasheets/82599-10-gbe-controller-datasheet.pdf Datasheet

0 Kudos
st4
New Contributor III
1,699 Views

Hi Tobyfan,

Thank you for the clarification. Is there any update from Vmware as you menitioned you also contacted them.

rgds

wb

0 Kudos
st4
New Contributor III
1,699 Views

Hi Tobyfan,

Good day. Here is a suggestion you can use the iproute2 available at to change MAC address, use the version dated March 14, 2016.

https://www.kernel.org/pub/linux/utils/net/iproute2/ https://www.kernel.org/pub/linux/utils/net/iproute2/

Here is another reference information from Vmware about MAC address, you can check this link out:

https://pubs.vmware.com/vsphere-55/index.jsp?topic=/com.vmware.vsphere.networking.doc/GUID-1C9C9FA5-2D2D-48DA-9AD5-110171E8FD36.html https://pubs.vmware.com/vsphere-55/index.jsp?topic=/com.vmware.vsphere.networking.doc/GUID-1C9C9FA5-2D2D-48DA-9AD5-110171E8FD36.html

Please feel free to update me. Thank you.

rgds,

wb

0 Kudos
Reply