Ethernet Products
Determine ramifications of Intel® Ethernet products and technologies
4810 Discussions

Which bonding mode does Intel SR-IOV NIC support?

idata
Employee
5,786 Views

Hi, Expert

I've investigated this for some days.

My purpose is launching a VM which has two VFs come from two physical NICs, and then do the bonding inside the VM.

I've tried all the 7 bonding modes, only the mode 1(active-backup) works, all the others cannot work.

From the following articles, the reason why other bonding mode cannot work is VM cannot change the VF's MAC address successfully.

https://bugzilla.redhat.com/show_bug.cgi?id=1164224 https://bugzilla.redhat.com/show_bug.cgi?id=1164224

/message/246183# 246183 https://communities.intel.com/message/246183# 246183

From the igb driver code, I also notice the HOST refuse VM's change VF's MAC address request.

case E1000_VF_SET_MAC_ADDR:

retval = -EINVAL;

if (!(vf_data->flags & IGB_VF_FLAG_PF_SET_MAC))

retval = igb_set_vf_mac_addr(adapter, msgbuf, vf);

else

dev_warn(&pdev->dev, "VF %d attempted to override administratively set MAC address\nReload the VF driver to resume operations\n", vf);

break;

So, My question is

* Does Intel's SR-IOV NIC only support bonding mode active-backup? only this mode not request VM change it's interface's MAC address.

* If I want to use active-active bonding mode, such as mode 0/2/4/5/6, any solution/walkaround to make it work?

Thanks.

0 Kudos
8 Replies
st4
New Contributor III
3,536 Views

Hi henry_wang,

Thank you for the post. Can you tell me are you using an onboard 82599 NIC or an X520 series (82599) network adapter?

For the meantime, I wll check on this.

rgds,

wb

0 Kudos
idata
Employee
3,536 Views

Thanks for reply quickly.

The NICs I use in my product are 82579 and I350.

# lspci | grep Eth

07:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)

07:00.1 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)

08:10.1 Ethernet controller: Intel Corporation I350 Virtual Function (rev 01)

08:10.5 Ethernet controller: Intel Corporation I350 Virtual Function (rev 01)

08:11.1 Ethernet controller: Intel Corporation I350 Virtual Function (rev 01)

08:11.5 Ethernet controller: Intel Corporation I350 Virtual Function (rev 01)

08:12.1 Ethernet controller: Intel Corporation I350 Virtual Function (rev 01)

08:12.5 Ethernet controller: Intel Corporation I350 Virtual Function (rev 01)

08:13.1 Ethernet controller: Intel Corporation I350 Virtual Function (rev 01)

82:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)

82:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)

82:10.0 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)

82:10.2 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)

82:10.4 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)

82:10.6 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)

82:11.0 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)

82:11.2 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)

82:11.4 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)

And here is more info about kernel and driver:

# uname -r

2.6.32-504.23.4.el6.centos.plus.x86_64

# ethtool -i eth0

driver: igb

version: 5.0.5-k

firmware-version: 1.61, 0x8000090e

bus-info: 0000:07:00.0

supports-statistics: yes

supports-test: yes

supports-eeprom-access: yes

supports-register-dump: yes

supports-priv-flags: no

# ethtool -i bond0

driver: bonding

version: 3.6.0

firmware-version: 2

bus-info:

supports-statistics: no

supports-test: no

supports-eeprom-access: no

supports-register-dump: no

supports-priv-flags: no

 

# ethtool -i eth21

driver: igbvf

version: 2.0.2-k

firmware-version:

bus-info: 0000:08:11.5

supports-statistics: yes

supports-test: yes

supports-eeprom-access: no

supports-register-dump: yes

supports-priv-flags: no

0 Kudos
st4
New Contributor III
3,536 Views

Hi henry_wang,

Thank you for the additional info. 82579 ethernet controller does not support SR-IOV but I350 does. Here is the website for reference:

http://www.intel.com/content/www/us/en/support/network-and-i-o/ethernet-products/000005722.html FAQ for Intel® Ethernet Server Adapters with SR-IOV

Please refer to the http://www.intel.com/content/www/us/en/network-adapters/10-gigabit-network-adapters/config-qos-with-flexible-port-partitioning.html Configure QoS with Intel® Flexible Port Partitioning regarding the bonding mode.

rgds,

wb

0 Kudos
idata
Employee
3,536 Views

I mean 82576 and I350, sorry for the typo error. Both types of NIC support SRIOV, I've already using the VF.

Thanks for the links share to me, I've also read these links before I asked this question.

From the reference http://www.intel.com/content/www/us/en/network-adapters/10-gigabit-network-adapters/config-qos-with-flexible-port-partitioning.html Configure QoS with Intel® Flexible Port Partitioning , seems VM is able to change the VF's MAC address, but when I do bonding according to the reference, error message report and bonding not success, the error message is:

Master "bond0": Error: SIOCBONDENSLAVE failed: Cannot assign requested address

Check on the HOST,

# dmesg | tail

igb 0000:82:00.0: VF 5 attempted to override administratively set MAC address

Reload the VF driver to resume operations

But from the reference , Patrick said: "One caveat – User can use IPROUTE2 utility to assign a unique MAC address to a VF from within the host Operating System. Once the new MAC address is assigned, VM that has this particular VF assigned to will not be able to alter its MAC address. This is called "administratively assigned MAC" and is a security feature."

And it's really what the igb driver's behavior:

 

case E1000_VF_SET_MAC_ADDR:

retval = -EINVAL;

if (!(vf_data->flags & IGB_VF_FLAG_PF_SET_MAC))

retval = igb_set_vf_mac_addr(adapter, msgbuf, vf);

else

dev_warn(&pdev->dev, "VF %d attempted to override administratively set MAC address\nReload the VF driver to resume operations\n", vf);

break;

 

So, I'm confused with these reference and the test result.

 

PS. I suppose the hardware NIC should have no limit for the bonding mode selection, but the driver does.

My current igb driver is 5.0.5-k and igbvf driver is 2.0.2-k.

 

 

0 Kudos
idata
Employee
3,536 Views

Thanks for your links.

I checked the latest driver code 5.3.3.5, it handle the VF MAC address change request the same as the driver 2.0.2:

if the VF's MAC address has been set by the PF from HOST, will not allow VM to set it again.

The related code in igb_main.c:

6852 case E1000_VF_SET_MAC_ADDR:

6853 retval = -EINVAL;

6854 # ifndef IGB_DISABLE_VF_MAC_SET

6855 if (!(vf_data->flags & IGB_VF_FLAG_PF_SET_MAC))

6856 retval = igb_set_vf_mac_addr(adapter, msgbuf, vf);

6857 else

6858 DPRINTK(DRV, INFO,

6859 "VF %d attempted to override administratively set MAC address\nReload the VF driver to resume operations\n",

6860 vf);

6861 # endif

6862 break;

Also I checked the ixgb driver code, also do the same thing for the change VF MAC address request:

if (adapter->vfinfo[vf].pf_set_mac &&

!ether_addr_equal(adapter->vfinfo[vf].vf_mac_addresses, new_mac)) {

e_warn(drv,

"VF %d attempted to override administratively set MAC address\n"

"Reload the VF driver to resume operations\n",

vf);

return -1;

}

I've tried to modify the code to delete the check, allow change the VF's MAC address, and after load the new driver module,

all bonding modes work fine.

But as Patrick said, allow VM to modify the VF's MAC address, is a security issue.

Do you know whether there any other potential issue except mac spoofing?

0 Kudos
st4
New Contributor III
3,536 Views

Hi henry_wang,

Thank you for the update and good to know the bonding mode works fine.

Let me check on you other inquiry about the potential issue.

rgds,

wb

0 Kudos
JHill9
Beginner
3,536 Views

Has this been fixed yet in the Intel driver? I would think that if a vf had spoofchk disabled that it would allow the guest to change the MAC, but it seems to still prevent it.

0 Kudos
Reply