11 Replies Latest reply on Aug 24, 2016 7:48 PM by Intel Corporation

    X710-DA4 - SR-IOV not working correctly

    jailbird

      Host OS: Ubuntu 16.04 LTS

      Hypervisor: Xen 4.6.0

      Kernel version: 4.4.0-22-generic

      Driver version: i40e-1.5.18

      NVM version: 5.02

      Guest OS: Windows Server 2012 R2

      Guest Driver version: 20.7.1

       

      I'm trying to forward the ports of the card to a guest Windows VM.  An example of forwarding just one port:

       

      # echo 1 > /sys/class/net/ens2f0/device/sriov_numvfs

      # /sbin/ip link set ens2f0 vf 0 mac 02:00:00:00:00:00

      # xl pci-assignable-add 02:02.0

      Added to Xen VM .cfg:

      pci = ['0000:02:02.0']

       

      All of that works and the VM starts fine and sees the card.  Xen reports that it added it:

       

      [26326.586765] i40e 0000:02:00.0 ens2f0: adding 68:05:ca:33:0a:59 vid=0

      [26326.586774] i40e 0000:02:00.0 ens2f0: adding 68:05:ca:33:0a:59 vid=1

      [26326.595995] i40e 0000:02:00.0: Allocating 1 VFs.

      [26326.697778] pci 0000:02:02.0: [8086:154c] type 00 class 0x020000

      [26326.697973] pci 0000:02:02.0: Max Payload Size set to 256 (was 128, max 2048)

      [26329.893582] i40e 0000:02:00.0: Setting MAC 02:00:00:00:00:00 on VF 0

      [26329.965339] i40e 0000:02:00.0: Reload the VF driver to make this change effective.

      [26366.200750] pciback 0000:02:02.0: seizing device

      [26366.200815] pciback 0000:02:02.0: enabling device (0000 -> 0002)

      [26472.867101] xen_pciback: vpci: 0000:02:02.0: assign to virtual slot 0

      [26472.867643] pciback 0000:02:02.0: registering for 4

       

      I install the drivers and Windows 'see' the card.  The problem is the reported state goes into a cycle:

       

      Connected @ 10GB

      Connected @ 40GB

      Disconnected

       

      When this happens, in dmesg on the host:

       

      [26624.137415] i40e 0000:02:00.0: VF 0 failed opcode 11, retval: -10

      [26635.075640] i40e 0000:02:00.0: VF 0 failed opcode 11, retval: -10

      [26645.903762] i40e 0000:02:00.0: VF 0 failed opcode 11, retval: -10

      [26656.669642] i40e 0000:02:00.0: VF 0 failed opcode 11, retval: -10

      [26667.904191] i40e 0000:02:00.0: VF 0 failed opcode 11, retval: -10

      [26679.013381] i40e 0000:02:00.0: VF 0 failed opcode 11, retval: -10

      [26689.966331] i40e 0000:02:00.0: VF 0 failed opcode 11, retval: -10

      [26701.028906] i40e 0000:02:00.0: VF 0 failed opcode 11, retval: -10

      [26712.372668] i40e 0000:02:00.0: VF 0 failed opcode 11, retval: -10

      [26723.154159] i40e 0000:02:00.0: VF 0 failed opcode 11, retval: -10

      [26733.935320] i40e 0000:02:00.0: VF 0 failed opcode 11, retval: -10

      [26744.685130] i40e 0000:02:00.0: VF 0 failed opcode 11, retval: -10

      [26756.075844] i40e 0000:02:00.0: VF 0 failed opcode 11, retval: -10

       

      This same issue happens using the built in kernel i40e and also i40e-1.5.18. Am I doing something wrong?