4 Replies Latest reply on Sep 12, 2016 2:39 PM by wahmad00

    Problem with assign SR-IOV Intel NIC to QEMU Virtual Machine.

    enigma_lee

      Hello,

           I am using an Intel 82599 NIC and would like to use the SR-IOV function to assign the virtual functions to my virtual machine. I actually follow the on-line Intel document to add the "max_vfs=2" parameter when the ixgbe module is loaded. I can see the virtual function of NIC by now:

      [root@vt-host2 enigmaLi@vt-host2]# lspci

      00:00.0 Host bridge: Intel Corporation 4th Gen Core Processor DRAM Controller (rev 06)

      00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x16 Controller (rev 06)

      00:02.0 VGA compatible controller: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor Integrated Graphics Controller (rev 06)

      00:03.0 Audio device: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor HD Audio Controller (rev 06)

      00:14.0 USB controller: Intel Corporation 8 Series/C220 Series Chipset Family USB xHCI (rev 05)

      00:16.0 Communication controller: Intel Corporation 8 Series/C220 Series Chipset Family MEI Controller #1 (rev 04)

      00:19.0 Ethernet controller: Intel Corporation Ethernet Connection I217-V (rev 05)

      00:1a.0 USB controller: Intel Corporation 8 Series/C220 Series Chipset Family USB EHCI #2 (rev 05)

      00:1b.0 Audio device: Intel Corporation 8 Series/C220 Series Chipset High Definition Audio Controller (rev 05)

      00:1d.0 USB controller: Intel Corporation 8 Series/C220 Series Chipset Family USB EHCI #1 (rev 05)

      00:1f.0 ISA bridge: Intel Corporation C220 Series Chipset Family H81 Express LPC Controller (rev 05)

      00:1f.2 IDE interface: Intel Corporation 8 Series/C220 Series Chipset Family 4-port SATA Controller 1 [IDE mode] (rev 05)

      00:1f.3 SMBus: Intel Corporation 8 Series/C220 Series Chipset Family SMBus Controller (rev 05)

      00:1f.5 IDE interface: Intel Corporation 8 Series/C220 Series Chipset Family 2-port SATA Controller 2 [IDE mode] (rev 05)

      01:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

      01:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

      02:10.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)

      02:10.1 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)

      02:10.2 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)

      02:10.3 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)

       

      and the "ip link show" gives me following result (I remove the kernel module ixgbevf):

      [root@vt-host2 enigmaLi@vt-host2]# ip link show

      1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1

          link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

      2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000

          link/ether e0:3f:49:4b:c1:82 brd ff:ff:ff:ff:ff:ff

      13: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default qlen 1000

          link/ether 52:54:00:2b:19:36 brd ff:ff:ff:ff:ff:ff

      14: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc fq_codel master virbr0 state DOWN mode DEFAULT group default qlen 1000

          link/ether 52:54:00:2b:19:36 brd ff:ff:ff:ff:ff:ff

      35: enp1s0f0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000

          link/ether 90:e2:ba:46:0a:30 brd ff:ff:ff:ff:ff:ff

          vf 0 MAC 00:00:00:00:00:00, spoof checking on, link-state auto

          vf 1 MAC 00:00:00:00:00:00, spoof checking on, link-state auto

      38: enp1s0f1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000

          link/ether 90:e2:ba:46:0a:31 brd ff:ff:ff:ff:ff:ff

          vf 0 MAC 00:00:00:00:00:00, spoof checking on, link-state auto

          vf 1 MAC 00:00:00:00:00:00, spoof checking on, link-state auto

       

      But when I try to assign the virtual function(PCI dev assign) to my virtual machine, the virt-manager gives me following error:

      Error starting domain: internal error: qemu unexpectedly closed the monitor: 2016-08-26T15:39:40.880928Z qemu-system-x86_64: -device vfio-pci,host=02:10.0,id=hostdev0,bus=pci.0,addr=0x9: vfio: error, group 1 is not viable, please ensure all devices within the iommu_group are bound to their vfio bus driver.

      2016-08-26T15:39:40.880946Z qemu-system-x86_64: -device vfio-pci,host=02:10.0,id=hostdev0,bus=pci.0,addr=0x9: vfio: failed to get group 1

      2016-08-26T15:39:40.880964Z qemu-system-x86_64: -device vfio-pci,host=02:10.0,id=hostdev0,bus=pci.0,addr=0x9: Device initialization failed

       

      Traceback (most recent call last):

        File "/usr/share/virt-manager/virtManager/asyncjob.py", line 88, in cb_wrapper

          callback(asyncjob, *args, **kwargs)

        File "/usr/share/virt-manager/virtManager/asyncjob.py", line 124, in tmpcb

          callback(*args, **kwargs)

        File "/usr/share/virt-manager/virtManager/libvirtobject.py", line 83, in newfn

          ret = fn(self, *args, **kwargs)

        File "/usr/share/virt-manager/virtManager/domain.py", line 1404, in startup

          self._backend.create()

        File "/usr/lib/python2.7/site-packages/libvirt.py", line 1035, in create

          if ret == -1: raise libvirtError ('virDomainCreate() failed', dom=self)

      libvirtError: internal error: qemu unexpectedly closed the monitor: 2016-08-26T15:39:40.880928Z qemu-system-x86_64: -device vfio-pci,host=02:10.0,id=hostdev0,bus=pci.0,addr=0x9: vfio: error, group 1 is not viable, please ensure all devices within the iommu_group are bound to their vfio bus driver.

      2016-08-26T15:39:40.880946Z qemu-system-x86_64: -device vfio-pci,host=02:10.0,id=hostdev0,bus=pci.0,addr=0x9: vfio: failed to get group 1

      2016-08-26T15:39:40.880964Z qemu-system-x86_64: -device vfio-pci,host=02:10.0,id=hostdev0,bus=pci.0,addr=0x9: Device initialization failed

       

      So anybody know what is the problem? BTW, I can use PCI passthrough perfectly to my virtual machine, but for SR-IOV mode, it just does not work.

      And also, in SR-IOV is the module vfio, vfio_pci, vfio_iommu_type1 and vfio_virqfd still needed?

      Thank you!