Intel Communities: Message List - Guest OS enumeration for VF PCIe device for 82599

Re: Guest OS enumeration for VF PCIe device for 82599

webadmin@intel.com — Fri, 15 Jun 2012 14:36:36 GMT

Great, I am happy it working and to be able to help.

I do not know what is happening with this DMAR issue you have described. We suspect it may be a BIOS issue, however I will pass along the information you provided to our SR-IOV experts and relay back anything they may offer by way of explaination.

Re: Guest OS enumeration for VF PCIe device for 82599

webadmin@intel.com — Thu, 14 Jun 2012 04:54:49 GMT

It is working with the latest source code Thnaks for the suppot

I am curious to know what is reason for the DMAR issue????

DRHD: handling fault status reg 2

DMAR:[DMA Read] Request device [03:00.0] fault addr 619c00000 DMAR:[fault reason 01] Present bit in root entry is clear

Re: Guest OS enumeration for VF PCIe device for 82599

webadmin@intel.com — Fri, 25 May 2012 06:03:59 GMT

Hi Patrick,

Thanks for the support................

I will download and test the lastest driver and let u know the result.

Veena

Re: Guest OS enumeration for VF PCIe device for 82599

webadmin@intel.com — Thu, 24 May 2012 16:10:22 GMT

That is a very old driver. I would recommend getting the latest versions (PF and VF) - which is available here:

http://sourceforge.net/projects/e1000/files/ixgbe%20stable/

Please give this a try and let us know how it works.

thanks,

Patrick

Re: Guest OS enumeration for VF PCIe device for 82599

webadmin@intel.com — Thu, 24 May 2012 05:41:37 GMT

I am using the kernel version 3.2.13 rt23.

Re: Guest OS enumeration for VF PCIe device for 82599

webadmin@intel.com — Tue, 22 May 2012 22:17:49 GMT

Thanks for trying that and reporting your findings. It would appear to be some kind of DMAR issue - we are preparing to investigate. It makes sense if there is a DMAR problem that VF's assigned to a VM would no work.

To aid in the investigation, our engineers have asked if you are using the inbox (kernel) drivers, or the latest from sourceforge.

thanks,

Patrick

Re: Guest OS enumeration for VF PCIe device for 82599

webadmin@intel.com — Tue, 22 May 2012 05:33:10 GMT

Hi Patrick,

I tried on RHEL 6.2 the previous issue was not there,new issue has been arrived.

Configuration:

CONFIG_DMAR_TABLE=y in kernel Config menu,
intel_iommu=on in grub.config
VT-d enabled in bios.

Scenarios 1:

Loaded the ixgbe max_vfs=0, and started the testing.
Below are dmesg from the kernel.
DRHD: handling fault status reg 2
DMAR:[DMA Read] Request device [03:00.0] fault addr 619c00000 DMAR:[fault reason 01] Present bit in root entry is clear

Scenarios 2:

loaded the ixgbe max_vfs=2, and started the testing.
Assigned the Vfs to the GuestOs.VFs are working fine.

Didn't receive any DMAR error.

If intel_iommu=off or CONFIG_DMAR_TABLE=n,the above issue is not seen and ethernet driver is working fine(only PF driver ).But VFs cannot be assigned to the guest OS.

Can u please tell where it went worng ......................

Re: Guest OS enumeration for VF PCIe device for 82599

webadmin@intel.com — Fri, 18 May 2012 22:02:31 GMT

My SR-IOV Guru's believe this is likely a Fedora 14 problem. We would like to suggest using the latest Fedora release, or try to reproduce the problem with RHEL 6.2.

It takes a several components to make SR-IOV work, BIOS, platform,end-point device (the Intel 82599) and of course the OS. There are lots and lots of patches to the linux kernel for SR-IOV support. Sometimes not all of those patches make it into a distro release due to any number of reasons.

Please update us on your efforts.

thanks,

Patrick

Re: Guest OS enumeration for VF PCIe device for 82599

webadmin@intel.com — Wed, 09 May 2012 08:36:33 GMT

Thanks for the support i will try to load the RHEL 6.x an d thanks for the document

Mean time the information on interrupts

Interrupts for PF

[root@veena /]# lspci | grep 82599

83:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit Network Connection (rev 01)

83:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit Network Connection (rev 01)

[root@veena /]# modprobe ixgbe max_vfs=1

[root@veena /]# lspci | grep 82599

83:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit Network Connection (rev 01)

83:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit Network Connection (rev 01)

83:10.0 Ethernet controller: Intel Corporation 82559 Ethernet Controller Virtual Function (rev 01)

83:10.1 Ethernet controller: Intel Corporation 82559 Ethernet Controller Virtual Function (rev 01)

[root@veena /]# cat /proc/interrupts

115: 2 0 0 0 0 0 0 0 IR-PCI-MSI-edge eth4-rx-0

116: 2 0 0 0 0 0 0 0 IR-PCI-MSI-edge eth4-tx-0

117: 9 0 0 0 0 0 0 0 IR-PCI-MSI-edge eth4:lsc

118: 2 0 0 0 0 0 0 0 IR-PCI-MSI-edge eth5-rx-0

119: 2 0 0 0 0 0 0 0 IR-PCI-MSI-edge eth5-tx-0

120: 8 0 0 0 0 0 0 0 IR-PCI-MSI-edge eth5:lsc

121: 2 0 0 0 0 0 0 0 IR-PCI-MSI-edge eth6-rx-0

122: 2 0 0 0 0 0 0 0 IR-PCI-MSI-edge eth6-tx-0

123: 3 0 0 0 0 0 0 0 IR-PCI-MSI-edge eth6:mbx

124: 1 0 0 0 0 0 0 0 IR-PCI-MSI-edge eth7-rx-0

125: 1 0 0 0 0 0 0 0 IR-PCI-MSI-edge eth7-tx-0

126: 3 0 0 0 0 0 0 0 IR-PCI-MSI-edge eth7:mbx

I have assigned 83:10.0 device to the guest OS

Interrupts on VF

[root@vm1 VM1]# lspci | grep 82599

00:06.0 Ethernet controller: Intel Corporation 82559 Ethernet Controller Virtual Function (rev 01)

[root@vm1 VM1]# cat /proc/interrupts

CPU0

0: 278 IO-APIC-edge timer

1: 199 IO-APIC-edge i8042

2: 0 XT-PIC-XT cascade

4: 2 IO-APIC-edge

8: 0 IO-APIC-edge rtc0

12: 130 IO-APIC-edge i8042

14: 8818 IO-APIC-edge ata_piix

15: 878 IO-APIC-edge ata_piix

34: 459 IO-APIC-fasteoi virtio1

35: 1104 IO-APIC-fasteoi uhci_hcd:usb1, Intel 82801AA-ICH

40: 0 PCI-MSI-edge virtio0-config

41: 8 PCI-MSI-edge virtio0-input

42: 1 PCI-MSI-edge virtio0-output

43: 0 PCI-MSI-edge eth1-rx-0

44: 0 PCI-MSI-edge eth1-tx-0

45: 0 PCI-MSI-edge eth1:mbx

NMI: 0 Non-maskable interrupts

LOC: 36375 Local timer interrupts

SPU: 0 Spurious interrupts

PMI: 0 Performance monitoring interrupts

PND: 0 Performance pending work

RES: 0 Rescheduling interrupts

CAL: 0 Function call interrupts

TLB: 0 TLB shootdowns

TRM: 0 Thermal event interrupts

THR: 0 Threshold APIC interrupts

MCE: 0 Machine check exceptions

MCP: 1 Machine check polls

ERR: 0

MIS: 0

[root@vm1 VM1]# lspci

00:06.0 Ethernet controller: Intel Corporation 82559 Ethernet Controller Virtual Function (rev 01)

Subsystem: Intel Corporation Device 0003

Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+

Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-

Latency: 0

Region 0: Memory at f2040000 (32-bit, non-prefetchable) [size=16K]

Region 3: Memory at f2044000 (32-bit, non-prefetchable) [size=16K]

Capabilities: [40] MSI-X: Enable+ Count=3 Masked-

Vector table: BAR=3 offset=00000000

PBA: BAR=3 offset=00002000

Kernel driver in use: ixgbevf

Kernel modules: ixgbevf

Re: Guest OS enumeration for VF PCIe device for 82599

webadmin@intel.com — Tue, 08 May 2012 21:31:26 GMT

I asked around about your post and heard back from a few developers. In our labs MSI-X capability is displayed in the correct offset of [70]. We do not know why you are seeing something different. Of course there could be something differentwith QEMU or the guest on your setup.

You might consider using RHEL 6.x. The tech brief at http://www.intel.com/content/www/us/en/network-adapters/10-gigabit-network-adapters/ethernet-x520-sr-iov-red-hat-tech-brief.html covers the configuration and this is a configuration that is known to work. Maybe going through the paper you can find something that will help you in your setup. On the other hand, if the issue is related to differences with any part of the distribution, then this paper might not help you.

When you say, "the VF interrupt is not functioning properly," what is happening? Is the VF able to pass traffic? Are you expecting a different number of interrupts to be used? Please explain further.

What is in the output of the 'cat /proc/interrupts` for both the PF and VF?

Mark H