6 Replies Latest reply on Oct 15, 2013 1:55 AM by NFN

    How to enable SR-IOV in I350-T2 NIC

    NFN

      Hello,

       

      I purchased from local reseller Intel I350-T2 NIC but it is missing SR-IOV capability.

      According to I350-T2 specification SR-IOV capability should be on-board. How can I enable SR-IOV?

      Is there an option to enable it in EEPROM or replace EEPROM?

       

      pci id of interface:

      8086:1521 (rev 01)  Subsystem: 8086:5002

       

      Thank you

       

      e0thtool -i eth7

      driver: igb

      version: 3.2.10-k

      firmware-version: 1.5-2

      bus-info: 0000:04:00.0

      supports-statistics: yes

      supports-test: yes

      supports-eeprom-access: yes

      supports-register-dump: yes

       

       

      $ sudo lspci -s 04:00.0 -v

      04:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)

              Subsystem: Intel Corporation Ethernet Server Adapter I350-T2

              Flags: bus master, fast devsel, latency 0, IRQ 38

              Memory at df200000 (32-bit, non-prefetchable) [size=1M]

              Memory at df1f8000 (32-bit, non-prefetchable) [size=16K]

              Expansion ROM at df100000 [disabled] [size=512K]

              Capabilities: [40] Power Management version 3

              Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+

              Capabilities: [70] MSI-X: Enable+ Count=10 Masked-

              Capabilities: [a0] Express Endpoint, MSI 00

              Capabilities: [e0] Vital Product Data

              Capabilities: [100] Advanced Error Reporting

              Capabilities: [140] Device Serial Number a0-36-9f-ff-ff-04-80-70

              Capabilities: [150] Alternative Routing-ID Interpretation (ARI)

              Capabilities: [1a0] Transaction Processing Hints

              Capabilities: [1c0] Latency Tolerance Reporting

              Capabilities: [1d0] Access Control Services

              Kernel driver in use: igb

              Kernel modules: igb

       

      More detailed output:

       

       

      $ sudo lspci -s 04:00.0 -vvv -n

      04:00.0 0200: 8086:1521 (rev 01)

              Subsystem: 8086:5002

              Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+

              Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-

              Latency: 0, Cache Line Size: 64 bytes

              Interrupt: pin A routed to IRQ 38

              Region 0: Memory at df200000 (32-bit, non-prefetchable) [size=1M]

              Region 3: Memory at df1f8000 (32-bit, non-prefetchable) [size=16K]

              Expansion ROM at df100000 [disabled] [size=512K]

              Capabilities: [40] Power Management version 3

                      Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)

                      Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=1 PME-

              Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+

                      Address: 0000000000000000  Data: 0000

                      Masking: 00000000  Pending: 00000000

              Capabilities: [70] MSI-X: Enable+ Count=10 Masked-

                      Vector table: BAR=3 offset=00000000

                      PBA: BAR=3 offset=00002000

              Capabilities: [a0] Express (v2) Endpoint, MSI 00

                      DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us

                              ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+

                      DevCtl: Report errors: Correctable- Non-Fatal+ Fatal+ Unsupported+

                              RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ FLReset-

                              MaxPayload 256 bytes, MaxReadReq 512 bytes

                      DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr+ TransPend-

                      LnkCap: Port #0, Speed 5GT/s, Width x4, ASPM L0s L1, Latency L0 <4us, L1 <32us

                              ClockPM- Surprise- LLActRep- BwNot-

                      LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+

                              ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-

                      LnkSta: Speed 5GT/s, Width x4, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-

                      DevCap2: Completion Timeout: Range ABCD, TimeoutDis+

                      DevCtl2: Completion Timeout: 65ms to 210ms, TimeoutDis-

                      LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB

                               Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-

                               Compliance De-emphasis: -6dB

                      LnkSta2: Current De-emphasis Level: -6dB

              Capabilities: [e0] Vital Product Data

                      Product Name: I350 GbE Controller

                      Read-only fields:

                              [V0] Vendor specific: FFV13.0.0

                              [PN] Part number: G15136

                              [MN] Manufacture ID: 31 30 32 38

                              [V1] Vendor specific: DSV1028VPDR.VER1.0

                              [V3] Vendor specific: DTINIC

                              [V4] Vendor specific: DCM1001008FC52101008FC53201008FC54301008FC5

                              [V5] Vendor specific: NPY4

                              [V6] Vendor specific: PMT13

                              [V7] Vendor specific: NMVIntel Corp

                              [RV] Reserved: checksum good, 0 byte(s) reserved

                      End

              Capabilities: [100 v2] Advanced Error Reporting

                      UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-

                      UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt+ UnxCmplt+ RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-

                      UESvrt: DLP+ SDES+ TLP+ FCP+ CmpltTO+ CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC+ UnsupReq- ACSViol-

                      CESta:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+

                      CEMsk:  RxErr+ BadTLP+ BadDLLP+ Rollover+ Timeout+ NonFatalErr+

                      AERCap: First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-

              Capabilities: [140 v1] Device Serial Number a0-36-9f-ff-ff-04-80-70

              Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI)

                      ARICap: MFVC- ACS-, Next Function: 1

                      ARICtl: MFVC- ACS-, Function Group: 0

              Capabilities: [1a0 v1] Transaction Processing Hints

                      Device specific mode supported

                      Steering table in TPH capability structure

              Capabilities: [1c0 v1] Latency Tolerance Reporting

                      Max snoop latency: 0ns

                      Max no snoop latency: 0ns

              Capabilities: [1d0 v1] Access Control Services

                      ACSCap: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-

                      ACSCtl: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-

              Kernel driver in use: igb

              Kernel modules: igb

        • 1. Re: How to enable SR-IOV in I350-T2 NIC
          Patrick_Kutch

          Thanx for posting your question.

           

          That device definitely supports SR-IOV - I've used it myself .

           

          For SR-IOV to function, it requires a number of components to all work together, including the platform BIOS, Chipset, Operating System and the SR-IOV device itself.

           

          If the BIOS or OS doesn't support SR-IOV it will never show up.  Both must suppor the feature for even lspci to show the functionality.  I'd recommend that you check to see if all the other pieces support SR-IOV.

           

          thanx,

           

          Patrick

          • 2. Re: How to enable SR-IOV in I350-T2 NIC
            NFN

            Hello Patrick,

             

            I used Intel 10GB interfaces with SR-IOV capability on same machine running Ubntu 12.04.3 server.

            Now I evaluating i350-T2 on same machine and I double checked BIOS -- SR-IOV global setting is "enable".

            I boot system with "intel_iommu=on" kernel parameter.

            Any idea why i350-T2 device does not report its SR-IOV capability?

            Can I check it in EEPROM?

             

            Thanks for your help,

            Igor

            • 3. Re: How to enable SR-IOV in I350-T2 NIC
              Patrick_Kutch

              There is an additional requirment, one that is not obvous and is very frustrating...

               

              The device - in your case the I350-T2, must be in a slot that support SR-IOV.  Some server designs have some PCIe connecors not connected directly to the Chipset and did not use a PCIe switch that supports SR-IOV.

               

              This doc is for 10G, but explains some of these issues:

              Intel® 10 GbE, PCI-SIG* SR-IOV, and Sharing Spec on Linux*

               

              Try putting the device in a different slot (try the one you used for 10G).  If that doesn't work then please provide more information on the plat form you are using, uncluding Model, Bios Version, driver versions and kernel log and we'll see what we can do.

               

              - Patrick

              • 4. Re: How to enable SR-IOV in I350-T2 NIC
                NFN

                Hello Patrick,

                 

                I have checked other slot -- same result. Both slots are PCIe v 2, x8.

                Platform is Dell PowerEdge R610, BIOS version 6.1.0.

                It is running Ubuntu server 12.04.3 LTS x86_64 with kernel 3.2.0-54-generic.

                Driver: igb version: 3.2.10-k firmware-version: 1.5-2

                Please see attached link to log and debug files.

                Temp Share : debug.tgz | Free temporary file hosting

                 

                Thanks for your help,

                Igor

                • 5. Re: How to enable SR-IOV in I350-T2 NIC
                  Patrick_Kutch

                  Hi Igor,

                   

                  I am assuming you purchased this NIC from Dell?  I've other posts on this.  The bottom-line is that currently (I hear rumors this may be changing) the Intel I350 NICs that Dell sells have SR-IOV disabled.

                   

                  We (Intel) has no control over this.  We make custom versions of our devices for several OEMs, who determine which features they wish to use.  In this case the OEM decided not to initially support SR-IOV on Intel's I350 devices.

                   

                  If you were to purchase a retail version of an I350 you would find that SR-IOV would work fine.  Though that might void your system warranty with you server manufacturer since it wouldn't be officially blessed hardware.

                   

                  I would recommend you contact Dell for advice and instructions.

                   

                  - Patrick

                  1 of 1 people found this helpful
                  • 6. Re: How to enable SR-IOV in I350-T2 NIC
                    NFN

                    I thought OEM have to change the subsystem vendor ID like in other posts on same issue.

                    Ok, I will check with my local reseller regarding this NIC OEM..

                     

                    Thanks a lot Patrick for your help

                    Igor