1 2 Previous Next 17 Replies Latest reply on Nov 2, 2016 10:04 AM by JacksonM

    Why LnkSta and LnkCtl show different with X710?

    JacksonM

      Hi,

       

      My lspci -vvv output for the X710DA4 Ethernet Adapter shows LnkSta to be 8GT/s but LnkCtl is 2.5GT/s.

       

      Motherboard used is the S2600CW with dual E5-2697Av4 CPU.

       

      Thanks....

        • 1. Re: Why LnkSta and LnkCtl show different with X710?
          Intel Corporation
          This message was posted on behalf of Intel Corporation

          Hi JacksonM,

          Thank you for posting your query in Wired Ethernet Communities.

          X710-DA4 should work according to specs which is PCI-E Gen 3 x8 (8GT/s), as to why the LnkCtl shows 2.5GT/s. Kindly share additional information below for us to further investigate.

          1. ethtool - i output to show the driver version and firmware version of your X710-DA4
          2. BIOS and FW version of your S2600CW
          3. Have you tried to install the X710-DA4 on other PCI-E Gen 3 x8 or x16 slots?
          4. Have you tried installing other PCI-E Gen3 x8 on the same slot on where the X710-DA4 was installed?

          Regards,
          Vince

          • 2. Re: Why LnkSta and LnkCtl show different with X710?
            JacksonM

            Thanks, Vince. I captured a few more output for you to see.

             

            The driver loaded is the i40e.

             

            DPDK output: PMD:
            eth_i40e_dev_init(): FW5.0 API 1.5 NVM 05.00.04 eetrack 800024cd

            (But we have also disabled the DPDK and put driver to kernel. Output from below is without DPDK).

             

            Yes, we have tried plugging into other slots as well.

             

            Motherboard BIOS (Note: I am using the S2600CO board here. And also upgraded my BIOS for the board. The same results with Cottonwood Pass board)

            SE5C600.86B.02.06.0005.080320162159

             

            Ethtool -I output

            driver: i40e

            version: 1.5.16

            firmware-version: 5.04 0x800024cd 0.0.0

            bus-info: 0000:04:00.0

            supports-statistics: yes

            supports-test: yes

            supports-eeprom-access: yes

            supports-register-dump: yes

            supports-priv-flags: yes

             

            Here's the lspci output (LnkSta and LnkCtl2)

                            LnkSta: Speed 8GT/s, Width x8, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-

                            DevCap2: Completion Timeout: Range ABCD, TimeoutDis+, LTR-, OBFF Not Supported

                            DevCtl2: Completion Timeout: 4s to 13s, TimeoutDis-, LTR-, OBFF Disabled

                            LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-

                                     Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-

                                     Compliance De-emphasis: -6dB

                            Kernel driver in use: i40e

                            d0: 01 00 1e 00 00 00 00 00 00 00 00 00 00 00 00 00

             

            It seems that the motherboard and X710 card cannot negotiate the speed. On another system (Advantech motherboard), where I was able to see 8GT/s on LnkCtl2, the first byte of the d0 register is 03.

             

            Hope it is clear. Let me know if you need more details.

             

            Appreciate your help

            Jackson

            • 3. Re: Why LnkSta and LnkCtl show different with X710?
              Intel Corporation
              This message was posted on behalf of Intel Corporation

              Hi Jackson, thanks for sharing the information, i'm currently checking your concern and will update this thread as soon as possible.

              regards,
              Vince

              • 4. Re: Why LnkSta and LnkCtl show different with X710?
                JacksonM

                Hi, Just a bit more info to update:

                 

                I have another system (Advantech ASMB-913) which came up with the Gen3 on LnkCtl2. It's the same Ubuntu OS, same batch of 10G NIC (though not the exact same X710 card).

                 

                Any leads?

                 

                 

                Thanks,
                Jackson

                • 5. Re: Why LnkSta and LnkCtl show different with X710?
                  Intel Corporation
                  This message was posted on behalf of Intel Corporation

                  Hi Jackson, 

                  Thanks for the additional details, we're still investigating your issue. Please share which PCIe slot  the X710 was installed on S2600CO.


                  Regards,
                  Vince

                  • 6. Re: Why LnkSta and LnkCtl show different with X710?
                    JacksonM

                    Hi, Vince:

                     

                    On the S2600CO, it was connected to Slot 5 on the motherboard.

                     

                    Thanks,
                    Jackson

                    • 7. Re: Why LnkSta and LnkCtl show different with X710?
                      Intel Corporation
                      This message was posted on behalf of Intel Corporation

                      Thanks Jackson, have you tried connecting to other slots such as slot 4 for further isolation? are there other add-in cards installed on the S2600CO?

                      regards,
                      Vince

                      • 8. Re: Why LnkSta and LnkCtl show different with X710?
                        JacksonM

                        Yes, Vince, I have installed the X710 card on other slots as well. It was the only card I plugged into the system.

                        • 9. Re: Why LnkSta and LnkCtl show different with X710?
                          Intel Corporation
                          This message was posted on behalf of Intel Corporation

                          Thanks Jackson, have you tried to install one of the supported PCIe* Gen 3 add-in adapters mentioned in the site below to further isolate the issue?

                          http://www.intel.com/content/www/us/en/support/boards-and-kits/000007457.html

                          Under your system's BIOS, Advanced, Processor Configuration, the processor has label of "V1" or "V2"? Example - E52640 V2 @ 2.00GHz

                          Regards,
                          Vince

                          • 10. Re: Why LnkSta and LnkCtl show different with X710?
                            JacksonM

                            Thanks for the reply, Vince.

                             

                             

                            No, I was not aware of this list of PCI-e peripherals.

                             

                            The processors are the v4 Type. It's the E5-2697Av4 with S2600CW-R motherboard. Since the X710DA4 card is also from Intel, I just assume all will come up as Gen3 without issue. They should work as advertised...

                             

                            Also, I have plugged in a Coletto Creek (Intel 8950 QAT) card and it showed up on lspci as Gen2 (5.0GT/s). Again, it was detected as Gen3 x16.

                             

                            This leads me to think maybe it's a motherboard problem? How does the motherboard affect the speed of the PCIe?

                             

                             

                            Thanks,
                            Jackson

                            • 11. Re: Why LnkSta and LnkCtl show different with X710?
                              JacksonM

                              Here's the Coleto Creek output: (Sorry about the misspell earlier)

                              83:00.0 Co-processor: Intel Corporation Coleto Creek PCIe Endpoint

                                      Subsystem: Intel Corporation Device 35c8

                                      Physical Slot: 0-1

                                      Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-

                                      Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-

                                      Latency: 0

                                      Interrupt: pin A routed to IRQ 10

                                      Region 0: Memory at 38007fe00000 (64-bit, prefetchable) [size=512K]

                                      Region 2: Memory at c8000000 (64-bit, non-prefetchable) [size=256K]

                                      Region 4: Memory at c8040000 (64-bit, non-prefetchable) [size=256K]

                                      Capabilities: [b0] MSI: Enable- Count=1/1 Maskable+ 64bit+

                                              Address: 0000000000000000  Data: 0000

                                              Masking: 00000000  Pending: 00000000

                                      Capabilities: [60] MSI-X: Enable- Count=33 Masked-

                                              Vector table: BAR=2 offset=0003b000

                                              PBA: BAR=2 offset=0003b800

                                      Capabilities: [6c] Power Management version 3

                                              Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)

                                              Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-

                                      Capabilities: [74] Express (v2) Endpoint, MSI 00

                                              DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <128ns, L1 <1us

                                                      ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset+

                                              DevCtl: Report errors: Correctable- Non-Fatal+ Fatal+ Unsupported-

                                                      RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+ FLReset-

                                                      MaxPayload 256 bytes, MaxReadReq 512 bytes

                                              DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq+ AuxPwr- TransPend-

                                              LnkCap: Port #0, Speed 5GT/s, Width x16, ASPM L0s, Exit Latency L0s <512ns, L1 unlimited

                                                      ClockPM- Surprise- LLActRep- BwNot-

                                              LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk-

                                                      ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-

                                              LnkSta: Speed 5GT/s, Width x16, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-

                                              DevCap2: Completion Timeout: Range B, TimeoutDis+, LTR-, OBFF Not Supported

                                              DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled

                                              LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-

                                                       Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-

                                                       Compliance De-emphasis: -6dB

                                              LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-

                                                       EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-

                              • 12. Re: Why LnkSta and LnkCtl show different with X710?
                                Intel Corporation
                                This message was posted on behalf of Intel Corporation

                                Thanks Jackson, kindly confirm if I've captured the correct details from your testing.

                                1. S2600CO and X710 - LnkSta = 8GT/s, LnkCtl2 = 2.5GT/s
                                2. S2600CW and X710 - LnkSta = 8GT/s, LnkCtl2 = 2.5GT/s
                                3. S2600CW and QAT 8950 - LnkSta = 5GT/s, LnkCtl2 = 5GT/s
                                4. Advantech ASMB-913 and X710 - LnkSta = 8GT/s, LnkCtl2 = 8GT/s


                                regards,
                                Vince
                                 

                                • 13. Re: Why LnkSta and LnkCtl show different with X710?
                                  JacksonM

                                  Hi, Vince:

                                   

                                  Yes, you have captured the details perfectly.

                                   

                                  Thanks,
                                  Jackson

                                  • 14. Re: Why LnkSta and LnkCtl show different with X710?
                                    JacksonM

                                    Hi, Vince:

                                     

                                    Just to keep updated, I have the PCIe Parameters on the setup page set to both <AUTO> and <Gen3>. Neither change how the "lspci"  reports the state.

                                     

                                    Thanks.

                                    1 2 Previous Next