1 2 Previous Next 16 Replies Latest reply on Jul 20, 2016 11:03 PM by taira

    82599EN repeats Link Up and Down

    taira

      Hi all

       

      We use ixgbe driver for 82599EN on linux. driver version is 4.1.2

      diriver is downloaded at https://downloadcenter.intel.com/download/25463/Network-Adapter-Driver-for-PCI-E-Intel-10-Gigabit-Ethernet-Network-Connections-under-Linux-.

       

      [test condition]:

        OS:Linux kernel 3.10.31-ltsi

        driver:Network Adapter Driver for PCI-E* Intel® 10 Gigabit Ethernet Network Connections under Linux* ver4.1.2

        device:Intel 82599EN

        PHY I/F:SFP module(FINISAR FTLX8573D3BTL) + MMF Fiber cable

        EEPROM data:82599EN_SFI_NO_MNG_4.40.bin(modiy MAC address)

      [steps]:

        1.insmod mdio.ko

        2.insmod ixgbe.ko

        3.ifconfig eth2 up

        4.ifconfig eth2 192.168.2.100

        5.connect MMF cable to 10GbE Tester(SPIRENT C1).

       

      [result]

      <step2 result>

      ------------------------------------------------------------------------------

      root@socfpga:/usr/fsim# insmod ixgbe.ko
      ixgbe: Intel(R) 10 Gigabit PCI Express Network Driver - version 3.13.10-k
      ixgbe: Copyright (c) 1999-2013 Intel Corporation.
      ixgbe_probe
      PCI: enabling device 0000:01:00.0 (0140 -> 0142)
      PCI resource c0000000 + 00020000
      ixgbe 0000:01:00.0: Multiqueue Disabled: Rx Queue count = 1, Tx Queue count = 1
      ixgbe 0000:01:00.0: (PCI Express:5.0GT/s:Width x4) 00:a0:c9:12:34:56
      ixgbe 0000:01:00.0: MAC: 2, PHY: 12, SFP+: 5, PBA No: FFFFFF-0FF
      ixgbe 0000:01:00.0: PCI-Express bandwidth available for this card is not sufficient for optimal performance.
      ixgbe 0000:01:00.0: For optimal performance a x8 PCI-Express slot is required.
      ixgbe 0000:01:00.0: Intel(R) 10 Gigabit Network Connection
      platform leds.7: Driver leds-gpio requests probe deferral

      ---------------------------------------------------------------------------------

       

      <result after step4>

      -----------------------------------------------------------------------------------------

      root@socfpga:/usr/fsim# ifconfig eth2
      eth2      Link encap:Ethernet  HWaddr 00:a0:c9:12:34:56
                inet addr:192.168.2.100  Bcast:192.168.2.255  Mask:255.255.255.0
                UP BROADCAST MULTICAST  MTU:1500  Metric:1
                RX packets:0 errors:0 dropped:0 overruns:0 frame:0
                TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
                collisions:0 txqueuelen:1000
                RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

      ------------------------------------------------------------------------------------------

       

      <when MMF cable is connected to Tester>

      ---------------------------------------------------------------------------------------------

      root@socfpga:/usr/fsim# ixgbe 0000:01:00.0 eth2: NIC Link is Up 10 Gbps, Flow Control: RX/TX
      ixgbe 0000:01:00.0 eth2: NIC Link is Down
      ixgbe 0000:01:00.0 eth2: NIC Link is Up 10 Gbps, Flow Control: RX/TX
      ixgbe 0000:01:00.0 eth2: NIC Link is Down
      ixgbe 0000:01:00.0 eth2: NIC Link is Up 10 Gbps, Flow Control: RX/TX
      ixgbe 0000:01:00.0 eth2: NIC Link is Down
      ixgbe 0000:01:00.0 eth2: NIC Link is Up 10 Gbps, Flow Control: RX/TX
      ixgbe 0000:01:00.0 eth2: NIC Link is Down
      ixgbe 0000:01:00.0 eth2: NIC Link is Up 10 Gbps, Flow Control: RX/TX
      ixgbe 0000:01:00.0 eth2: NIC Link is Down

      --------------------------------------------------------------------------------------------------

       

      Our issue is that link up and down is repeated. How should we resolve this issue ?

       

      BR,

      Taira Suzuki

        • 1. Re: 82599EN repeats Link Up and Down
          Intel Corporation
          This message was posted on behalf of Intel Corporation
          • 2. Re: 82599EN repeats Link Up and Down
            taira

            Hi wb

             

            I tried to use ixgbe ver4.3.15. But the issue is occured again.

            I can not resolve this issue. eth2(target eth) status is as below.

             

            [staus result]

            root@socfpga:/usr/fsim# ethtool eth2

            Settings for eth2:

                    Supported ports: [ FIBRE ]

                    Supported link modes:   10000baseT/Full

                    Supported pause frame use: No

                    Supports auto-negotiation: No

                    Advertised link modes:  10000baseT/Full

                    Advertised pause frame use: Symmetric

                    Advertised auto-negotiation: No

                    Speed: Unknown!

                    Duplex: Unknown! (255)

                    Port: FIBRE

                    PHYAD: 0

                    Transceiver: external

                    Auto-negotiation: off

                    Supports Wake-on: d

                    Wake-on: d

                    Current message level: 0x00000007 (7)

                                           drv probe link

                    Link detected: no

            root@socfpga:/usr/fsim# ethtool -i eth2

            driver: ixgbe

            version: 4.3.15

            firmware-version: 0x80000707

            bus-info: 0000:01:00.0

            supports-statistics: yes

            supports-test: yes

            supports-eeprom-access: yes

            supports-register-dump: yes

            supports-priv-flags: no

            root@socfpga:/usr/fsim#

             

            I doubt that " Supports auto-negotiation: No" is a reason for repeating link up and down.

            Could you tell a next step to resolve ?

             

            BR,

            Taira

            • 3. Re: 82599EN repeats Link Up and Down
              Intel Corporation
              This message was posted on behalf of Intel Corporation

              Hi Taira,

               Thank you for the update. Let me further check on this issue.

              rgds,
              wb
               

              • 4. Re: 82599EN repeats Link Up and Down
                Intel Corporation
                This message was posted on behalf of Intel Corporation

                Hi Taira,

                  
                Checking on the fiber module used, FINISAR FTLX8573D3BTL is not a supported fiber module for Intel X520 network adapter. Here is the website for reference: 

                http://www.intel.com/content/www/us/en/support/network-and-i-o/ethernet-products/000005688.html

                 If you are using Intel X520 server adapter, you need to check on the supported fiber module. If not, then please confirm the following information:

                1) Are you using an onboard NIC? If yes, what is the brand and model of your system?
                2) If it is onboard NIC, have you contacted the system vendor to check with them?
                3) With regards to the autonegotiation, as stated on our README.txt 
                https://downloadmirror.intel.com/13663/eng/README.txt

                "An Intel(R) Network Adapter using fiber-based connections, however, will not
                attempt to auto-negotiate with its link partner since those adapters operate
                only in full duplex and only at their native speed."

                4) Have you try testing on another switch to isolate the issue?


                   Please feel free to update me.

                Thanks,
                wb
                   

                • 5. Re: 82599EN repeats Link Up and Down
                  taira

                  Hi wb

                   

                  I answer your questions.

                  1) Are you using an onboard NIC? If yes, what is the brand and model of your system?

                   

                  No, I use 82599EN for our PCIe exntended board. This PCI extended board is connected to mother board via PCI slot.

                   

                  2) If it is onboard NIC, have you contacted the system vendor to check with them?

                   

                  It is not  onboard NIC.

                   

                  3) With regards to the autonegotiation, as stated on our README.txt

                   

                  I understand that  fiber-based connections can not be auto-negotiated.

                   

                   

                  4) Have you try testing on another switch to isolate the issue?

                   

                  No, but I used a tester(SPIRENT C1). The SFP modules and cable which I use can be linked between tester port A and tester port B.

                   

                  Regurding to SFP module, I refer the site which you direct.

                  According to the site, FTLX8571D3BCV-IT works with 82599. So I order FTLX8571D3BCV without -IT.

                  When I will get FTLX8571D3BCV, I will try and update soon.

                   

                  BR,

                  Taira

                  • 6. Re: 82599EN repeats Link Up and Down
                    Intel Corporation
                    This message was posted on behalf of Intel Corporation

                    Hi Taira,

                     Thank you for the clarification. Please update me the test result of using FTLX8571D3BCV fiber module.

                     Looking forward to your update.

                    rgds,
                    wb
                     

                    • 7. Re: 82599EN repeats Link Up and Down
                      taira

                      Hi! wb

                       

                      I tried FTLX8571D3BCV SFP module. But the issue can not be solved.

                      Same messages appear on linux console.

                       

                      If you have another method for effecive debug, Please tell.

                       

                      BR,

                      Taira

                      • 8. Re: 82599EN repeats Link Up and Down
                        Intel Corporation
                        This message was posted on behalf of Intel Corporation

                        Hi Taira,

                           Thank you for the update, can you provide me the serial number of your X520 server adapter? You can locate the serial number on the network adapter's white sticker (NOT on the box).
                        Please provide me the number that has the format as follow:

                        12 alphanumeric + 5 digits + 6 digits -3 digits 

                        Thanks,
                        wb


                         

                        • 9. Re: 82599EN repeats Link Up and Down
                          taira

                          Hi! wb

                           

                          No, We use 82599EN on the board which is designed by ourself.

                          This board is connected to mother board via PCI-e.

                          ixgbe driver runs on Linux with ALTERA FPGA(ArriaV SoC).

                           

                          We performed ethtool -t eth2. The result is below.

                          The test result is FAIL

                          The test extra info:

                          Register test  (offline)         0

                          Eeprom test    (offline)         0

                          Interrupt test (offline)         0

                          Loopback test  (offline)         0

                          Link test   (on/offline)         1

                           

                          Link test is failed. What should we do for preventing to fail link test ?

                           

                          BR,

                          Taira

                          • 10. Re: 82599EN repeats Link Up and Down
                            Intel Corporation
                            This message was posted on behalf of Intel Corporation

                            Hi Taira,

                             Thank you for the clarification. As this is an ethernet controller used on the board designed by your company, which is a setup involves third party board. Please contact your Intel Field Application engineer for the support or contact the point of purchase .

                            Thanks,
                            wb

                            • 11. Re: 82599EN repeats Link Up and Down
                              taira

                              Hi wb

                               

                              Sorry for generating another thread.

                              I contacted  to Inetl Japan and local distributor.

                              But they can not supprt my issue. Because they do not have experience to  support  a device which is implemented on non-Intel board.

                              So  I wish you to help my issue.

                               

                              I constructed test enviroment for 1Gb ether link as below,

                              *[82599EN]-->[FNISAR FTLX8571D3BCV(1G/10G)]-->[Fiber cable]-->[FNISAR FTLX8571D3BCV(1G/10G)]-->[Tester(SPIRENT)]

                              82599EN can link up on this enviroment.

                               

                              And next,I constructed test enviroment for 10Gb ether link as below,

                              *[82599EN]-->[FNISAR FTLX8573D3BTL(10G)]-->[Fiber cable]-->[FNISAR FTLX8573D3BTL(10G)]-->[Tester(SPIRENT)]

                              The issue(link up/down) is occured on this enviroment.

                               

                              [Detail]

                              When I run an ixgbe dirver and enter "ifconfig eth2 up", link up and down are displaied repeatly in linux console.

                              Then, I read the Link Status Register — LINKS (0x042A4; RO).

                              read value is 0x79081113=0111_1001_0000_1000_0001_0001_0001_0011.

                               

                              I organize above value to the register field of LINKS.

                               

                              [11:8]KX4_SIG_DET     0001b

                              Signal Detect of 10 GbE Parallel (KX4, CX4 or XAUI).Bit[11, 10, 9, 8] shows lane <3,2,1,0> status, respectively.

                              For each bit:0b = A signal is not present (failed). 1b = A signal is present (good).

                               

                              [12]KR_SIG_DET          1b

                              Signal Detect of 10 GbE serial (KR or SFI)  success.

                               

                              [17]10G Align Status     0b

                              10 GbE Align_status failed (deskew process not complete).

                               

                              I change SFP module  to  FINISAR FTLX8571D3BCV. FTLX8571D3BCV can work  for 1GbE or 10GbE.

                              When I connect to 1GbE tester via this SFP module, 82599EN can keep a link up status.

                               

                              If I should confirm the some registers, please tell.

                              Could you estimate causes which occur link up/down repeatly ?

                               

                              BR,

                              Taira

                              • 12. Re: 82599EN repeats Link Up and Down
                                Intel Corporation
                                This message was posted on behalf of Intel Corporation

                                HiTaira,

                                 Thank you for the update and taking time to check. Let me further check for you.

                                rgds,
                                wb
                                 

                                • 13. Re: 82599EN repeats Link Up and Down
                                  taira

                                  Hi wb

                                   

                                  Thank you for your support continuously.

                                   

                                  I notice that the register "LINKS" has  strange  value.

                                  bit 30 "Link up" is 1b which means "Link is up."

                                  bit 7 "Link Status is 0b which means "Link is currently down or link was down since last time read."

                                   

                                  I read register value as below, when the issue was occured.

                                  reg[00004200] = 80000008

                                  reg[00004208] = 0204000e

                                  reg[0000420c] = 0000000e

                                  reg[00004218] = 000001a0

                                  reg[0000421c] = 00000000

                                  reg[00004220] = 00000000

                                  reg[00004224] = 00000000

                                  reg[00004240] = 08002fff

                                  reg[00004244] = 00000381

                                  reg[00004248] = 0000ffff

                                  reg[0000425c] = 10000000

                                  reg[00004260] = 00000000

                                  reg[00004268] = 05ee0000

                                  reg[00004288] = 00000080

                                  reg[0000428c] = 00008400

                                  reg[00004290] = 04000000

                                  reg[00004298] = 1b1b0000

                                  reg[0000429c] = 461f02f0

                                  reg[000042a0] = c09c6084

                                  reg[000042a4] = 79081113

                                  reg[00004324] = 00000012

                                  reg[000042a8] = 000a0000

                                  reg[000042b0] = 00090000

                                  reg[000042b4] = 00000000

                                  reg[000042d0] = 00000000

                                  reg[000042d4] = 00000000

                                  reg[000042d8] = 00000000

                                  reg[000042e0] = 10010000

                                  reg[000042e4] = 822ffff8

                                  reg[000042e8] = 00000000

                                  reg[000042ec] = 00000000

                                  reg[00014f00] = 0000006d

                                  reg[00014f10] = 00000000

                                  reg[00004294] = 0000000a

                                  reg[00004314] = 00020000

                                  reg[00004000] = 00000000

                                  reg[00004004] = 00000000

                                  reg[00004008] = 00000000

                                  reg[00004034] = 00000013

                                  reg[00004038] = 0010e891

                                  reg[00004040] = 00000000

                                  reg[00008780] = 00000000

                                  reg[000041a4] = 00000000

                                  reg[000041a8] = 00000000

                                  reg[00004140] = 00000000

                                  reg[00004144] = 00000000

                                  reg[00004148] = 00000000

                                  reg[0000414c] = 00000000

                                  reg[00004150] = 00000000

                                  reg[00004154] = 00000000

                                  reg[00004158] = 00000000

                                  reg[0000415c] = 00000000

                                  reg[00004160] = 00000000

                                  reg[00004164] = 00000000

                                  reg[00004168] = 00000000

                                  reg[0000416c] = 00000000

                                  reg[00004170] = 00000000

                                  reg[00004174] = 00000000

                                  reg[00004178] = 00000000

                                  reg[0000417c] = 00000000

                                  reg[0000405c] = 00000000

                                  reg[00004060] = 00000000

                                  reg[00004064] = 00000000

                                  reg[00004068] = 00000000

                                  reg[0000406c] = 00000000

                                  reg[00004070] = 00000000

                                  reg[00004078] = 00000000

                                  reg[0000407c] = 00000000

                                  reg[00004074] = 00000000

                                  reg[00004088] = 00000000

                                  reg[0000408c] = 00000000

                                  reg[000041b0] = 00000000

                                  reg[000041b4] = 00000000

                                  reg[000041b8] = 00000000

                                  reg[00002f50] = 00000000

                                  reg[00002f54] = 00000000

                                  reg[00002f58] = 00000000

                                  reg[00002f5c] = 00000000

                                  reg[00002f60] = 00000000

                                  reg[00002f64] = 00000000

                                  reg[00002f68] = 00000000

                                  reg[00002f6c] = 00000000

                                  reg[00002f70] = 00000000

                                  reg[00002f74] = 00000000

                                  reg[00002f78] = 00000000

                                  reg[00002f7c] = 00000000

                                  reg[00004080] = 00000000

                                  reg[00004090] = 00000000

                                  reg[00004094] = 00000000

                                  reg[000087a0] = 00000000

                                  reg[000087a4] = 00000000

                                  reg[000087a8] = 00000000

                                  reg[000040a4] = 00000000

                                  reg[000040a8] = 00000000

                                  reg[000040ac] = 00000000

                                  reg[000040b0] = 00000000

                                  reg[000040b4] = 00000000

                                  reg[000040b8] = 00000000

                                  reg[000040c0] = 00000000

                                  reg[000040c4] = 00000000

                                  reg[000040d0] = 00000000

                                  reg[000040d4] = 00000000

                                  reg[000040d8] = 00000000

                                  reg[000040dc] = 00000000

                                  reg[000040e0] = 00000000

                                  reg[000040e4] = 00000000

                                  reg[000040e8] = 00000000

                                  reg[000040ec] = 00000000

                                  reg[000040f0] = 00000000

                                  reg[000040f4] = 00000000

                                  reg[00004010] = 00000000

                                  reg[00004120] = 00000000

                                  reg[00002300] = 00000000

                                  reg[00002304] = 00000000

                                  reg[00002308] = 00000000

                                  reg[0000230c] = 00000000

                                  reg[00002310] = 00000000

                                  reg[00002314] = 00000000

                                  reg[00002318] = 00000000

                                  reg[0000231c] = 00000000

                                  reg[00002320] = 00000000

                                  reg[00002324] = 00000000

                                  reg[00002328] = 00000000

                                  reg[0000232c] = 00000000

                                  reg[00002330] = 00000000

                                  reg[00002334] = 00000000

                                  reg[00002338] = 00000000

                                  reg[0000233c] = 00000000

                                  reg[00002340] = 00000000

                                  reg[00002344] = 00000000

                                  reg[00002348] = 00000000

                                  reg[0000234c] = 00000000

                                  reg[00002350] = 00000000

                                  reg[00002354] = 00000000

                                  reg[00002358] = 00000000

                                  reg[0000235c] = 00000000

                                  reg[00002360] = 00000000

                                  reg[00002364] = 00000000

                                  reg[00002368] = 00000000

                                  reg[0000236c] = 00000000

                                  reg[00002f40] = 00000000

                                  reg[00005118] = 00000000

                                  reg[0000241c] = 00000000

                                  reg[00002424] = 00000000

                                  reg[00002428] = 00000000

                                  reg[0000242c] = 00000000

                                  reg[00008784] = 00000000

                                  reg[00008788] = 00000000

                                   

                                  I hope that above result is useful for yur analysis.

                                   

                                  BR,

                                  Taira

                                  • 14. Re: 82599EN repeats Link Up and Down
                                    Intel Corporation
                                    This message was posted on behalf of Intel Corporation

                                    Hi Taira ,

                                     Just want to clarify, I noticed that you mentioend. 
                                    "I change SFP module to FINISAR FTLX8571D3BCV. FTLX8571D3BCV can work for 1GbE or 10GbE. 
                                    When I connect to 1GbE tester via this SFP module, 82599EN can keep a link up status. "

                                      Are you saying that you are able to keep the link up status with FINISAR FTLX8271D38CV?

                                      If yes, then please continue used the supported fiber modules.
                                      
                                      You may also tried newer driver package 4.4.6 available at sourcefore

                                    https://sourceforge.net/projects/e1000/files/ixgbe%20stable/ 



                                    Thanks,
                                    wb
                                     

                                    1 2 Previous Next