4 Replies Latest reply on Nov 19, 2014 5:27 PM by Sandy_Intel

    Problems with 10Gb NICs and Nexus switches

    gnik

      Hi all,

      We are facing a peculiar problem when connecting particular NICs to particular switches. The NICs will go off-line for 4-5 seconds, in irregular intervals, and then return to service as if nothing had happened. The weird thing is, we see this only on particular combinations of NICs and switches:

       

      - Intel 82599EB, 8086:151c, only work reliably on a Nexus 4900M switch. When used on a Nexus 3064, we get a lot of those:

       

      Sep  2 14:06:16 host kernel: ixgbe 0000:04:00.0: eth0: NIC Link is Down
      Sep  2 14:06:21 host kernel: ixgbe 0000:04:00.0: eth0: NIC Link is Up 10 Gbps, Flow Control: RX/TX
      Sep  2 14:06:21 host kernel: ixgbe 0000:04:00.0: eth0: NIC Link is Down
      Sep  2 14:06:23 host kernel: ixgbe 0000:04:00.0: eth0: NIC Link is Up 10 Gbps, Flow Control: RX/TX

       

      - Intel X540-AT2, 8086:1528, only work reliably on a Nexus 3064, When used on a Nexus 4900M, we observe the above.

       

      All servers in question are running CentOS6. Those that are Dell have all the latest firmware and BIOS updates installed. We even tested the (then) latest Linux drivers, downloaded directly from Intel, over the ones that come with the OS, but it made no difference. Other machines in the mix are HP and custom-built, and all have one or the other NIC card.

       

      Are there any debug tools for the NICs that could be useful here? Are there any particular options that should or should not be set?

       

      Thanks.