0 Replies Latest reply on Dec 11, 2012 1:09 AM by ramesh

    Intel 82599 tcpdump problem.

    ramesh

      Hello,

       

      Wonder if anyone here has seen this problem, and have any suggestions. Our setup is for an astronomy research instrument. We have an high performance node with dual 10GbE ports. The two ports are connected to an HP5412zl switch. A custom instrument sends 8208-byte UDP packets at 5.2 Gbps on two more ports of the switch. These UDP packets are tagged with IP addresses 10.0.0.1 and 10.0.0.101.

       

      What I expect to see in my linux box is that the packets with IP address 10.0.0.101 appearing on eth3 when I run 'tcpdump -i eth3' and the stream tagged with 10.0.0.1 appear on eth2. But I see both UDP streams on eth2 and I lose a lot of packets on one of the streams randomly.

       

      My question is: Is the ixgbe driver bonding the two ports on the card? Do I have to use RSS or hardware filtering to bind the streams to a given interface? I am a bit lost on this issue - so suggestions or comments are highly appreciated. Thanks!

       

      I've attached relevant information below.

       

      Cheers,

      Ramesh

       

      ##########

      root@psrfb0:~# /sbin/ifconfig eth2   

      eth2      Link encap:Ethernet  HWaddr 00:1b:21:95:23:a4

                inet addr:10.0.0.1  Bcast:10.0.0.255  Mask:255.255.255.0

                inet6 addr: fe80::21b:21ff:fe95:23a4/64 Scope:Link

                UP BROADCAST RUNNING MULTICAST  MTU:9000  Metric:1

                RX packets:6148 errors:0 dropped:0 overruns:0 frame:0

                TX packets:54 errors:0 dropped:0 overruns:0 carrier:0

                collisions:0 txqueuelen:1000

                RX bytes:393472 (384.2 KiB)  TX bytes:2484 (2.4 KiB)

       

      root@psrfb0:~# /sbin/ifconfig eth3

      eth3      Link encap:Ethernet  HWaddr 00:1b:21:95:23:a5

                inet addr:10.0.0.101  Bcast:10.0.0.255  Mask:255.255.255.0

                inet6 addr: fe80::21b:21ff:fe95:23a5/64 Scope:Link

                UP BROADCAST RUNNING MULTICAST  MTU:9000  Metric:1

                RX packets:6164 errors:0 dropped:0 overruns:0 frame:0

                TX packets:54 errors:0 dropped:0 overruns:0 carrier:0

                collisions:0 txqueuelen:1000

                RX bytes:394496 (385.2 KiB)  TX bytes:2484 (2.4 KiB)

       

       

      ###############

      root@psrfb0:~# lspci | tail -2

      07:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ Network Connection (rev 01)

      07:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ Network Connection (rev 01)

      root@psrfb0:~#

      ###############

      dmesg output (trimmed)

      [6000434.612724] ixgbe: Intel(R) 10 Gigabit PCI Express Network Driver - version 2.0.44-k2

      [6000434.632655] ixgbe: Copyright (c) 1999-2009 Intel Corporation.

      [6000434.652732] ixgbe 0000:07:00.0: PCI INT A -> Link[LNEB] -> GSI 19 (level, low) -> IRQ 19

      [6000434.673421] ixgbe 0000:07:00.0: setting latency timer to 64

      [6000434.814220] ixgbe 0000:07:00.0: irq 28 for MSI/MSI-X

      ....

      ....

      [6000435.209685] ixgbe 0000:07:00.0: irq 52 for MSI/MSI-X

      [6000435.221213] ixgbe: 0000:07:00.0: ixgbe_init_interrupt_scheme: Multiqueue Enabled: Rx Queue count = 12, Tx Queue count = 12

      [6000435.233806] ixgbe 0000:07:00.0: (PCI Express:2.5Gb/s:Width x8) 00:1b:21:95:23:a4

      [6000435.246617] ixgbe 0000:07:00.0: MAC: 2, PHY: 11, SFP+: 5, PBA No: e66560-003

      [6000435.323191] ixgbe 0000:07:00.0: Intel(R) 10 Gigabit Network Connection

      [6000435.335946] ixgbe 0000:07:00.1: PCI INT B -> Link[LNEC] -> GSI 18 (level, low) -> IRQ 18

      [6000435.349022] ixgbe 0000:07:00.1: setting latency timer to 64

      [6000435.482172] ixgbe 0000:07:00.1: irq 53 for MSI/MSI-X

      ....

      ....

      [6000435.717155] ixgbe 0000:07:00.1: irq 77 for MSI/MSI-X

      [6000435.724868] ixgbe: 0000:07:00.1: ixgbe_init_interrupt_scheme: Multiqueue Enabled: Rx Queue count = 12, Tx Queue count = 12

      [6000435.734073] ixgbe 0000:07:00.1: (PCI Express:2.5Gb/s:Width x8) 00:1b:21:95:23:a5

      [6000435.743793] ixgbe 0000:07:00.1: MAC: 2, PHY: 11, SFP+: 5, PBA No: e66560-003

      [6000435.819138] ixgbe 0000:07:00.1: Intel(R) 10 Gigabit Network Connection

      [6000776.126439] ADDRCONF(NETDEV_UP): eth2: link is not ready

      [6000776.139056] ixgbe: eth2: ixgbe_change_mtu: changing MTU from 1500 to 9000

      [6000776.388200] ixgbe: eth2 NIC Link is Up 10 Gbps, Flow Control: None

      [6000776.402248] ADDRCONF(NETDEV_CHANGE): eth2: link becomes ready

      [6000777.978389] ADDRCONF(NETDEV_UP): eth3: link is not ready

      [6000777.992922] ixgbe: eth3: ixgbe_change_mtu: changing MTU from 1500 to 9000

      [6000778.236201] ixgbe: eth3 NIC Link is Up 10 Gbps, Flow Control: None

      [6000778.252047] ADDRCONF(NETDEV_CHANGE): eth3: link becomes ready

      [6000786.988194] eth2: no IPv6 routers present

      [6000788.916195] eth3: no IPv6 routers present