2 Replies Latest reply on Mar 28, 2011 2:48 PM by

    intel X7542 6 cores better than 12?

    allexch

      HI.

      I have supermicro server with :

      • Intel® 7500 (Boxboro-EX) chipset
      • ICH10R

      4 processors with 6 cores in each (Intel(R) Xeon(R) CPU           X7542  @ 2.67GHz)

       

      I installed 2 10G intel NICs(82599EB) and the latest ixgbe driver (v3.2.10).

       

      Yesterday I made a test with pktgen and got some strange results(I generated traffic on server (A) and flooded it through supermicro(B)):

       

      1) While I spread interrupts on B 10G nic among 12 cores (the 1-st and the 2-d processors) I recieved 100% utilization on all 12 cores at rate 5Gbit/s (~1 mln pps)

       

      2) While I spread interrupts on B 10G nic among 8 cores (the 1-st (6cores) and the 2-d(2cores) processors) I recieved 100% utilization on all 8 cores at rate 7Gbit/s (~1 mln pps)

       

      3) While I spread interrupts on B 10G nic among 6 cores (the 1-st (6cores)  processors) I  DID NOT recieved 100% utilization and I could forward 9.5Gbit/s (~1 mln pps)

       

      Of course I have heard that spreading interrupts among different processors could make things worth, but not so dramatically.

       

      I made test on 2.6.37-gentoo-r1.

       

      So can you tell me, how to overcome this problem?

        • 1. Re: intel X7542 6 cores better than 12?
          jbrandeb

          okay, it is good you're running a newish kernel (2.6.37)

           

          Please make sure to just allow the driver to load with driver defaults (no module options)

          rmmod ixgbe

          killall irqbalance

          modprobe ixgbe

          ip ad ad <address/maskbits> dev ethM

          ...

          ip link set ethM up

          ...

          then run the (included in ixgbe-3.2.10.tar.gz) set_irq_affinity.sh ethM ethN ethO ethP script

          run your test

           

          this will spread all interrupts from each port across all cores.  It is probably better to limit the cores in use to the ones directly connected to the I/O controller that is only one hop from your processors, but that is difficult to do currently.

           

          If you want to try the second suggestion, reload the driver like so

           

          rmmod ixgbe

          modprobe ixgbe FdirMode=0,0,0,0 RSS=12,12,12,12

          • 2. Re: intel X7542 6 cores better than 12?

            sorry could u please a basic and interface setting method??

             

            im not and advanced user as im has been first time face this..

             

            tq