13 Replies Latest reply on Dec 1, 2015 11:07 PM by wb_Intel

    FreeBSD and Intel XL710 10G

    John_obn

      Hi all,

      I've installed FreeBSD 10.2-STABLE on a server with 2*CPU E5-2643v3 (with HyperThreading on) and Intel XL710 4*10G SFP+ card.

      At first I updated FreeBSD drivers to 1.4.0 (from download.intel.com).

      I see next strange thing - Every acive ixl interface creates 24 queues (6 core *2 (HT) *2 CPU) but uses only 16-17 of them:

      irq284: ixl0:q0                164383663       1941

      irq285: ixl0:q1                371238730       4384

      irq286: ixl0:q2                378286557       4468

      irq287: ixl0:q3                365073427       4312

      irq288: ixl0:q4                371116376       4383

      irq289: ixl0:q5                372589584       4400

      irq290: ixl0:q6                361879025       4274

      irq291: ixl0:q7                354607200       4188

      irq292: ixl0:q8                223602267       2641

      irq293: ixl0:q9                199067474       2351

      irq294: ixl0:q10               212598000       2511

      irq295: ixl0:q11               202534854       2392

      irq296: ixl0:q12               212050675       2504

      irq297: ixl0:q13               209106917       2469

      irq298: ixl0:q14               201452403       2379

      irq299: ixl0:q15               203896634       2408

      irq300: ixl0:q16                76328643        901

      irq301: ixl0:q17                    6030          0

      irq302: ixl0:q18                    5433          0

      irq303: ixl0:q19                    6804          0

      irq304: ixl0:q20                    6098          0

      irq305: ixl0:q21                    6603          0

      irq306: ixl0:q22                    6476          0

      irq307: ixl0:q23                    7141          0

      irq309: ixl1:q0                161169757       1903

      irq310: ixl1:q1                402042077       4748

      irq311: ixl1:q2                399166615       4714

      irq312: ixl1:q3                389702886       4602

      irq313: ixl1:q4                383371508       4528

      irq314: ixl1:q5                388621686       4590

      irq315: ixl1:q6                385533771       4553

      irq316: ixl1:q7                390478220       4612

      irq317: ixl1:q8                232313544       2743

      irq318: ixl1:q9                248387076       2933

      irq319: ixl1:q10               233942388       2763

      irq320: ixl1:q11               237794942       2808

      irq321: ixl1:q12               227292626       2684

      irq322: ixl1:q13               222151566       2623

      irq323: ixl1:q14               234209020       2766

      irq324: ixl1:q15               217878026       2573

      irq325: ixl1:q16                80177041        947

      irq326: ixl1:q17                      83          0

      irq327: ixl1:q18                      74          0

      irq328: ixl1:q19                     201          0

      irq329: ixl1:q20                      98          0

      irq330: ixl1:q21                      95          0

      irq331: ixl1:q22                      91          0

      irq332: ixl1:q23                      87          0

       

      # top -aSCHP

       

       

      last pid: 28661;  load averages:  7.06,  6.35,  6.23                                                                                                                                                                      up 0+23:35:07  17:15:12

      391 processes: 31 running, 215 sleeping, 145 waiting

      CPU 0:   0.0% user,  0.0% nice,  0.0% system, 39.4% interrupt, 60.6% idle

      CPU 1:   0.0% user,  0.0% nice,  0.0% system, 48.8% interrupt, 51.2% idle

      CPU 2:   0.0% user,  0.0% nice,  0.0% system, 42.1% interrupt, 57.9% idle

      CPU 3:   0.0% user,  0.0% nice,  0.0% system, 40.2% interrupt, 59.8% idle

      CPU 4:   0.0% user,  0.0% nice,  0.4% system, 41.3% interrupt, 58.3% idle

      CPU 5:   0.0% user,  0.0% nice,  0.0% system, 37.0% interrupt, 63.0% idle

      CPU 6:   0.0% user,  0.0% nice,  0.0% system, 35.8% interrupt, 64.2% idle

      CPU 7:   0.0% user,  0.0% nice,  0.0% system, 39.0% interrupt, 61.0% idle

      CPU 8:   0.0% user,  0.0% nice,  0.0% system, 22.0% interrupt, 78.0% idle

      CPU 9:   0.0% user,  0.0% nice,  0.0% system, 26.0% interrupt, 74.0% idle

      CPU 10:  0.0% user,  0.0% nice,  0.0% system, 17.7% interrupt, 82.3% idle

      CPU 11:  0.0% user,  0.0% nice,  0.0% system, 19.3% interrupt, 80.7% idle

      CPU 12:  0.0% user,  0.0% nice,  0.4% system, 25.2% interrupt, 74.4% idle

      CPU 13:  0.0% user,  0.0% nice,  0.0% system, 23.6% interrupt, 76.4% idle

      CPU 14:  0.0% user,  0.0% nice,  0.0% system, 22.4% interrupt, 77.6% idle

      CPU 15:  0.0% user,  0.0% nice,  0.0% system, 26.8% interrupt, 73.2% idle

      CPU 16:  0.0% user,  0.0% nice,  1.2% system,  1.6% interrupt, 97.2% idle

      CPU 17:  0.0% user,  0.0% nice,  0.8% system,  0.0% interrupt, 99.2% idle

      CPU 18:  0.0% user,  0.0% nice,  1.2% system,  0.0% interrupt, 98.8% idle

      CPU 19:  0.0% user,  0.0% nice,  0.4% system,  0.0% interrupt, 99.6% idle

      CPU 20:  0.0% user,  0.0% nice,  3.5% system,  0.0% interrupt, 96.5% idle

      CPU 21:  0.0% user,  0.0% nice,  1.6% system,  0.0% interrupt, 98.4% idle

      CPU 22:  0.0% user,  0.0% nice,  0.8% system,  0.0% interrupt, 99.2% idle

      CPU 23:  0.0% user,  0.0% nice,  2.0% system,  0.0% interrupt, 98.0% idle

       

      # netstat -I ixl0 -w1 -h

                  input           ixl0           output

         packets  errs idrops      bytes    packets  errs      bytes colls

            235K     0     0       126M       300K     0       321M     0

            233K     0     0       114M       297K     0       312M     0

            232K     0     0       116M       300K     0       315M     0

            227K     0     0       108M       297K     0       316M     0

       

      Where is the source of problem that network adapter doesn't utilize all queues ?