1 2 Previous Next 16 Replies Latest reply on Nov 16, 2017 7:38 PM by Intel Corporation

    X710-DA4 not respecting Base Processor

    Tilla

      Have a dual socket HP Z840 with NIC attached to CPU1 runing windows server 2016.

       

      Have a X710-DA4 with 22.6 driver installed.

       

      Utilizing 2 ports on NIC, trying to set 1 port to Numa 1, core 0 and Numa 1 core 3.

       

      Using Closest static profile.

       

      Only the first NIC port for Numa 1 core 0 is being "respected.

       

      The second NIC port for Numa 1, core 3 is landing on core 2...

       

      I have tried changing around the second port to be on other cores as well to no avail...

       

      Any idea if there is a bug to numa node base processor core binding?

       

      Each NIC set to max process or 1 and queue 1...

       

      Any suggestions would be appreciated...

        • 1. Re: X710-DA4 not respecting Base Processor
          Intel Corporation
          This message was posted on behalf of Intel Corporation

          Hi Tilla,

             Thank you for posting in Wired Communities. Can you provide the tool use to check on the status of the RSS output or where did you see the NIC port that core 3 is landed on Core 2? If possible can you share the steps on how you configure? 

          Thanks,
          Sharon




              

          • 2. Re: X710-DA4 not respecting Base Processor
            Tilla

            Thank you so much for responding, I appreciate any help that you can offer...

             

            This is a bit long, but I will give you as much info on the physical setup and app as I can.

             

            Physical Hardware Setup:

             

            So I have a 2 physical processor HP Z840.

             

            I have an X710-DA4 in PCIe slot #3, which is "attached" to CPU1.

             

            I have a device that streams heavy UDP data over 1-N of the ports on the X710-DA4.

             

            I want each port of the 710-DA4 to "land" on a different core of CPU1.

             

            Application:

             

            I have an application that reads the streams that I have affinitize to CPU0 (this is to observe the receive load placed on CPU1 by the socket guaranteeing that the load is not from the app).

             

            I then initiate a data stream from the device to my application and observe CPU load on task manager.

             

            NIC Setup, Test #1:

             

            Preferred NUMA Node: 1

            Max Num RSS Processors: 1

            Max Num RSS Queues: 1

            RSS: enabled

            RSS Base Processor Number: 0

            RSS Load Balancing profile: ClosestProcessorStatic

             

             

            NIC Setup, Test #2:

             

            Preferred NUMA Node: 1

            Max Num RSS Processors: 1

            Max Num RSS Queues: 1

            RSS: enabled

            RSS Base Processor Number: 1

            RSS Load Balancing profile: ClosestProcessorStatic

             

             

            Expectations:

             

            In the graphs above, CPU0 is the top 4 graphs, CPU1 is the bottom 4 graphs.

             

            By changing the base processor number, I expected the socket receive CPU load to shift from processor 0 to processor 1 on CPU#1.

             

            Nothing else is running on the machine.

             

            As you can see form the graphs, the recv socket load did not move to CPU1 from CPU0 despite the change in configuration...

             

            Thanks for your time...

            • 3. Re: X710-DA4 not respecting Base Processor
              Tilla

              I have tried many different permutations of configs as well, perhaps there is something about the config that I am not understanding or interpreting properly...

              • 4. Re: X710-DA4 not respecting Base Processor
                Intel Corporation
                This message was posted on behalf of Intel Corporation

                Hi Tilla,

                 Thank you for the detail information. I will check on this then update you.

                Regards,
                Sharon
                 

                • 5. Re: X710-DA4 not respecting Base Processor
                  Intel Corporation
                  This message was posted on behalf of Intel Corporation

                  Hi  Tilla,

                    Can you provide the output of Get-NetAdapterRss for test#1 and test#2 ? Thank you.

                  Regards,
                  Sharon

                  • 6. Re: X710-DA4 not respecting Base Processor
                    Intel Corporation
                    This message was posted on behalf of Intel Corporation

                    Hi Tilla,

                      Please feel free to provide the information requested for me to better investigate. 

                    Thanks,
                    Sharon
                     

                    • 7. Re: X710-DA4 not respecting Base Processor
                      Tilla

                      Sorry for late response, I was on vaca last week...

                       

                      The box has gone to the field, so I do not have direct access to it any more

                       

                      I will see if I can find another box to recreate.

                      • 8. Re: X710-DA4 not respecting Base Processor
                        Intel Corporation
                        This message was posted on behalf of Intel Corporation

                        Hi Tilla,
                         
                           Thank you for the update and hope you had a great vacation. Feel free to update me when you find another system for the setup. 

                        Thanks,
                        Sharon
                         

                        • 9. Re: X710-DA4 not respecting Base Processor
                          Intel Corporation
                          This message was posted on behalf of Intel Corporation

                          Hi Tilla,

                           Please feel free to update me in case you already have the setup.

                          Thanks,
                          Sharon
                           

                          • 10. Re: X710-DA4 not respecting Base Processor
                            Tilla

                            Test #1:

                             

                            Name                                            : 10GB-1-10.1

                            InterfaceDescription                            : Intel(R) Ethernet Converged Network Adapter X710

                            Enabled                                         : True

                            NumberOfReceiveQueues                           : 1

                            Profile                                         : ClosestStatic

                            BaseProcessor: [Group:Number]                   : :0

                            MaxProcessor: [Group:Number]                    : :

                            MaxProcessors                                   : 1

                            RssProcessorArray: [Group:Number/NUMA Distance] :

                            IndirectionTable: [Group:Number]                :

                             

                            Test #2

                             

                            Name                                            : 10GB-1-10.1

                            InterfaceDescription                            : Intel(R) Ethernet Converged Network Adapter X710

                            Enabled                                         : True

                            NumberOfReceiveQueues                           : 1

                            Profile                                         : ClosestStatic

                            BaseProcessor: [Group:Number]                   : :1

                            MaxProcessor: [Group:Number]                    : :

                            MaxProcessors                                   : 1

                            RssProcessorArray: [Group:Number/NUMA Distance] :

                            IndirectionTable: [Group:Number]                :

                            • 11. Re: X710-DA4 not respecting Base Processor
                              Intel Corporation
                              This message was posted on behalf of Intel Corporation

                              Hi Tilla,

                               Thank you for the information. I will further check on this.

                              Regards,
                              Sharon
                               

                              • 12. Re: X710-DA4 not respecting Base Processor
                                Intel Corporation
                                This message was posted on behalf of Intel Corporation

                                Hi Tilla,
                                 
                                Further checking, please set Max Num RSS Processors = 2 and  "RSS Base Processor Number" to 1. 

                                Please feel free to update me. 

                                Thanks,
                                Sharon
                                 

                                • 13. Re: X710-DA4 not respecting Base Processor
                                  Tilla

                                  I could try that, but in essence isnt that saying "use either core 1 or core 2" (base processor=1 and max processors = 2)?

                                   

                                  I desire to be able to pin CPU load for a specific NIC port to a single specific processor in order to efficiently load balance my high network traffic.

                                  • 14. Re: X710-DA4 not respecting Base Processor
                                    Intel Corporation
                                    This message was posted on behalf of Intel Corporation

                                    Hi Tilla,

                                    Thank you for the reply. If you set "Max Num RSS Processors" to 1, it  only allows RSS to use CPU 0.  Setting "Max Num RSS Processors" to 2 and "RSS Base Processor Number" to 1 would force all to CPU 1. 

                                    Max processors = 1 can use only CPU 0

                                    Max processors = 2 can use CPU 0 and 1

                                    Max processors = 3 can use CPU 0,1,2

                                    Max processors = 4 can use CPU 0,1,2,3

                                         Please feel free to update me.

                                    Thanks,
                                    Sharon
                                     

                                    1 2 Previous Next