13 Replies Latest reply on Nov 9, 2016 4:46 PM by Intel Corporation

    Poor UDP performance over ad-hoc network

    AU_yzy0050

      Hi,

       

      I am using iperf3 to send UDP packets over ad-hoc network with Atheros  AR9280 in 2.4GHz.

      The performance is pretty poor, see the following screenshots:

      1.

      The Galileo can only send about 700 packets within 10 sends for ad-hoc network, which is supposed to send much more than this.

       

      2. Then, I tested it through Ethernet, two Galileos.

      The Galileo can send over 50k packets within 10 sends, which demonstrates that Galileo can send and handle this many packets.

       

      We are expecting that Galileo can send at least 500 packets per second over ad-hoc network. However, the performance is quite limited. It cannot achieve what we need.

       

      I have no idea why the performance is so poor. Is it because Galileo's limited hardware capability, wireless network configuration or something else?

      Any ideas are appreciated.

       

      P.S. 1. The two Galileos are vary close, and both are connected with proper antennas, so there shouldn't be much interference between them.

             2. The option -l 12 with iperf3 means the length of a packet. It has been the shortest length.

             3. The default bandwidth is 1M configured for iperf3, it should be more than enough to send 500 or more packets in a second.

       

       

       

      Updated:

      We tested the same scheme with old devices, FreeBSD, Atom x86 32bit, Atheros AR9280.

      Old devices are working fine.

      Old device is powerful than Galileo, but the performance should not be a such difference.

        • 1. Re: Poor UDP performance over ad-hoc network
          FGT

          Hi AU_yzy0050 ,

           

          What happens with larger packets, e.g. 400 or 500?

           

          What Yocto image (and Atheros driver) are you using?

           

          Fernando.

          • 2. Re: Poor UDP performance over ad-hoc network
            AU_yzy0050

            Hi, FGT

             

            I haven't tried larger packets, but tried default packets with the length of 1470 bytes, it was not working as well as old devices. It send about 200 packets within 10 seconds, while old devices could send much more than this number, not exactly remembered the number.

             

            For the image, I am using latest one, BSP 1.2.1, and I believe we are using ath9k driver pre-installed in BSP 1.2.1. Only did minor modifications, i.e. some kernel printings to trace some messages.

             

            Thank you.

            • 3. Re: Poor UDP performance over ad-hoc network
              FGT

              AU_yzy0050 wrote:

               

              Hi, FGT

               

              I haven't tried larger packets, but tried default packets with the length of 1470 bytes, it was not working as well as old devices. It send about 200 packets within 10 seconds, while old devices could send much more than this number, not exactly remembered the number.

               

              For the image, I am using latest one, BSP 1.2.1, and I believe we are using ath9k driver pre-installed in BSP 1.2.1. Only did minor modifications, i.e. some kernel printings to trace some messages.

               

              Thank you.

              What do you mean by "old devices"? If you refer to Atom as an "old device":

              Intel® Atom™ Processor

               

              Then you could try:

              a) Easy experiments, at least for documentation: different packet sizes and compare those with other processors and Ethernet

              b) Given you have already recompiled the kernel maybe you should try with a better network driver.

               

              Fernando.

              • 4. Re: Poor UDP performance over ad-hoc network
                AU_yzy0050

                Hi, FGT

                 

                Yes, by old devices, I mean our devices with Atom Processor N270:  Intel® Atom™ Processor N270 (512K Cache, 1.60 GHz, 533 MHz FSB) Specifications

                In the old devices (FreeBSD), we are also using the same driver.

                 

                And I will try other packet sizes.

                 

                Thank you.

                • 5. Re: Poor UDP performance over ad-hoc network
                  FGT

                  AU_yzy0050 wrote:

                   

                  Hi, FGT

                   

                  Yes, by old devices, I mean our devices with Atom Processor N270: Intel® Atom™ Processor N270 (512K Cache, 1.60 GHz, 533 MHz FSB) Specifications

                  In the old devices (FreeBSD), we are also using the same driver.

                   

                   

                  I see... I know it does not necessarilly helps for the problem you have, but

                  a) At Intel® Quark™ SoC X1000 Series—Get Started , about Intel® Quark™ SoC X1000 Series:

                  Intel® Pentium® processor instruction set architecture (ISA) enables applications to scale from the Intel Quark SoC to platforms based on Intel® Atom™ and Intel® Core™ processors, without recompiling code.

                  b) And comparing Intel® Atom™ Processor N270 (512K Cache, 1.60 GHz, 533 MHz FSB) Specifications with Intel® Quark™ SoC X1000 (16K Cache, 400 MHz) Specifications you are right taking into account Lithography, for example, but taking into account GHz, Quark is 1/4 behind Atom...

                   

                  Anyhow, now that you mention that you are using the same driver maybe there is some other I/O and/or CPU-memory interaction that strongly penalizes the device in the ntel Galileo and not in other platforms... would you please send the details of those old devices (PC? laptop?, memory size?).

                   

                  Fernando.

                  • 6. Re: Poor UDP performance over ad-hoc network
                    Intel Corporation
                    This message was posted on behalf of Intel Corporation

                    Hi AU,

                    Have you been able to keep working on this? Do you have any updates?

                    Regards,
                    -Pablo

                    • 7. Re: Poor UDP performance over ad-hoc network
                      AU_yzy0050

                      Hi, FGT

                       

                       

                      Anyhow, now that you mention that you are using the same driver maybe there is some other I/O and/or CPU-memory interaction that strongly penalizes the device in the ntel Galileo and not in other platforms... would you please send the details of those old devices (PC? laptop?, memory size?).

                       

                      The memory is DDR2 4GB, and we are using SATA3 SSD.

                       

                      As you mentioned I/O writes and reads, I guess it could be the cause. The Galileo now frequently reads and writes kernel messages, so the speed of SD card could be a limit. Anyway, I don't know much about this hardware specification.

                      I will test a pre-compiled binary image by Intel to see if there is much difference.

                       

                      Thank you.

                      • 8. Re: Poor UDP performance over ad-hoc network
                        AU_yzy0050

                        Hi, Pablo,

                         

                        Thanks for your reply.

                        I'm not sure if I/O reads and writes limit the performance, but as far as I understand, once the OS is loaded, kernel messages should be handled in memory instead of I/O reads and writes.

                        Anyway, as I mentioned in the above reply, I will test a pre-compiled binary image by Intel, which does not have frequent reads and writes to kernel messages.

                         

                        Will let you know the results.

                         

                        Thank you.

                        • 9. Re: Poor UDP performance over ad-hoc network
                          AU_yzy0050

                          Hi, Pablo and FGT

                           

                          Just tested the same scheme with a pre-compiled kernel image without any kernel modification, the performance is acceptable, see the below screenshot:

                           

                          Except for our kernel modification, which are some printings, the rest should be the same.

                          In our modification, every packet sent and received will issue some printings in kernel. Therefore, if there are a lot of packets, there will be a lot of printings in kernel.

                           

                          Now my question is: are these kernel messages I/O reads and writes? If so, are they associated with SD card? Frequent SD card reads and writes will decrease performance for sure.

                          Like I said in the previous reply: as far as I understand, once the OS is loaded, kernel messages should be handled in memory. Please let me know if I was wrong.

                           

                          Thank you.

                          • 10. Re: Poor UDP performance over ad-hoc network
                            FGT

                            AU_yzy0050 wrote:

                             

                            Hi, Pablo and FGT

                             

                            Just tested the same scheme with a pre-compiled kernel image without any kernel modification, the performance is acceptable, see the below screenshot:

                             

                            Except for our kernel modification, which are some printings, the rest should be the same.

                            In our modification, every packet sent and received will issue some printings in kernel. Therefore, if there are a lot of packets, there will be a lot of printings in kernel.

                             

                            Now my question is: are these kernel messages I/O reads and writes? If so, are they associated with SD card? Frequent SD card reads and writes will decrease performance for sure.

                            Like I said in the previous reply: as far as I understand, once the OS is loaded, kernel messages should be handled in memory. Please let me know if I was wrong.

                             

                            Thank you.

                            Experiment seems to indicate that's not true (or a kernel expert should explain what really happens, but kernel experts are not usually at hand). I'd sujest you modify your printings, so that you do not print something on each packet... e.g. average values, or store data in some array to print it all at once (you should be careful about array length, though...), or something similar...

                             

                            HTH,

                             

                            Fernando.

                            • 11. Re: Poor UDP performance over ad-hoc network
                              Intel Corporation
                              This message was posted on behalf of Intel Corporation

                              Hi AU,

                              I’m wondering if you have updates on this case.

                              Regards,
                              -Pablo

                              • 12. Re: Poor UDP performance over ad-hoc network
                                AU_yzy0050

                                So far, no updates. The issue could be from too many I/O writes and reads.

                                Will update later on if possible.

                                 

                                Thanks

                                • 13. Re: Poor UDP performance over ad-hoc network
                                  Intel Corporation
                                  This message was posted on behalf of Intel Corporation

                                  Hi Au, 

                                  Were you able to run some more tests? 

                                  Regards, 
                                  -Pablo