9 Replies Latest reply on Dec 7, 2015 12:07 AM by qq19773585

    Uninterruptible sleep during Edison SPI

    jbromley

      Hello,

       

      I am having an SPI problem on the Edison. I have found that trying to do multiple rapid-fire SPI transfers of more than 32 bytes will eventually result in the process going into an uninterruptible sleep state while doing the ioctl call to do the SPI transfer. I have attached a program that allows the problem to be demonstrated. I have modified the spidev module spidev_test.c sample to allow multiple transfers of a specified number of bytes. When I run this program as ./spidev_test -n 64 -r 96 (that's 96 transfers of 64 bytes each), I can always induce the uninterruptible sleep state. Eventually the kernel will detect that the process is hung and produces the following call trace (in dmesg):

       

      [  840.594159] spidev_test     D c18b49c6  7000   368    242 0x00000000

      [  840.594201]  f5053dd8 00000086 f5053d6c c18b49c6 f667c2c0 f5053d94 c126ee63 c1cd7940

      [  840.594261]  f5052000 f6c83d30 f73fd940 f54dac80 f667c2c0 f5d83b44 f6c0e000 f5053da0

      [  840.594316]  c126ef4f f5cc8c00 f5053da8 c1257dae f5053dc0 c125938e f6c0e000 f5d43c00

      [  840.594373] Call Trace:

      [  840.594415]  [<c18b49c6>] ? _raw_spin_unlock_irqrestore+0x26/0x50

      [  840.594448]  [<c126ee63>] ? try_to_wake_up+0x173/0x240

      [  840.594478]  [<c126ef4f>] ? wake_up_process+0x1f/0x40

      [  840.594505]  [<c1257dae>] ? wake_up_worker+0x1e/0x30

      [  840.594531]  [<c125938e>] ? insert_work+0x4e/0x80

      [  840.594557]  [<c18b4947>] ? _raw_spin_unlock+0x17/0x40

      [  840.594583]  [<c12598cb>] ? __queue_work+0x10b/0x320

      [  840.594610]  [<c18b3c23>] schedule+0x23/0x60

      [  840.594638]  [<c18b1305>] schedule_timeout+0x165/0x290

      [  840.594671]  [<c18b8125>] ? sub_preempt_count+0x55/0xe0

      [  840.594699]  [<c18b2bd3>] wait_for_completion+0xa3/0xe0

      [  840.594727]  [<c126ef90>] ? wake_up_state+0x20/0x20

      [  840.594757]  [<c15a58c0>] spidev_sync+0x80/0xb0

      [  840.594787]  [<c15a61e9>] spidev_ioctl+0x619/0x6a0

      [  840.594815]  [<c18b805d>] ? add_preempt_count+0x7d/0xf0

      [  840.594846]  [<c15a5990>] ? spidev_write+0xa0/0xa0

      [  840.594875]  [<c15a5bd0>] ? spidev_probe+0x180/0x180

      [  840.594902]  [<c132a836>] do_vfs_ioctl+0x2f6/0x540

      [  840.594933]  [<c146d53a>] ? inode_has_perm.isra.41.constprop.78+0x3a/0x50

      [  840.594962]  [<c146d5d7>] ? file_has_perm+0x87/0x90

      [  840.594992]  [<c1263140>] ? hrtimer_get_res+0x40/0x40

      [  840.595020]  [<c146d99c>] ? selinux_file_ioctl+0x4c/0xf0

      [  840.595046]  [<c132aae0>] SyS_ioctl+0x60/0x80

      [  840.595075]  [<c18b5038>] syscall_call+0x7/0xb

       

      Unfortunately, the device I am trying to use requires numerous transfers in its interface, so I cannot rearrange the transfers into one large transfer. Is this a known problem? Is there a workaround?

       

      Thanks,

      Jay Bromley

        • 1. Re: Uninterruptible sleep during Edison SPI
          Intel_Peter

          Hello jbromley,

           

          I just tested your code and I believe I'm got the same behavior. However I didn't get the same error message, I got this:

           

          [ 2880.326243] INFO: task spidev_test:397 blocked for more than 120 seconds.
          [ 2880.326338] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
          [ 3000.325423] INFO: task spidev_test:397 blocked for more than 120 seconds.
          [ 3000.325519] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
          [ 3120.320661] INFO: task spidev_test:397 blocked for more than 120 seconds.
          [ 3120.320754] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
          [ 3240.327690] INFO: task spidev_test:397 blocked for more than 120 seconds.
          [ 3240.327785] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
          [ 3360.336382] INFO: task spidev_test:397 blocked for more than 120 seconds.
          [ 3360.336477] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
          [ 3480.340076] INFO: task spidev_test:397 blocked for more than 120 seconds.
          [ 3480.340170] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
          [ 3600.340587] INFO: task spidev_test:397 blocked for more than 120 seconds.
          [ 3600.340683] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
          [ 3720.339292] INFO: task spidev_test:397 blocked for more than 120 seconds.
          [ 3720.339387] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
          
          

           

          Let me take this a closer look at this. I will get back to you as soon as possible.

           

          Peter.

          • 2. Re: Uninterruptible sleep during Edison SPI
            jbromley

            Hi Peter,

             

            I just wanted to add a couple of pieces of information. First, I only see the message you quote above on the console terminal where the program is running. The call trace does appear in dmesg output. Also, I am using Edison version 146 as reported by configure_edison --version. I am using the stock image (edison-image-ww18-15) without any modifications. In my case, if I lower the SPI clock speed to 26 kHz, the transfer will always work, but 27.5 kHz always fails, so perhaps there is some magic number somewhere with a value around 40 microseconds?

             

            Thanks,

            Jay

            • 3. Re: Uninterruptible sleep during Edison SPI
              Intel_Peter

              Hello jbromley,

               

              We have been running some more tests on our Edison. We were able to find the error messages on dmesg (thanks by the way), and on a deeper research we found out that this is a known issue. And right now there is no workaround for it.

               

              Peter.

              • 4. Re: Uninterruptible sleep during Edison SPI
                KurtE

                Hi @Intel_Peter,

                 

                If I were a guessing man, I would guess that there is probably a race condition?  Something like: the code is looking at some state, but the state happens before we are ready and so we loop...

                 

                Do you have an internal fix for it yet?  If so could you release a patch that those of us who do or can build their own kernel can get it up and running?

                 

                Thanks

                • 5. Re: Uninterruptible sleep during Edison SPI
                  Intel_Peter

                  Hello KurtE,

                   

                  Unfortunately there's no fix for this issue. We are currently working on fixing it, however there's no ETA on when it'll be available. My only suggestion would be for you to be aware of the new features added in future releases.

                   

                  Peter.

                  • 6. Re: Uninterruptible sleep during Edison SPI
                    xiaoyang

                    Hi, has this issue being solved yet?


                    • 7. Re: Uninterruptible sleep during Edison SPI
                      iceowl

                      I have been using the Arduino interface and trying to get the Adafruit ST7735 1.8" TFT

                       

                      1.8 Color TFT LCD display with MicroSD Card Breakout [ST7735R] ID: 358 - $19.95 : Adafruit Industries, Unique & fun DIY …

                       

                      to "work" without going into uninterruptible sleep mode  - and the only workaround I have found is to do the following things


                      1) modify the third party code Adafruit_ST77753.cpp by changing:

                       

                      #define X86_BUFFSIZE 32

                      void inline Adafruit_ST7735::spiwriteN(uint32_t count, uint16_t c) {


                      to


                       

                      #define X86_BUFFSIZE 16

                      void inline Adafruit_ST7735::spiwriteN(uint32_t count, uint16_t c) {


                      2)  i in Adafruit_ST77735.cpp find:


                      void Adafruit_ST7735::commonInit(const uint8_t *cmdList) {

                          colstart  = rowstart = 0; // May be overridden in init func


                      and add


                      pinMode(_cs, OUTPUT); 





                      Or you can put that in the Arduino sketch as long as it's very close to the top of the Setup block.

                       

                      The result is that you don't go into uninterruptable sleep - BUT - it is slow as heck.


                      Hope any of that is useful


                      Cheers


                      Joe





                      1 of 1 people found this helpful
                      • 8. Re: Uninterruptible sleep during Edison SPI
                        ElijahParker

                        Any update on this?  I just ran into this issue...after 8 hours of trying things I found this thread.  Has anything been done?  Even if not, where can I start looking for a fix?  Is the issue in the kernel driver?  I tried using ioctl instead of mraa, as well as write() but the results are the same.

                        • 9. Re: Uninterruptible sleep during Edison SPI
                          qq19773585

                          Hi Peter,

                           

                          I got the same error message when the SPI doing a massive transmission. This may be a kernel driver bug: http://blog.csdn.net/williamwanglei/article/details/8971181


                          I'm a hardware Engineer, can only simply translate this page:


                          When the SPI doing a massive transmission, the sleep in spi_sync() function may cause the SPI ISR can't get the authorization of mutex lock. In the final, this thread will stall.

                           

                           

                          I tested the MRAA libs, and get the same error message.

                           

                          You can try to limit the transmission length to less than 64, it may work.

                           

                          Regards

                          Arnold