6 Replies Latest reply on May 13, 2015 7:08 AM by KurtE

    SPI on 2.1 - new issue/Bug

    Stan_Gifford

      I have been using this http://www.freetronics.com.au/collections/display/products/128x128-pixel-oled-module#.VVFs860cRmM using the FTOLED library - specifically the example sketch all_drawing_operations.

      (see https://github.com/freetronics/FTOLED/tree/master/examples ).

       

      The program compiles with no issues in the Intel Edison arduino IDE.

       

      Under the previous version I had this running continually for two weeks without it missing a beat.

       

      Under 2.1 however (and I have repeated the test three times) after 4-5 hours (ish) the sketch will 'freeze' when clearing the oled to a solid color.

       

      I am not panicking about this because I am still awaiting a kernel with acceptable SPI speed - so I can then write the UPM module for Eclipse however someone in Intel may be interested in this bug (personally I think a memory leak may be implicated).

       

      Stan

        • 1. Re: SPI on 2.1 - new issue/Bug
          KurtE

          Out of curiosity I looked at the link you mentioned and the sketch file you mentioned, looks like it simply calls their graphic operations once in setup() and the loop function does nothing,

          So my guess is that once this program finishes drawing the first stuff, it should more or less eat one processor (100%).  Am I missing something.

           

          So not sure how you are running it, that it freezes in 4-5 hours...  When you say freeze, is the whole processor hung up or just that program?  If just that program, is there anything interesting shown in error logs? dmesg?   If I remember correctly and looking at current released ide code everything written to stdout goes to a file (/tmp/log.txt) and everything written to stderr goes to the file (/tmp/log_er.txt) so if you can still look at processor without reboot, see if anything interesting in these files...

           

          I don't have their display so can not try it... I should probably try out one of the Adafruit displays to see how it is doing with this release.  I noticed the release notes did not mention anything about SPI, so I have not done much yet with it.

          • 2. Re: SPI on 2.1 - new issue/Bug
            Stan_Gifford

            Kurt (and others) - apolegies.

             

            I realise now I did modify the program to continually cycle thru the drawing operations.

             

            I haven't spent a lot of time debugging/looking at error logs as yet - I believe however the more interesting thing is that this ran 'forever' on previous version and karked it on 2.1.

             

            BTW - to answer Kurt's question - the system keeps running - just the program/sketch stops.

             

            I will post the code later (am at work ATM) - but as Kurt rightly points out - it will not do a lot of good unless you have this board!

             

            Stan

            • 3. Re: SPI on 2.1 - new issue/Bug
              KurtE

              HI Stan,

               

              No problem from me, just trying to understand what is happening. 

               

              Again I don't have your hardware or setup, but I do have my Adafruit ILI9341 TFT code base.  So I thought I would try running some of the graphic test stuff for it that I have to see if any better than previous builds. Nope!!!  I tried it a couple different ways (Arduino, Eclipse and Native and my test programs really hang now!

               

              For example if you use my native stuff, that builds on the Edison using mraa, which is part of my code base: KurtE/Raspberry_Pi · GitHub

              If you build the directory testAdafruit_ILI9341C, it will build the other directories it depends on.  Note: I did this on my machine using current MRAA (that is I built it from current sources).

              When the build is done,  I then run the command: ./graphictest

              The test starts up and then stops.  Note: the command does not exit, but SPI stops begin output (as seen by Logic Analyzer) as well as debug text that gets output.  You can not ctrl+c out of this program.  If you go to a different putty window, you can not exit the program using the kill command.

              the ps command still shows it there:

                516 root  3744 D./graphictest

               

              The top command (or htop which is installed in alext's library) shows one processor is running at 100%, yet none of the shown process shows any CPU usage except htop... So I think it is hung in the kernel, probably in some hard loop waiting for something.

               

              Wondered if it was again the SPI power management stuff, so I tried the stuff mentioned for previous kernel (echo on > /sys/devices/pci0000:00/0000:00:07.1/power/control)

              Again did not help.  So far the only way I have found to exit that program is to reboot the processor.

              • 4. Re: SPI on 2.1 - new issue/Bug
                Stan_Gifford

                Hmmm - when mine hung, I threatened it with kill -9 (sketch.elf) and it did successfully die.

                If however I reran the sketch, nothing happened - so I am tending to agree with you that something in kernel code is spinning.

                 

                Reboot obviously (like your experience) did fix the problem - until the sketch hung again.

                 

                I suppose then at this juncture the question is - did any changes to SPI get incorporated into 2.1?

                 

                Stan

                • 5. Re: SPI on 2.1 - new issue/Bug
                  Stan_Gifford

                  Hi Kurt,

                   

                  did the demise thang and believe it or not, SPI is implicated.

                   

                  Get lots of;

                   

                  [ 3840.074982] INFO: task sketch.elf:202 blocked for more than 120 seconds.    

                  [ 3840.075077] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this

                  [ 3840.075156] sketch.elf      D c18b49c6  6364   202    201 0x00000000        

                  [ 3840.075197]  f5035dd8 00000086 f5035d6c c18b49c6 f4e50000 f8940672 000002e9 c

                  [ 3840.075256]  f5034000 f5f24de0 f73fd940 f5f20000 f4e50000 f5d35344 f6c0e000 f

                  [ 3840.075312]  c126ef4f f5ccfc00 f5035da8 c1257dae f5035dc0 c125938e f6c0e000 f

                  [ 3840.075369] Call Trace:                                                     

                  [ 3840.075412]  [<c18b49c6>] ? _raw_spin_unlock_irqrestore+0x26/0x50           

                  [ 3840.075451]  [<c126ef4f>] ? wake_up_process+0x1f/0x40                       

                  [ 3840.075478]  [<c1257dae>] ? wake_up_worker+0x1e/0x30                        

                  [ 3840.075503]  [<c125938e>] ? insert_work+0x4e/0x80                           

                  [ 3840.075530]  [<c18b4947>] ? _raw_spin_unlock+0x17/0x40                      

                  [ 3840.075556]  [<c12598cb>] ? __queue_work+0x10b/0x320                        

                  [ 3840.075582]  [<c18b3c23>] schedule+0x23/0x60                                

                  [ 3840.075612]  [<c18b1305>] schedule_timeout+0x165/0x290                      

                  [ 3840.075645]  [<c18b8125>] ? sub_preempt_count+0x55/0xe0                     

                  [ 3840.075672]  [<c18b2bd3>] wait_for_completion+0xa3/0xe0                     

                  [ 3840.075700]  [<c126ef90>] ? wake_up_state+0x20/0x20                         

                  [ 3840.075731]  [<c15a58c0>] spidev_sync+0x80/0xb0                             

                  [ 3840.075762]  [<c15a61e9>] spidev_ioctl+0x619/0x6a0                          

                  [ 3840.075791]  [<c1206d7b>] ? native_sched_clock+0x2b/0xd0                    

                  [ 3840.075822]  [<c15a5990>] ? spidev_write+0xa0/0xa0                          

                  [ 3840.075851]  [<c15a5bd0>] ? spidev_probe+0x180/0x180                        

                  [ 3840.075877]  [<c132a836>] do_vfs_ioctl+0x2f6/0x540                          

                  [ 3840.075909]  [<c146d53a>] ? inode_has_perm.isra.41.constprop.78+0x3a/0x50   

                  [ 3840.075937]  [<c146d5d7>] ? file_has_perm+0x87/0x90                         

                  [ 3840.075968]  [<c146d99c>] ? selinux_file_ioctl+0x4c/0xf0                    

                  [ 3840.075994]  [<c132aae0>] SyS_ioctl+0x60/0x80                               

                  [ 3840.076023]  [<c18b5038>] syscall_call+0x7/0xb                              

                  [ 3840.076053]  [<c18b0000>] ? input_proc_exit+0x10/0x36                       

                  [ 3960.062687] INFO: task sketch.elf:202 blocked for more than 120 seconds.    

                  [ 3960.062781] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this

                  [ 3960.062860] sketch.elf      D c18b49c6  6364   202    201 0x00000000        

                  [ 3960.062901]  f5035dd8 00000086 f5035d6c c18b49c6 f4e50000 f8940672 000002e9 c

                  [ 3960.062960]  f5034000 f5f24de0 f73fd940 f5f20000 f4e50000 f5d35344 f6c0e000 f

                  [ 3960.063016]  c126ef4f f5ccfc00 f5035da8 c1257dae f5035dc0 c125938e f6c0e000 f

                  [ 3960.063073] Call Trace:

                   

                  Nothing of great interest in /tmp/log*

                   

                  Stan

                  • 6. Re: SPI on 2.1 - new issue/Bug
                    KurtE

                    Yep same hang here...

                    [ 1080.529865] INFO: task graphictest:350 blocked for more than 120 seconds.
                    [ 1080.529959] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
                    [ 1080.530038] graphictest     D c18b49c6  6280   350    343 0x00000004
                    [ 1080.530079]  f5ce1dd8 00000086 f5ce1d6c c18b49c6 f6fe4850 f5ce1d94 c126ee63 c1cd7940
                    [ 1080.530138]  f5ce0000 c1b9c120 f73ef940 f5f237a0 f6fe4850 f5d9bb44 f6c0e000 f5ce1da0
                    [ 1080.530193]  c126ef4f f5cf2300 f5ce1da8 c1257dae f5ce1dc0 c125938e f6c0e000 f5d2dc00
                    [ 1080.530251] Call Trace:
                    [ 1080.530293]  [<c18b49c6>] ? _raw_spin_unlock_irqrestore+0x26/0x50
                    [ 1080.530326]  [<c126ee63>] ? try_to_wake_up+0x173/0x240
                    [ 1080.530355]  [<c126ef4f>] ? wake_up_process+0x1f/0x40
                    [ 1080.530382]  [<c1257dae>] ? wake_up_worker+0x1e/0x30
                    [ 1080.530408]  [<c125938e>] ? insert_work+0x4e/0x80
                    [ 1080.530434]  [<c18b4947>] ? _raw_spin_unlock+0x17/0x40
                    [ 1080.530460]  [<c12598cb>] ? __queue_work+0x10b/0x320
                    [ 1080.530487]  [<c18b3c23>] schedule+0x23/0x60
                    [ 1080.530516]  [<c18b1305>] schedule_timeout+0x165/0x290
                    [ 1080.530549]  [<c18b8125>] ? sub_preempt_count+0x55/0xe0
                    [ 1080.530576]  [<c18b2bd3>] wait_for_completion+0xa3/0xe0
                    [ 1080.530605]  [<c126ef90>] ? wake_up_state+0x20/0x20
                    [ 1080.530636]  [<c15a58c0>] spidev_sync+0x80/0xb0
                    [ 1080.530667]  [<c15a61e9>] spidev_ioctl+0x619/0x6a0
                    [ 1080.530694]  [<c18b8125>] ? sub_preempt_count+0x55/0xe0
                    [ 1080.530725]  [<c15a5990>] ? spidev_write+0xa0/0xa0
                    [ 1080.530754]  [<c15a5bd0>] ? spidev_probe+0x180/0x180
                    [ 1080.530781]  [<c132a836>] do_vfs_ioctl+0x2f6/0x540
                    [ 1080.530812]  [<c146d53a>] ? inode_has_perm.isra.41.constprop.78+0x3a/0x50
                    [ 1080.530841]  [<c146d5d7>] ? file_has_perm+0x87/0x90
                    [ 1080.530872]  [<c146d99c>] ? selinux_file_ioctl+0x4c/0xf0
                    [ 1080.530898]  [<c132aae0>] SyS_ioctl+0x60/0x80
                    [ 1080.530927]  [<c18b5038>] syscall_call+0x7/0xb
                    
                    

                    The good news is there are at least two easy to reproduce cases here so hopefully Intel can release a fix soon.