1 2 Previous Next 16 Replies Latest reply on Apr 30, 2015 11:28 PM by Browndav

    Weird one; mraa ISR, threading, and opening a serial port = weird problem

    Browndav

      Ok, this is going to sound weird, but here's what I'm dealing with.  I'm really not sure how else to put it

       

      I have the seeed studio arduino shield and an attached grove button on the Edison/arduino breakout board.  I have this set up in code with an interrupt so I can do things when the button is pressed.  This is working fine.

       

      Then I add some code that creates a new thread.  In the new thread, I open the serial port, and loop to read a constant stream of data and write it a file on the SD card.  This also works just fine.

       

      Except, the ISR does not work if it's the first time I ran the program after rebooting the edison

       

      Sequence of events:

      1. boot the edison (either cold boot, or by 'reboot' from the command line)
      2. log in
      3. Execute the program
      4. Press the button, the ISR does not fire (also ran in the debugger with a breakpoint, the ISR function is never called) (MRAA_SUCCESS is returned, even though it doesn't fire)
      5. ctrl+C to close the program
      6. Arrow up, and run the program again
      7. Press the button, and the ISR does fire this time (and every other time I run the program until cold/reboot).

       

      Below is the entirety of my stripped down program.  If I comment out line 38, the ISR will fire every time the program runs, including after the initial reboot. 

       

      If I move the opening of the serial port outside the new thread, to the original thread, then everything works fine. I came across this issue since I wanted to kick off the new thread, and have it do all of it's initializing itself, rather than in the main thread.  I'm fine with that workaround, but this issue is just so... weird.  It seems to me like it should just work, but I've never used threads before, so I may be overlooking something simple.  open() should be thread safe, but looking around online, sometimes the file descriptor table isn't?

       

      I've tested on two different Edisons, one using the latest image available, the other is a little older.  mraa version is 0.6.1 (I believe, the newest available).

       

      Anyways, I'm out of ideas

       

      #include <iostream>
      #include <mraa.h>
      
      using namespace std;
      
      mraa_gpio_context gpioIn, gpioLED;
      
      //Just print a character in the button ISR
      
      void ISR_button4(void *)
      {
          printf("I\n");
      }
      
      //initialize the button input
      //The button is pulled down, then tied high when pressed
      void ISR_Button_Init()
      {
          gpioIn = mraa_gpio_init(4);
      
          mraa_gpio_dir(gpioIn, MRAA_GPIO_IN);
          mraa_gpio_mode(gpioIn, MRAA_GPIO_PULLDOWN);
      
          int iRet = mraa_gpio_isr(gpioIn, MRAA_GPIO_EDGE_RISING, &ISR_button4, NULL);    //create the ISR function
      
          if (iRet == MRAA_SUCCESS)
              printf("ISR good\n");
          else
              printf("ISR bad\n");
      }
      
      void *ML_Run(void*ptr)
      {
          printf("Hello from thread\n");
          char portname[] = "/dev/ttyMFD1";
      
          //Open the serial port...
          int fu = open (portname, O_RDWR);
      
          // Opening the serial port like this doesn't work either
          //FILE * fdsp = fopen(portname, "w+");
      
          while (1)
              sleep(1);
      }
      
      int main()
      {
          pthread_t tLogThread;
      
          mraa_init();
          mraa_uart_init(0);
      
          ISR_Button_Init();
      
          gpioLED = mraa_gpio_init(13);                    //initialize the LED output
          mraa_gpio_dir(gpioLED, MRAA_GPIO_OUT);            //make the LED an output
      
          //set up ISR call
          mraa_gpio_isr(gpioIn, MRAA_GPIO_EDGE_RISING, &ISR_button4, NULL);
      
          pthread_create (&tLogThread, NULL, ML_Run, NULL);        //create the thread to loop and do nothing
      
          cout << "Hello world" << endl;
      
          //now just sit and wait for stuff to happen
          while (1)
          {
              sleep(1);
              printf("0\n");
          }
          return 0;
      }
      
        • 1. Re: Weird one; mraa ISR, threading, and opening a serial port = weird problem
          Browndav

          Ok, I feel like I'm taking crazy pills.

           

          So I figure I have it working with my work around from above - don't open serial ports in a separate thread.  So, I test my other program that sets up an mraa_isr that triggers on both edges of an 50Hz RC PWM signal to measure the pulse width (pulse width is 1-2mS).  (yes, the signal is going through the proper conditioning before going into the shield board I'm using).

           

          Except now, reliably, I do not get any interrupts if I call the isr function with MRAA_GPIO_EDGE_BOTH.  If I recompile/execute the program with MRAA_GPIO_EDGE_RISING, I get interrupts.  If I then re-recompile the code back to the original _BOTH, then I get both interrupts, and the code works like it should.

           

          To recap:

          1.  Compile/run with interrupt in _BOTH mode - no interrupts at all.

          2.  Compile/run with interrupt in _FALLING (or rising) mode - yay, interrupts on the specified edge.

          3.  Compile/run with interrupt in _BOTH mode, just like #1, yay, interrupts on both edges, and I can measure the pulse width just fine.

          4.  Me: confused

           

          My mraa version is 0.6.1.

           

          Anybody have any hint of an idea what's going on?

           

          Edit:

          Sigh, back to the 'some interrupts don't work at all the first run after a reboot'.

          Here's a screenshot of what I'm looking at 2015-03-29_2359 -

          And here's my current workaround, that is shown working in the screenshot I just linked.  create the _isr() call with a _FALLING edge.  call _isr_exit().  Then call _isr() again with the _BOTH edge.  That seems to make it actually interrupt on each edge, but only the second time I run the program after a reboot.

          • 2. Re: Weird one; mraa ISR, threading, and opening a serial port = weird problem
            Intel_Alvarado

            Hi,

             

            Let me try and run your code to see if I get the same output. I’ll post my results shortly.

             

            Sergio

            • 3. Re: Weird one; mraa ISR, threading, and opening a serial port = weird problem
              Browndav

              And the plot thickens... Just tried another test on my second program (used in my second post).  In addition to the PWM isr, I have the same button set up in an ISR.

               

              If I initialize the PWM ISR after I make the call to set up the button ISR, then the PWM isr works immediately after a reboot.

               

              If I move the PWM init function before the call to make the button ISR, it does not work after a reboot - only works the second time I run the program.

               

              I have a couple buttons here, so I'll try to modify my first program to see if I can duplicate it in a way it can be tested somewhere else.

               

              Edit: Yep, by changing the order I call the functions to set up the ISRs, the PWM isr either works, or doesn't work.  I also don't need to do the work around of creating the isr, exiting it, then recalling it with _BOTH edges selected.

               

              I should note that I have two different Edison's I'm using to test on, both having the same results.  One was flashed a couple days ago with the latest image.  The other one was flashed a month ago.  Is there a way to get the current build info from the command line?

              • 4. Re: Weird one; mraa ISR, threading, and opening a serial port = weird problem
                Browndav

                Ok, here's my updated version of the code in the first post.  This one has two buttons, connected to pins 4 and 5 on the arduino shield.

                 

                If I do not open the serial port in the thread, both ISRs fire just fine when the buttons are pressed during the first run after a reboot.

                 

                If I do open the serial port, only one of the ISRs fire the first time the program runs after a reboot.  On second run, they both work.

                 

                Thanks for testing this for me, Alvarado. 

                 

                #include <iostream>
                #include <mraa.h>
                
                using namespace std;
                
                mraa_gpio_context gpioIn, gpioLED, gpioIn2;
                
                //Just print a character in the button ISR
                
                void ISR_button4(void *)
                {
                    printf("4\n");
                }
                void ISR_button5(void *)
                {
                    printf("5\n");
                }
                //initialize the button input
                //The button is pulled down, then tied high when pressed
                void ISR_Button_Init4()
                {
                    gpioIn = mraa_gpio_init(4);
                
                    mraa_gpio_dir(gpioIn, MRAA_GPIO_IN);
                    mraa_gpio_mode(gpioIn, MRAA_GPIO_PULLDOWN);
                
                    int iRet = mraa_gpio_isr(gpioIn, MRAA_GPIO_EDGE_RISING, &ISR_button4, NULL);    //create the ISR function
                
                    if (iRet == MRAA_SUCCESS)
                        printf("ISR good\n");
                    else
                        printf("ISR bad\n");
                }
                
                void ISR_Button_Init5()
                {
                    gpioIn2 = mraa_gpio_init(5);
                
                    mraa_gpio_dir(gpioIn2, MRAA_GPIO_IN);
                    mraa_gpio_mode(gpioIn2, MRAA_GPIO_PULLDOWN);
                
                    int iRet = mraa_gpio_isr(gpioIn2, MRAA_GPIO_EDGE_RISING, &ISR_button5, NULL);    //create the ISR function
                
                    if (iRet == MRAA_SUCCESS)
                        printf("ISR good\n");
                    else
                        printf("ISR bad\n");
                
                }
                
                void *ML_Run(void*ptr)
                {
                    printf("Hello from thread\n");
                    char portname[] = "/dev/ttyMFD1";
                
                    //Open the serial port...
                    int fu = open (portname, O_RDWR);
                
                    // Opening the serial port like this doesn't work either
                    //FILE * fdsp = fopen(portname, "w+");
                
                    while (1)
                        sleep(1);
                }
                
                int main()
                {
                    pthread_t tLogThread;
                
                    mraa_init();
                    mraa_uart_init(0);
                
                    ISR_Button_Init4();
                    ISR_Button_Init5();
                
                    gpioLED = mraa_gpio_init(13);                    //initialize the LED output
                    mraa_gpio_dir(gpioLED, MRAA_GPIO_OUT);            //make the LED an output
                
                    pthread_create (&tLogThread, NULL, ML_Run, NULL);        //create the thread to loop and do nothing
                
                    cout << "Hello world" << endl;
                
                    //now just sit and wait for stuff to happen
                    while (1)
                    {
                        sleep(1);
                        printf("0\n");
                    }
                    return 0;
                }
                
                • 5. Re: Weird one; mraa ISR, threading, and opening a serial port = weird problem
                  Browndav

                  @Intel_Alvarado, where you able to test this?

                   

                  I did another test where I hooked up four of the grove buttons.  On first run after a reboot, two of them would work, the other two would not.  Re-run the program, and a third one would start working.  The fourth never did work?  I didn't look into it too much though.

                  • 6. Re: Weird one; mraa ISR, threading, and opening a serial port = weird problem
                    Intel_Alvarado

                    Hi,

                     

                    Yes, we’ve been working on your case but still no positive results. We will post a more complete answer to you as soon as possible.

                     

                    Sergio  

                    • 7. Re: Weird one; mraa ISR, threading, and opening a serial port = weird problem
                      Tingleby@Intel

                      @Browndav This is a very curious issue.

                       

                      You are trying to time a PWM signal yes? Doing this on a non-realtime system your success will vary.

                       

                      A few questions, just to help flesh out the background:

                      • What sort of PWM frequency are you trying to decode here?
                      • Is the signal into the pwm detecting isr constant during start up?
                      • Is there any output to syslog during the startup. Check with journalctl -f.

                       

                      Ill try and reproduce and debug this afternoon.

                      • 8. Re: Weird one; mraa ISR, threading, and opening a serial port = weird problem
                        Browndav

                        I agree, it is weird.

                         

                        1.  It's a standard RC servo signal.  I was testing with a 50Hz (20ms period) signal, that is high for 1-2ms.  Final system would be a 300Hz signal, also high for 1-2ms.  I know it won't be truly accurate, I just needed to detect a short or long pulse.  We've also since found a different way to acquire this signal, so I don't need to decode it any more.

                         

                        2.  I tried having the signal both active, and low (inactive) during Edison/program start up.  Neither made a difference.  The only difference was whether it was the first or second time the program ran, or whether I opened the Serial port in a thread.

                         

                        3.  I'll have to check that later.

                         

                        The issue wasn't specific to the PWM; it was specific to making and using an ISR.  I was also able to reliably duplicate it using a button with the PWM completely disconnected and not in the program.  With four buttons hooked up, some of them would work right after reboot, while the others would only work during the second run after a reboot.

                         

                        All this being said, we found the workaround for opening the serial port in the main thread, and the final program won't have any IO in ISRs, but still, it's a goofy problem having ISRs work dependent on when they were run after bootup.

                         

                        I can also take a video of the entire sequence if that would be helpful.

                        • 9. Re: Weird one; mraa ISR, threading, and opening a serial port = weird problem
                          Tingleby@Intel

                          Have you noticed it happening on some but not others? Do you happen to have ones it doesn't work on and others that do. It might help find an underlying cause.

                           

                          Videos are always fun.

                          • 10. Re: Weird one; mraa ISR, threading, and opening a serial port = weird problem
                            Browndav

                            IIRC, when I had a program running with 4 buttons, 2 would always work, 1 would work on the second run, and 1 wouldn't work at all.  I thought that was tied to the order they were initialized, but that did not seem to be the case.

                             

                            I'll try to do some more testing this afternoon, and try to make a video at the same time.

                            • 11. Re: Weird one; mraa ISR, threading, and opening a serial port = weird problem
                              Browndav

                              Ok, here's a short video of me running the second program I posted (with button 4 and 5).  Hopefully I don't give anyone motion sickness

                               

                              Edison ISR issue - YouTube

                              • 12. Re: Weird one; mraa ISR, threading, and opening a serial port = weird problem
                                Tingleby@Intel

                                I don't have a grove shield in front of me at this time,

                                Could you please try this with 5v set on the grove shield? I'm not expecting a change but just to eliminate the possibility

                                 

                                Ok, so It could be an issue with the pinmuxes not being set on the first try. Could you please give me the output of journalctl -xn just after the run it for the first time after a reboot.

                                • 13. Re: Weird one; mraa ISR, threading, and opening a serial port = weird problem
                                  Browndav

                                  After a reboot, and running the program once:

                                   

                                  journalctl -xn

                                  -- Logs begin at Sat 2000-01-01 00:00:10 UTC, end at Wed 2015-04-08 15:21:14 UTC. --

                                  Apr 07 20:43:24 Edison1 udhcpc[311]: Lease of 192.168.1.129 obtained, lease time 86400

                                  Apr 07 20:43:24 Edison1 wpa_cli[284]: Lease of 192.168.1.129 obtained, lease time 86400

                                  Apr 07 20:43:24 Edison1 systemd-timesyncd[159]: Network configuration changed,trying to establish connection.

                                  Apr 07 20:43:24 Edison1 kernel: [[1;39mip (320) used greatest stack depth: 5216 bytes left[[0m

                                  Apr 07 20:43:24 Edison1 wpa_cli[284]: /etc/udhcpc.d/50default: Adding DNS 192.168.1.1

                                  Apr 07 20:43:46 Edison1 login[226]: [[1;39mROOT LOGIN  on '/dev/ttyMFD2'[[0m

                                  Apr 07 20:43:52 Edison1 libmraa[326]: [[1;39mlibmraa version v0.6.1 initialised by user 'root' with EUID 0[[0m

                                  Apr 07 20:43:55 Edison1 systemd-timesyncd[159]: Using NTP server 216.239.36.15:123 (time3.google.com).

                                  Apr 08 15:21:14 Edison1 systemd[1]: Time has been changed

                                  Apr 08 15:21:14 Edison1 systemd-timesyncd[159]: interval/delta/delay/jitter/drift 32s/+67039.284s/0.115s/0.000s/+0ppm

                                   

                                  ~

                                  ~

                                  standard input

                                   

                                   

                                  root@Edison1:/home/test# journalctl -f

                                   

                                  -- Logs begin at Sat 2000-01-01 00:00:10 UTC. --

                                  Apr 07 20:43:24 Edison1 systemd-timesyncd[159]: Network configuration changed, trying to establish connection.

                                  Apr 07 20:43:24 Edison1 kernel: ip (320) used greatest stack depth: 5216 bytes left

                                  Apr 07 20:43:24 Edison1 wpa_cli[284]: /etc/udhcpc.d/50default: Adding DNS 192.168.1.1

                                  Apr 07 20:43:46 Edison1 login[226]: ROOT LOGIN  on '/dev/ttyMFD2'

                                  Apr 07 20:43:52 Edison1 libmraa[326]: libmraa version v0.6.1 initialised by user 'root' with EUID 0

                                  Apr 07 20:43:55 Edison1 systemd-timesyncd[159]: Using NTP server 216.239.36.15:123 (time3.google.com).

                                  Apr 08 15:21:14 Edison1 systemd[1]: Time has been changed

                                  Apr 08 15:21:14 Edison1 systemd-timesyncd[159]: interval/delta/delay/jitter/drift 32s/+67039.284s/0.115s/0.000s/+0ppm

                                  Apr 08 15:21:46 Edison1 systemd-timesyncd[159]: interval/delta/delay/jitter/drift 64s/+0.013s/0.140s/0.005s/+0ppm

                                  Apr 08 15:22:51 Edison1 systemd-timesyncd[159]: interval/delta/delay/jitter/drift 128s/-0.024s/0.173s/0.018s/-95ppm

                                   

                                   

                                  Edit: I'll try it later today with the shield at 5v.  Could it potentially damage anything if the Edison was set to 3.3v, and the shield was set to 5v?

                                  • 14. Re: Weird one; mraa ISR, threading, and opening a serial port = weird problem
                                    Browndav

                                    So I did try it with the breakout board and the shield at 5v, no change in behavior.

                                     

                                    You mentioned the pinmuxes.  Are these the I2C port expanders on the breakout board?  I wonder if sending the commands twice would make a difference... Is there any other way I can test the pinmuxes?  I'm not checking return values on all commands, yet.

                                     

                                    Anyways, our application has changed slightly so we're not reading the PWM signal anymore, but it's still (un)reliable enough (and doing the same thing on two different Edisons), that I don't quite trust it to work every time yet.

                                     

                                    If there are any other diagnostics/message logs you want to gather, just let me know the exact commands you need me to run.

                                     

                                    Thanks.

                                    1 2 Previous Next