10 Replies Latest reply on Mar 10, 2017 12:10 PM by Intel Corporation

    Is there Watch Dog Capability?

    floydg

      Have not seen anything mentioned in MRAA or Intel Joule Documents, other than:

       

      IoT - Troubleshooting and FAQ | Intel® Software

      • It may help to disable the runtime watchdog. To do so, locate and delete the /etc/systemd/system.conf.d/00-runtime-watchdog.conf file on your development platform and reboot the development platform. Alternatively, you can comment out the two lines in the file and reboot.

       

      Is there special watch dog hardware?

      Are there any documents regarding the Watch Dog hardware and Software?

      How do you strobe/Kick the Watch Dog?

       

      Will the Intel Joule (hardware) Watch Dog work for both Ostro, and Ubuntu?

        • 1. Re: Is there Watch Dog Capability?
          Intel Corporation
          This message was posted on behalf of Intel Corporation

          Hello floydg,

          Thanks for reaching out!

          As you mentioned, there is no documentation about Joule's watchdog aside from the document you mentioned. Nevertheless, Joule indeed has a watchdog timer as one can confirm by simply running the command ls /dev/watchdog*.

          However, you will not be able to access this device with mraa as this library does not have any watchdog timer capabilities. So, if you would like to use this feature, I suggest you to try with Linux's standard libraries. I would recommend you to start by reading these documents:

          https://www.kernel.org/doc/Documentation/watchdog/watchdog-api.txt
          https://linux.die.net/man/8/watchdog

          I'm not sure if this device is also enabled in Ubuntu, for this specific question I believe you might get a more accurate answer if you ask them directly. You can do so in https://www.ubuntu.com/about/contact-us.

          I hope this information helps you,
          Pedro M.

          • 2. Re: Is there Watch Dog Capability?
            floydg

            Pedro,

             

            this was selected as a correct answer. Not sure who selected and verified the answer, was this selected by you and or Intel?

             

             

            1. After the watchdog is configured, and the kernel freezes (gets stuck in a loop forever, or NOP for ever), or crashes, have you verified that the Joule and all associated peripherals are reset due to the watchdog timeout?
            2. Does the Joule Reset line also get set to active on the hardware Watchdog timeout?

             

            Please share your test scenario/example with the community. Thanks in advance.

            • 3. Re: Is there Watch Dog Capability?
              Intel Corporation
              This message was posted on behalf of Intel Corporation

              Hello floydg,

              I've never seen Joule's watchdog be triggered before, as I've never experienced a kernel crash on the board. So, in order to test it, I looked how to intentionally create a kernel panic (I found how in http://unix.stackexchange.com/questions/66197/how-to-cause-kernel-panic-with-a-single-command in case you are interested). 

              I made the kernel panic and the board was stuck for a few seconds and then the watchdog reset the system. I then ran some tests to see if everything was working and I was not able to find anything abnormal. Have you seen anything out of the ordinary after a kernel crash reset? If so, could you please let us know how to replicate it?

              I'm afraid I don't understand your second question, could you please explain us what you mean?

              We will be waiting for your response.
              Pedro M.

              • 4. Re: Is there Watch Dog Capability?
                floydg

                Pedro,

                 

                Thanks, I have performed a preliminary fast paced review of the info you provided. Thanks in advance.

                 

                We are not interested in a "Software Watchdog" which requires that the kernel be "alive" in-order for the Watchdog to timeout, and restart the system. Although it is rare, there is always the potential that the kernel is not running correctly, executing garbage code, stuck in a loop, or stopped.

                 

                For that reason I was hoping to get clarification that the Intel Joule has adequate hardware support, integrated and configured for the linux kernel to provide a timeout/reset/reboot in the rare case as stated above.

                 

                Clarification for the second question:

                Is there an external reset output which can be used to provide a reset to peripheral chips?

                And are all of the Intel Joule SOM peripherals reset when the Watch Dog time out occurs?

                 

                Thanks again for your input.

                 

                We have decided to use proprietary hardware to accomplish the Watch Dog feature, due to our current concerns, and the possible limitations addressed.

                • 5. Re: Is there Watch Dog Capability?
                  Intel Corporation
                  This message was posted on behalf of Intel Corporation

                  I completely understand what you are saying and your situation. As you are aware, there is no documentation about Joule's Watchdog aside from the one document you shared above. Joule is a recently released product and we are currently working on improving its documentation, hopefully more documentation will be released soon that can provide us a bigger inside on Joule's capabilities, nevertheless, there's no ETA for when that might happen.
                   
                  However, I will try to see if I can find some information about this that you can use. But I must point out that since this is directly related to the hardware of the module, this might not be public information. If I find any useful information, I will make sure to post it here.
                   
                  Pedro M.

                  • 6. Re: Is there Watch Dog Capability?
                    floydg

                    Pedro,

                     

                    thanks for the update.

                     

                    I am not interested in the Intel Intellectual Property of the Watch Dog hardware.

                     

                    What is needed is the high level functionality of the Watch Dog hardware, and any associated API capability for configuration and control.

                     

                    And if the Intel Watch Dog hardware is integrated with Linux, just mention that it is integrated, and provide a simple test scenario, with results.

                     

                    Thanks in advance.

                    • 7. Re: Is there Watch Dog Capability?
                      Intel Corporation
                      This message was posted on behalf of Intel Corporation

                      Hi floydg,
                       
                      I apologize for the delay in our response, however, we have an update for this case.
                       
                      Joule supports iTCO watchdog. You can find a very useful document about how to test an iTCO watchdog in http://www.madore.org/~david/linux/iTCO-wdt-test.html. I’d suggest that you pay special attention to steps 5 and 6 as they show you how test the watchdog. For example, please read the following:
                       
                      (Taken from step #5) “…Now type the following commands at the administrator prompt:modprobe i2c-i801modprobe i2c-smbusmodprobe iTCO-wdt
                       
                      (note that capitalization matters: it's iTCO-wdt and not itco-wdt; on the other hand, the minus sign can be replaced by an underscore in any or all of these lines if it's somehow easier to type). The first two load the I²C bus drivers and the third loads the watchdog driver itself. None of these commands will produce any output.
                       
                      Optional: check whether the watchdog has correctly been detected by typing dmesg to view the Linux kernel logs. Look for lines such as the following (preceded by numerical timestamps):iTCO_wdt: Intel TCO WatchDog Timer Driver v1.11iTCO_wdt: Found a Intel PCH TCO device (Version=4, TCOBASE=0x0400)iTCO_wdt: initialized. heartbeat=120 sec (nowayout=0)
                      If only the first appears, the watchdog was not detected (and it is useless to continue). If all three appear, the watchdog was correctly detected…”
                       
                      (Taken from step #6) “…Finally, type the following at the administrator prompt:cat >> /dev/watchdogand press enter twice. This will cause the watchdog to start counting down. The prompt will not return (do not press control-c nor control-d), simply wait.
                      After a few minutes (specifically, twice the heartbeat value indicated in the above optional step), the system will perform a hard reboot.
                       
                      Results: If the system performs a successful reboot (as if the reset button had been pressed), congratulations!, the watchdog is functioning correctly. (Please report a success to me.) If the system fails to reboot (remains forever hung in POST), turn the computer off, disconnect the power supply for a few minutes, and then perform a cold reboot to return to normal operation. Please report this as a failure…”
                       
                      I believe this will be of help for you, let me know if you still have any doubts and we’ll do our best effort to help you with them.
                       
                      Pedro M.

                      • 8. Re: Is there Watch Dog Capability?
                        floydg

                        A colleague and I tried this with no success.

                         

                        Here is his observations:

                         

                        I tried this and found that it is NOT implemented:

                         

                            The Intel support suggested the following page to try:

                        http://www.madore.org/~david/linux/iTCO-wdt-test.html

                         

                        ; sudo modprobe i2c-i801

                        ; sudo modprobe i2c-smbus

                        ; sudo modprobe iTCO-wdt

                         

                        ; dmesg | tail -2

                        [ 1207.985628] iTCO_vendor_support: vendor-support=0

                        [ 1207.988640] iTCO_wdt: Intel TCO WatchDog Timer Driver v1.11

                         

                        ; ls /dev/watchdog

                        ls: cannot access '/dev/watchdog': No such file or directory

                         

                        Appears that Ubuntu does NOT support the iTCO driver

                         

                        The documentation page includes the following which suggests that we’re missing the necessary lines to prove that it loaded:

                         

                        Optional: check whether the watchdog has correctly been detected by typing dmesg to view the Linux kernel logs. Look for lines such as the following (preceded by numerical timestamps):

                        iTCO_wdt: Intel TCO WatchDog Timer Driver v1.11 iTCO_wdt: Found a Intel PCH TCO device (Version=4, TCOBASE=0x0400) iTCO_wdt: initialized. heartbeat=120 sec (nowayout=0)

                         

                        If only the first appears, the watchdog was not detected (and it is useless to continue). If all three appear, the watchdog was correctly detected.

                        • 9. Re: Is there Watch Dog Capability?
                          Intel Corporation
                          This message was posted on behalf of Intel Corporation

                          I understand, please let me go over your findings, I will get back to you as soon as possible.

                          Pedro M.
                           

                          • 10. Re: Is there Watch Dog Capability?
                            Intel Corporation
                            This message was posted on behalf of Intel Corporation

                            We checked the input you provided us and analyzed it. We can confirm that our answer is valid for Ostro, if you'd like to test this on Ubuntu, you will have to get in contact with Canonical. You can do so in https://www.ubuntu.com/support/contact-us.

                            We apologize for any inconvenience this might cause.
                            Pedro M.