5 Replies Latest reply on Jul 1, 2017 4:42 AM by FerryT

    Edison abnormal behavior in long run

    vinu_gk

      Hi Intel,

      We have come across some very wierd behaviour from Intel Edison module and now it seems to not working at all. This has occured after consistent testing of  the device for more than 2 months. I have attached the boot up logs here:

       

      ******************************
      PSH KERNEL VERSION: b0182b2b
        WR: 20104000
      ******************************
      SCU IPC: 0x800000d0 0xfffce92c
      PSH miaHOB version: TNG.B0.VVBD.0000000c
      microkernel built 11:24:08 Feb 5 2015
      ******* PSH loader *******
      PCM page cache size = 192 KB
      Cache Constraint = 0 Pages
      Arming IPC driver ..
      Adding page store pool ..
      PagestoreAddr(IMR Start Address) = 0x04899000
      pageStoreSize(IMR Size) = 0x00080000
      *** Ready to receive application ***
      U-Boot 2014.04 (Jun 06 2016 - 14:40:07)
        Watchdog enabled
      DRAM: 980.6 MiB
      MMC: tangier_sdhci: 0
      In: serial
      Out: serial
      Err: serial
      Hit any key to stop autoboot: 0
      Target:blank
      Partitioning already done...
      Flashing already done...
      GADGET DRIVER: usb_dnl_dfu
      reading vmlinuz
      5461344 bytes read in 135 ms (38.6 MiB/s)
      Valid Boot Flag
      Setup Size = 0x00003c00
      Magic signature found
      Using boot protocol version 2.0c
      Linux kernel version 3.10.98-poky-edison+ (neck@flax) #1 SMP PREEMPT Mon Jun 6 14:32:08 PDT 2016
      Building boot_params at 0x00090000
      Loading bzImage at address 00100000 (5445984 bytes)
      Magic signature found
      Kernel command line: "rootwait root=PARTUUID=012b3303-34ac-284d-99b4-34e03a2335f4 rootfstype=ext4 console=ttyMFD2 earlyprintk=ttyMFD2,keep loglevel=4 g_multi.ethernet_config=rndis systemd.unit=multi-user.target hardware_id=00 g_multi.iSerialNumber=2c3b9ec04de9c3ca87850002f0ec1424 g_multi.dev_addr=02:00:86:ec:14:24 platform_mrfld_audio.audio_codec=dummy"
      Starting kernel ...
      [ 0.735747] pca953x 1-0020: failed reading register
      [ 0.740861] pca953x 1-0021: failed reading register
      [ 0.746007] pca953x 1-0022: failed reading register
      [ 0.751054] pca953x 1-0023: failed reading register
      [ 1.647010] snd_soc_sst_platform: Enter:sst_soc_probe
      [ 2.131085] pmic_ccsm pmic_ccsm: Error reading battery profile from battid frmwrk
      [ 2.140445] pmic_ccsm pmic_ccsm: Battery Over heat exception
      Welcome to Linux!
      [ 2.799721] systemd[1]: [/lib/systemd/system/wyliodrin-hypervisor.service:3] Failed to add dependency on redis, ignoring: Invalid argument
      [ 2.812773] systemd[1]: [/lib/systemd/system/wyliodrin-server.service:3] Fail Expecting device dev-ttyMFD2.device...d argument
      [ OK ] Reached target Remote File Systems.
        Expecting device sys-subsystem-net-devices-wlan0.device...
        Expecting device dev-disk-by\x2dpartlabel-factory.device...
      [ OK ] Reached target Paths.
      [ OK ] Reached target Swap.
      [ OK ] Set up automount boot.automount.
      [ OK ] Created slice Root Slice.
      [ OK ] Listening on Journal Socket.
      [ OK ] Listening on Delayed Shutdown Socket.
      [ OK ] Listening on udev Control Socket.
      [ OK ] Listening on /dev/initctl Compatibility Named Pipe.
      [ OK ] Listening on udev Kernel Socket.
      [ OK ] Created slice User and Session Slice.
      [ OK ] Created slice System Slice.
        Starting Apply Kernel Variables...
        Starting Create list of required static device nodes...rrent kernel...
        Mounting Debug File System...
        Starting Load Kernel Modules...
        Mounting POSIX Message Queue File System...
        Starting udev Coldplug all Devices...
        Starting Journal Service...
      [ OK ] Started Journal Service.
        Starting Remount Root and Kernel File Systems...
        Mounting Temporary Directory...
      [ OK ] Created slice system-serial\x2dgetty.slice.
      [ OK ] Created slice system-getty.slice.
      [ OK ] Reached target Slices.
      [ OK ] Set up automount home.automount.
      [ OK ] Mounted POSIX Message Queue File System.
      [ OK ] Mounted Debug File System.
      [ OK ] Mounted Temporary Directory.
      [ OK ] Started Apply Kernel Variables.
      [ OK ] Started Create list of required static device nodes ...current kernel.
      [ OK ] Started Load Kernel Modules.
      [ OK ] Started Remount Root and Kernel File Systems.
      [ OK ] Started udev Coldplug all Devices.
        Starting Load/Save Random Seed...
        Mounting FUSE Control File System...
        Mounting Configuration File System...
        Starting Create Static Device Nodes in /dev...
      [ OK ] Mounted Configuration File System.
      [ OK ] Mounted FUSE Control File System.
      [ OK ] Started Load/Save Random Seed.
      [ OK ] Started Create Static Device Nodes in /dev.
        Starting udev Kernel Device Manager...
      [ OK ] Reached target Local File Systems (Pre).
        Mounting /var/volatile...
      [ OK ] Started udev Kernel Device Manager.
      [ OK ] Mounted /var/volatile.
      [ OK ] Reached target Local File Systems.
        Starting Create Volatile Files and Directories...
        Starting Trigger Flushing of Journal to Persistent Storage...
      [ OK ] Started Create Volatile Files and Directories.
      [ OK ] Found device /sys/subsystem/net/devices/wlan0.
      [ OK ] Found device /dev/ttyMFD2.
      [ OK ] Started Trigger Flushing of Journal to Persistent Storage.
      [ OK ] Found device /dev/disk/by-partlabel/factory.
        Mounting Mount for factory...
      [ OK ] Created slice system-systemd\x2drfkill.slice.
        Starting Load/Save RF Kill Switch Status of rfkill2...
        Starting Load/Save RF Kill Switch Status of rfkill1...
        Starting Load/Save RF Kill Switch Status of rfkill0...
        Starting Network Time Synchronization...
        Starting Update UTMP about System Boot/Shutdown...
      [ OK ] Mounted Mount for factory.
      [ OK ] Started Network Time Synchronization.
      [ OK ] Started Load/Save RF Kill Switch Status of rfkill2.
      [ OK ] Started Load/Save RF Kill Switch Status of rfkill1.
      [ OK ] Started Load/Save RF Kill Switch Status of rfkill0.
      [ OK ] Started Update UTMP about System Boot/Shutdown.
      [ OK ] Reached target Sound Card.
      [ OK ] Created slice system-systemd\x2dfsck.slice.
        Starting File System Check on /dev/disk/by-partlabel/home...
      [ OK ] Reached target System Initialization.
      [ OK ] Listening on RPCbind Server Activation Socket.
      [ OK ] Listening on D-Bus System Message Bus Socket.
      [ OK ] Reached target Timers.
        Starting Restore Sound Card State...
        Starting Console System Startup Logging...
      [ OK ] Listening on sshd.socket.
      [ OK ] Started Console System Startup Logging.
      [ 8.180765] systemd-fsck[180]: /dev/mmcblk0p10: recovering journal
      [ OK ] Reached target Sockets.
      [ OK ] Reached target Basic System.
        Starting Edison PWR button handler...
      [ 8.236671] systemd-fsck[180]: /dev/mmcblk0p10: clean, 900/87120 files, 17607/348155 blocks
      [ OK ] Started Edison PWR button handler.
        Starting Edison sketch check service...
      [ OK ] Started Edison sketch check service.
        Starting Hostap daemon service...
        Starting Wyliodrin server...
        Starting Telephony service...
        Starting OpenSSH Key Generation...
        Starting Daemon to handle arduino sketches...
      [ OK ] Started Daemon to handle arduino sketches.
        Starting Daemon to reset sketches...
      [ OK ] Started Daemon to reset sketches.
        Starting Start or stop WiFI AP Mode in Edison...
      [ OK ] Started Start or stop WiFI AP Mode in Edison.
        Starting Wyliodrin hypervisor...
        Starting Daemon to load edison mcu app binary...
      [ OK ] Started Daemon to load edison mcu app binary.
        Starting Bluetooth rf kill event daemon...
      Application available at (physical) address 0x04819000
        VRL mapped to 0xff217000
        App size = 11508 bytes
        App Authentication feature is disabled!
        Resetting IPC
      *** Ready to receive application ***
      [ OK ] Started Bluetooth rf kill event daemon.
        Starting WPA supplicant service...
        Starting D-Bus System Message Bus...
      [ OK ] Started D-Bus System Message Bus.
      [ OK ] Started Telephony service.
        Starting Network Service...
        Starting Login Service...
        Starting Permit User Sessions...
        Starting Cleanjournal service...
      [ OK ] Started Cleanjournal service.
        Starting Crashlog service...
      [ OK ] Started Crashlog service.
        Starting Edison Arduino board LED Blinker...
      [ OK ] Started Edison Arduino board LED Blinker.
        Starting Watchdog sample daemon...
      [ OK ] Started Watchdog sample daemon.
        Starting AP_startup.service...
      [ OK ] Started Network Service.
      [ OK ] Started File System Check on /dev/disk/by-partlabel/home.
      [ OK ] Started OpenSSH Key Generation.
      [ OK ] Started Permit User Sessions.
        Starting Bluetooth service...
        Expecting device /sys/subsystem/net/devices/wlan0...
        Starting Serial Getty on ttyMFD2...
      [ OK ] Started Serial Getty on ttyMFD2.
        Starting Getty on tty1...
      [ OK ] Started Getty on tty1.
      [ OK ] Reached target Login Prompts.
        Mounting /home...
      [ OK ] Reached target Network.
        Starting Mosquitto - lightweight server implementati...SN protocols...
        Starting Zero-configuration networking...
        Starting Network Name Resolution...
      [ OK ] Mounted /home.
      [ OK ] Started Network Name Resolution.
      [ OK ] Started Hostap daemon service.
      [ OK ] Started Mosquitto - lightweight server implementatio...T-SN protocols.
      [ OK ] Started Login Service.
      [ OK ] Found device /sys/subsystem/net/devices/wlan0.
      [ OK ] Started Bluetooth service.
      [ OK ] Started Restore Sound Card State.
        Starting PulseAudio Sound System...
        Starting Hostname Service...
        Starting Daemon to receive the wpa_supplicant event...
      [ OK ] Started Daemon to receive the wpa_supplicant event.
        Starting Load/Save RF Kill Switch Status of rfkill4...
        Starting Load/Save RF Kill Switch Status of rfkill3...
        Starting udhcpd daemon for hostapd...
      [ OK ] Started Load/Save RF Kill Switch Status of rfkill4.
      [ OK ] Started Load/Save RF Kill Switch Status of rfkill3.
      [ OK ] Started udhcpd daemon for hostapd.
      [ OK ] Started Zero-configuration networking.
      [ OK ] Started Hostname Service.
        Starting Daemon to receive the wpa_supplicant event...
      [ OK ] Started Daemon to receive the wpa_supplicant event.
        Starting The Edison status and configuration service...
      [ OK ] Started The Edison status and configuration service.
        Starting Intel_XDK_Daemon...
      [ OK ] Started Intel_XDK_Daemon.
      [ OK ] Started Wyliodrin server.
      [ OK ] Started Wyliodrin hypervisor.
      [ OK ] Started PulseAudio Sound System.
        Stopping Daemon to receive the wpa_supplicant event...
      [ OK ] Stopped Daemon to receive the wpa_supplicant event.
      [ OK ] Created slice user-0.slice.
        Starting User Manager for UID 0...
        Stopping Daemon to receive the wpa_supplicant event...
      [ OK ] Stopped Daemon to receive the wpa_supplicant event.
      [FAILED] Failed to start WPA supplicant service.
      See 'systemctl status wpa_supplicant.service' for details.
      [ OK ] Started User Manager for UID 0.

       

      I have seen some users reporting the same issue in the community : pca953x 1-0020: failed reading register

      We are using the compute module mounted on a custom board. I have seen wifi , i2c and simple file operations failing off late. This has been very unfortunate behaviour from the module. Please see if there are any fixes, or how can i recover this module.

       

      thanks in advance,

      Vinu

        • 1. Re: Edison abnormal behavior in long run
          Intel Corporation
          This message was posted on behalf of Intel Corporation

          Hi Vinu,

          Thanks for your interest in the Intel® Edison™ Platform.

          I'm sorry to hear that your Edison is not working. Could you be so kind to specify what exactly is not working with your Edison? Also, it would be helpful if you could give us more detail regarding the weird behavior of the Edison.

          We will be waiting for your reply, have a nice day!

          Best Regards,
          -Jose P.

          • 2. Re: Edison abnormal behavior in long run
            vinu_gk

            Hi Jose,

            We have figured out what went wrong with the Edison. The MRAA library was not successfully compiled before running the application and that made Edison to crash. We are pretty sure that this was the primary reason. I will get back to you if we are not able to resolve this.

            Thanks

            Vinu

            • 3. Re: Edison abnormal behavior in long run
              Intel Corporation
              This message was posted on behalf of Intel Corporation

              Hi Vinu,

              I'm glad to know that your Edison is working now! Please don't hesitate to come back if any questions come up.

              Have a great weekend!

              Best Regards,
              -Jose P.

              • 4. Re: Edison abnormal behavior in long run
                0andriy

                FerryT used to have similar issue once. My narration of mine words to him: I'm pretty sure that the issue is in SDA hold timeouts which are hard coded into the driver. I copied them to vanilla kernel blindly and rarely saw an issue with GPIO extender initialization. Besides that I would recommend to check current you supply to the board. It might be just a power issue (not enough power / current comes to the board).

                • 5. Re: Edison abnormal behavior in long run
                  FerryT

                  0andriy My issue was different: 1-0020 failed to initialize while 1-0021 .. 1-0023 initialized fine. Moreover, that was on the Edison Arduino, using the vanilla kernel from your tree. Just when I started to suspect a hardware problem, I booted with original image and found the same error. I then had the bright idea to do a cold start (unpower) instead of warm (shutdown, followed by pressing the reset button).

                   

                  My conclusion is: the kernel (same for vanilla and original edison image kernel) does not initialize all hardware properly on boot. In this case the i2c slave.

                   

                  We already found similar problems in the the hsu (high speed uart) a year ago on the original kernel. There appear to be ways to crash the Edison (writing a string larger than buffer size to the uart) that will only recover with a cold start.

                   

                  Nothing wrong with your kernel though. Apart from the stuff not implemented yet, what is implemented is just fantastic.