2 Replies Latest reply on Aug 11, 2015 11:31 PM by isvissu

    Galileo Gen 2 Facial Recognition project update

    Toni_K

      I started out with the lofty goal of attempting to build a webcam interface that would use the Galileo to take a picture of a person, process this image, compare it to a known compilation of ID pictures, and spit out on an LCD screen a greeting to the person currently standing there. Unfortunately, there were simply too many holes in the documentation currently available for the Galileo for this to succeed properly at this time, and after spending well over 80 hours between image builds, research time and beating my head against the desk, I'm no further than I was at the beginning of this project, except frustrated.  This is a summary of what I tried, what info I was able to find, and what bits were helpful. I do recommend however for anyone looking at this in the future to look at this demo project for inspiration of this Face Recognition with Python, in under 25 lines of code - Real Python. The Python code is a slick interface that Shatnu Tiwari does a great job explaining, and modifying it to work with OpenCV for eigenface recognition would not be an extremely difficult undertaking. The problem comes down to implementing Python + OpenCV on the Galileo.

       

      Here's what I was able to accomplish on this project so far.

      Upon first receiving the board, plug the board in over the microB connection to the computer, and plug in the 5V power supply that is included.

      Update the USB drivers on your machine, and then update the Galileo firmware via the Arduino IDE. (These instructions can be found in-depth here: Galileo Getting Started Guide - learn.sparkfun.com).

      The gen 2 board can freeze up the terminal if attempting to use the sketch described for communicating with the gen 1 in the SparkFun tutorial, so I recommend against using that. Set the IOREF jumper on the Galileo to match your FTDI board, and plug in the FTDI board on the Galileo's header (only applicable to gen 2).

       

      I then attempted to get a webcam functioning with the Galileo. I initially started with this Webcam - USB - SEN-11957 - SparkFun Electronics.

      After plenty of digging, I was able to find this helpful resource for Linux device drivers Linux UVC driver & tools. After running $dmesg | tail  in the Galileo terminal connection, I was able to look at all of the device stats on the board. I was then able to verify whether or not the webcam enumerated as a USB device with the command $lsusb. The original camera I attempted was ID'ed as 0c45:62f1 and after searching the Linux driver site, determined it was labeled as an Avatec CMA-L688 HueHD, which is not currently supported. Switching to another webcam, I was able to to determine the device ID was 1871:0306, which is supported by the driver.


      The next step is rebuilding the image.

      The current firmware and SD image provided by Intel do not currently support running OpenCV. In order to get OpenCV running on your Galileo, you will need to rebuild the full Linux image to boot off of the SD card.

      NOTE: There are several different sets of build instructions available, including [this post](http://www.malinov.com/Home/sergey-s-blog/intelgalileo-buildinglinuximage) by Sergey Kiselev, which was very helpful in getting started with rebuilding the image.

       

      Build Environment

      Currently, the build is not compatible with Ubuntu 14.04 LTS (you can find more info on that here Bitbake error). If you do not have access to a Linux machine, you will need to create a Virtual Image.

      If you don't already have it installed, you will need to download and install [VirtualBox](https://www.virtualbox.org/wiki/Downloads). This is a freeware version that is compatible on all major operating systems.

      Once you have it downloaded and installed, you will need to create a virtual instance of Linux. I chose the [Ubuntu 12.04](http://releases.ubuntu.com/12.04/ubuntu-12.04.4-desktop-amd64.iso) image.

      For more detailed information of installing a virtual image, please check out the [VirtualBox User Manual](https://www.virtualbox.org/manual/UserManual.html).

      One thing to note when setting up your virtual machine is that you will need to allocate enough memory and processors to the virtual box in order to prevent memory overflows during the build process.

      These may need to vary for your particular pc that you are running the virtual machine on, but you do want to bump up the memory on the virtual machine from the preset of 512MB.

      Downloads

      Once you have your Linux machine configured, you then need to download the [Board Support Package for Intel Quark](http://downloadcenter.intel.com/confirm.aspx?httpDown=http://downloadmirror.intel.com/23197/eng/Board_Support_Package_Sources_for_Intel_Quark_v1.0.1.7z&Lang=eng&Dwnldid=23197).

      You will also want to download the [Board Support Patches and Build Instructions](http://downloadcenter.intel.com/confirm.aspx?httpDown=http://downloadmirror.intel.com/24000/eng/BSP-Patches-and-Build_Instructions.tar.gz&Lang=eng&Dwnldid=24000).

      Build the Image

      To begin with the rebuild process, open Terminal on your virtual machine. You need to then install all the necessary packages that will assist during the rebuild process.

          $sudo apt-get install build-essential gcc-multilib vim-common git diffstat gawk chrpath 7z file texinfo

      If you receive any errors running the previous commands, you are likely missing dependencies for the packages above. Install the missing dependencies, and then try again to install the packages listed above.

      The next step is to unzip the Board Support Packages and Patches.

          $7z x Board_Support_Package_Sources_for_Intel_Quark_v1.0.1.7z  

          $tar xzvf meta-clanton_v1.0.1.tar.gz

          $tar xzvf BSP-Patches-and-Build_Instructions.tar.gz

      Once you have the files all unzipped, it's time to install the patches. Please double check that the patches.txt file has not been updated to include additional or different steps.

      First, you will be installing the x264 patch:

          $cd meta-clanton_v1.0.1/meta-oe/meta-oe/recipes-multimedia/x264/

          $sed -i 's|1cffe9f406cc54f4759fc9eeb85598fb8cae66c7|bfed708c5358a2b4ef65923fb0683cefa9184e6f|' x264_git.bb

      The second step is to install the UART patches:

          $cd meta-clanton_v1.0.1/meta-clanton-bsp/recipes-kernel/linux/files/

          $cp ~/patches/uart-1.0.patch .

          $echo 'SRC_URI += "file://uart-1.0.patch"' >> ../linux-yocto-clanton_3.8.bb

          $cp ~/patches/uart-reverse-8.patch .

          $cd meta-clanton_v1.0.2

          $patch -p1 < ../uart-reverse-8.patch

      Third, include the USB patch.

          $cd meta-clanton_v1.0.1/meta-clanton-distro/recipes-galileo/galileo-target/files/

          $cp ~/patches/usb_improv_patch-1.patch .

          $echo 'SRC_URI += "file://usb_improv_patch-1.patch"' >> ../galileo-target_0.1.bb

      Finally, you need to include the OpenSSL patch.

          $cd $WORKSPACE/meta-clanton_v1.0.1/poky/meta/recipes-connectivity/openssl/

          $mv openssl-1.0.1e    openssl-1.0.1h

          $mv openssl_1.0.1e.bb openssl_1.0.1h.bb

          $sed -i 's|66bf6f10f060d561929de96f9dfe5b8c|8d6d684a9430d5cc98a62a5d8fbda8cf|' openssl_1.0.1h.bb

          $sed -i 's|f74f15e8c8ff11aa3d5bb5f276d202ec18d7246e95f961db76054199c69c1ae3|9d1c8a9836aa63e2c6adb684186cbd4371c9e9dcc01d6e3bb447abf2d4d3d093|' openssl_1.0.1h.bb

          $sed -i 's|file://openssl-fix-doc.patch||' openssl_1.0.1h.bb

          $sed -i 's|file://0001-Fix-for-TLS-record-tampering-bug-CVE-2013-4353.patch||' openssl_1.0.1h.bb

          $sed -i 's|file://0001-Fix-DTLS-retransmission-from-previous-session.patch||' openssl_1.0.1h.bb

          $sed -i 's|file://0001-Use-version-in-SSL_METHOD-not-SSL-structure.patch||' openssl_1.0.1h.bb

          $sed -i 's|file://CVE-2014-0160.patch||' openssl_1.0.1h.bb

      Once you have the patches properly installed, it's time to modify the image to run eglibs, instead of ulibs. This is necessary for OpenCV to function inside the image once everything is compiled.

          $cd meta-clanton_v1.0.1/yocto_build/

          $sudo gedit conf/local.conf

      Once the conf/local.conf file opens, you need to edit DISTRO from clanton-tiny to clanton-full. The file should read like this once you are finished:

          DISTRO ?= "clanton-full"

      Save this file and exit.

      Next, you need to edit out the uClibc patch:

          $cd meta-clanton_v1.0.1/meta-clanton-distro/recipes-multimedi/v4l2apps/

          $sudo gedit v4l-utils_0.8.8.bbappend   

      Each line in this file needs to be commented out. The file should look like this once you are finished.

          #FILESEXTRAPATHS_prepend := "${THISDIR}/files:"

          #SRC_URI += "file://uclibc-enable.patch"

          #DEPENDS += "libiconv"   

      Again, save this file and exit.

      Finally, you also want to update the OpenCV library loader.

          $cd meta-clanton_v1.0.1/meta-oe/meta-oe/recipes-support/opencv/

          $sudo gedit opencv_2.4.3.bb   

      Update the following lines:

          http://sourceforge.net/projects/opencvlibrary/files/opencv-unix/2.4.10/opencv-2.4.10.zip/download

          SRC_URI[md5sum] = "ec63952d3a3dff965d5fdde765926821"

          SRC_URI[sha256sum] = "1bf4cb87283798fd91669d4f90b622a677a903c2

       

      .....Aaaaand this is about where the project fell apart.

       

      I found a few more useful links of things to try on the forums ( such as running the bitbake commands here and attempting the build again: Bitbake Errors), but I still could not get it to work.

       

      I even tried following the advice here Intel galileo eglibc image regarding trying to use the devkit-daisy-multilibc to rebuild the image, but the git repo for yocto was bricked as soon as I downloaded it. It seems like this link did work at one point, as the suggestion to use that image was repeated in several places on the forums (such as here Trouble connecting Galileo to access point.), but it, like many other things in the documentation and files on this system, appear to have been abandoned for the last few months. There was also the fix mentioned here failed to build yocto project for Galileo, however that also led to rebuilding errors.

       

      Another thing I discovered while trying various options with different images and such was that the Galileo being plugged in over USB actually bricked out my other USB ports on my computer. It would lead to errors on the insertion of any USB device stating that the device had malfunctioned. Needless to say, when attempting to debug several items at once, this turns into quite a pain. This was on a Windows machine, so I'm guessing it is something to do with the USB drivers for the Galileo, but I'm not sure of the exact error.

       

      As I mentioned, I'm incredibly frustrated with this. There's a lot of potential for makers out there with the Galileo, but currently the documentation is so spotty ( or simply in conflict with itself - for example, two pages showing "the-most-up-to-date" downloads for the Galileo that are different version numbers), that I couldn't recommend this as a product to anyone who asked my opinion. Considering how many different feature implementations would require a rebuild of the image, the fact that there aren't more up-to-date patches available or more clear, explicit directions of which downloads are the proper ones would make this impossible for a beginner in embedded electronics (and thus far, impossible for me, who isn't even a beginner). It's great that Intel wants to build up the community on this product and encourage it as a useful tool to the community, but that doesn't mean they are off the hook for documentation/firmware updates/etc. The only positive thing I can say regarding working with the Galileo is that I now feel that rebuilding entire Linux images should be no problem for me in the general case - but that wasn't at all the purpose of working with this hardware.