1 2 3 Previous Next 87 Replies Latest reply on Dec 1, 2015 12:44 PM by AlexT_Intel

    How to enable the Yocto-provided package management capability

    AlexT_Intel

      UPDATE 19-Sep-2014: The post is now two-part, first one describes how to create your own repo, second one has information about the existing repo

       

      Part 1: Creating your own repo

      As some of you may know, the Linux image you create after following the BSP Build Guide is not the only result of the process.

      It also builds a lot of additional software packages, only some of which are making it into the image. You can install those additional ones using Yocto's package management feature (which is being added by default) and the respective tool called opkg.

       

      The idea is pretty much the same as with those customary package managers you may be using today, like yum, zypper, apt-get and so on. You need to have a package manager tool, repository of packages and a way to access it over the network using e.g. HTTP protocol.

       

      The repository is created by Yocto during BSP build, the package manager is in your Linux image by default and here's the instruction on how to enable the HTTP sharing of the repo and what settings you need on the Galileo.

       

      This is for Apache, you'll need to adjust accordingly for your favorite HTTP server. I also assume you have network connectivity on your board and can access the console either via serial port or SSH. For instructions on enabling those please see other threads - search is your friend, we have excellent guides in this community already.

       

      1) Create a VirtualHost for your package web site, use the below config excerpt as a template. There may be a lot of details, depending on your distro and preferences, but this should give you an idea (if it doesn't - please ask).

      NameVirtualHost *:80

       

      <Directory "/srv/www/htdocs">

          Options FollowSymLinks

          AllowOverride None

      </Directory>

       

      <VirtualHost *:80>

              DocumentRoot "/srv/www/htdocs/pkgrepo.my.local"

              ServerName pkgrepo.my.local

              UseCanonicalName off

              <Directory "/srv/www/htdocs/pkgrepo.my.local">

                      Options +Indexes

                      AllowOverride none

                      Order allow,deny

                      Allow from all

              </Directory>

      </VirtualHost>

       

      Here /srv/www/htdocs/pkgrepo.my.local is a symlink to the directory where Yocto puts the packages and package indexes. On my machine the full path to it is: /home/dev/yocto/meta-clanton_v0.8.0/yocto_build/tmp/deploy/ipk/

      That is, I've unpacked the BSP into /home/dev/yocto/meta-clanton_v0.8.0 and yocto_build is the build directory created by default.

       

      Make sure Apache user, typically www, has read access to that ipk directory. Generally you can just run

      chmod -R o+rx /home/dev/yocto/meta-clanton_v0.8.0

      to get that.

       

      2) Reload Apache to have it pick up the config (/etc/init.d/apache2 reload, or whatever your distro's equivalent is)

       

      3) In the output of the below command you should see the newly created VirtualHost and you should be able to access it using the browser and see the listing of the directories beneath that ipk one. Here I have only one VirtualHost:

      lnx:~ # apache2ctl -S

      VirtualHost configuration:

      wildcard NameVirtualHosts and _default_ servers:

      *:80 is a NameVirtualHost

      default server pkgrepo.my.local (/etc/apache2/vhosts.d/pkgrepo.my.local.conf:11)

      port 80 namevhost pkgrepo.my.local (/etc/apache2/vhosts.d/pkgrepo.my.local.conf:11)

      Syntax OK

       

      3) On Galileo (and you should be running off of the SD card, not from the SPI "read-only" image) add the address and the name of the package repo server into /etc/hosts, so that we can use a name instead of the plain IP address (the line in bold is what you need to add):

      root@clanton:~# cat /etc/hosts

      127.0.0.1       localhost.localdomain localhost

      192.168.10.100  pkgrepo.my.local      pkgrepo

       

      4) Add the repository information into /etc/opkg/base-feeds.conf. By default you get three directories under ipk, each needs a separate line:

      root@clanton:~# cat /etc/opkg/base-feeds.conf

      src/gz all     http://pkgrepo.my.local/all

      src/gz clanton http://pkgrepo.my.local/clanton

      src/gz i586    http://pkgrepo.my.local/i586

       

      5) Refresh the package index on the Galileo and you're good to go. Here's the expected output:

      root@clanton:~# opkg update

      Downloading http://pkgrepo.my.local/all/Packages.gz.

      Inflating http://pkgrepo.my.local/all/Packages.gz.

      Updated list of available packages in /var/lib/opkg/all.

      Downloading http://pkgrepo.my.local/clanton/Packages.gz.

      Inflating http://pkgrepo.my.local/clanton/Packages.gz.

      Updated list of available packages in /var/lib/opkg/clanton.

      Downloading http://pkgrepo.my.local/i586/Packages.gz.

      Inflating http://pkgrepo.my.local/i586/Packages.gz.

      Updated list of available packages in /var/lib/opkg/i586.

       

      On the basic level opkg is pretty similar to apt-get, you can search, list installed and remote repo packages, etc. Here's a nice guide to opkg: http://wiki.openwrt.org/doc/techref/opkg

       

      The above is based on the Yocto's manual, you can find some additional details here: http://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#using-runtime-package-management

       

      Part 2: Existing repo

      If you don't want to create your own one and just need a couple of packages, there's a repo I've created and which Daveman is generously hosting. See the opkg config details on this page: Package Repo Configuration Instructions. The page will be kept up to date with any repo relocations or changes.

       

      If you have any questions to the above - feel free to ask in this thread and your feedback is more than welcome.

        • 1. Re: How to enable the Yocto-provided package management capability
          AlexanderMerz

          I really love to see this explanation here! Thank you!

          This should be go straight to official docs.

           

          I really ask myself why Intel does not setup a regular package server? This would make your live much easier. Especially for all the guys out there like me, who don't have a dedicated build machine for the BSP stuff but. I'm on a Mac running Linux in a VM for building the BSP. Booting the VM and do all the network magic to make the VM to act as a webserver just for installing a package is not always an option.

          • 2. Re: How to enable the Yocto-provided package management capability
            zJack

            Great info. Now just need to find a server that everyone can use

            • 3. Re: How to enable the Yocto-provided package management capability
              Thomas_Faust_(Intel)

              I wrote this text some hours ago and put it in the queue for the admin to review my post and publish it. It never happened and now I can post without admin review again … arg … what happened to my post? Do I need to write it all over again? (Or should I ask the NSA for an backup of that missing post?)

               

              Hi AlexT.

              Thanks a lot for getting this tutorial together. Before I was copying my ipk pacakges to the target manually – with your approach, at least the dependencies are solved. Don’t get me wrong, I think the Yocto system is cool and works to create an embedded image. But here we are makers and want to have a little bit more freedom in our choice.

               

              The functionality of opkg and apt-get (APS) is the same – no doubt. The difference between Yocto/opkg/ipk and a full mature Linux distro (like debian – or name any other) is the availability of ready-to-use packages.

              Here is my real world example:

              A while ago I build a CCTV with an raspberry - http://www.raspberrypi.org/archives/tag/motion-detector

              How to make a DIY home alarm system with a raspberry pi and a webcam - https://medium.com/p/2d5a2d61da3d

               

              I tried to do that with the Galileo board as well, so I needed ‘motion’ - http://motion.sourceforge.net/

              So, I compile this on the full Yocto image with devtools, but failed because of some dependencies.

               

              If the debian would work, it would just be an ‘apt-get install motion’ – done.

               

              To get an ipk package for opkg on the Yocto build, I need to:

              1. create an bb (bitbake recipe) for motion
              2. figure out what dependencies are needed
              3. create bb recipes for the dependencies as well
              4. run the full-image bitbake again (took me about 4 hours the last time)
              5. on Galileo: opkg install motion

              Please correct me if I’m wrong on the steps.

              • 4. Re: How to enable the Yocto-provided package management capability
                mdelgert

                Hi guys I agree with Thomas_Faust_(Intel).

                 

                Over the last several weeks I been trying to get Yocto to add one simple packages like mono to work!

                 

                In Debian & Ubuntu it was "$sudo apt-get install mono" one line that was it completely done.

                 

                I've gone through hundreds of hours trying to get Yocto to package "meta-mono" on my Galileo with no success.

                 

                C# is a popular language it should be reasonable that someone else would want to install these libraries from http://git.yoctoproject.org/cgit/cgit.cgi/meta-mono

                 

                Has anyone found a active Yocto support forum or active community that could help?

                 

                The article above "How to enable the Yocto-provided package management capability" does Thomas and I no good for the reason "meta-mono" recipe errors out in the bitbaske process and Thomas is trying to add one simple library from the linux command prompt. Why can't we have vanilla linux "with apt-get or public popular package manager" on the Galileo SD boards as a simple download from Intel?

                 

                I'm currently able to get Debian to run on my Galileo and would like to share my image on Dropbox for the other developers that need "apt-get".

                 

                May Intel please provide the instructions how to install all the Galileo board dependencies on a vanilla linux distribution like"Debian, Ubuntu and or FreeBSD"?

                 

                When I install Debian on SD card I lose "Sketch", "GPIO" and "CLLOADER". Basically end up with a Galileo board running linux with no Arduino support.

                 

                I see in the latest version http://downloadmirror.intel.com/23197/eng/Board_Support_Package_Sources_for_Intel_Quark_v0.9.0.7z image-full.bb the following dependencies I'm going to build one virtual add all dependencies if successful will post and share this SD image with other developers that need full vanilla linux. So that other developers that don't have the large time investment and learning curve for Yocto & Bitbake build process. I would be a fan of Yocto if I could just get some simple libraries "meta-mono", "mono-xsp", "monotools-server" & "mod-mono" up and running with out Bitbake errors during the build.

                 

                MAGE_INSTALL += "ethtool pciutils"

                IMAGE_INSTALL += "strace"

                IMAGE_INSTALL += "linuxptp"

                IMAGE_INSTALL += "libstdc++"

                IMAGE_INSTALL += "dmidecode"

                IMAGE_INSTALL += "opencv nodejs"

                IMAGE_INSTALL += "python python-modules python-numpy python-opencv"

                IMAGE_INSTALL += "alsa-lib alsa-utils alsa-tools"

                IMAGE_INSTALL += "connman wireless-tools wpa-supplicant bluez4"

                IMAGE_INSTALL += "ppp openssh"

                 

                This Galileo board has allot of promise and is way more appealing to developers than the ARM based processors that is losing linux support. Intel gets much more market share and respect with this board all us developers need now is vanilla linux with public package manager like apt-get so we can write applications in other languages like C# etc.

                • 5. Re: How to enable the Yocto-provided package management capability
                  AlexT_Intel

                  Hi Thomas,

                   

                  Yes, the process you describe would be more or less the way to get the package not included into any of the existing Yocto or OpenEmbedded layers (and there are a lot of them) built.

                   

                  And no, I not saying that's the best or the only way to do things :-) I was just interested in getting this Yocto-provided capability working and published the results of my exploration.

                   

                  I definitely agree the well-established distros have a very big advantage of having an elaborated package ecosystem when a lot of packages are available right away in publicly available repos. That's why I appreciate the work you and others do in those Debian-related threads and I have mentioned that there.

                   

                  I believe there's a place for both approaches. For instance I personally don't need a full-blown distro on Galileo, because I don't need an arbitrary package on it at arbitrary moment. The typical distro will most probably be slower than trimmed-down Yocto-generated one and have tons of unnecessary bloat wasting the SD card space I don't need on a board which will do just several pretty much specified functions in my projects and I will prepare packages or plain compiled binaries for those using Yocto or cross-compilation toolchain or a temporary installed SDK image in advance. I prefer to have an optimized system able to run required functions at maximum possible performance level with only those functions available (for security and again performance), that's where Yocto with its capability of generating a tailored Linux distro comes in handy.

                   

                  On the other hand, you and other people working in that thread, as well as many other people, I'm pretty sure, would want to have a generic Linux playground or avoid all those compilation steps and be able to install whatever at whatever moment, or have some other reasons I just can't think right now - and that's IMHO absolutely ok, that's the Maker and Open Source spirit. My personal guess is that with time, as community grows and Quark-based product become more commonplace, we'll see distros catching up on that and rolling out versions tailored for it, which wouldn't require any major work on our side. Especially given the fact that's x86 after all and Linux is used from the very beginning as the OS - so I guess they'll upstream all the specific tweaks to the vanilla kernel.

                  • 6. Re: How to enable the Yocto-provided package management capability
                    Thomas_Faust_(Intel)

                    Hi AlexT. (BTW: Shoot me an Intel email, would like to share some thoughts)

                     

                    You are absolutely right. The debian approach is not for the performance. Compared to the Yocto build, it’s very slow.

                     

                    The beauty about the Quark/Galileo ecosystem is that the Irish team managed to make all information public. From board schematics, to architecture specification, even the UEFI source code and the JTAG debug interface (afaik first time ever for an IA processor). That means that we are not limited to the approach that is given to the community in first place. Yocto is a great environment and most likely fits to the most for an embedded system where Quark wants to play. But everyone can build on its own SW stack. Maybe I should try MINUX or FreeRTOS, just for the fun of it.

                    What I want to say is that I’m happy that we have these options on Galileo and I’m really looking forward to see what the community is bringing up. May there be no limits!

                     

                    I got a little off-topic. Sorry. Please keep posting the Yocto recipes. It fuels all the ideas out there to build a great IA/Quark/Galileo ecosystem.

                    • 7. Re: How to enable the Yocto-provided package management capability
                      AlexT_Intel

                      Thomas, you know what - the things you've just mentioned are exactly my opinion/view too! :-) I had them in mind writing the above reply, but didn't want to make it any longer than it already was.

                       

                      > Shoot me an Intel email, would like to share some thoughts

                      Will do.

                      • 8. Re: How to enable the Yocto-provided package management capability
                        AlexT_Intel

                        Okay, here goes the publicly accessible repo :-) This is an initial attempt, we'll see how it works out, because it's not ideal from my point of view, but it required minimal effort from me so far.

                         

                        I have used bintray.com to create a repository with all the packages, which "bitbake image-full" compiles (+newer node.js, redis and not included by default ntp packages). I've tested it to work fine with my board and your feedback is welcome.

                         

                        The file management features at Bintray are somewhat limited and they only provide 500MB for free, but at least they offer a plain repo-type web hosting without those surplus website templates and file management limitations typical web hosters do. If someone knows any better alternative - I'm all attention :-)

                         

                        To configure your Galileo to fetch packages from there, do the following:

                        1) Make sure you're working with SD card Linux image and not the SPI one;

                        2) Replace anything you have in /etc/opkg/base-feeds.conf with the following (other opkg config files don't need any changes):

                        root@clanton:~# cat /etc/opkg/base-feeds.conf

                        src/gz all     http://dl.bintray.com/alext-mkrs/galileo-packages/all

                        src/gz clanton http://dl.bintray.com/alext-mkrs/galileo-packages/clanton

                        src/gz i586    http://dl.bintray.com/alext-mkrs/galileo-packages/i586

                        3) Run opkg update command and you should see the below output, which means you're successfully communicating with the repo:

                        root@clanton:~# opkg update

                        Downloading http://dl.bintray.com/alext-mkrs/galileo-packages/all/Packages.gz.

                        Inflating http://dl.bintray.com/alext-mkrs/galileo-packages/all/Packages.gz.

                        Updated list of available packages in /var/lib/opkg/all.

                        Downloading http://dl.bintray.com/alext-mkrs/galileo-packages/clanton/Packages.gz.

                        Inflating http://dl.bintray.com/alext-mkrs/galileo-packages/clanton/Packages.gz.

                        Updated list of available packages in /var/lib/opkg/clanton.

                        Downloading http://dl.bintray.com/alext-mkrs/galileo-packages/i586/Packages.gz.

                        Inflating http://dl.bintray.com/alext-mkrs/galileo-packages/i586/Packages.gz.

                        Updated list of available packages in /var/lib/opkg/i586.

                         

                        If you don't see that or have other problems/questions - feel free to post here, I'll see if I can help.

                         

                        Just to make sure no one is mislead - this is not any kind of an official or Intel-endorsed repo. While I do work for Intel, I'm not related to Galileo by any means and in this sense I'm the same Maker as anyone else here, doing that in my spare time as a hobby project. I see a rising demand for such a repo and that's what I came up with to help the community. Let's see how it works out.

                         

                        The packages were compiled with BSP v0.8.0, but theoretically should work with Linux images based on any of the current BSPs (0.7.5, 0.8.0, 0.9.0), but there's definitely no guarantees, as with everything in the Makers world you use that on your own risk :-)

                         

                        I recommend you to back up your image-full-clanton.ext3 file, which contains the OS filesystem image, before installing any packages, that way you should be able to always get back to known working state if something goes wrong.

                        • 9. Re: How to enable the Yocto-provided package management capability
                          Thomas_Faust_(Intel)

                          Thanks a lot AlexT for setting up the repository. I hope the available binary pacakges will increase over time and more and more useful tools will be available to the maker comunity.

                          My first request (as already discussed in the other thread) is 'motion'.

                           

                          BTW: Nice disclaimer.

                          Keep the good work up.

                          • 10. Re: How to enable the Yocto-provided package management capability
                            AlexT_Intel

                            Thanks :-)

                             

                            Motion happens to be on my list to after bundled avconv/avserver provided less than desired stability :-) I've almost started yesterday, but eventually didn't have time to look to any serious degree. I'll be posting stuff as I come up with something meaningful.

                            • 11. Re: How to enable the Yocto-provided package management capability
                              AlexanderMerz

                              Amazing! Thank you! I will give it a try as fast as possible!

                              • 12. Re: How to enable the Yocto-provided package management capability
                                AlexT_Intel

                                For everyone's information, due to ridiculously deficient file management capabilities at Bintray.com (probably package repository use case is not what they're after), I've decided to move the repo to another hosting. After some selection process I've picked Altervista as they seem to offer the most features and disk space (2x compared to Bintray) for free.

                                 

                                Please find the Galileo configuration instructions here: Package Repo Configuration Instructions and feel free to let me know in this thread if you have any troubles with the new repo.

                                 

                                The old one on Bintray.com will be deleted in about a week.

                                • 13. Re: How to enable the Yocto-provided package management capability
                                  Copernicus

                                  I'd like to install GCC. I've recompiled the 0.9.0 BSP and also connected to the Altervister repo. (Thank you for putting this together - it's great to have!)

                                   

                                  Is there a chance you'll add GCC to the repository?

                                  • 14. Re: How to enable the Yocto-provided package management capability
                                    rmm200

                                    Alex's package repository is great - but gcc is so basic, you really should have it in your yocto build.

                                    Sergey has a really good blog on adding tools to your build - take a look at his image-sdk mods.

                                    If you are building your own bsp, image-sdk is no harder.

                                    I did not even make image-sdk, I just modified the image-full recipe.

                                     

                                    Sergey's Blog - Malinov Family Web Presence

                                    1 2 3 Previous Next