7 Replies Latest reply on Jan 10, 2012 8:53 AM by Nil

    Barrelfish on the SCC

    tedk

      Is anyone currently looking at Barrelfish on the SCC? Locally we just recently built the SCC version, but haven't yet done anything with it. We didn't follow the SCC Barrelfish instructions exactly. What we did is described in http://marcbug.scc-dc.com/bugzilla3/show_bug.cgi?id=324

        • 1. Re: Barrelfish on the SCC
          tedk

          To build barrelfish, look at Bug 324. To run barrelfish on the SCC you need to install a new crbif (the PCIe driver). Look at Bug 355.

           

          The source for the patched crbif driver is available from http://www.dcl.hpi.uni-potsdam.de/research/scc/serial.htm Thanks to Jan-Arne Sobania for this driver. A built kernel module crbif.ko is available on our public SVN, http://marcbug.scc-dc.com/svn/repository/trunk/crbif-barrelfish/

           

          When you download the source from the HPI site, you get a file called crbif-dkms-1.1.3-tty.tar.gz. Open up that file and you get a directory called crbif-dkms-1.1.3. Enter that directory and issue the following make.

           

          $ make -C /lib/modules/`uname -r`/build M=`pwd` modules
          make: Entering directory `/usr/src/linux-headers-2.6.32-28-generic'
            CC [M]  /home/username/CRBIF/crbif-dkms-1.1.3/mcedev_main.o
            CC [M]  /home/username/CRBIF/crbif-dkms-1.1.3/mcedev_common.o
            CC [M]  /home/username/CRBIF/crbif-dkms-1.1.3/crbif_main.o
            CC [M]  /home/username/CRBIF/crbif-dkms-1.1.3/crbif_file.o
            CC [M]  /home/username/CRBIF/crbif-dkms-1.1.3/crbif_net.o
            CC [M]  /home/username/CRBIF/crbif-dkms-1.1.3/crbdev_dma.o
            CC [M]  /home/username/CRBIF/crbif-dkms-1.1.3/crbdev_misc.o
            CC [M]  /home/username/CRBIF/crbif-dkms-1.1.3/crbtty.o
            LD [M]  /home/username/CRBIF/crbif-dkms-1.1.3/crbif.o
            Building modules, stage 2.
            MODPOST 1 modules
            CC      /home/username/CRBIF/crbif-dkms-1.1.3/crbif.mod.o
            LD [M]  /home/username/CRBIF/crbif-dkms-1.1.3/crbif.ko
          make: Leaving directory `/usr/src/linux-headers-2.6.32-28-generic'
          $ ls -l *.ko
          -rw-r--r-- 1 username username 103190 2011-12-09 10:27 crbif.ko

          $

           

          Note that you must disable SoftRAM (and associated virtual UARTs) in sccGui for the patched crbif to work. The problem is that the patches assume that FPGA automatically generates responses to write requests from cores. Hence, if you enable SoftRAM, sccGui automatically disables automatic write responses, and subsequent write operations to I/O ports hang the corresponding SCC core.

           

          The virtual UARTs in sccGui cannot work if the patched crbif is active. It might be possible to use SoftRAM, but only if the software running on the SCC cores does not access the UART I/O ports at all.

          • 2. Re: Barrelfish on the SCC
            Nil

            Hi,

             

             

            In order to install this new crbif do i need standalone SCC system or can it be done on datacenter system as well?

             

             

            Thank you.

            • 3. Re: Barrelfish on the SCC
              tedk

              This is a new thing. Yes, you can do it on a data center system but you need root access. We don't give that out, but we can give you permission to sudo certain commands. If you want to try out the crbif used for barrelfish, let us know through Bugzilla..

               

              The default crbif is installed when we run intall.csh under /opt/sccKit. The crbif source is in a deb file. What we're loooking into is having two possible deb files or just providing an optional crbif.ko.

              • 4. Re: Barrelfish on the SCC
                Nil

                Hi,

                 

                As you suggested i have filled a bug under marc administration needed (366) on 12-12-2011 but not received any response yet. Can someone look into this please.

                 

                 

                Thank you.

                • 5. Re: Barrelfish on the SCC
                  tedk

                  There has actually been a lot of behind-the-scenes work on barrefish on the SCC. Please look at

                  http://marcbug.scc-dc.com/bugzilla3/show_bug.cgi?id=324

                   

                  We are certainly interested in getting barrelfish working on marc systems. We have successfully built barrefilsh on marc037. We now know what is needed in the SCC environment to be able to do this on other marc systems.

                   

                  First of all, you need an older version of GHC, namely GHC 6.10.4. You can read about this on

                  http://wiki.barrelfish.org/Getting_Started

                  We installed this GHC 6,10.4 on marc037  and will also on your marc022. Installing it is not enough. As a non-root user, you must also issue "cabal update" and "cabal install ghc-paths". This is described in the wiki link above.

                   

                  As you point out, you also need hg. Then, we issued the following (these steps are not exactly what you see on the barrelfish.org site, but close). If as you progress in your research, you see an improvement in this build method, please let us know.

                  hg clone http://hg.barrelfish.org

                  cd hg.barrelfish.org

                  mkdir build

                  cd build

                  ../hake/hake.sh -s .. -a scc

                  make

                  <you get lots of output ... be patient>

                  make scc

                  <you get an error ... this is ok, but this procedure can certainly be improved at this step>

                  tools/bin/dite -32 -o bigimage.dat menu.lst

                  cp ../tools/scc/bootvector.dat .

                  bin2obj -m ../tools/scc/bigimage.map -o barrelfish0.obj

                  <you now have a barrelfish0.obj amd barrelfish mt files. The mt files are in ./tools/scc>

                   

                  We have not yet tried to boot barrelfish on the SCC. Once you succeed in doing this, please let us know. But we do know that you need a new crbif, the PCIe driver. Please look at

                  http://www.dcl.hpi.uni-potsdam.de/research/scc/serial.htm

                  and download the

                  bz2 for crbif 1.1.3 (complete). Unzip it (you get a directory called crbif-dkms-1.1.3). Enter that direcotry and build a new crbif.ko by issuing

                  make -C /lib/modules/`uname -r`/build M=`pwd` modules

                   

                  Please look at bug 355 for details about the use of this new crbif driver.

                  http://marcbug.scc-dc.com/bugzilla3/show_bug.cgi?id=355

                   

                  Now as a non-root user, we're still trying to figure out how you can have this new crbif.ko replace the default version. What do you think is the best way? We were thinking of just issuing insmod for you or setting up some setuid script for you or having a cron install the patched crbif for certain times of the day. The default crbif gets installed with install.csh in /opt/sccKit/current/firmware. You have to be root to run this install.csh. It gets the crbif from a deb file in /opt/sccKit/current/firmware/PCIeDriver. We don't see the advantage of making a new deb file with this new crbif driver, but are willing to discuss the possibility.

                   

                  So that's where we are. Now that our tests on marc037 have worked out, we hope to make marc022 configured correctly so that you can build barrelefish for SCC today. Booting it is another matter, but it is something we want to do.

                   

                  Message was edited by: Ted Kubaska ... typo

                  • 6. Re: Barrelfish on the SCC
                    tedk

                    About that need for a patched crbif. sccKit 1.4.2 (not yet available) should come with a crbif that supports barrelfish. sccKit 1.4.2 will be in beta very soon.

                    • 7. Re: Barrelfish on the SCC
                      Nil

                      According to bug 355 in order to relode new driver i have to do

                       

                      /sbin/rmmod crbif
                      /sbin/insmod
                      /opt/sccKit/current/firmware/PCIeDriver/130_svn/mcpc_driver/crbif.ko

                      but under  "PCIeDriver" i have only "crbif-dkms_1.1.3.deb"

                      i am stuck here

                       

                       

                      Any help?

                       

                      Thank you.