1 2 Previous Next 18 Replies Latest reply: Sep 2, 2011 8:42 PM by hoffman RSS

    Queries on ETI Baremetal framework




      I wanted to run barmetal apps using the ETI SCC development framework. I have installed it and have just read the user manual. However

      I have few queries about the usage of the framework. If anyone has already worked on the framework please answer.

      - Does the launcher resets the cores by itself or do we have to reset the cores explicitly before launching any app?  I want to run an app

      on cores running linux.

      - Is there a way to specify the cores selectively on which to run an application ? I only see flags in the manual that take a single number and then perform operations on cores from 0 to that number. (e.g. -z#)

      - Do the MPI and streams api use DRAM by default ? If yes, then how can I use MPB for message passing ?

      - How can the scc_find_region function be actually used ?

        • 1. Re: Queries on ETI Baremetal framework


          I'm a new baremetal user as well, but I'm confused by your question

          I want to run an app on cores running linux.

          If it's baremetal, then there is no OS running on the cores. All you got running on the cores is your baremetal program. I thought that's what baremetal was all about.

          • 2. Re: Queries on ETI Baremetal framework

            Sorry for the confusion. Actually I am sharing the SCC with other users and it has currently linux loaded on all the cores.

            So just wanted to ask if I should explicitly reset the cores or the framework will do it automatically when it launches a baremetal app?



            • 3. Re: Queries on ETI Baremetal framework

              Well, I booted SCC Linux on all my cores. I looked ay sccPerf in the Gui. All the arrows were green, indicating that SCC Linux was running on the cores. I then had to quit the Gui because the launcher will not run at the same time as sccGui. Then, I ran "launcher -z8 hello". Then, I brought up the Gui again and now all the cores have gray arrows. It looks as if the launcher reset all the cores.


              You can use the Gui even as a remote user although many users prefer not to. Remotely, VNC works better than X11 forwarding, If you do not use the Gui, you can check whether SCC Linux is running by issuing "sccBoot -s".


              I think it makes sense that running a baremetal program resets Linux on the cores on which you run the program. But it does look as if it resets all the cores.

              • 4. Re: Queries on ETI Baremetal framework


                • 5. Re: Queries on ETI Baremetal framework

                  - as ted kindly pointed out, the eti launcher does reset all the cores and

                     runs your application without an operating system. linux would need to

                     be rebooted on the cores in order to use it after using the launcher


                  - no, there currently isn't any way to specify which nodes get used. the

                     cores are assigned in a contiguous block starting at core 0.

                     i can see why it might be interesting. this is something we can consider



                  - mpb should be the default mode for streams and mpi communication


                  - scc_find_region was intended to allow the user to query the local core address

                     for the various resources that the launcher maps into the core's memory. the

                     original reason for exposing this was to allow users to specify the '-r' flag

                     to the launcher, and then use the mpbs for their own communications rather

                     than the streams interface


                     unfortunately, the rest of the resources (other cores gpbs, etc) are rather

                     haphazardly represented, and this might be cleaned up in later releases

                  • 6. Re: Queries on ETI Baremetal framework

                    its not really necessary to reset all the cores....but while brining it up we've had problems with

                    rouge programs still executing from earlier runs and screwing things up


                    this is behaviour that can of course be changed

                    • 7. Re: Queries on ETI Baremetal framework

                      Thanks Hoffman!

                      I just have one more query. I looked at this old post which mentions that ETI's framework uses

                      DRAM for messaging.


                      I know the post is quite old and things might have changed but just wanted to confirm

                      if they have changed the framework so that both MPI and streams api use mpb.

                      I am using the version compatible with scc 1.3



                      Vaibhav Jain

                      • 8. Re: Queries on ETI Baremetal framework

                        ok, sorry, I misspoke. its been a few months and our use of the terminology is confusing.


                        the ETI code currently uses dram for the message bodies, and the MPB for the

                        synchronization pointers. with this arrangement we get similar bandwidth numbers

                        as RCCE, but are able to cover larger control transfer latencies. I suspect

                        though this leaves us more subject to contention problems.


                        this would be easy to add as a flag in a later release. you can also explore the use of the -r flag

                        which maps the MPBs and lets you do what you want with them. with -r the channels use

                        to communicate with the launcher are allocated (pointers and bodies) entirely out of DRAM,

                        and no channels are set up between the backend nodes

                        • 9. Re: Queries on ETI Baremetal framework

                          Hi hoffman,

                          I wish to experiment with message passing entirely using mpb. Can you please suggest how to proceed after

                          using the -r option. The user manual says that after using -r  MPI and streams are unavailable.



                          • 10. Re: Queries on ETI Baremetal framework

                            with the -r option, the messaging that the ETI software uses doesn't use the MPBs, you're free to

                            play with them as you like. which implies you'd need to write your own messaging layer in

                            order to use them.


                            it would be possible to add another mode which drastically reduces the MPI mtu and

                            allocates the message buffers out of the MPB space. we dont really have alot of resources

                            to do this, but are looking at various options for ongoing support of the software, so implementing

                            this is a possiblity


                            how important is the MPI layer to your work? or are you just interested in looking at messaging


                            • 11. Re: Queries on ETI Baremetal framework

                              I am interested in looking at the messaging performance and don't care much about MPI.

                              • 12. Re: Queries on ETI Baremetal framework

                                it should be pretty straightforward to use the '-r' mode and set up ring buffers in the mpbs

                                in order to run benchmarks. is it acceptable to poll for message arrival, or would you prefer

                                to use asynchronous interrupts?

                                • 13. Re: Queries on ETI Baremetal framework

                                  Actually  right now my aim is to gain a better understanding of the system and of the tradeoffs involved in message-passing(such as dram vs mpb).

                                  So to begin with I was thinking of implementing few simple baremetal applications which can use mpb for message passing.

                                  This is basically primitive work. So I was hoping to get some directions and may be some sample code so that I can get started.

                                  I am not looking into running benchmarks and collecting statistics right now (though it might come at a later stage).





                                  • 14. Re: Queries on ETI Baremetal framework

                                    well, in ETI land, assuming you are doing polling, all you'd need to do is run


                                      memory_region m;

                                      scc_find_region(MEMORY_REGION_MPB, target, &m)


                                    where m is defined in memory_region.h, and target is the name of the remote node.

                                    m will contain a pointer in your address space to that node's 8k MPB base.


                                    static inline void invalidate_l1_mpb()
                                        __asm__ volatile ( ".byte 0x0f; .byte 0x0a;\n" );


                                    would then be used to flush the lines with the MPB tag

                                    1 2 Previous Next