I wanted to run barmetal apps using the ETI SCC development framework. I have installed it and have just read the user manual. However
I have few queries about the usage of the framework. If anyone has already worked on the framework please answer.
- Does the launcher resets the cores by itself or do we have to reset the cores explicitly before launching any app? I want to run an app
on cores running linux.
- Is there a way to specify the cores selectively on which to run an application ? I only see flags in the manual that take a single number and then perform operations on cores from 0 to that number. (e.g. -z#)
- Do the MPI and streams api use DRAM by default ? If yes, then how can I use MPB for message passing ?
- How can the scc_find_region function be actually used ?
I'm a new baremetal user as well, but I'm confused by your question
I want to run an app on cores running linux.
If it's baremetal, then there is no OS running on the cores. All you got running on the cores is your baremetal program. I thought that's what baremetal was all about.
Well, I booted SCC Linux on all my cores. I looked ay sccPerf in the Gui. All the arrows were green, indicating that SCC Linux was running on the cores. I then had to quit the Gui because the launcher will not run at the same time as sccGui. Then, I ran "launcher -z8 hello". Then, I brought up the Gui again and now all the cores have gray arrows. It looks as if the launcher reset all the cores.
You can use the Gui even as a remote user although many users prefer not to. Remotely, VNC works better than X11 forwarding, If you do not use the Gui, you can check whether SCC Linux is running by issuing "sccBoot -s".
I think it makes sense that running a baremetal program resets Linux on the cores on which you run the program. But it does look as if it resets all the cores.
- as ted kindly pointed out, the eti launcher does reset all the cores and
runs your application without an operating system. linux would need to
be rebooted on the cores in order to use it after using the launcher
- no, there currently isn't any way to specify which nodes get used. the
cores are assigned in a contiguous block starting at core 0.
i can see why it might be interesting. this is something we can consider
- mpb should be the default mode for streams and mpi communication
- scc_find_region was intended to allow the user to query the local core address
for the various resources that the launcher maps into the core's memory. the
original reason for exposing this was to allow users to specify the '-r' flag
to the launcher, and then use the mpbs for their own communications rather
than the streams interface
unfortunately, the rest of the resources (other cores gpbs, etc) are rather
haphazardly represented, and this might be cleaned up in later releases
I just have one more query. I looked at this old post which mentions that ETI's framework uses
DRAM for messaging.
I know the post is quite old and things might have changed but just wanted to confirm
if they have changed the framework so that both MPI and streams api use mpb.
I am using the version compatible with scc 1.3
ok, sorry, I misspoke. its been a few months and our use of the terminology is confusing.
the ETI code currently uses dram for the message bodies, and the MPB for the
synchronization pointers. with this arrangement we get similar bandwidth numbers
as RCCE, but are able to cover larger control transfer latencies. I suspect
though this leaves us more subject to contention problems.
this would be easy to add as a flag in a later release. you can also explore the use of the -r flag
which maps the MPBs and lets you do what you want with them. with -r the channels use
to communicate with the launcher are allocated (pointers and bodies) entirely out of DRAM,
and no channels are set up between the backend nodes
with the -r option, the messaging that the ETI software uses doesn't use the MPBs, you're free to
play with them as you like. which implies you'd need to write your own messaging layer in
order to use them.
it would be possible to add another mode which drastically reduces the MPI mtu and
allocates the message buffers out of the MPB space. we dont really have alot of resources
to do this, but are looking at various options for ongoing support of the software, so implementing
this is a possiblity
how important is the MPI layer to your work? or are you just interested in looking at messaging
Actually right now my aim is to gain a better understanding of the system and of the tradeoffs involved in message-passing(such as dram vs mpb).
So to begin with I was thinking of implementing few simple baremetal applications which can use mpb for message passing.
This is basically primitive work. So I was hoping to get some directions and may be some sample code so that I can get started.
I am not looking into running benchmarks and collecting statistics right now (though it might come at a later stage).
well, in ETI land, assuming you are doing polling, all you'd need to do is run
scc_find_region(MEMORY_REGION_MPB, target, &m)
where m is defined in memory_region.h, and target is the name of the remote node.
m will contain a pointer in your address space to that node's 8k MPB base.
static inline void invalidate_l1_mpb()
__asm__ volatile ( ".byte 0x0f; .byte 0x0a;\n" );
would then be used to flush the lines with the MPB tag