Ted Kubaska wrote:
Is anyone in the community running baremetal applications with RCCE? Baremetal means not using an operating system but running your applications directly on the hardware. This would most likely involve modifications to the RCCE source downloadable from our public SVN. If you are not doing baremetal RCCE applications, is anyone doing baremetal message passing?
1.) ETI is 'doing baremetal message passing' in our SCC Development Framework via non-cached DRAM ring buffers between each core, currently running with the ring buffer control data also resident in non-cached DRAM. We also have a mode that puts the ring buffer control data in the MPBs but is not enabled in the current SCC Development Framework distribution because it needs further stability work.
2.) ETI's SCC Development Framework does not use the inter-processor interrupts: Each core polls the ring buffer control data via a 'progress' function to see if there are messages available for it to process.
3.) ETI's design philosophy is to allow message passing without any preemption: If the destination ring buffers are full then the sender queues its outgoing message until there is room in the destination's ring buffer.
Thanks. By using non-cached DRAM, does that mean you are not making use of the MPBT memory type?
The reason I am asking is that we had some internal discussion about CR4. It turns out that in order to enable PMB-mapping, one must set bit 11 in CR4. This does not seem to be documented anywhere. You can see it happen by looking at rckmem.c in rckos/rckApps. This bit is then set by SCC Linux when you open /dev/rckmpb, the message-passing buffer. The reason I asked about baremetal was because with baremental, users are not running SCC Linux and if they wanted to use the properties of MPBT memory, they would have to set bit 11 themselves. Memory typed as MPBT bypasses L2.
But I don't think that applies to ETI because I don't think you are using MPBT ... if I understood what you wrote correctly. Is that true?
Is anyone using MPBT with baremetal or possibly with a customized SCC Linux? If you are, how are you dealing with bit 11 in CR4?
Hi,
we are also working on a BareMetal Environment. So far we set up
a valid C++ Environment in which each core boots. Communication
is next. Thanks for pointing out the CR4 issue. Would have been very
annoying to ran into that.
Furthermore, I do have two questions:
Does the SCC Cores have MTRRs? And if yes, how do we program them (normally MTRRs are programmed via I/O Ports)?
And secondly, does the MMU snoop the L1? I made some experiments and I would say: no. Is that correct?
There's no snooping going on.
I don't know if MTRR is supported. I don't think so. The core is essentially P54C. I think MTRR came later. What leads me to believe this is because it's EDX[12] (after a CPUID) that tells you if MTRR is supported and that's a reserved bit in the P54C docs. I checked with one of the chip engineers, and he said he was 99% sure that MTRR was not supported on this chip.
No, the SCC cores definitely don't have MTRRs. That feature came in with Pentium Pro.
MTTR registers are set (on later processors) with RDMSR WRMSR instructions (read and write model specific register respectively).
Ted Kubaska wrote:
Thanks. By using non-cached DRAM, does that mean you are not making use of the MPBT memory type?
The reason I am asking is that we had some internal discussion about CR4. It turns out that in order to enable PMB-mapping, one must set bit 11 in CR4. This does not seem to be documented anywhere. You can see it happen by looking at rckmem.c in rckos/rckApps. This bit is then set by SCC Linux when you open /dev/rckmpb, the message-passing buffer. The reason I asked about baremetal was because with baremental, users are not running SCC Linux and if they wanted to use the properties of MPBT memory, they would have to set bit 11 themselves. Memory typed as MPBT bypasses L2.
But I don't think that applies to ETI because I don't think you are using MPBT ... if I understood what you wrote correctly. Is that true?
Is anyone using MPBT with baremetal or possibly with a customized SCC Linux? If you are, how are you dealing with bit 11 in CR4?
We have a mode that enables several throughput related features including using the MPBT on DRAM to avoid L2: This mode is still being debugged so the results are still TBD... But we did not know about CR4 bit 11 !
After the SC2010 fray we will enable it and reevaluate our results.
I updated the SCC EAS to mention the need for bit 11 in CR4. My ability to fix the link on the front page of this site is currently broken, but you can get the new EAS under the Documents tab. This is Rev 1.1. There's not much more there other than to say you have to set this bit called MPE (for message-passing enable). Rev 1.1 also has a couple of new lines to clarify the use of the write-combine buffer. What was there was not incorrect.
This bit 11 is 11 from the right starting at 0. I mentioned that because some of our internal email referred to it as bit 12 (I guess because someone started counting at 1), but it's really 0x800.
You might want to look in rckmem.c under rckos/rckApps to see how SCC Linux handles this.
Hi,
I'm just starting up with my SCC related work and I want to catch uup with others who already did baremetal programming.
Would you be able to share more experience and/or source code regarding baremetal SCC programming?
Yes,
I was thinking of a simple but complete runnable example.
After reading this http://communities.intel.com/message/93238#93238
I would bet that you may already have written some boot code
which makes it simple to create a loadable object which
in the end can execute a simple C function (which does not required libc call).
Do you have such example with may be a some makefile ?
To go a step further do you use RCCE with you baremetal example
or did you implement your own may be basic communication lib?
I don't think anyone has recently used RCCE with baremetal. But I would be very interested in hearing of any. I think RCCE development has focused on power management and shared memory under Linux. I think RCCE with baremetal has lagged, but I'm not sure of that. If you are using RCCE and baremetal, can tell us what you are doing?
Hi Ted,
Sure I will.
Until now I wasn't doing anything, but I am starting up so be sure I will
keep you all informed of what we are doing.
Our goal is fairly simple, we want to use SCC to implement
control-like critical program (this will be a pseudo-application not a real one).
This kind of applications consists of a set of multi-periodic functions.
Since each function is "critical", one must ensure that their execution
is "deterministic", that's the main reason we want to go baremetal.
The proof of concept would be to map 1 function on 1 core,
each such function would have to be baremetal.
Those functions have to communicate with each other so
being able to use RCCE would be nice. Even if we may not
need the whole RCCE API.
