12 Replies Latest reply: Feb 28, 2011 12:48 AM by erk RSS

    Baremetal Applications

    tedk

      Is anyone in the community running baremetal applications with RCCE? Baremetal means not using an operating system but running your applications directly on the hardware. This would most likely involve modifications to the RCCE source downloadable from our public SVN. If you are not doing baremetal RCCE applications, is anyone doing baremetal message passing?

        • 1. Re: Baremetal Applications
          mdeazley

          Ted Kubaska wrote:

           

          Is anyone in the community running baremetal applications with RCCE? Baremetal means not using an operating system but running your applications directly on the hardware. This would most likely involve modifications to the RCCE source downloadable from our public SVN. If you are not doing baremetal RCCE applications, is anyone doing baremetal message passing?

           

           

          1.) ETI is 'doing baremetal message passing' in our SCC Development Framework via non-cached DRAM ring buffers between each core, currently running with the ring buffer control data also resident in non-cached DRAM. We also have a mode that puts the ring buffer control data in the MPBs but is not enabled in the current SCC Development Framework distribution because it needs further stability work.

           

          2.) ETI's SCC Development Framework does not use the inter-processor interrupts: Each core polls the ring buffer control data via a 'progress' function to see if there are messages available for it to process.

           

          3.) ETI's design philosophy is to allow message passing without any preemption: If the destination ring buffers are full then the sender queues its outgoing message until there is room in the destination's ring buffer.

          • 2. Re: Baremetal Applications
            tedk

            Thanks. By using non-cached DRAM, does that mean you are not making use of the MPBT memory type?

             

            The reason I am asking is that we had some internal discussion about CR4. It turns out that  in order to enable PMB-mapping, one must set bit 11 in CR4. This does not seem to be documented anywhere. You can see it happen by looking at rckmem.c in rckos/rckApps. This bit is then set by SCC Linux when you open /dev/rckmpb, the message-passing buffer. The reason I asked about baremetal was because with baremental, users are not running SCC Linux and if they wanted to use the properties of MPBT memory, they would have to set bit 11 themselves. Memory typed as MPBT bypasses L2.

             

            But I don't think that applies to ETI because I don't think you are using MPBT ... if I understood what you wrote correctly. Is that true?

             

            Is anyone using MPBT with baremetal or possibly with a customized SCC Linux? If you are, how are you dealing with bit 11 in CR4?

            • 3. Re: Baremetal Applications
              tpresche

              Hi,

               

              we are also working on a BareMetal Environment. So far we set up

              a valid C++ Environment in which each core boots. Communication

              is next. Thanks for pointing out the CR4 issue. Would have been very

              annoying to ran into that.

               

              Furthermore, I do have two questions:

              Does the SCC Cores have MTRRs? And if yes, how do we program them (normally MTRRs are programmed via I/O Ports)?

              And secondly, does the MMU snoop the L1? I made some experiments and I would say: no. Is that correct?

              • 4. Re: Baremetal Applications
                tedk

                There's no snooping going on.

                 

                I don't know if MTRR is supported. I don't think so. The core is essentially P54C.  I think MTRR came later. What leads me to believe this is because it's EDX[12] (after a CPUID) that tells you if MTRR is supported and that's a reserved bit in the P54C docs. I checked with one of the chip engineers, and he said he was 99% sure that MTRR was not supported on this chip.

                • 5. Re: Baremetal Applications
                  jheld

                  No, the SCC cores definitely don't have MTRRs.  That feature came in with Pentium Pro.

                  MTTR registers are set (on later processors) with RDMSR WRMSR instructions (read and write model specific register respectively).

                  • 6. Re: Baremetal Applications
                    mdeazley

                    Ted Kubaska wrote:

                     

                    Thanks. By using non-cached DRAM, does that mean you are not making use of the MPBT memory type?

                     

                    The reason I am asking is that we had some internal discussion about CR4. It turns out that  in order to enable PMB-mapping, one must set bit 11 in CR4. This does not seem to be documented anywhere. You can see it happen by looking at rckmem.c in rckos/rckApps. This bit is then set by SCC Linux when you open /dev/rckmpb, the message-passing buffer. The reason I asked about baremetal was because with baremental, users are not running SCC Linux and if they wanted to use the properties of MPBT memory, they would have to set bit 11 themselves. Memory typed as MPBT bypasses L2.

                     

                    But I don't think that applies to ETI because I don't think you are using MPBT ... if I understood what you wrote correctly. Is that true?

                     

                    Is anyone using MPBT with baremetal or possibly with a customized SCC Linux? If you are, how are you dealing with bit 11 in CR4?

                     

                    We have a mode that enables several throughput related features including using the MPBT on DRAM to avoid L2: This mode is still being debugged so the results are still TBD... But we did not know about CR4 bit 11 !

                     

                    After the SC2010 fray we will enable it and reevaluate our results.

                    • 7. Re: Baremetal Applications
                      tedk

                      I updated the SCC EAS to mention the need for bit 11 in CR4. My ability to fix the link on the front page of this site is currently broken, but you can get the new EAS under the Documents tab. This is Rev 1.1. There's not much more there other than to say you have to set this bit called MPE (for message-passing enable). Rev 1.1 also has a couple of new lines to clarify the use of the write-combine buffer. What was there was not incorrect.

                       

                      This bit 11 is 11 from the right starting at 0. I mentioned that because some of our internal email referred to it as bit 12 (I guess because someone started counting at 1), but it's really 0x800.

                       

                      You might want to look in rckmem.c under rckos/rckApps to see how SCC Linux handles this.

                      • 8. Re: Baremetal Applications
                        erk

                        Hi,

                         

                        I'm just starting up with my SCC related work and I want to catch uup with others who already did baremetal programming.

                        Would you be able to share more experience and/or source code regarding baremetal SCC programming?

                        • 9. Re: Baremetal Applications
                          tpresche

                          Of course. Do you have any specific question?

                          • 10. Re: Baremetal Applications
                            erk

                            Yes,

                             

                            I was thinking of a simple but complete runnable example.

                             

                            After reading this http://communities.intel.com/message/93238#93238

                             

                            I would bet that you may already have written some boot code

                            which makes it simple to create a loadable object which

                            in the end can execute a simple C function (which does not required libc call).

                             

                            Do you have such example with may be a some makefile ?

                             

                            To go a step further do you use RCCE with you baremetal example

                            or did you implement your own may be basic communication lib?

                            • 11. Re: Baremetal Applications
                              tedk

                              I don't think anyone has recently used RCCE with baremetal. But I would be very interested in hearing of any. I think RCCE development has focused on power management and shared memory under Linux. I think RCCE with baremetal has lagged, but I'm not sure of that. If you are using RCCE and baremetal, can tell us what you are doing?

                              • 12. Re: Baremetal Applications
                                erk

                                Hi Ted,

                                 

                                Sure I will.

                                Until now I wasn't doing anything, but I am starting up so be sure I will

                                keep you all informed of what we are doing.

                                 

                                Our goal is fairly simple, we want to use SCC to implement

                                control-like critical program (this will be a pseudo-application not a real one).

                                This kind of applications consists of a set of multi-periodic functions.

                                Since each function is "critical", one must ensure that their execution

                                is "deterministic", that's the main reason we want to go baremetal.

                                 

                                The proof of concept would be to map 1 function on 1 core,

                                each such function would have to be baremetal.

                                 

                                Those functions have to communicate with each other so

                                being able to use RCCE would be nice. Even if we may not

                                need the whole RCCE API.