We are trying to play a bit with sharing the off-chip memory between the cores, but there are a few things we haven't been able to figure out easily:
1) When using POPSHM, or the shared memory API provided by RCCE, what is the default memory mapping used? I guess it's cached non-MPBT but just to check.
2) Is there an easy way to change the mappings without having to hack the kernel? What do you propose?
the original SccKit had three special devices /dev/rckncm (non-cached mode), /dev/rckmpb (MPBT mode), and /dev/rckdcm (definitively cached mode). Thus, RCCE and others can only access shared memory in these modes. For our project we added /dev/rckmpbwt (for MPBT+write through) by recompiling the whole system. You use them by opening the device and using it as file in mmap(). The offset parameter of mmap is the physical address that you want to access, but be careful with the LUT mapping, otherwise different cores see different memory at the same physical address.
The SMC project has a nicer approach. They extended the mprotect system call so that you can change the mode after mmap() to any desired variant. Unfortunately, their approach to modify mprotect is quite elaborate. I would like to have this feature either in the standard SccKit or as a loadable kernel module. That would lower the entry barrier
I don't know what devices are available in SccKit-1.4.2.