RCCE does not know about POP-SHM. All the RCCE shared memory calls refer to either the default shared memory or hijacked shared memory (which is just a way of getting the default shared memory to be larger). It would be ideal to put in some POP-SHM intelligence into RCCE, but that has not yet happened. I think you could run RCCE and POP-SHM together if you built RCCE without the -DSHMADD -DSHMADD_CACHEABLE. But I don't know of anyone who has actually tried this yet.
POP-SHM is working. There a simple test program posted in the tarballs section of our public SVN. http://marcbug.scc-dc.com/svn/repository/tarballs/
Not to say that it hasn't happened, but I don't know of anyone yet having done anything with POP-SHM beyond a simple test. It's time.
POP-SHM was designed to ortogonal to all other available libraries. It will work together with RCCE and any other communication library you may like to use. It should work with RCCE without any modification, just add the popshm.h and popshm.c to your proyect (scc_api.c and .h are also required, RCCE already includes them).
The reason is: it does not use legacy shared memory and it does not use the MPB.
The kernel finds and pins 16MB "pop-shm pages" from its _private_ memory pool (currently 640MB in sccLinux, so if you allocate 16MB per core for POP-SHM, then you are left with 624MB of usable private memory). This memory is then used by the POP-SHM library, and the aggregated total is available to all cores.
When you configure RCKMPI to use the sccshm or sccmulti channels, POP-SHM is built in as a dependency. If you use one of those channels, then POP-SHM is not available for your application since it will be used by the MPI library.