RCCE_malloc() and RCCE_free() are assumed to be called collectively by all cores, in the same program order, to facilitate message passing via RCCE_put() and RCCE_get().
Suppose you call RCCE_malloc() to allocate a buffer b in the MPB of core i. How can you access the buffer on, say, core j? RCCE maintains an array where j can look up the address at which i's portion of MPB memory starts, but that's about it. You would need to determine the buffer offset (in bytes) on i's side and communicate that offset to j, to be able to perform remote memory accesses. By relying on collective allocations, and thus a symmetric name space, you basically sidestep that communication. If both i and j call RCCE_malloc(), the buffer offset can be determined implicitly by just looking at j's MPB. No need to communicate information from i to j.
Internally, RCCE_put() and RCCE_get() include code like this:
int offset = (int)b - RCCE_comm_buffer[myID]; char *p = (char *)(RCCE_comm_buffer[otherID] + offset); // put/get using p
This works precisely because we assume collective allocations. So, yes, as long as you perform identical allocations on every core, you can be sure of identical buffer offsets and take advantage of it.
Just one more question. Does it mean that even when only two needs to exchange a variable the (empty) space for it is allocated by all the cores in their mpbs ?
-Thanks
Just one more question. Does it mean that even when only two needs to exchange a variable the (empty) space for it is allocated by all the cores in their mpbs ?
-Thanks
If you do it collectively, yes. If you want to save space and not replicate the buffer, that is, if you want to call RCCE_malloc() only once, you have to explicitly pass the buffer offset to every core that needs access (for example, by setting a shared variable in off-chip memory). But note that RCCE_put() and RCCE_get() assume collective allocations. If you break this assumption, you must write your own put and get functions.
Thanks! There is just one more thing that I need to be clear about. I was looking at the code for RCCE_malloc() and I saw that it uses standard malloc() for allocating memory.
b2 = (RCCE_BLOCK *) malloc(sizeof(RCCE_BLOCK));
So is the standard malloc() modified to allocate memory from MPB instead of private memory?
-Thanks
No, MPB memory is actually allocated (memory-mapped) at the beginning in RCCE_init(). Whenever you RCCE_malloc(), you mark a block of MPB memory as used (and return this block), and vice versa, whenever you RCCE_free(), you mark a block of MPB memory as free. The RCCE_BLOCKs that are allocated here are used to keep track of which portions of the MPB are currently in use. It's required for internal bookkeeping.
