7 Replies Latest reply: Aug 19, 2012 8:44 AM by aprell RSS

RCCE symmetric memory model

vjain27 Community Member
Currently Being Moderated

Hi,

 

I am unable to understand how the symmetric memory model and the collective operation RCCE_malloc work

and how they  are useful. Following are some of the queries that I have -

 

- When RCCE_malloc() is executed , the offset of memory area allocated on a core's MPB from the start of the MPB is same for all the cores.

Am I right ? 

- Is it correct that the contents of the memory areas  created by a single  RCC_malloc() are not meant to be identical ?r

- How do RCC_put and RCC_get use this symmetric model ?

 

It would be great if an example is used to explain.

 

Thanks

Vaibhav Jain

  • 1. Re: RCCE symmetric memory model
    aprell Community Member
    Currently Being Moderated

    RCCE_malloc() and RCCE_free() are assumed to be called collectively by all cores, in the same program order, to facilitate message passing via RCCE_put() and RCCE_get().

     

    Suppose you call RCCE_malloc() to allocate a buffer b in the MPB of core i. How can you access the buffer on, say, core j? RCCE maintains an array where j can look up the address at which i's portion of MPB memory starts, but that's about it. You would need to determine the buffer offset (in bytes) on i's side and communicate that offset to j, to be able to perform remote memory accesses. By relying on collective allocations, and thus a symmetric name space, you basically sidestep that communication. If both i and j call RCCE_malloc(), the buffer offset can be determined implicitly by just looking at j's MPB. No need to communicate information from i to j.

     

    Internally, RCCE_put() and RCCE_get() include code like this:

     

    int offset = (int)b - RCCE_comm_buffer[myID];
    char *p = (char *)(RCCE_comm_buffer[otherID] + offset);
    // put/get using p
    

     

    This works precisely because we assume collective allocations. So, yes, as long as you perform identical allocations on every core, you can be sure of identical buffer offsets and take advantage of it.

  • 2. Re: RCCE symmetric memory model
    vjain27 Community Member
    Currently Being Moderated

    Well explained. Thanks!

  • 3. Re: RCCE symmetric memory model
    vjain27 Community Member
    Currently Being Moderated

    Just one more question. Does it mean that even when only two needs to exchange a variable the (empty) space for it is allocated by all the cores in their mpbs ?

    -Thanks

  • 4. Re: RCCE symmetric memory model
    vjain27 Community Member
    Currently Being Moderated

    Just one more question. Does it mean that even when only two needs to exchange a variable the (empty) space for it is allocated by all the cores in their mpbs ?

    -Thanks

  • 5. Re: RCCE symmetric memory model
    aprell Community Member
    Currently Being Moderated

    If you do it collectively, yes. If you want to save space and not replicate the buffer, that is, if you want to call RCCE_malloc() only once, you have to explicitly pass the buffer offset to every core that needs access (for example, by setting a shared variable in off-chip memory). But note that RCCE_put() and RCCE_get() assume collective allocations. If you break this assumption, you must write your own put and get functions.

  • 6. Re: RCCE symmetric memory model
    vjain27 Community Member
    Currently Being Moderated

    Thanks! There is just one more thing that I need to be clear about. I was looking at the code for RCCE_malloc() and I saw that it uses standard malloc() for allocating memory.

     

    b2 = (RCCE_BLOCK *) malloc(sizeof(RCCE_BLOCK));

     

    So is the standard malloc() modified  to allocate memory from MPB instead of private memory?

     

    -Thanks

  • 7. Re: RCCE symmetric memory model
    aprell Community Member
    Currently Being Moderated

    No, MPB memory is actually allocated (memory-mapped) at the beginning in RCCE_init(). Whenever you RCCE_malloc(), you mark a block of MPB memory as used (and return this block), and vice versa, whenever you RCCE_free(), you mark a block of MPB memory as free. The RCCE_BLOCKs that are allocated here are used to keep track of which portions of the MPB are currently in use. It's required for internal bookkeeping.

More Like This

  • Retrieving data ...

Legend

  • Correct Answers - 4 points
  • Helpful Answers - 2 points