4 Replies Latest reply on Dec 20, 2011 12:33 AM by ohntz

    Some flags questions




      Maybe someone can help me out with some uncertainties. The documentation, this forum, and the source code left me quite confused:


      1. When SINGLEBITFLAGS=0, using only one flag means that we have to allocate 32 bytes from the MPBs. Correct?

      2. When SINGLEBITFLAGS=0, using 32 flags means that we have to allocate, again, 32 bytes?
        (32 bytes for the 1st flag, and 31 flags get 1 byte each from that already allocated cache line).
        Is this correct?

      3. If 1&2 are true, then in the code why isn't there a lock to protect the whole cache line when writing to a flag? Is there a trick that I am missing?
      4. When SINGLEBITFLAGS=1, as can be seen in RCCE_lib - we always allocate 256 1-bit flags and use that struct to choose which one to access. This saves space, but as I understand, each flag write is locking out the other 255 flags in the cache line.
        If all correct, then why doesn't RCCE use the trick from question 3 to Read-Modify-Write only 1 byte (8 flags) instead of locking all 256 flags?


      Thanks in advance!



        • 1. Re: Some flags questions

          Hi Ohn,


          Have you seen this discussion?
          Yes, you are correct. Remember that the RCCE allocator works with cache line granularity, so even if we use only a single bit or byte flag, we have to allocate an entire cache line. Regarding 4), I think this is a necessary restriction because the bit twiddling part requires mutual exclusion, but there is only a very limited number of locks to choose from.
          1 of 1 people found this helpful
          • 2. Re: Some flags questions

            Thanks for your answer Andreas!


            Yes I read it a couple of times but still the bottom line is not 100% clear to me..


            Still regarding 3) and 4) :


            Assuming SINGLEBITFLAGS=0 and we allocated 32 flags in a single cache line.


            Can core X and core Y write to flag A and flag B at the same time without any problems?

            Can you please explain in simple words how it is being done?


            Regarding 4):

            Why bit twiddling, when SINGLEBITFLAGS=0, is different from when SINGLEBITFLAGS=1?

            If we can twiddle 8-bits in the first case without locking the cache line, then we can twiddle 8-bits in the second case without locking it. No? (and thus lock only 8 flags instead of 256 flags)



            • 3. Re: Some flags questions

              Yes, there is no problem with writing single bytes (see RCCE_put_char in RCCE_put.c). The only difficulty here is that MPB writes within a cache line are being accumulated in the write combine buffer, and we have to make sure that the data is actually flushed to the MPB and doesn't just sit in the buffer until some time in the future. I remember there was an interesting discussion about this topic... Here it is:



              There is no bit twiddling in the case of byte flags! You just write a '1' to set the flag and a '0' to unset it, that's it. But for bit flags, you have to copy the corresponding byte into private memory, change the flag bit, and copy the byte back to the target MPB. And that sequence, instead of doing a single write, requires locking.

              • 4. Re: Some flags questions

                It is very clear now...

                Thank you!