What pattern of accesses are you using?
I'd need more information about how you are doing the reads and writes to understand your issue.
You are working on a data center machine and so I am able to look at your code ... and I did take a peek. I noticed that you are using a /dev/rckwtmem on the cores. This device is not part of the standard SCC Linux, so did you make your own Linux? But then you also have code that refers to a /dev/wtd that I cannot find anywhere.
I think what you are doing is more related to the P54C than it is to the SCC, but I'm not sure.
Yes, I just modified some linux code, because I cannot modify page table attributes in application level.
The code of /dev/rckwtmem is located to trunk/linuxkernel/linux-2.6.16-mcemu/drivers/char/rckmem.c
I just added some code to the original code of /dev/rckncm code.
I made three versions of rcklinux.obj -- rcklinux_WT.obj, rcklinux_WB.obj, and rcklinux_UC.obj.
mmap with /dev/rckwtmem defines the mmaped area as WT, WB, or UC according to which linux obj file used.
Anyway P54C is a part of SCC, actually.
Jim// The code I wrote is below.
arr = mmap();
alloc_size = 32*1024;
cacheline_size = 32;
for( i = 0; i < alloc_size / sizeof(int); i++ )
arr[i] = 0;
// read 32 KB
for( i = 0; i < alloc_size / sizeof(int); i+= cacheline_size / sizeof(int) )
temp = arr[i];
// write 16KB
for( i = 0; i < alloc_size / sizeof(int) / 2; i+= cacheline_size / sizeof(int) )
arr[i+0] = i+0;
arr[i+1] = i+1;
arr[i+2] = i+2;
arr[i+3] = i+3;
You are correct. There is no WT policy for L2 on Rock Creek. You can disable L2 and configure L1 for WT.