Re: Slow Raid5 Performance on H67 chipset w/2600k CPU

webadmin@intel.com — Wed, 25 Apr 2012 01:23:44 GMT

I have an H67-B3 (MSI) board with the 2600K, and I'm running a 5-disk raid 5 with 2TB hitachi deskstars (which have the 512e advanced format sectors). It is configured as a 1TB volume and a 6.3TB volume (both raid 5), and I have typical read performance > 480 MB/sec and typical write performance > 350 MB/sec (N.B. writes are SLOW with write caching disabled). I copied 100 GB from the first volume to the second volume and the average transfer rate was about 110 MB/sec (note that this is reading and writing on the same array).

I have the stripe size and ntfs cluster size both at 64k, and the volumes and partitions are properly aligned to a 4k boundary (at the physical disk level) - you can boot a live linux cd and use mdadm to examine the volume alignment. BTW, I didn't do anything special to align the array. In fact, I migrated it over from an old core 2 board with the G45 chipset (still ICH10R) in a 4 disk RAID 10 configuration, added the 5th drive, and converted it to RAID 5. It's actually all around faster (and twice as big) as the 4 disk RAID 10.

It is important to note that it is generally not good to run a raid 5 with an even number of disks. Use 3 or 5. For example, in my configuration, a 64KB stripe consists of 4 16 KB (4 4K sectors) data blocks and 1 16KB parity block. This aligns nicely across 5 disks. In a four disk configuration, the 64K data is divided over 3 drives, and the parity block will be (I think) 64K / 3, so basically you have 21 1/3 KB "chunks" per drive. I actually have no idea how the RAID ROM manages this (maybe rounds to the nearest sector, or maybe "packs" partial sectors together?), but I imagine what ever it does leads to extremely poor performance, as any reads will surely result in an unaligned access on one or more drives. This is pretty much true for any kind of RAID controller (h/w, s/w or "fake").

But seriously, I have a 256GB SATA III SSD (samsung 830) that I just installed and moved my OS over to, and while it smokes the RAID in access latency (I don't ever even see the "starting windows" screen during boot), the transfer rates are only about 15% better. Add another disk and grow your RAID, it'll probably start screaming!

Edit: Also, I used the 64KB ntfs cluster size to match the stripe size for a very particular reason. Note that any writes to the array 5 that are smaller than the stripe size will force the controller to read the whole stripe, modify it, calculate new parity, and write that all back to disk. I don't know how windows is issuing the actual writes to the drive, or if the Intel controller is smart enough to "know" that a full stripe write does not require a read-modify-write cycle, but if you do use the default 4k cluster and have lots of small writes, it will definitely not have optimal performance.

Intel Communities: Message List

Re: Slow Raid5 Performance on H67 chipset w/2600k CPU