Lately I’ve been dealing mostly with storage and disk performance. We have several MySQL / MongoDB databases, which can be very I/O consuming. Usually people benchmark storage when they setup server, but running a real service on it, will actually show how really optimized the server is.
Tips for optimizing your disk storage
I’ll give you few tips for optimizing your storage and what to check. They are not all the aspects I think of, when setting up a storage, but they sure are the major ones.
Be sure to enable this one on your RAID controller. It’s usually On by default, but it’s worth to check. You should also check that your Linux actually uses disk cache policy by executing:
hdparm -W /dev/sda , also check:
hdparm -a /dev/sda to see readahead value. It might be useful to set this parameter to at least 1024. Test different readahead values by testing read performance with dd and choose best one that fits your storage.
Write Policy on RAID Controller
Very important thing to check. Be sure it’s writeback for best performance. It uses BBU unit to make sure if server crashes or there’s power outage it will still flush data on disks.
Read Policy on RAID Controller
I usually keep it Adaptive readahead. Some controllers allow you to set this, some don’t. Set it if you have it.
Check Raid’s level write penalties before you choose it. It’s very important aspect for performance.
Check maximum input/output operations per second of your disk. For databases it’s most important aspect. It’s all about IOPS.
Example calculating Max IOPS for raid 10 with 8 disks:
SAS 15k rpm disk: 175-210 IOPS ~ 200 IOPS
200 * 8 = 1600 IOPS for only read performance
200 * 8 / 2 = 800 IOPS for only write performance (becouse of write performance penalty).
Explaination: Raid 10 with 8 disks consists of 4 raid 1 arrays connected with raid 0. Raid 1 has to write on both disks and raid 0 just writes on one, meaning you have to write on 2 disks for one write. Therefore write penalty of 2. Here’s Raid 10 architecture:
You should usually set this to 65KB or 128KB max.
For databases you should use xfs or at least ext4. Filesystem optimized for your usecase can help a lot.
There are several I/O schedulers availiable in Linux. Default one is cfq. Even if you think this is not important aspect, I will try to convince you otherwise.
We have write heavy database server, that’s totally draining our storage. 100% utilization of disks and large await for read/write requests. I was testing Linux I/O schedulers the other day and came to great results. First deadline (at around 9-11 o’clock), which showed minor improovement and afterwards noop (at 11:30 o’clock). You can see the difference when I changed scheduler to noop and left it there on these graphs:
To check availiable Linux I/O schedulers:
To set certain Linux I/O scheduler:
echo noop > /sys/block/sda/queue/scheduler
I optimized this storage a little and we prolonged the time needed to buy a new one. You are very welcome to comment under this post and tell everyone your own ways of improoving performance!