Linux RAID: lessons learned
Friday January 12, 2007 by Derek Young
I had the opportunity to add a second drive to my machine at work this week and decided to make a RAID volume. Like with most Linux projects I undertake, now that I’m done it seems simple but at each turn I made a number of mistakes.
“Hardware” RAID is usually fake
I saw that the BIOS on my machine (a Dell Dimension 9200) allowed RAID configuration so my first attempt was to use the BIOS to create a new RAID volume. I expected to see a single drive 500GB when I booted up, but instead saw two normal 250GB drives.
Apparently like most onboard controllers, the BIOS tags the drives in a special way when you create a RAID volume but then it’s up to the operating system to treat the drives as RAID. Windows drivers look for these special tags and make the drives look like a single volume.
The Linux community calls this fake raid. To get both Windows and Linux to see the drives as a single volume you can use a tool called dmraid to recognize the tags and create a virtual device. Instructions for Ubuntu are here. I didn’t care about Windows seeing the RAID volume so I decided to use the more flexible software RAID instead.
mdadm and software RAID
Multiple Device (MD) support is built into the kernel for creating software based RAID volumes. mdadm creates and manages these volumes. You can combine partitions in anyway you want, including mixing RAID 0 and RAID 1. To create a RAID 0 volume out of two partitions:
mdadm --create --verbose /dev/md0 --level=0 --raid-devices=2 /dev/sdb1 /dev/sdc1 --auto=md
This starts the creation of the volume, which will be referred to as /dev/md0. You can watch the creation with
watch cat /proc/mdstat
I found the auto=md flag necessary because it will create /dev/md0 (or md1 etc.) if it doesn’t yet exist.
Creating a RAID 0 volume is very fast. Creating a RAID 1 volume has to synchronize the two partitions (even though they’re not yet formatted).
Once the volume has been created, you can format and mount it like a normal disk.
Ubuntu install on RAID
First, you’ll need the alternate install CD to install onto a RAID device. The live CD doesn’t support this. If you do need mdadm or need to mount a RAID volume from the live CD you can sudo apt-get install mdadm. Existing devices should be recognized automatically.
Once you’ve booted into the alternate install you won’t immediately see any newly created volumes. Request to partition the disk manually, then choose to create a RAID device, then immediately go back. After this the volume will be visible. I chose not to create the volume from the install because it takes so long (20 minutes for a 50GB RAID 1). I created it first from the live CD, then rebooted into the install.
I recommend creating a separate non-RAID /boot partition. Grub and Lilo both have some issues with RAID (only installing on RAID 1), and some issues with different file systems (XFS). I created a 4GB ext3 /boot partition and left the root partition as RAID. I chose to format it as XFS.
My final setup
I had two 250GB drives to work with. A common warning about RAID 0 is that you’re twice as likely to lose data. I wanted a portion of my system to be RAID 1 to hold frequent backups and critical data, then the rest RAID 0 for speed. I also have a working windows partition I didn’t want to lose.
I ended up going with 50GB RAID 1, (50GB per drive), 295GB RAID 0 (about 150GB per drive). Then I left my 30GB XP partition and used the rest for swap and unformatted space.
So far timings are positive. Some hdparm timings:
RAID 1:
Timing buffered disk reads: 186 MB in 3.02 seconds = 61.59 MB/sec
RAID 0:
Timing buffered disk reads: 350 MB in 3.00 seconds = 116.64 MB/sec
You may wish to investigate lvm2 (man lvm, and the howto at tldp.org and articles at ibm developerworks). It works at a higher level than mdadm, allowing you to do things like resize your partitions on the fly, move them from device to device, etc. To take real advantage of this you also need a filesystem you can resize on the fly, dunno if xfs does this or not. Anyhow, lvm2 lets you do mdadm stuff and more.
As you note, it’s easy once you learn the vocabulary. But to learn the vocabularly you almost always have to stumble through actually using the tools at least once.
I can’t help with Ubuntu, but with Debian if you use the installer in expert mode it lets you setup lvm2 as part of the partitioning.
Even better than lvm2 is http://evms.sourceforge.net, but that’s (AFIK) not in the ‘stock’ kernels yet.
— Karl O. Pinc Jan 12, 01:42 PM #