During Ubuntu installation

Partitions creation

* From the “Partitions Disks” dialog box, select “Manual”. * Select ”FREE SPACE” under the first disk (”sda”). If you don't have free space you can delete unwanted partitions or select an existing one. * Select “Create a new partition” * Enter the size of the new partition (keep space for swap, around 1-2GB) * Select “Primary“ or “Logical“ if you have > 3 partitions * For “Use as” select “physical volume for RAID“, not the default “Ext3 journaling file system” * Make the partition bootable : “Bootable flag: on“. Now select “Done setting up the partition“ * Use the dialogs to create another partition in the free space. We will use that partition later for swap so make it small (1-2GB) * For “How to use this partition” select “physical volume for RAID“, not the default “Ext3 journaling file system” and not “swap area” * Repeat the above steps to create identical partitions on the second drive. Remember to mark partition as “bootable”.

Assembling partitions to software RAID

* Once the partitions are configured, at the top of the “Partition Disks” main dialog select “Configure Software RAID” * When asked “Write the changes to the storage devices and configure RAID” select “Yes”. * For “Multidisk configuration actions” select “Create MD device” * For “Multidisk device type” select “RAID1″ * For “Number of active devices for the RAID1 array” enter “2″ * For Number of spare devices for the RAID1 array” enter “0″ (zero) * When asked to select “Active devices for the RAID1 multidisk device” select both /dev/sda1 and /dev/sdb1 * From the next dialog select “create MD device” * Repeat the above steps to create an MD device that contains /dev/sda2 and /dev/sdb2 * Finally, from the dialog “Mulidisk configuration actions” select “Finish”

RAID partitions configuration

Next configure device md0 to be mounted as the “/” filesystem and device md1 to be mounted as swap:

* From the “Partition Disks” dialog, move the cursor bar under “RAID device #0″ and select “#1” * Configure the device as an “Ext3 filesystem“ mounted on / * Format the partition * From the “Partition Disks” dialog, move the cursor bar under “RAID device #1″ and select “#1” * Configure the device as “swap area“

Now you can select “Finish partitioning and write changes to disk“ and continue installation.

Care and feeding

Having two drives configured in a RAID1 mirror allows the server to continue to function when either drive fails. When a drive fails completely, the kernel RAID driver automatically removes it from the array.

However, a drive may start having seek errors without failing completely. In that situation the RAID driver may not remove it from service and performance will degrade. Luckily you can manually remove a failing drive using the “mdadm” command. For example, to manually mark both of the RAID devices on drive sda as failed:

mdadm /dev/md0 –-fail /dev/sda1 mdadm /dev/md1 –-fail /dev/sda2

The above removes both RAID devices on drive sda from service, leaving only the partitions on drive sdb active.

Removing a failed drive

When a drive fails it is vital to act immediately. RAID drives have an eerie habit of all failing around the same time, especially when they are identical models purchased together and put into service at the same time. Even drives from different manufacturers sometimes fail at nearly the same time… probably because they all experience the same environmental factors (power events, same number of power downs, the same janitor banging the vacuum into the server every night, etc.)

When Ubuntu sees that RAID has been configured, it automatically runs the mdadm command in “monitor mode” to watch each device and send email to root when a problem is noticed. You can also manually inspect RAID status using commands like the following:

cat /proc/mdstat mdadm –-query –-detail /dev/md0 mdadm –-query –-detail /dev/md1

It’s also wise to use “smartctl” to monitor each drive’s internal failure stats. However as noted in a recent analysis by Google, drives are perfectly capable to dying without any warning showing in their SMART monitors.

To replace a drive that has been marked as failed (either automatically or by using “mdadm -–fail”), first remove all partitions on that drive from the array. For example to remove all partitions from drive sda:

mdadm –-remove /dev/md0 /dev/sda1 mdadm –-remove /dev/md1 /dev/sda2

Once removed it is safe to power down the server and replace the failed drive.

Preparing the new drive

Once system as been rebooted with the new unformatted replacement drive in place, some manual intervention is required to partition the drive and add it to the RAID array.

The new drive must have an identical (or nearly identical) partition table to the other. You can use fdisk to manually create a partition table on the new drive identical to the table of the other, or if both drives are identical you can use the “sfdisk” command to duplicate the partition. For example, to copy the partition table from the second drive “sdb” onto the first drive “sda”, the sfdisk command is as follows:

sfdisk -–d /dev/sdb | sfdisk /dev/sda

Warning: be careful to specify the right source and destinations drives when using sfdisk or your could blank out the partition table on your good drive.

Once the partitions have been created, you can add them to the corresponding RAID devices using “mdadm –add” commands. For example:

mdadm -–add /dev/md0 /dev/sda1 mdadm -–add /dev/md1 /dev/sda2

Once added, the Linux kernel immediately starts re-syncing contents of the arrays onto the new drive. You can monitor progress via “cat /proc/mdstat”. Syncing uses idle CPU cycles to avoid overloading a production system, so performance should not be affected too badly. The busier the server (and larger the partitions), the longer the re-sync will take.

Note that you don’t have to wait until all partitions are re-synced… servers can be on-line and in production while syncing is in progress: no data will be lost and eventually all drives will become synchronized.

Creating a new array under Linux with free space

First use fdisk to create a partition on both hard drive in the remaining space (exact space). You must make sure the filesystem is set to Raid (code fd).

Then do something like :

sudo mdadm --create --verbose /dev/md3 --level=1 --raid-devices=2 /dev/sda5 /dev/sdb5

Array restoration

Let say you have 2 partitions that are RAID array from a previous installation or whatever the case may be and you want to restore them. Simply do :

mdadm --assemble /dev/md3 /dev/sda4 /dev/sdb4

Mdadm email notifications when a drive failed

Simply edit mdadm.conf :

vi /etc/mdadm/mdadm.conf

... and add a MAILADDR line (with your email address) to the file, e.g. like this:

... MAILADDR you@yourdomain.com ...

and restart mdadm

/etc/init.d/mdadm restart

Summary

Linux software RAID is far more cost effective and flexible than hardware RAID, though it is more complex and requires manual intervention when replacing drives. In most situations, software RAID performance is as good (and often better) than an equivalent hardware RAID solution, all at a lower cost and with greater flexibility. When all you need are mirrored drives, software RAID is often the best choice.