Setting up RAID on an existing Debian/Ubuntu installation

I run RAID1 on all of the machines I support. While such hard disk mirroring is not a replacement for having good working backups, it means that a single drive failure is not going to force me to have to spend lots of time rebuilding a machine.

The best possible time to set this up is of course when you first install the operating system. The Debian installer will set everything up for you if you choose that option and Ubuntu has alternate installation CDs which allow you to do the same.

This post documents the steps I followed to retrofit RAID1 into an existing Debian squeeze installation. Getting a mirrored setup after the fact.

Overview

Before you start, make sure the following packages are installed:

apt-get install mdadm rsync initramfs-tools

Then go through these steps:

  1. Partition the new drive.
  2. Create new degraded RAID arrays.
  3. Install GRUB2 on both drives.
  4. Copy existing data onto the new drive.
  5. Reboot using the RAIDed drive and test system.
  6. Wipe the original drive by adding it to the RAID array.
  7. Test booting off of the original drive.
  8. Resync drives.
  9. Test booting off of the new drive.
  10. Reboot with the two drives and resync the array.

(My instructions are mostly based on this old tutorial but also on this more recent one.)

1- Partition the new drive

Once you have connected the new drive (/dev/sdb), boot into your system and use one of cfdisk or fdisk to display the partition information for the existing drive (/dev/sda on my system).

The idea is to create partitions of the same size on the new drive. (If the new drive is bigger, leave the rest of the drive unpartitioned.)

Partition types should all be: fd (or "linux raid autodetect").

2- Create new degraded RAID arrays

The newly partioned drive, consisting of a root and a swap partition, can be added to new RAID1 arrays using mdadm:

mdadm --create /dev/md0 --level=1 --raid-devices=2 missing /dev/sdb1  
mdadm --create /dev/md1 --level=1 --raid-devices=2 missing /dev/sdb2

and formatted like this:

mkswap /dev/md1  
mkfs.ext4 /dev/md0

Specify these devices explicitly in /etc/mdadm/mdadm.conf:

DEVICE /dev/sda* /dev/sdb*

and append the RAID arrays to the end of that file:

mdadm --detail --scan >> /etc/mdadm/mdadm.conf  
dpkg-reconfigure mdadm

You can check the status of your RAID arrays at any time by running this command:

cat /proc/mdstat

3- Install GRUB2 on both drives

The best way to ensure that GRUB2, the default bootloader in Debian and Ubuntu, is installed on both drives is to reconfigure its package:

dpkg-reconfigure grub-pc

and select both /dev/sda and /dev/sdb (but not /dev/md0) as installation targets.

This should cause the init ramdisk (/boot/initrd.img-2.6.32-5-amd64) and the grub menu (/boot/grub/grub.cfg) to be rebuilt with RAID support.

4- Copy existing data onto the new drive

Copy everything that's on the existing drive onto the new one using rsync:

mkdir /tmp/mntroot  
mount /dev/md0 /tmp/mntroot  
rsync -auHxv --exclude=/proc/* --exclude=/sys/* --exclude=/tmp/* /* /tmp/mntroot/

5- Reboot using the RAIDed drive and test system

Before rebooting, open /tmp/mntroot/etc/fstab, and change /dev/sda1 and /dev/sda2 to /dev/md0 and /dev/md1respectively.

Then reboot and from within the GRUB menu, hit "e" to enter edit mode and make sure that you will be booting off of the new disk:

set root='(md/0)'  
linux /boot/vmlinuz-2.6.32-5-amd64 root=/dev/md0 ro quiet

Once the system is up, you can check that the root partition is indeed using the RAID array by running mount and looking for something like:

/dev/md0 on / type ext4 (rw,noatime,errors=remount-ro)

6- Wipe the original drive by adding it to the RAID array

Once you have verified that everything is working on /dev/sdb, it's time to change the partition types on /dev/sda to fd and to add the original drive to the degraded RAID array:

mdadm /dev/md0 -a /dev/sda1  
mdadm /dev/md1 -a /dev/sda2

You'll have to wait until the two partitions are fully synchronized but you can check the sync status using:

watch -n1 cat /proc/mdstat

7- Test booting off of the original drive

Once the sync is finished, update the boot loader menu:

update-grub

and shut the system down:

shutdown -h now

before physically disconnecting /dev/sdb and turning the machine back on to test booting with only /dev/sda present.

After a successful boot, shut the machine down and plug the second drive back in before powering it up again.

8- Resync drives

If everything works, you should see the following after running cat /proc/mdstat:

md0 : active raid1 sda1[1]  
280567040 blocks [2/1] [_U]

indicating that the RAID array is incomplete and that the second drive is not part of it.

To add the second drive back in and start the sync again:

mdadm /dev/md0 -a /dev/sdb1

9- Test booting off of the new drive

To complete the testing, shut the machine down, pull /dev/sda out and try booting with /dev/sdb only.

10- Reboot with the two drives and resync the array

Once you are satisfied that it works, reboot with both drives plugged in and re-add the first drive to the array:

mdadm /dev/md0 -a /dev/sda1

Your setup is now complete and fully tested.

Ongoing maintenance

I recommend making sure the two RAIDed drives stay in sync by enabling periodic RAID checks. The easiest way is to enable the checks that are built into the Debian package:

dpkg-reconfigure mdadm

but you can also create a weekly or monthly cronjob which does the following:

echo "check" > /sys/block/md0/md/sync_action

Something else you should seriously consider is to install the smartmontools package and run weekly SMART checks by putting something like this in your /etc/smartd.conf:

/dev/sda -a -d ata -o on -S on -s (S/../.././02|L/../../6/03)  
/dev/sdb -a -d ata -o on -S on -s (S/../.././02|L/../../6/03)

These checks, performed by the hard disk controllers directly, could warn you of imminent failures ahead of time. Personally, when I start seeing errors in the SMART log (smartctl -a /dev/sda), I order a new drive straight away.

Installing Fallout 3 on Ubuntu Lucid

Here are some notes on what I did to install Fallout 3 (Game of the year edition) on Ubuntu Lucid using Wine.

Prerequisites

Basic Installation

  • follow the WineHQ instructions

  • after installing both DVDs, apply the patch

  • set the appropriate performance settings for your hardware (make sure you use the same resolution as the one you use in Xorg)

  • click on "Data Files" and enable all of the Fallout 3 add-ons you have installed

Performance Tweaks

Change these settings in your ~/My Games/Fallout 3/FALLOUT.INI:

bUseThreadedParticleSystem=1  
bUseThreadedBlood=1  
bUseThreadedMorpher=1  
bUseThreadedAI=1  
iNumHWThreads=2  
iNumHavokThreads=5  
bDoSpecularPass=0  
bUseRefractionShader=0  
bInvalidateOlderFiles=1

Then create a direct3d.reg file (replacing 512 with the correct amount of video memory you have, in MB):

Windows Registry Editor Version 5.00  
  
[HKEY_CURRENT_USER\SOFTWARE\Wine\Direct3D]  
"OffscreenRenderingMode"="backbuffer"  
"VideoMemorySize"="512"  
  
[HKEY_CURRENT_USER\Control Panel\Desktop]  
"FontSmoothing"="2"  
"FontSmoothingType"=dword:00000002  
"FontSmoothingGamma"=dword:00000578  
"FontSmoothingOrientation"=dword:00000001

and import these settings (which come from the Wine wiki and its appdb) into the registry:

regedit file.reg

Savegame corruption

At some point, I did run into savegame corruption (mostly due to my hard disk filling up) and I created this cronjob in /etc/cron.hourly/fallout3-savegames to detect corrupt savegame:

#!/bin/sh  
find /path/to/Fallout3/Savegames/ -type f -name "*.tmp" -print

Installing mods

If you want to install user-contributed mods (for example, if you can't be bothered with the lockpicking and terminal hacking mini-games), you will most likely want to install FOMM.

To do that, follow these instructions to install dotnet20 and gdiplus using winetricks otherwise you may run into this bug.

Finally, pay attention to the order in which your selected mods are loaded since some of the mods will overwrite files provided by another.

Hints and tips

There is an incredibly dedicated community maintaining a wiki for this game as well as all of the other "chapters" of the Fallout series. Use it wisely!