[PLUG] RAID 1 suggestions

Tue Nov 25 11:57:02 UTC 2003

On Mon, 24 Nov 2003, Josh Orchard josh-at-emediatedesigns.com |PDX Linux| wrote:

> I don't know what else I would want to take into account. Here is what
> I'm trying to resolve:
>
> 1. Backups
> 2. Little but preferred no maintenance
> 3. Easy recovery
>
> It just seemed that RAID 1 was a simple solution with little worries.
> The other option is to RSYN the drives minus the obvious /proc and some
> others.  But this doesn't give you hot swap ability.  You would have to
> turn off the drive do a lilo or other grub loader thingy and then it
> would work again.  Maybe.

For a really simple, low-effort way to achieve this, I would think about
the following:

Allocate 50% of each drive for the usual filesystems, software mirrored.
This protects you from hardware failure but not from accidental deletes,
programs going nuts, filesystem bugs, etc. etc.

For example, on a pair of hypothetical 40GB drives you could have half the
drive used like this:

200MB  /dev/md0 /boot
512MB  /dev/md1 <swap>
8192MB /dev/md2 /
8192MB /dev/md3 /home

The rest of the drives serves as a mirrored local to-disk backup area:

16384MB /dev/md4 /backups

I suggest setting up a script that uses "dump" to back up the mirrored
partitions to the backups area.  A sample script is attached.

The first level 0 dump should be copied to a couple tapes, a stack of
CD-Rs, or a hard drive that is removed from the system.  (Alternatively,
only allocate 25% of your disks, and make the backup area the other 75%,
but that wastes a lot of your online storage on static data.)

The level 0 backup would only be needed if a file that has not changed
since that level 0 backup was created needed to be restored.  I've never
had to do this, since I seem to only damage/delete recently created files.
However, you'll want that level 0 backup somewhere-- just in case.  (I
usually use stacks of CD-Rs for this.)

The attached backup script sample schedule never does a level 0 (full)
backup for this reason.

Easy recovery is the reason I like the dump utility.  Using "restore -i" I
can use the bash-like interface to mark the files I want restored, then
restore them.  (Because RAID provides basic protection, I'm seldom doing a
full restore.)

The "Tower of Hanoi" sequence mentioned in the script makes it a bit
harder to find the files you want restored but saves lots of disk space.
If you want a truly easy recovery, just make every backup a level 1 and
you'll never need to hunt through multiple backups to find the files you
want.

As always, do a test restore to familiarize yourself with the process so
you don't have to learn it under pressure.  Also, periodic test restores
help ensure that the backups are valid and that you've included all the
devices that should be included.

  -- Steve
-------------- next part --------------
#!/bin/sh

# Backs up system to a second local drive using "dump".  The only parameter
# is the "dump" incremental level.  This script has only been tested
# on Linux using ext2 dump.

# YOU MUST RUN A LEVEL 0 BACKUP MANUALLY BEFORE USING THIS SCRIPT.  It will
# still work without this, but your first "level 1" will be huge.

# Below is a sample schedule of system backups with level 1 on 
# Saturday.  Adjust schedule as makes the most sense for your 
# activity schedule.  The below "Tower of Hanoi" sequence
# is highly recommended since it allows for a maximum number of points
# in time available for recovery with a minimum amount of storage usage.
# (At the expense of some additional complexity during restores.)

# To restore a given day's data:
# DoW     Levels needed
# ------ --------------
# Sat(6): 1
# Sun(0): 1,3
# Mon(1): 1,2
# Tue(2): 1,3,5
# Wed(3): 1,2,4
# Thu(4): 1,3,5,7
# Fri(5): 1,2,4,6

# Sample crontab entries for the above schedule:

#00 06 * * 6 root /root/backup.sh 1 >> /var/tmp/backup.cron.out 2>&1
#00 06 * * 0 root /root/backup.sh 3 >> /var/tmp/backup.cron.out 2>&1
#00 06 * * 1 root /root/backup.sh 2 >> /var/tmp/backup.cron.out 2>&1
#00 06 * * 2 root /root/backup.sh 5 >> /var/tmp/backup.cron.out 2>&1
#00 06 * * 3 root /root/backup.sh 4 >> /var/tmp/backup.cron.out 2>&1
#00 06 * * 4 root /root/backup.sh 7 >> /var/tmp/backup.cron.out 2>&1
#00 06 * * 5 root /root/backup.sh 6 >> /var/tmp/backup.cron.out 2>&1

# The simplest way to restore a file is to use 
# "restore -t -A <archive file>" to list the contents of the dump and
# grep for the file(s) that need to be restored.  Then use "restore -i"
# to interactively restore them.

# Note that this script overwrites each level's previous backup with the
# new one.  It also aborts on the first error so any subsequent 
# filesystems will NOT BE BACKED UP.

# Steve Bonds
# Email: rn5o6kw02 at sneakemail.com
# Copyright 2003, All rights reserved
# Redistribution permitted under the terms of the GNU General Public License
# version 2.0.  http://www.gnu.org/copyleft/gpl.html

bail() {
	MAIL_NEEDED=yes
	echo `date` "[$$]: Error: $1"
	echo `date` "[$$]: Error: $1" >> $MAIL_BODY
	MAIL_SUBJECT="Fatal backup error on $HOSTNAME"
	send_mail
	exit 1
}	

send_mail() {
	if [ "${MAIL_NEEDED}x" != "x" -a "${MAIL_TO}x" != "x" ]; then
		cat $MAIL_BODY | $MAIL_CMD "$MAIL_SUBJECT" $MAIL_TO
		cat $MAIL_BODY
	elif [ "${MAIL_NEEDED}x" != "x" ]; then
		echo "Subject: $MAIL_SUBJECT"
		cat $MAIL_BODY
	fi
	rm -f $MAIL_BODY
}

############################################################
# Customize these as needed
MAIL_TO=root

# This script assumes that the following structure is in place:
# ${BACKUP_AREA}/level0
# ${BACKUP_AREA}/level1
# ...
# ${BACKUP_AREA}/level9
BACKUP_AREA=/backups

# List of devices to back up without the /dev in front.  Don't
# try to back up the area where $BACKUP_AREA is located or you
# will waste a LOT of space.
DEVICES="md0 md1 md2 md3"

############################################################
# These should be fine as-is

HOSTNAME=`hostname`
MAIL_CMD="mail -s"
MAIL_BODY=`mktemp backup.XXXXXX`

############################################################
# Actual commands

LEVEL=$1
: ${LEVEL:?"Please specify backup level (0-9), 0=full"}
shift

# Backups contain every file on the system, so it's real important to be sure
# that only root can read them.
umask 077

case $LEVEL in
	[0123456789]) # OK to proceed
	;;
	*) echo "Invalid level, must be 0 through 9"
	exit
esac

for device in $DEVICES; do
	# If you like, you can create cd-sized images via -M -B 665600
	# This is handy for burning them onto CD-R media

	# The filenames are used several places, so only figure them
	# out once.  Do not embed a datestamp so each level's backup
	# overwrites the previous one.
	DUMP_FILE=${BACKUP_AREA}/level${LEVEL}/${device}.dump
	ARCHIVE_FILE=${BACKUP_AREA}/level${LEVEL}/${device}.archive

	# Remove existing backup at the same level from last week
	rm -f $DUMP_FILE $ARCHIVE_FILE

	dump -${LEVEL} -f $DUMP_FILE -A $ARCHIVE_FILE -q -u -z1 /dev/${device} && echo `date` "$device backup succeeded" || bail "Problem backing up $device" 

done