This document will show how I moved the contents of an old ATA disc to two SATA discs in a raid1 system. The SATA discs are attached to a Promise SATA300 TX2 card. It is partly written after I did the exchange of hard discs so use this guide as a guide for ideas, not as a step by step guide. Please inform me of any faults, or things that could be done in a better way.
In the old system I had two hard discs, /dev/hda, and
/dev/hdc, setup as follows
/dev/hda1 /boot ext3 defaults 1 2 /dev/hda2 / ext3 defaults 1 1 /dev/hda3 swap swap defaults,pri=10 0 0 # /dev/hdc1 /srv ext3 defaults 1 2 /dev/hdc5 /samba ext3 defaults 1 2 /dev/hdc6 /home ext3 defaults 1 2 /dev/hdc7 swap swap defaults,pri=10 0 0
The idea is to get the following
Raid HD1 HD2 MP Size Type md0 sda1 sdb1 /boot 512 MB fd 1 2 2 swap 1 GB fd 2 3 3 / 20 GB fd - 4 4 extended 3 5 5 /srv 60 GB fd 4 6 6 /home 165 GB fd
(In retrospect I should have done the /srv partition
slightly larger, and the swap larger, say 4 GB)
Use fdisk to partition /dev/sda. Then use
sfdisk to copy that layout to /dev/sdb
(note that I use two identical drives)
sfdisk -d /dev/sda | sfdisk /dev/sdb
Now create raid1 units.
mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sd[ab]1 mdadm --create /dev/md1 --level=1 --raid-devices=2 /dev/sd[ab]2 mdadm --create /dev/md2 --level=1 --raid-devices=2 /dev/sd[ab]3 mdadm --create /dev/md3 --level=1 --raid-devices=2 /dev/sd[ab]5 mdadm --create /dev/md4 --level=1 --raid-devices=2 /dev/sd[ab]6
Check status with
cat /proc/mdstat
Wait until the resync is finished (watch -n 1 "cat
/proc/mdstat") Must say that I am not sure this is really
needed.
Now create the file systems ...
mkfs.ext3 -j /dev/md0 mkswap /dev/md1 mkfs.ext3 -j /dev/md2 mkfs.ext3 -j /dev/md3 mkfs.ext3 -j /dev/md4
... and mount them
mkdir ~/newboot ~/newroot ~/newsrv ~/newhome mount /dev/md0 ~/newboot mount /dev/md2 ~/newroot mount /dev/md3 ~/newsrv mount /dev/md4 ~/newhome
Now it is time to copy the data from the old hard disc to the new raid system.
[root@server ~]# dir / bin dev home lost+found mnt proc samba selinux sys usr boot etc lib media opt root sbin srv tmp var cd ~/newroot mkdir boot dev home proc srv sys cp -a /bin /etc /lib /media /mnt /opt /root /sbin /selinux /tmp /usr /var . cd ~/newsrv cp -a /samba /srv/bittorrent /srv/cvsroot /srv/ftp /srv/www . cd ~/newhome cp -a /home/* .
(It is also possible to use rsync, i.e. rsync
-auHxv /var/ /root/newroot/var/, and so on.)
Now change ~/newroot/etc/fstab to include the new
raid devices, and remove all references to /dev/hd?
hard disc devices (keep the CD-ROMS etc, and for the time being
the /boot partition)
# /dev/hda1 /boot ext3 defaults 1 2 #/dev/md0 /boot ext3 defaults 1 2 /dev/md1 swap swap defaults 0 0 /dev/md2 / ext3 defaults 1 1 /dev/md3 /srv ext3 defaults 1 2 /dev/md4 /home ext3 defaults 1 2 #
Now create an /etc/mdadm.conf file.
echo "DEVICE /dev/sda\[12356\] /dev/sdb\[12356\]" > ~/newroot/etc/mdadm.conf mdadm -D --scan >> ~/newroot/etc/mdadm.conf echo "MAILADDR root@localhost.localdomain" >> ~/newroot/etc/mdadm.conf
Make an initrd file, do not forget to use the correct
/etc/fstab file, and include raid1 and promise
support (not sure that we explicitly have to include these
modules.)
mkinitrd -v --fstab=~/newroot/etc/fstab --preload raid1 --preload sata_promise \\ --preload sd_mod -f /boot/initrd-2.6.17-1.2139_FC4_RAID.img 2.6.17-1.2139_FC4
Update grub.conf to be able to use the raid system.
title Fedora Core (2.6.17-1.2139_FC4) RAID
root (hd0,0)
kernel /vmlinuz-2.6.17-1.2139_FC4 ro root=/dev/md2 selinux=disable
initrd /initrd-2.6.17-1.2139_FC4_RAID.img
Copy the /boot partition to the new one.
cp -a /boot/* ~/newboot/
Restart the computer. We now have the following situation.
/dev/hda1 on /boot type ext3 (rw) /dev/md0 on /root/newboot type ext3 (rw) /dev/md2 on / type ext3 (rw) /dev/md3 on /srv type ext3 (rw) /dev/md4 on /home type ext3 (rw) [root@server ~]# cat /proc/swaps Filename Type Size Used Priority /dev/md1 partition 987896 0 -1 [root@server ~]#
We now have to install grub on the MBR on /dev/sda
and /dev/sdb. Start grub and do the following.
[root@server ~]# grub grub> device (hd0) /dev/sda grub> root (hd0,0) grub> setup (hd0) grub> device (hd0) /dev/sdb grub> root (hd0,0) grub> setup (hd0) grub> quit
This means that we have grub installed on MBR on both SATA discs.
We must now update /etc/fstab to mount
/boot from /dev/md0
# /dev/md0 /boot ext3 defaults 1 2 /dev/md1 swap swap defaults 0 0 /dev/md2 / ext3 defaults 1 1 /dev/md3 /srv ext3 defaults 1 2 /dev/md4 /home ext3 defaults 1 2 #
Change the contents of
/boot/grub/devices.map into
(hd0) /dev/sda (hd1) /dev/sdb
Restart computer, and in the BIOS disable the ATA disc. Hopefully your computer will now start from the new SATA discs and the raid will be operating. If all is well, shutdown and remove the old ATA disc.
Finally, add scsi_hostadapter to
/etc/modprobe.conf so that future calls to
mkintrd will include the sata_promise
module
That should be it...
mdadm commands
Always zero the superblock of a device before adding it to a RAID device. Why? Because the disks decide what array they are in based on the disk-id information written on them. Zero the superblock first in case the disk was part of a previous RAID device. Also, if a partition was part of a previous RAID device, it appears to store the size of it's previous partition in the signature. Zeroing the superblock before adding it to a new RAID device takes care of cleaning up that, too.
Erase the MD superblock from a device
mdadm --zero-superblock /dev/hdx
Remove disk from array
mdadm --set-faulty /dev/md1 /dev/hda1 mdadm --remove /dev/md1 /dev/hda1
Replace failed disk or add disk to array
mdadm --add /dev/md1 /dev/hda1
(that will format the disk and copy the data from the existing disk to the new disk.)
Create mdadm config file: echo "DEVICE /dev/hda /dev/hdc" > /etc/mdadm/mdadm.conf mdadm --brief --detail --verbose /dev/md0 >> /etc/mdadm/mdadm.conf mdadm --brief --detail --verbose /dev/md1 >> /etc/mdadm/mdadm.conf
To stop the array completely
mdadm -S /dev/md0