I spent a considerable amount of time over the last couple of months testing different restore processes. This is my documentation for restoring Veritas/Symantec backups to a Linux server.
The general outline is this:
1. Create a LiveUSB drive to boot CentOS with a persistent overlay.
2. Install Symantec backupexec on the LiveUSB drive.
3. Recreate the drive layout on the new server.
4. Restore to the new server.
You must install CentOS LiveUSB on an ext2/3/4 formatted USB drive in order for Symantec to work. If you leave the VFat partition Symantec will nowork properly and you will get the error “An unknown error occurred within the NDMP subsystem.” Once I reformatted the USB drive as ext3 and installed a new LiveUSB with persistent overlay Symantec worked. My guess is it has to do with permission bits but that is only a guess.
I downloaded the LiveCD tools for Centos here.
Here is some of my history from that server:
umount /mnt fdisk /dev/sdb mkfs -t ext3 /dev/sdb1 mkfs -t ext3 /dev/sdb2 livecd-iso-to-disk --overlay-size-mb 1500 CentOS-5.5-i386-LiveCD.iso /dev/sdb1 mount /dev/sdb1 /mnt ls /mnt
I wanted to give it a persistent name and IP address for use in our data center. For some of this I was also shooting in the dark in order to get Symantec working, for thoroughness I include it here.
vi /etc/sysconfig/network HOSTNAME=recovery.chainringcircus.org
vi /etc/sysconfig/networking/devices/ifcfg-eth0 DEVICE=eth0 BOOTPROTO=none ONBOOT=yes NETMASK=255.255.255.0 IPADDR=192.168.1.200 GATEWAY=192.168.1.1 TYPE=Ethernet USERCTL=no IPV6INIT=no PEERDNS=yes DNS1=192.168.1.201 DNS2=192.168.1.202 DOMAIN=chainringcircus.org
After I configured the hostname and network settings I rebooted to make sure that the persistent overlay worked. I also turned on sshd and set it to runlevel 3 in /etc/inittab because I did not want to mess with a gui, but that is your choice. When everything came up properly I installed Symantec and we did a test restore.
I cover installing Symantec on Linux in another post here. You need to install an older package for compatibility:
yum install compat-libstdc++-296-2.96-138.i386
The specific Symantec rpms I installed are listed below. I did try a newer package from Symantec but it did not allow us to restore erroring with a different message. I will also say that was when we were on a VFat partition. Once I got everything working on an ext3 partition I quit testing.
Recreate Drive Layout
For thoroughness I am going to cover creating the logical volumes that are default for CentOS and RHEL.
First I need to lay out the drive mappings. This is from the old server which I am cloning onto a similar server. In this section I am just going to show the output of a number of commands that confirm the file system layout of the server.
File layout on the old server
From the file /etc/fstab:
LABEL=/boot /boot ext3 defaults 1 2 /dev/VolGroup00/LogVol00 / ext3 defaults 1 1 /dev/VolGroup00/LogVol01 swap swap defaults 0 0
From the mount command:
/dev/sda1 on /boot type ext3 (rw) /dev/mapper/VolGroup00-LogVol00 on / type ext3 (rw)
From the fdisk command:
Disk /dev/sda: 219.8 GB, 219823472640 bytes 255 heads, 63 sectors/track, 26725 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System /dev/sda1 * 1 13 104391 83 Linux /dev/sda2 14 26725 214564140 8e Linux LVM
Working my way up from the bottom of the LVM stack with the physical volume, the volume group and finally the logical volume.
--- Physical volume --- PV Name /dev/sda2 VG Name VolGroup00 PV Size 204.62 GB / not usable 31.29 MB Allocatable yes PE Size (KByte) 32768 Total PE 6547 Free PE 4 Allocated PE 6543 PV UUID jAuzGO-3Zpz-4T3K-mqcI-Ql6D-1dqf-wj917q
--- Volume group --- VG Name VolGroup00 System ID Format lvm2 Metadata Areas 1 Metadata Sequence No 3 VG Access read/write VG Status resizable MAX LV 0 Cur LV 2 Open LV 2 Max PV 0 Cur PV 1 Act PV 1 VG Size 204.59 GB PE Size 32.00 MB Total PE 6547 Alloc PE / Size 6543 / 204.47 GB Free PE / Size 4 / 128.00 MB VG UUID LJc2HJ-D7Gr-ketA-5TSe-ppQM-m5di-4YMEgZ
--- Logical volume --- LV Name /dev/VolGroup00/LogVol00 VG Name VolGroup00 LV UUID HcyaVT-DOEs-1Rdy-h7af-7i0t-P0EF-K2cCxy LV Write Access read/write LV Status available # open 1 LV Size 202.53 GB Current LE 6481 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 253:0 --- Logical volume --- LV Name /dev/VolGroup00/LogVol01 VG Name VolGroup00 LV UUID ZpAnvu-Of5D-PoEO-HaDN-2krv-zIXp-1fF5av LV Write Access read/write LV Status available # open 1 LV Size 1.94 GB Current LE 62 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 253:1
On the old server the drive is broken into two partitions, sda1 and sda2:
sda1 /boot 100MB
sda2 Volume Group ~200GB
The volume group on the old server on the sda2 partition is broken into two logical volumes:
LogVol00 / ~200GB
LogVol01 swap ~2GB
It is important to remember that the drive mappings on the old server will not necessarily match the mappings on the new one. For instance on the old server the raid was on /dev/sda and on the new server the raid drive is mapped on /dev/sdb. That is only because I am booting from /dev/sda on the LiveUSB, under normal circumstance it will come back up as /dev/sda.
Working on the new server recreate the partitions
fdisk /dev/sdb Command (m for help): p Disk /dev/sdb: 1199.9 GB, 1199906488320 bytes 255 heads, 63 sectors/track, 145880 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 1 First cylinder (1-145880, default 1): 1 Last cylinder or +size or +sizeM or +sizeK (1-145880, default 145880): +200M Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 2 First cylinder (26-145880, default 26): Using default value 26 Last cylinder or +size or +sizeM or +sizeK (26-145880, default 145880): Using default value 145880 Command (m for help): a Partition number (1-4): 1 Command (m for help): t Partition number (1-4): 2 Hex code (type L to list codes): 8e Changed system type of partition 2 to 8e (Linux LVM) Command (m for help): p Disk /dev/sdb: 1199.9 GB, 1199906488320 bytes 255 heads, 63 sectors/track, 145880 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System /dev/sdb1 * 1 25 200781 83 Linux /dev/sdb2 26 145880 1171580287+ 8e Linux LVM Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. WARNING: Re-reading the partition table failed with error 16: Device or resource busy. The kernel still uses the old table. The new table will be used at the next reboot. Syncing disks.
/usr/sbin/lvmdiskscan /dev/ramdisk [ 16.00 MB] /dev/live-squashed [ 669.80 MB] /dev/root [ 4.00 GB] /dev/ram [ 16.00 MB] /dev/live-osimg [ 4.00 GB] /dev/live [ 7.45 GB] /dev/ram2 [ 16.00 MB] /dev/live-overlay [ 1.46 GB] /dev/sda2 [ 7.47 GB] LVM physical volume /dev/ram3 [ 16.00 MB] /dev/ram4 [ 16.00 MB] /dev/ram5 [ 16.00 MB] /dev/ram6 [ 16.00 MB] /dev/ram7 [ 16.00 MB] /dev/ram8 [ 16.00 MB] /dev/ram9 [ 16.00 MB] /dev/ram10 [ 16.00 MB] /dev/ram11 [ 16.00 MB] /dev/ram12 [ 16.00 MB] /dev/ram13 [ 16.00 MB] /dev/ram14 [ 16.00 MB] /dev/ram15 [ 16.00 MB] /dev/sdb1 [ 196.08 MB] /dev/sdb2 [ 1.09 TB] 7 disks 16 partitions 0 LVM physical volume whole disks 1 LVM physical volume
Turn off the the LVM in order to make changes, this is just a precautionary step if you have repartitioned your drive.
lvm vgchange -an
Create the LVM.
vgscan Reading all physical volumes. This may take a while... pvcreate -ff /dev/sdb2 Physical volume "/dev/sdb2" successfully created
Create and activate the volume groups.
vgcreate VolGroup00 -l 0 -p 0 -s 32m /dev/sdb2 Volume group "VolGroup00" successfully created vgchange -ay VolGroup00 0 logical volume(s) in volume group "VolGroup00" now active
Finally, create the logical volumes. Even though I have 1.1T I decided to start using 800G, leaving myself room if I want to add another mount point.
lvcreate -L 800000m -r auto -n LogVol00 VolGroup00 Logical volume "LogVol00" created lvcreate -L 4096m -r auto -n LogVol01 VolGroup00 Logical volume "LogVol01" created
Read in the new volume groups.
vgscan Reading all physical volumes. This may take a while... Found volume group "VolGroup00" using metadata type lvm2
Format all of the partitions:
mkfs -t ext3 /dev/sdb1 mke2fs 1.39 (29-May-2006) Filesystem label= OS type: Linux Block size=1024 (log=0) [output removed for brevity]
mkfs -t ext3 /dev/VolGroup00/LogVol00 mke2fs 1.39 (29-May-2006) Filesystem label= OS type: Linux Block size=4096 (log=2) [output removed for brevity]
mkswap /dev/VolGroup00/LogVol01 Setting up swapspace version 1, size = 4294963 kB
Here are some quick commands if you mess up and need to delete any of the LVM stack.
lvremove -f /dev/VolGroup00/LogVol00 lvm lvremove -f /dev/VolGroup00/LogVol01 lvm vgchange -a n VolGroup00 lvm vgremove VolGroup00
Mount the root under /mnt and then have Veritas restore to that mount point.
mount -t ext3 /dev/VolGroup00/LogVol00 /mnt
If you have made it this far then you must really need the information. Here are a couple of screenshots from our backup guru to help in the restore process.
“Preserve Tree”, by default it is selected.
Select “Restore over existing files”, “Restore all information for files and directories” and “Preserve tree”.
Install grub on the new machine
During the restore we restored all of the files and directories to /mnt, including /boot. In order to get everything working again we need to setup the boot directory and then grub. But Red Hat and CentOS 4.X uses legacy grub.
Copy all of /mnt/boot to the real /boot directory.
mkdir /mnt/newboot mount /dev/sdb1 /mnt/newboot cp -r /mnt/boot/* /mnt/newboot/ umount /mnt/newboot mount /dev/sdb1 /boot grub> root (hd1,0) Filesystem type is ext2fs, partition type 0x83 grub> find /grub/stage1 (hd1,0) grub> setup (hd1) Checking if "/boot/grub/stage1" exists... no Checking if "/grub/stage1" exists... yes Checking if "/grub/stage2" exists... yes Checking if "/grub/e2fs_stage1_5" exists... yes Running "embed /grub/e2fs_stage1_5 (hd1)"... 16 sectors are embedded. succeeded Running "install /grub/stage1 (hd1) (hd1)1+16 p (hd1,0)/grub/stage2 /grub/grub.conf"... succeeded Done. grub > quit
Restore /dev and /tmp
Depending upon your backup options you may need to restore the /dev directory and create a tmp directory. You need to set the sticky bit on /tmp.
cp devices.tar /mnt/lvm/VolGroup00-LogVol00/ cd /mnt/lvm/VolGroup00-LogVol00/ tar -tvf devices.tar tar -xvf devices.tar chroot /mnt/lvm/VolGroup00-LogVol00/ mkdir /tmp ls -al / chmod a+rwx /tmp chmod +t /tmp exit
Finally you need to set up your ethernet interfaces by editing the file,