I spent a considerable amount of time over the last couple of months testing different restore processes. This is my documentation for restoring Veritas/Symantec backups to a Linux server.

The general outline is this:
1. Create a LiveUSB drive to boot CentOS with a persistent overlay.
2. Install Symantec backupexec on the LiveUSB drive.
3. Recreate the drive layout on the new server.
4. Restore to the new server.

Create LiveUSB
CentOS makes a LiveCD toolset for CentOS. They also have directions for how to create a LiveUSB drive with persistent overlay. Please follow those links for more in depth directions.

You must install CentOS LiveUSB on an ext2/3/4 formatted USB drive in order for Symantec to work. If you leave the VFat partition Symantec will nowork properly and you will get the error “An unknown error occurred within the NDMP subsystem.” Once I reformatted the USB drive as ext3 and installed a new LiveUSB with persistent overlay Symantec worked. My guess is it has to do with permission bits but that is only a guess.

I downloaded the LiveCD tools for Centos here.

Here is some of my history from that server:

umount /mnt
fdisk /dev/sdb
mkfs -t ext3 /dev/sdb1
mkfs -t ext3 /dev/sdb2
livecd-iso-to-disk --overlay-size-mb 1500 CentOS-5.5-i386-LiveCD.iso /dev/sdb1
mount /dev/sdb1 /mnt
ls /mnt

LiveUSB Setup
I wanted to give it a persistent name and IP address for use in our data center. For some of this I was also shooting in the dark in order to get Symantec working, for thoroughness I include it here.

vi /etc/sysconfig/network
HOSTNAME=recovery.chainringcircus.org

vi /etc/sysconfig/networking/devices/ifcfg-eth0
DEVICE=eth0
BOOTPROTO=none
ONBOOT=yes
NETMASK=255.255.255.0
IPADDR=192.168.1.200
GATEWAY=192.168.1.1
TYPE=Ethernet
USERCTL=no
IPV6INIT=no
PEERDNS=yes
DNS1=192.168.1.201
DNS2=192.168.1.202
DOMAIN=chainringcircus.org

After I configured the hostname and network settings I rebooted to make sure that the persistent overlay worked. I also turned on sshd and set it to runlevel 3 in /etc/inittab because I did not want to mess with a gui, but that is your choice. When everything came up properly I installed Symantec and we did a test restore.

Install Symantec
I cover installing Symantec on Linux in another post here. You need to install an older package for compatibility:

yum install compat-libstdc++-296-2.96-138.i386

The specific Symantec rpms I installed are listed below. I did try a newer package from Symantec but it did not allow us to restore erroring with a different message. I will also say that was when we were on a VFat partition. Once I got everything working on an ext3 partition I quit testing.

VRTSvxmsa-4.2.1-211.i386.rpm
VRTSralus-10.00.5629-0.i386.rpm

Recreate Drive Layout
For thoroughness I am going to cover creating the logical volumes that are default for CentOS and RHEL.

First I need to lay out the drive mappings. This is from the old server which I am cloning onto a similar server. In this section I am just going to show the output of a number of commands that confirm the file system layout of the server.

File layout on the old server
From the file /etc/fstab:

LABEL=/boot             /boot                   ext3    defaults        1 2
/dev/VolGroup00/LogVol00 /                       ext3    defaults        1 1
/dev/VolGroup00/LogVol01 swap                    swap    defaults        0 0

From the mount command:

/dev/sda1 on /boot type ext3 (rw)
/dev/mapper/VolGroup00-LogVol00 on / type ext3 (rw)

From the fdisk command:

Disk /dev/sda: 219.8 GB, 219823472640 bytes
255 heads, 63 sectors/track, 26725 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          13      104391   83  Linux
/dev/sda2              14       26725   214564140   8e  Linux LVM

Working my way up from the bottom of the LVM stack with the physical volume, the volume group and finally the logical volume.

From pvdisplay:

  --- Physical volume ---
  PV Name               /dev/sda2
  VG Name               VolGroup00
  PV Size               204.62 GB / not usable 31.29 MB
  Allocatable           yes
  PE Size (KByte)       32768
  Total PE              6547
  Free PE               4
  Allocated PE          6543
  PV UUID               jAuzGO-3Zpz-4T3K-mqcI-Ql6D-1dqf-wj917q

From vgdisplay:

  --- Volume group ---
  VG Name               VolGroup00
  System ID
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  3
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                2
  Open LV               2
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               204.59 GB
  PE Size               32.00 MB
  Total PE              6547
  Alloc PE / Size       6543 / 204.47 GB
  Free  PE / Size       4 / 128.00 MB
  VG UUID               LJc2HJ-D7Gr-ketA-5TSe-ppQM-m5di-4YMEgZ

From lvdisplay:

  --- Logical volume ---
  LV Name                /dev/VolGroup00/LogVol00
  VG Name                VolGroup00
  LV UUID                HcyaVT-DOEs-1Rdy-h7af-7i0t-P0EF-K2cCxy
  LV Write Access        read/write
  LV Status              available
  # open                 1
  LV Size                202.53 GB
  Current LE             6481
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           253:0

  --- Logical volume ---
  LV Name                /dev/VolGroup00/LogVol01
  VG Name                VolGroup00
  LV UUID                ZpAnvu-Of5D-PoEO-HaDN-2krv-zIXp-1fF5av
  LV Write Access        read/write
  LV Status              available
  # open                 1
  LV Size                1.94 GB
  Current LE             62
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           253:1

On the old server the drive is broken into two partitions, sda1 and sda2:
sda1 /boot 100MB
sda2 Volume Group ~200GB

The volume group on the old server on the sda2 partition is broken into two logical volumes:
LogVol00 / ~200GB
LogVol01 swap ~2GB

It is important to remember that the drive mappings on the old server will not necessarily match the mappings on the new one. For instance on the old server the raid was on /dev/sda and on the new server the raid drive is mapped on /dev/sdb. That is only because I am booting from /dev/sda on the LiveUSB, under normal circumstance it will come back up as /dev/sda.

Working on the new server recreate the partitions

fdisk /dev/sdb
Command (m for help): p

Disk /dev/sdb: 1199.9 GB, 1199906488320 bytes
255 heads, 63 sectors/track, 145880 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-145880, default 1): 1
Last cylinder or +size or +sizeM or +sizeK (1-145880, default 145880): +200M

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 2
First cylinder (26-145880, default 26):
Using default value 26
Last cylinder or +size or +sizeM or +sizeK (26-145880, default 145880):
Using default value 145880

Command (m for help): a
Partition number (1-4): 1

Command (m for help): t
Partition number (1-4): 2
Hex code (type L to list codes): 8e
Changed system type of partition 2 to 8e (Linux LVM)

Command (m for help): p

Disk /dev/sdb: 1199.9 GB, 1199906488320 bytes
255 heads, 63 sectors/track, 145880 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1   *           1          25      200781   83  Linux
/dev/sdb2              26      145880  1171580287+  8e  Linux LVM

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.

WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table.
The new table will be used at the next reboot.
Syncing disks.

  /usr/sbin/lvmdiskscan
  /dev/ramdisk       [       16.00 MB]
  /dev/live-squashed [      669.80 MB]
  /dev/root          [        4.00 GB]
  /dev/ram           [       16.00 MB]
  /dev/live-osimg    [        4.00 GB]
  /dev/live          [        7.45 GB]
  /dev/ram2          [       16.00 MB]
  /dev/live-overlay  [        1.46 GB]
  /dev/sda2          [        7.47 GB] LVM physical volume
  /dev/ram3          [       16.00 MB]
  /dev/ram4          [       16.00 MB]
  /dev/ram5          [       16.00 MB]
  /dev/ram6          [       16.00 MB]
  /dev/ram7          [       16.00 MB]
  /dev/ram8          [       16.00 MB]
  /dev/ram9          [       16.00 MB]
  /dev/ram10         [       16.00 MB]
  /dev/ram11         [       16.00 MB]
  /dev/ram12         [       16.00 MB]
  /dev/ram13         [       16.00 MB]
  /dev/ram14         [       16.00 MB]
  /dev/ram15         [       16.00 MB]
  /dev/sdb1          [      196.08 MB]
  /dev/sdb2          [        1.09 TB]
  7 disks
  16 partitions
  0 LVM physical volume whole disks
  1 LVM physical volume

Turn off the the LVM in order to make changes, this is just a precautionary step if you have repartitioned your drive.

lvm vgchange -an

Create the LVM.

  vgscan
  Reading all physical volumes.  This may take a while...

  pvcreate -ff /dev/sdb2
  Physical volume "/dev/sdb2" successfully created

Create and activate the volume groups.

  vgcreate VolGroup00 -l 0 -p 0 -s 32m /dev/sdb2
  Volume group "VolGroup00" successfully created

  vgchange -ay VolGroup00
  0 logical volume(s) in volume group "VolGroup00" now active

Finally, create the logical volumes. Even though I have 1.1T I decided to start using 800G, leaving myself room if I want to add another mount point.

  lvcreate -L 800000m -r auto -n LogVol00 VolGroup00
  Logical volume "LogVol00" created

  lvcreate -L 4096m -r auto -n LogVol01 VolGroup00
   Logical volume "LogVol01" created

Read in the new volume groups.

  vgscan
  Reading all physical volumes.  This may take a while...
  Found volume group "VolGroup00" using metadata type lvm2

Format all of the partitions:

mkfs -t ext3 /dev/sdb1
mke2fs 1.39 (29-May-2006)
Filesystem label=
OS type: Linux
Block size=1024 (log=0)
[output removed for brevity]

mkfs -t ext3 /dev/VolGroup00/LogVol00
mke2fs 1.39 (29-May-2006)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
[output removed for brevity]

mkswap /dev/VolGroup00/LogVol01
Setting up swapspace version 1, size = 4294963 kB

Here are some quick commands if you mess up and need to delete any of the LVM stack.

lvremove -f /dev/VolGroup00/LogVol00
lvm lvremove -f /dev/VolGroup00/LogVol01
lvm vgchange -a n VolGroup00
lvm vgremove VolGroup00

Restore
Mount the root under /mnt and then have Veritas restore to that mount point.

mount -t ext3 /dev/VolGroup00/LogVol00 /mnt

If you have made it this far then you must really need the information. Here are a couple of screenshots from our backup guru to help in the restore process.

“Preserve Tree”, by default it is selected.

Select “Restore over existing files”, “Restore all information for files and directories” and “Preserve tree”.

Install grub on the new machine
During the restore we restored all of the files and directories to /mnt, including /boot. In order to get everything working again we need to setup the boot directory and then grub. But Red Hat and CentOS 4.X uses legacy grub.

Copy all of /mnt/boot to the real /boot directory.

mkdir /mnt/newboot
mount /dev/sdb1 /mnt/newboot
cp -r /mnt/boot/* /mnt/newboot/

umount /mnt/newboot

mount /dev/sdb1 /boot

grub> root (hd1,0)
 Filesystem type is ext2fs, partition type 0x83

grub> find /grub/stage1
 (hd1,0)

grub> setup (hd1)
 Checking if "/boot/grub/stage1" exists... no
 Checking if "/grub/stage1" exists... yes
 Checking if "/grub/stage2" exists... yes
 Checking if "/grub/e2fs_stage1_5" exists... yes
 Running "embed /grub/e2fs_stage1_5 (hd1)"...  16 sectors are embedded.
succeeded
 Running "install /grub/stage1 (hd1) (hd1)1+16 p (hd1,0)/grub/stage2 /grub/grub.conf"... succeeded
Done.

grub > quit

Restore /dev and /tmp
Depending upon your backup options you may need to restore the /dev directory and create a tmp directory. You need to set the sticky bit on /tmp.

cp devices.tar /mnt/lvm/VolGroup00-LogVol00/
cd /mnt/lvm/VolGroup00-LogVol00/
tar -tvf devices.tar
tar -xvf devices.tar

chroot /mnt/lvm/VolGroup00-LogVol00/
mkdir /tmp
ls -al /
chmod a+rwx /tmp
chmod +t /tmp
exit

Finally you need to set up your ethernet interfaces by editing the file,
/etc/sysconfig/networking/devices/ifcfg-eth0