HOWTO Custom Stage4
From Gentoo Linux Wiki
| Installation • Kernel & Hardware • Networks • Portage • Software • System • X Server • Gaming • Non-x86 • Emulators • Misc |
Contents |
[edit] Introduction
This article will demonstrate how to create a custom stage4 archive. A stage4 archive is an image of your entire root partition. The primary reason to create a stage4 archive is to provide for quick recovery in the event of disk failure. A stage4 archive is the same as a stage3, only you can select the CFLAGS you want to use and what other software you want installed. You can adapt this method to suit your personal uses.
If you want a generic stage4 that you can install on multiple systems, use Genkernel to set up your kernel. This will ensure that on boot, the kernel will work like the one on the livecd. You may also want to use less restrictive CFLAGS (i.e., MCPU instead of MARCH). You may still have to modify /etc/fstab and the USE flags after extracting the stage4, to suit the user.
This article assumes you have already installed Gentoo, if not go to the handbook and do the install first.
This article also assumes that you have already setup any other software you may want to have on your system. For example, on a desktop system you may want X, Xfce4, Sun's JDK, CVS, Emacs, Thunderbird, and Firefox.
[edit] (Optional) Tar /boot
Make a copy of your /boot partition:
# mount /boot # cp -R /boot /bootcpy # umount /boot
Alternatively, you can just mount /boot before you tar everything up. However, there may be problems with this alternate method if you use the stage4 on systems with different hardware configurations.
[edit] Clean Up
If you are cloning machines make sure you use different /etc/ssh/ssh_host_* keys.
rm /etc/ssh/ssh_host*
You can follow the Running out of disk space FAQ to free more space.
Use caution in clearing /var/tmp. On some systems, config information was stored in there for sound and portage. This can cause problems later, when restoring the system. It may be better to have a slightly larger tarball than to find that the restored system is incomplete.
(If you use an exclude file like the one shown in the next paragraph, this step can be omitted.)
[edit] Tar the System
Create an exclusion file that tells tar what not to pack into your stage 4 archive. Ending listed directories with /* instead of / will include the directory itself in the tar, but nothing inside the directory, thereby saving you the hassle of recreating them once you've restored.
stage4.excl contents:
.bash_history /mnt/* /tmp/* /proc/* /sys/* /dev/* /etc/ssh/ssh_host_* /usr/src/* /usr/portage/* /path/to/save/at/stage4.tar.bz2
After everything is set up correctly from the previous sections, we will create the archive:
# tar cvjf /path/to/save/at/stage4.tar.bz2 / -X stage4.excl tar options we used: c - create archive v - verbosely list files processed j - use bzip2 compression f - specify file name X - use the specified exclusion file tar option we DIDN'T use: p - preserve file attributes (necessary only during extraction of the tar, not during creation)
Be sure to look at the tar manpage
It will probably take a long time to create the archive depending on how much you have installed. I usually save the archive on a spare disk I have in my system that I use for backups. You can also burn it to a cd or dvd.
If it is too large to fit on a cd you will have to split it up. Large tar files can be split up with 'split'. The parts can be joined later with 'cat'. Read the manpages for these tools before you use them.
Here are some scripts to make this step a little easier.
[edit] Install from a stage4
- Boot live CD
- Set the system date to UTC and set the BIOS clock
# date MMDDHHmmCCYY # hwclock --systohc
- Create partitions, make filesystems and mount filesystems
- Copy your stage4 archive(s) to disk (if it is on another CD type "gentoo docache" at the boot prompt. Then you'll be able to umount/mount other CDs.)
# cd /mnt/gentoo # tar -xvjpf stage4.tar.bz2
- If you used the option to copy over your /boot to /bootcpy, then restore /boot from /bootcpy
# cp -R /mnt/gentoo/bootcpy/* /mnt/gentoo/boot/ Double check the boot dir after you copy the files over before deleting /bootcpy! # rm -rf /mnt/gentoo/bootcpy
- Make a few basic device nodes needed for booting
# mknod -m 660 /mnt/gentoo/dev/console c 5 1 # mknod -m 660 /mnt/gentoo/dev/null c 1 3 (optional) # mknod -m 660 /mnt/gentoo/dev/tty1 c 4 1
- Mount /proc and /dev into the chroot
# mount -t proc none /mnt/gentoo/proc # mount -o bind /dev /mnt/gentoo/dev
- Get the latest portage snapshot
# cd /mnt/gentoo/usr/ # wget http://gentoo.osuosl.org/snapshots/portage-latest.tar.bz2 # tar xjf portage-latest.tar.bz2
- Chroot into /mnt/gentoo
# chroot /mnt/gentoo /bin/bash # env-update; source /etc/profile
- Make sure the following files are correct for your system
- /boot/grub/grub.conf
- /etc/fstab
- /etc/conf.d/hostname
- /etc/hosts
- /etc/conf.d/net
- Make sure to install your boot loader into the mbr with either grub or lilo
[edit] Partition Table tip
To store partition table info:
dd if=/dev/(your_disk) of=mbr.save count=1 bs=512 sfdisk -d /dev/(your_disk) > partitions.save
The first of those saves the mbr and the second will store all partition info (including logical partitions, which aren't part of the mbr).
To restore partition info:
dd if=mbr.save of=/dev/(your_disk) sfdisk /dev/(your_disk) < partitions.save
[edit] GRUB tip
After chrooting:
root# grub grub> root (hd0,0) grub> setup (hd0) grub> quit
(hd may need to be changed depending on your setup)
[edit] udev tip
This took me 2 days to understand so listen carefully. If your eth0 controller doesn't come up after recovery, take a look at the following file:
/etc/udev/rules.d/70-persistent-net.rules
This took me way too long to find out by myself, so I hope this will help. The issue is that the MAC address probably changed on your NIC, so you need to update eth0 with the correct MAC, which is probably automatically setup as eth1.
[edit] Sample Stage 4 Build Scripts
Small script, which I use for making backup of my /home. (The /dev/hdX corresponds to the /mnt/tarbackup directory in /etc/fstab.)
lazy_bum removed:
- "--preserve-permissions" from "tarOptions", this is used with EXTRACT only.
| File: backupHome.sh |
#! /bin/bash
# Backup script for Gentoo Linux
# Author: fdavid
# Date: 2003.11.29.
# Making backup of home partition to cds
# options for the archive
tarOptions="--create --absolute-names --gzip --file"
# name of the archive
archive=/mnt/tarbackup/$(date +%Y%m%d)home.tar.gz
# mount the backup partition
mount /dev/hdaX
sleep 5
# create the archive
tar ${tarOptions} ${archive} /home/;
echo archive is done
# split the archive to cd size
# (use: "cat ${archive}.* >> ${archive}" to join the parts)
split --bytes=700000000 ${archive} ${archive}.
echo splitting is done
# unmount
sleep 5
umount /dev/hdaX
|
odessit added:
- Added "Public Key" and "Password" encryption methods using GnuPG
- Added instructions how to merge split files
- Added changelog-like comments to the top of the script.
Note: test restoration in your environment BEFORE using encryption in production!
BrianW added:
- Cleaned up the script a bit.
- Made it so the --exclude= for the archive doesn't have to be edited if you modify $stage4Location
- Added --exclude=/var/tmp/*
- Added --verbose to the tar options
- Added code to remove /bootcpy
- Moved the --exclude directories/files out of $tarOptions into their own variable's (thanks kamilian )
| File: mkstage4.sh |
#! /bin/bash
## Backup script for Gentoo Linux
## Change Log
## 2006.07.20 by odessit
## Added symmetric & asymmetric encryption options to the script using GnuPG
## to accomodate secure environments
## Date: 2006.03.05 by BrianW
## Initial Script
## Adapted from backupHome.sh by fdavid
## Adapted from mkstage4.sh by nianderson
## This is a script to create a custom stage 4 tarball (System and boot backup)
## I use this script to make a snapshot of my system. Meant to be done weekly in my case
## Please check the options and adjust to your specifics.
echo -=- Starting the Backup Script...
echo -=-
echo -=- Setting the variables...
## The location of the stage 4 tarball.
## Be sure to include a trailing /
stage4Location=/
## The name of the stage 4 tarball.
archive=$stage4Location$(hostname)-stage4.tar.bz2
## Directories/files that will be exluded from the stage 4 tarball.
##
## Add directories that will be recursively excluded, delimited by a space.
## Be sure to omit the trailing /
dir_excludes="/mnt/* /dev /proc /sys /tmp /usr/portage /var/tmp"
##
## Add files that will be excluded, delimited by a space.
## You can use the * wildcard for multiple matches.
## There should always be $archive listed or bad things will happen.
file_excludes="$archive"
##
## Combine the two *-excludes variables into the $excludes variable
excludes="$(for i in $dir_excludes; do if [ -d $i ]; then \
echo -n " --exclude=$i/*"; fi; done) $(for i in $file_excludes; do \
echo -n " --exclude=$i"; done)"
## The options for the stage 4 tarball.
tarOptions="$excludes --create --absolute-names --preserve-permissions --bzip2 --verbose --totals --file"
echo -=- Done!
echo -=-
## Mounting the boot partition
echo -=- Mounting boot partition, then sleeping for 5 seconds...
mount /boot
sleep 5
echo -=- Done!
echo -=-
## Creating a copy of the boot partition (copy /boot to /bootcpy).
## This will allow the archiving of /boot without /boot needing to be mounted.
## This will aid in restoring the system.
echo -=- Copying /boot to /bootcpy ...
cp -R /boot /bootcpy
echo -=- Done!
echo -=-
## Unmounting /boot
echo -=- Unmounting /boot then sleeping for 5 seconds...
umount /boot
sleep 5
echo -=- Done!
echo -=-
## Creating the stage 4 tarball.
echo -=- Creating custom stage 4 tarball \=\=\> $archive
echo -=-
echo -=- Running the following command:
echo -=- tar ${tarOptions} ${archive} /
tar ${tarOptions} ${archive} /;
echo -=- Done!
## Uncomment this portion to encrypt the stage4 using asymmetric encryption with GnuPG
## To get a list of available ciphers on your machine do #gpg --version
## Note: only aes aes192 aes256 and twofish should be used for large files (~1Gig & larger)
## To Restore: (cat together if split) and #gpg stage4.tar.bz2.gpg
#echo -=- Starting Encryption Process usung asymmetric encryption with a public key -=-
#cd $stage4Location
#gpg --encrypt --batch --recipient [MY_PUB_KEY] --cipher-algo twofish --output $archive.gpg $archive
#rm $archive
#archive=$stage4Location$(hostname)-stage4.tar.bz2.gpg
## Uncomment this portion to encrypt the stage4 using symmetric encryption with GnuPG
## ## To get a list of ciphers on your machine do #gpg --version
## Note: only aes aes192 aes256 and Twofish should be used for large files (~1 gig & larger)
## To Restore: (cat together if split) and #gpg stage4.tar.bz2.gpg
#echo -=- Starting Encryption Process usung symmetric encryption with a password -=-
#cd $stage4Location
#pass=PLEASE_CHANGE_ME_I_BEG_YOU!
#echo $pass | gpg --batch --cipher-algo twofish --passphrase-fd 0 --symmetric $archive
#rm $archive
#archive=$stage4Location$(hostname)-stage4.tar.bz2.gpg
## Split the stage 4 tarball in cd size tar files.
## To combine the tar files after copying them to your
## chroot do the following: "cat *.tar.bz2 >> stage4.tar.bz2".
## or if encrypted do the following: "cat *.tar.bz2.gpg >> stage4.tar.bz2.gpg".
## Uncomment the following lines to enable this feature.
#echo -=- Splitting the stage 4 tarball into CD size tar files...
#split --bytes=700000000 ${archive} ${archive}.
#echo -=- Done!
## Removing the directory /bootcpy.
## You may safely uncomment this if you wish to keep /bootcpy.
echo -=- Removing the directory /bootcpy ...
rm -rf /bootcpy
echo -=- Done!
echo -=-
## This is the end of the line.
echo -=- The Backup Script has completed!
|
Here is a little script to install from a stage4 created by either of the above two scripts. (Limited testing reported by rpcyan thanks please continue testing) should work though. A few changes were made by rpcyan to get the install script working correctly Please contact me with comments or improvements. nianderson
| File: installstage4.sh |
#! /bin/bash # Backup script for Gentoo Linux # Author: nianderson # Date: 2004.09.15. # #Used to install a stage4 created with mkstage4.sh #assumes you have already partitioned and formated your drive #assumes your booted from gentoo live cd #assumes you already decrypted your *.gpg archive (#gpg stage4.tar.bz2.gpg) #Define Disk layout #if using ide disks use hdax if using scsi use sdax rootPartition=/dev/sda3 bootPartition=/dev/sda1 #where to mount the disk partitions mntRootPartition=/mnt/gentoo mntBootPartition=/mnt/gentoo/boot #URL of stage4 #I put a copy of the tar on a webserver so i can #easily get it when a reinstall is needed urlToStage4=http://domain.com/ stage4=hostname-stage4.tar.bz2 #mount root partition echo mounting root partition $rootPartition to $mntRootPartition mount $rootPartition $mntRootPartition sleep 5 echo #not sure about this part yet #wget the stage4 to the mounted root partition cd $mntRootPartition echo wget $urlToStage4$stage4 to $mntRootPartition wget $urlToStage4$stage4 sleep 5 #untar the stage4 echo extract stage4 tar xjpf $stage4 sleep 5 echo #mount boot partiton echo mounting $bootPartition to $mntBootPartition mkdir $mntbootPartition mount $bootPartition $mntBootPartition sleep 5 echo #copy boot copy back to boot echo copy bootcpy back to boot cp -R $mntRootPartition/bootcpy $mntBootPartition sleep 5 #remove stage4 file rm -rf $mntRootPartition/$stage4 echo you need to check your fstab and install grub or lilo then echo all should be well echo Removing bootcpy rm -rf /bootcpy echo Enjoy |
For explanation and further infos refer to the Gentoo Post. A howto (Wiki) of the script can be found here.
| File: mkstage4.sh |
#!/bin/bash # Backup script for Gentoo Linux # Author: Reto Glauser aka blinkeye # Homepage: http://blinkeye.ch # Mailto: stage4 at blinkeye dot ch # Date: 23.03.2005 # If you need further infos check out this post: http://forums.gentoo.org/viewtopic.php?p=1751698#1751698 version=v1.2 # these are the commands we actually need for the backup command_list="echo tar hostname date split" # verify that each command we use exists for command in $command_list; do path=`which $command | grep "no $command in"` if [ ! -x `which $command` -a "$path" ]; then echo -e "\n\nERROR: $command not found! Check your commands and/or your \$PATH" exit -1 fi done # options for the tar command tarOptions="--create --absolute-names --preserve-permissions --totals --bzip2 --ignore-failed-read --verbose --file" # where to put the stage4 stage4Location=/mnt/backups/stage4 # name prefix stage4prefix=$(hostname)-stage4-`date +\%d.\%m.\%Y` # these files/directories are always excluded default_exclude_list=" --exclude=/tmp/* --exclude=/var/tmp/* --exclude=/lost+found/* --exclude=/dev/* --exclude=/proc/* --exclude=/mnt/* --exclude=/sys/* --exclude=/usr/portage/* --exclude=/var/log/* --exclude=$stage4Location" # depending on your choice these files or directories will additionally be excluded custom_exclude_list=" --exclude=/usr/src/* --exclude=/opt/mathematica --exclude=/usr/share/smssend --exclude=/home/*" # check the folder/files stored in $default_exclude_list exist for exclude in $default_exclude_list; do if [ ! -e "`echo "$exclude" | cut -d'=' -f2 | cut -d'*' -f1`" ]; then echo -e "\n\nERROR: `echo "$exclude" | cut -d'=' -f2` not found! Check your \$default_exclude_list" fi done # check the folder/files stored in $custom_exclude_list exist for exclude in $custom_exclude_list; do if [ ! -e "`echo "$exclude" | cut -d'=' -f2 | cut -d'*' -f1`" ]; then echo -e "\n\nERROR: `echo "$exclude" | cut -d'=' -f2` not found! Check your \$custom_exclude_list" fi done # print out the version echo -e "\nBackup script $version" echo -e "===================" # how do you want to backup? echo -e "\nWhat do you want to do? (Use CONTROL-C to abort)\n (1) Minimal backup (2) Interactive backup" while [ "$option" != '1' -a "$option" != '2' ]; do echo -en "\nPlease enter your option: " read option done case $option in 1) stage4Name=$stage4Location/$stage4prefix-minimal final_command="tar $default_exclude_list $custom_exclude_list $tarOptions $stage4Name.tar.bz2 / /var/log/emerge.log" ;; 2) for folder in $custom_exclude_list; do echo -en "Do you want to backup" `echo "$folder" | cut -d'=' -f2`"? (y/n) " read answer while [ "$answer" != 'y' -a "$answer" != 'n' ]; do echo "please enter y or n" read answer done if [ "$answer" == 'n' ]; then default_exclude_list="$default_exclude_list $folder" fi done stage4Name=$stage4Location/$stage4prefix-custom final_command="tar $default_exclude_list $tarOptions $stage4Name.tar.bz2 / /var/log/emerge.log" ;; esac # show what will be done echo -e "\n* creating the stage4 at $stage4Location with the following options:\n\n"$final_command # everything is set, are you sure to continue? echo -ne "\nDo you want to continue? (y/n) " read answer while [ "$answer" != 'y' ] && [ "$answer" != 'n' ]; do echo "please enter y or n" read answer done if [ "$answer" == 'y' ]; then # mount boot echo -e "\n* mount boot" mount /boot >/dev/null 2>&1 # if necessary, create the stage4Location if [ ! -d "$stage4Location" ] ; then echo "* creating directory $stage4Location" mkdir -p $stage4Location fi # check whether the file already exists if [ -a "$stage4Name.tar.bz2" ]; then echo -en "\nDo you want to overwrite $stage4Name.tar.bz2? (y/n) " read answer while [ "$answer" != 'y' ] && [ "$answer" != 'n' ]; do echo "please enter y or n" read answer done if [ "$answer" == 'n' ]; then echo -e "\n* There's nothing to do ... Exiting" exit 0; fi fi # do the backup time $final_command # copy the current world file to the stage4 location echo -e "\n* creating stage4 overview $stage4Name.txt" cp /var/lib/portage/world $stage4Name.txt >/dev/null 2>&1 # we finished, clean up echo "* stage4 is done" echo "* umounting boot" umount /boot else echo -e "\n* There's nothing to do ... Exiting" fi #Uncomment the following command if you want to split the archive in cd size chunks: #split --suffix-length=1 --bytes=670m $stage4Name.tar.bz2 "$stage4Name".tar.bz2_ && echo "* splitting is done" |
Here an alternative mkstage4 shell script:
| File: /sbin/mkstage4 |
#!/bin/sh
# checks if run as root
if ! [ "`whoami`" == "root" ]
then
echo "`basename $0`: must be root."
exit 1
fi
# checks for correct cmdline usage
if [ "$#" == "0" ]
then
echo "`basename $0`: too few arguments."
echo "syntax: \$ `basename $0` [-q|--quiet] <backup-filename> [custom-tar-options]"
echo "-q, --quiet: activates quiet mode (no confirmation)."
exit 1
fi
# checks for quiet mode (no confirmation)
if [ "$1" == "-q" ] || [ "$1" == "--quiet" ]
then
shift
AGREE="yes"
fi
# determines if filename was given with relative or absolute path
if [ "`echo $1 | grep -c '\/'`" -gt "0" ] && \
[ "`echo $1 | grep -c '^\/'`" -gt "0" ]
then
STAGE4_FILENAME="$1"
else
STAGE4_FILENAME="`pwd`/$1"
fi
# misc vars initialization block
shift;OPTIONS="$@"
EXCLUDES="\
--exclude=/dev/* \
--exclude=/proc/* \
--exclude=/sys/* \
--exclude=/tmp/* \
--exclude=/usr/portage \
--exclude=/var/tmp/* \
--exclude=/var/lock/* \
--exclude=/var/log/* \
--exclude=/var/run/* \
--exclude=.bash_history \
--exclude=lost+found \
--exclude=$STAGE4_FILENAME"
TAR_OPTIONS="-cjpP --ignore-failed-read -f"
# if not in quiet mode, this message will be displayed
if [ "$AGREE" != "yes" ]
then
echo "Are you sure that you want to make a complete system-backup now?"
echo ""
echo "NOTE: external filesystems which should not be saved must be"
echo "unmounted before backing up or their mountpoints excluded per cmdline."
echo "example: \$ `basename $0` /my-backup.tar.bz2 --exclude=/mnt/samba/* ..."
echo ""
echo "WARNING: since all data is saved by default the user should exclude all"
echo "security- or privacy-related files and directories manually per cmdline."
echo "example: \$ `basename $0` /my-backup.tar.bz2 --exclude=/etc/ssh/ssh_host*"
echo ""
echo -e "COMMAND LINE PREVIEW:\n\$ tar $TAR_OPTIONS $STAGE4_FILENAME / $EXCLUDES $OPTIONS"
echo ""
echo -n "Type \"yes\" to continue or anything else to quit: "
read AGREE
fi
# start stage4 creation
if [ "$AGREE" == "yes" ]
then
tar $TAR_OPTIONS $STAGE4_FILENAME / $EXCLUDES $OPTIONS
fi
exit 0
|
[edit] Credits
Original Forum Post by allucid.
Up-to-Date Forum Thread link you to Blinkeye's Wiki by Blinkeye.
[edit] Feedback
Concerns or Compliments? Please use the Discussion section.
