SECURITY LVM and EVMS over cryptsetup-LUKS

From Gentoo Linux Wiki

This article is part of the HOWTO series.

Installation • Kernel & Hardware • Networks • Portage • Software • System • X Server • Gaming • Non-x86 • Emulators • Misc

This article is part of the Security series.

1 Introduction to Using EVMS/LVM layered on top of DM-Crypt/LUKS encrypted disks
- 1.1 Preface
- 1.2 Reasons
2 Kernel Preparation
3 Disk partitioning
4 LUKS (Linux Unified Key Setup)
5 LVM (Logical Volume Manager)
- 5.1 Configuring LVM
- 5.2 Setting up, using, and managing with LVM
6 Summary of all steps using different invocation of lvm
7 EVMS (Enterprise Volume Management System)
8 Installing initrd
9 Todo

[edit] Introduction to Using EVMS/LVM layered on top of DM-Crypt/LUKS encrypted disks

[edit] Preface

I felt like doing a quick tutorial on setting up lvm (and evms) on top of cryptsetup-luks on top of a disk partition.

Warning: MAKE A BOOTABLE BACKUP UNLESS YOU ARE A DEITY!

I am a novice at the below programs, so this is fairly simple but probably incomplete and missing important safeguards in the instructions, and it might even be wrong, but it works.

The concepts are identical for most Linux distributions. I have two Linux distributions which I've done this with: Gentoo, then Debian. Since many of the concepts are identical or similar for both systems, I just leave documentation for both in. When it says "emerge", that's Gentoo; when it says something starting with "apt" or "dpkg", that's Debian. Otherwise, I'm speaking about both systems.

[edit] Reasons

Why? So:

No unencrypted volume information (metadata, partition sizes, etc.) besides necessary pre-encryption boot strap and the size of the whole large disks (and bootstrap);
Can manage everything (multiple volume objects, analogous to partitions) with LVM or EVMS (resize, move, copy, rename, add, delete volumes, etc.) within and across any number of large monolithic encrypted physical disk devices (without any unencrypted metadata), so not difficult to manage at all. Nearly all intended features of LVM and EVMS available to users of LUKS.

Both of these successful reasons contrast sharply to the case in which they do not hold true, i.e., LUKSing each filesystem one by one, which seems to be the old default support for LUKS and those users who also used LVM/EVMS. The above features should be enough in most cases that anyone who does want encryption, and previously either didn't or did use LVM/EVMS, would want to start doing so in the manner I describe now.

[edit] Kernel Preparation

First, make sure you have the correct kernel support. You can make sure you have the modules, get them, build the kernel with them, or make sure you have a kernel with it built in static or build a kernel with them static (inside the kernel). Here's what I think it needs for LUKS support, but it might need more:

Ok what modules in the kernel do we need? Well, we need to be able to use Device Mapper (DM), Device Mapper Crypt with LUKS (dm-crypt??), and finally LVM which is just Device Mapper. So, here I go to look at my .config file so you know what must be enabled.

CONFIG_DM_CRYPT=y or =m if module, but remember that module must be in your initrd if using initrd

so perhaps if your system includes the module (does it?? perhaps some default Gentoo or Debian systems have modules for DM_CRYPT and LVM included) you can just use that. But, there are many more that you have to have turned on too:

In case your kernel doesn't include those modules and you need to build a new kernel or modules, it is in:

Device Drivers -> Multi-device support -> Device mapper support -> Crypt target support

Mark all the ones up to that Module/Yes depending on your situation. Also, go to:

Cryptographic options ->

and set most of those on, including SHA256, AES, and whatever else you'll use.

This has been tested to work for Linux kernels 2.6.16 through 2.6.18.2.

[edit] Disk partitioning

Ok, well grab a partition for experimentation, say your swap partition. (Let's say your swap partition is /dev/hdb1: Just execute "swapoff /dev/hdb1") (Note that if you have a second disk you don't ever want to boot, you can just use the whole thing, so in this example, if you had a /dev/hdb, then you would just use /dev/hdb instead of /dev/hdb1 and just skip the nasty fdisk step (which requires an annoying reboot anyway). In theory as far as I know there's nothing at all wrong with it; I haven't tried it. But, there's no need for Microsoft/DOS/IBM partitioning if you are going to just encrypt the whole disk anyway, then put LVM or EVMS on top of that anyway, since those can manage your various sub pieces of the disk very nicely. But, if you need to boot onto it using, say, a GRUB kernel and initrd partition, then you need partitioning of some sort that the kernel, some utility like fdisk, and GRUB, will understand. This could be a file using BOOTP/DHCP served initrd, if you like! You could even have a diskless client and use NBD (network block device) to grab a remote disk, then put cryptsetup on top of the NBD (I'm currently doing that right now and it works quite well), then go from there, which means no NFS and no unencrypted LAN traffic if you layer it that way, and it's quite fast, no NFS nonsense, but only one client can do it readwrite per NBD device (readonly can do multiple).)

Anyway the theory is encrypting the whole disk except for an initrd -- but I'll leave setting up an initrd to another time; for now, just get enough for experiment. As you take this further, for fallback, keep your boot and root partitions until your initrd works right and with backups, then you can put a copy of root partition inside LVM and then try to boot from the initrd into that and if it works you can get rid of root, but be careful to keep backups for booting in case that doesn't work. Now, let's assume you have a partition to work with that we can erase.

You can cryptsetup encrypt as many physical devices as you want. Each one would get its own cryptsetup encryption, then it would be added to whatever volume group you want it to be in, and LVM/EVMS would handle it from there.

[edit] LUKS (Linux Unified Key Setup)

emerge cryptsetup-luks

If you like, you can also emerge lvm2 now, or later. (In Debian: "apt-get install lvm2 cryptsetup". There *might* be a problem in Debian Sarge that cryptsetup isn't good enough so you *might* have to go get a recent version and build and install that, but *I AM NOT SURE*.)

Let's say your partition you're partitioning is /dev/hdb1: something like (do all these cryptsetup steps for all physical devices you are going to use, using a new DM name for each device; this does delete data on those devices):

cryptsetup -c aes-cbc-essiv:sha256 luksFormat /dev/hdb1

You must chose a password. This is done once for persistent parts. Then,

cryptsetup luksOpen /dev/hdb1 HDB1

This must be done every time you wish to use it, so at boot. It goes in some script someplace early, before LVM. If you don't put this in an initrd and want to experiment for now and/or want to do some of this after your rc scripts start running, you could just put it into your LVM script before LVM. You must supply a password. You can either type it in, or some how or other deliver it securely. If you have a secure file (say, in another DM-crypted partition that you have already, somehow, via user intervention or something, unlocked), you can just redirect it:

cryptsetup luksOpen /dev/hdb1 HDB1 < /some/password/file/someplace

[edit] LVM (Logical Volume Manager)

emerge lvm2

[edit] Configuring LVM

You must edit /etc/lvm/lvm.conf to add a line with quotes in it:

types = [ "device-mapper", 16 ]

There's already a prototype line for types, so put it near that. And, in /etc/lvm/lvm.conf, you have to edit the filter line to be something like this: I use capitol letters for dm crypt mounted stuff, and small letters for other stuff:

filter = [ "a|^/dev/mapper/[A-Z].*|", "r|^/dev/mapper/.*|", "r|/dev/cdrom|" ]

(all one line). There are variations like:

filter = [ "a|^/dev/mapper/HDB|", "r|^/dev/mapper/.*|", "a|.*|" ]

filter = [ "a|^/dev/mapper/HDB1$|", "r/.*/" ]

The last one is the most secure and least error prone, since it is VERY specific. This gets used by pvscan and vgscan. Mine is a bit more general, using in one system the one above with [A-Z].* in it, and in another system just omitting the $ regexp piece and using a letter for my luks mounts:

filter = [ "a|^/dev/mapper/L|", "r/.*/" ]

That matches /dev/mapper/L, /dev/mapper/L1, /dev/mapper/L2, /dev/mapper/LEMMEDOIT, and so on. These edits to /etc/lvm/lvm.conf should need to be made once.

[edit] Setting up, using, and managing with LVM

Then, continue with the following:

For every time you wish to set up /dev, so perhaps at every boot since you might be using udev or devfs or tmpfs or something, and right now since you're setting things up:

dmsetup targets
dmsetup mknodes

Note! You know you have what we need if you get a line with crypt in it after executing "dmsetup targets": crypt v1.1.0 Then, during setup, do this:

pvcreate /dev/mapper/HDB1
vgcreate vg /dev/mapper/HDB1

pv stands for Physical Volume, and vg stands for Volume Group, a group of individual volumes (so vg contains, say, usr, home, var, root, video, distfiles, etc.) If you had more than one physical volume (set up first with cryptsetup-luks for each one), you can vgcreate for all of them at once:

vgcreate vg /dev/mapper/HDB1 /dev/mapper/HDC /dev/mapper/SDA

or you could do some, then extend later:

vgcreate vg /dev/mapper/HDB1 /dev/mapper/SDA
vgextend vg /dev/mapper/HDC

You can, of course, have multiple volume groups, but I don't see why you'd do that unless there were different physical disk qualities you wanted different volumes to go onto specifically (like fast unreliable swapping disks for tmp and swap volumes, and RAID arrays for more reliable stuff, or something like that). You can vgmerge and vgsplit later, too.

Then, during every time you wish to use them (so say at boot; in initrd for LVM or if you already havce rc scripts running at boot then before LVM):

vgscan --mknodes
vgchange -ay
lvcreate -L256M -n tmp2 vg
mkreiserfs /dev/vg/tmp2  OR  mkfs /dev/mapper/vg-tmp2    or whatever
  (mkswap, etc., whatever else you want with a partition)

Then, put an appropriate line in /etc/fstab, like:

/dev/mapper/vg-tmp2 /tmp2 reiserfs noatime 1 1

or if you use "vgscan --mknodes" with the --mknodes option and your "vgchange -ay", you can use:

/dev/vg/tmp2 /tmp2 reiserfs noatime 1 1

it's just a symlink to /dev/mapper/vg-tmp2. Anyway, to do this immediately by hand:

mount /dev/vg/tmp2 /tmp2

And, now you have a DM block device with reiserfs or something on it of 256MB. You can go ahead and make more LVM partitions:

lvcreate -L128M -n swap vg
mkswap /dev/vg/swap

and put that into /etc/fstab:

/dev/vg/swap none swap sw 0 0

and use it immediately:

swapon /dev/vg/swap

And more filesystems:

lvcreate -L1G -n usr2 vg
mkreiserfs /dev/vg/usr2
mount /dev/vg/usr2 /mnt
cp -a /usr/* /mnt/

and so on. Now, let's say you need to rename a reiserfs partition; well, that's easy:

umount /mnt
lvrename vg usr2 usr
mount /dev/vg/usr /mnt

Then, what if you need to expand it? Well, that's even easier:

lvresize -L+256M /dev/vg/usr
resize_reiserfs /dev/vg/usr

That can be done online with the filesystem mounted! Awesome, huh?

Ok, what if you need to make it smaller? Well, that's harder. Let's do it:

umount /dev/vg/usr
resize_reiserfs -s1152M /dev/vg/usr   # (answer the question y)
reiserfsck --fix-fixable /dev/vg/usr  # (paranoid step; not necessary)
lvresize -L1.125G /dev/vg/usr
mount /dev/vg/usr /mnt

To look at your LVM logical volumes, you have all sorts of status commands:

pvs
vgs
lvs
pvscan
lvscan
pvdisplay
vgdisplay
lvdisplay
vgcfgbackup
 then look at /etc/lvm/backup/vg in this example.

Ok, that wasn't so bad.

There's lots of obvious LVM commands for managing volumes.

[edit] Summary of all steps using different invocation of lvm

What if you want to use one executable named "lvm" and don't want to use a dozen little lvm executables like "vgscan", "vgchange", "lvcreate", etc.? Well, just prefix all of the pv, vg, lv commands with "lvm", like this (with a review of most steps taken in the above):

Device Drivers -> Multi-device support -> Device mapper support -> Crypt target support

Cryptographic options -> and set most of those on, including SHA256,

emerge cryptsetup-luks lvm2    #  apt-get install cryptsetup lvm2
cryptsetup -c aes-cbc-essiv:sha256 luksFormat /dev/hdb1
cryptsetup luksOpen /dev/hdb1 HDB1 < /some/password/file/someplace
/etc/lvm/lvm.conf:
  filter = [ "a|^/dev/mapper/HD|", "r/.*/" ]
  types = [ "device-mapper", 16 ]
dmsetup targets
dmsetup mknodes
lvm pvcreate /dev/mapper/HDB1
lvm vgcreate myfirstlvg /dev/mapper/HDB1
lvm vgscan --mknodes
lvm vgchange -ay
lvm lvcreate -L256M -n var myfirstlvg
mkreiserfs /dev/myfirstlvg/var
/etc/fstab:
  /dev/mapper/myfirstlvg-var /var reiserfs noatime 1 1
mount /dev/myfirstlvg/var /var
lvm lvresize -L+256M /dev/myfirstlvg/var
resize_reiserfs /dev/myfirstlvg/var
lvm pvs
lvm vgs
lvm lvs
lvm pvscan
lvm lvscan
lvm pvdisplay
lvm vgdisplay
lvm lvdisplay
lvm vgcfgbackup # then look at /etc/lvm/backup/myfirstlvg in this example.

[edit] EVMS (Enterprise Volume Management System)

Putting EVMS on this too in lieu of or along with LVM is done like this:

Warning: NOTE! You must make certain you have backups for this step!

EVMS can destroy volumes VERY FAST with this setup, so be PARTICUARILY PARANOID about backing things up! All three of my systems (two Gentoo and one Debian) use EVMS as described here, so I know it works, and use it constantly, but haven't touched the Gentoo setups for a while and I just installed it in Debian tonight, so I may be quite rusty. BE VERY CAREFUL and have BACKUPS that WORK, including a BOOTABLE BACKUP.

YOU HAVE BEEN WARNED.

[edit] EVMS preface

EVMS is straightforward if you already use EVMS. Basically, just patch EVMS as I am about to describe, then substitute LVM above with EVMS tools you already use (evmsn, /dev/evms/..., then evms_activate at boot, etc.), and you're set. You may wish to reorganize at this point, though, so you'll have to reconsider everything as you go through this. E.g., using dm-crypt, you'll want to put as much in encrypted partitions as possible, so what you used to consider an acceptable loss of EVMS (or LVM) control over your root partition logical voluming capabilities by being forced to chose a size for it may now be unacceptable to have unencrypted. Or, you could remount a new root partition over the old root partition after you're done loading the LVM/EVMS stuff. Who knows. It's up to you, but it can make a headache if you do it wrong, so decide now, and figure out how to do it simply. We can expand with that later in this Wiki. (I put root filesystem inside too (inside EVMS (with other volumes) inside LUKS) and use initrd.) For now, just EVMS.

If doing this in initrd, you must EVMS in initrd -- think of EVMS as a more advanced version of LVM, and do EVMS wherever you would do LVM in the above situation.

[edit] Patch EVMS

First, you must patch EVMS to actually look for volumes of DM (device mapper) (the stuff that shows up in /dev/mapper) since dm-crypt (with cryptsetup program) makes new DM devices but EVMS usually thinks that it already knows how to make all the DM devices on its own, but it doesn't. So, we have to:

A)  patch EVMS to look for these devices (or get a patched version), and
B)  edit /etc/evms.conf to turn on the functionality of the patch
    and to look for these particular devices.

These two steps are similar and analogous to editing /etc/lvm/lvm.conf above. Sorry, no immediate "emerge evms" (or "apt-get install evms") for this.

Patching EVMS is only slightly difficult. In order of difficulty from easiest to hardest is: (a) Getting the precompiled package for Debian Sarge (if it works); (b) setting up an overlay for emerge in Gentoo (this should work easily until Gentoo releases a new version of its EVMS ebuilds); and finally (c) patching and rebuilding the package for Debian if necessary. Enjoy.

[edit] Patching Gentoo EVMS

In Gentoo, we have to edit our portage overlays. Basically, it starts with the line in /etc/make.conf:

PORTDIR_OVERLAY="/etc/portage"

Some people use /var/local/lib/portage, others /usr/local/lib/portage. Get some of the files set and checked first. Do:

emerge -f evms

to get and verify the correct files all at once, since we're going to create digests in the next step, so it is important the file verification is already done before we tell Gentoo the files are already OK. Note the version it is emerging. Now:

. /etc/make.conf
dest="$PORTDIR_OVERLAY/sys-fs"
mkdir -p "$dest"
cp -a "$PORTDIR/sys-fs/evms" "$dest/"
cd "$dest/evms"

You may want to call your new ebuild evms-2.5.5-r90.ebuild to distinguish it from the standard ebuilds. /PjK 2007-02-21

You can then do any one of the following first three options, and sometimes the fourth:

# Use noted version from above (if you noted it) or get it one of these ways:
tail /var/log/emerge.log
emerge -p evms
# and sometimes:  equery which evms (but often (usually) this gives too recent a version)

The filename will be evms-versionsomething.ebuild, such as evms-2.5.5-r1.ebuild. The file should already be there in $PORTDIR_OVERLAY/sys-fs/evms. Edit it.

Following are the changes needed for the ebuild file. I include two versions, one for evms-2.5.5-r1.ebuild and one for evms-2.5.5-r2.ebuild. You can adapt the changes to other versions of the evms ebuild file, and it will probably work fine. (These modified ebuilds include another patch to fix zero size disk geometries that may be needed for this patch to work as used in this document, which I needed for Linux kernel 2.6.17 and 2.6.18 using EVMS 2.5.5 and before (this patch is included in EVMS CVS and probably in EVMS 2.5.6 and beyond); previously it worked fine without the geometry patch in Linux kernel 2.6.16, but it should also run fine in 2.6.16 with the patch): First, is the patch for ebuild file $PORTDIR_OVERLAY/sys-fs/evms/evms-2.5.5-r1.ebuild if that's what you're using (use -l option to patch to match tabs to spaces, or use:

source /etc/make.conf
curl http://www.sonic.net/~qm/gentoo/evms/evms-2.5.5-r1.ebuild.dminput.patch |
patch $PORTDIR_OVERLAY/sys-fs/evms/evms-2.5.5-r1.ebuild

Here is the patch to the ebuild:

--- /var/lib/portage/sys-fs/evms/evms-2.5.5-r1.ebuild   2006-08-20 14:36:16.000000000 -0700
+++ /etc/portage/sys-fs/evms/evms-2.5.5-r1.ebuild       2006-11-09 22:04:36.000000000 -0800
@@ -11,7 +11,7 @@
 LICENSE="GPL-2"
 SLOT="0"
 KEYWORDS="amd64 ia64 ppc ppc64 sparc x86"
-IUSE="ncurses gtk nls"
+IUSE="ncurses gtk nls evmsdminput"
 
 #EVMS uses libuuid from e2fsprogs
 RDEPEND="virtual/libc
@@ -25,9 +25,20 @@
 src_unpack() {
         unpack ${A}
         cd ${S}
-        epatch ${FILESDIR}/${PV}/raid5_degrade_fix.patch
         epatch ${FILESDIR}/${PV}/md_super_fix.patch
+        epatch ${FILESDIR}/${PV}/md_super_fix.patch
+        epatch ${FILESDIR}/${PV}/ntfs_unmkfs.patch
+        epatch ${FILESDIR}/${PV}/raid5_degrade_fix.patch
+        epatch ${FILESDIR}/${PV}/raid5_remove_spare_fix.patch
+        epatch ${FILESDIR}/${PV}/raid5_remove_spare_fix_2.patch
+        epatch ${FILESDIR}/${PV}/raid5_algorithm.patch
+        epatch ${FILESDIR}/${PV}/cli_reload_options.patch
+        epatch ${FILESDIR}/${PV}/cli_query_segfault.patch
+        epatch ${FILESDIR}/${PV}/get_geometry.patch
+        epatch ${FILESDIR}/${PV}/BaseName.patch
+
         epatch ${FILESDIR}/evms-2.5.5-as-needed.patch
+        use evmsdminput && epatch ${FILESDIR}/${PV}/2.5.5-dev-mapper- input.patch
 }
 
 src_compile() {

Second is the patch for ebuild $PORTDIR_OVERLAY/sys-fs/evms/evms-2.5.5-r2.ebuild if that's what you're using (use patch -l to match tabs to spaces, or just use:

source /etc/make.conf
curl http://www.sonic.net/~qm/gentoo/evms/evms-2.5.5-r2.ebuild.dminput.patch |
patch $PORTDIR_OVERLAY/sys-fs/evms/evms-2.5.5-r2.ebuild

Here is that patch:

--- /var/lib/portage/sys-fs/evms/evms-2.5.5-r2.ebuild   2006-08-20 14:36:16.000000000 -0700
+++ /etc/portage/sys-fs/evms/evms-2.5.5-r2.ebuild       2006-11-09 22:00:09.000000000 -0800
@@ -11,7 +11,7 @@
 LICENSE="GPL-2"
 SLOT="0"
 KEYWORDS="~amd64 ~ia64 ~ppc ~ppc64 ~sparc ~x86"
-IUSE="ncurses gtk nls"
+IUSE="ncurses gtk nls evmsdminput"
 
 #EVMS uses libuuid from e2fsprogs
 RDEPEND="virtual/libc
@@ -25,11 +25,19 @@
 src_unpack() {
         unpack ${A}
         cd "${S}"
-        epatch "${FILESDIR}"/${PV}/raid5_degrade_fix.patch
         epatch "${FILESDIR}"/${PV}/md_super_fix.patch
+        epatch "${FILESDIR}"/${PV}/ntfs_unmkfs.patch
+        epatch "${FILESDIR}"/${PV}/raid5_degrade_fix.patch
         epatch "${FILESDIR}"/${PV}/raid5_remove_spare_fix.patch
         epatch "${FILESDIR}"/${PV}/raid5_remove_spare_fix_2.patch
+        epatch "${FILESDIR}"/${PV}/raid5_algorithm.patch
+        epatch "${FILESDIR}"/${PV}/cli_reload_options.patch
+        epatch "${FILESDIR}"/${PV}/cli_query_segfault.patch
+        epatch "${FILESDIR}"/${PV}/get_geometry.patch
+        epatch "${FILESDIR}"/${PV}/BaseName.patch
+
         epatch "${FILESDIR}"/evms-2.5.5-as-needed.patch
+        use evmsdminput && epatch ${FILESDIR}/${PV}/2.5.5-dev-mapper-input.patch
 }
 
 src_compile() {

We need to have the patch itself (as well as the geometry divide by zero patch to prevent divide by zero faults from zero geometries (SIGFPE, floating point exception signal)). Go to the files subdirectory someplace appropriate, which here is files/2.5.5, then get the files:

cd files/2.5.5
wget http://www.sonic.net/~qm/debian/evms-dm/2.5.5-dev-mapper-input.patch
for i in md_super_fix.patch ntfs_unmkfs.patch raid5_degrade_fix.patch raid5_remove_spare_fix.patch raid5_remove_spare_fix_2.patch raid5_algorithm.patch cli_reload_options.patch cli_query_segfault.patch get_geometry.patch BaseName.patch;do
wget http://evms.sourceforge.net/patches/2.5.5/engine/$i;done

cd $PORTDIR_OVERLAY/sys-fs/evms/files/2.5.5
wget ftp://ftp.sonic.net/pub/users/qm/debian/evms-dm/2.5.5-dev-mapper-input.patch
wget http://evms.sourceforge.net/patches/2.5.5/engine/md_super_fix.patch
wget http://evms.sourceforge.net/patches/2.5.5/engine/ntfs_unmkfs.patch
wget http://evms.sourceforge.net/patches/2.5.5/engine/raid5_degrade_fix.patch
wget http://evms.sourceforge.net/patches/2.5.5/engine/raid5_remove_spare_fix.patch
wget http://evms.sourceforge.net/patches/2.5.5/engine/raid5_remove_spare_fix_2.patch
wget http://evms.sourceforge.net/patches/2.5.5/engine/raid5_algorithm.patch
wget http://evms.sourceforge.net/patches/2.5.5/engine/cli_reload_options.patch
wget http://evms.sourceforge.net/patches/2.5.5/engine/cli_query_segfault.patch
wget http://evms.sourceforge.net/patches/2.5.5/engine/get_geometry.patch
wget http://evms.sourceforge.net/patches/2.5.5/engine/BaseName.patch

Now, we need to compute the new digests:

ebuild `equery which evms` digest

where `equery which evms` is the file you just edited. You may need to specify its full path to get its correct version if equery returns the wrong one (like a version too new) or you don't have equery, E.g., "ebuild /etc/portage/sys-fs/evms/evms-2.5.5-r2.ebuild digest".

In /etc/portage/package.use put the line:

sys-fs/evms evmsdminput

Then, simply:

emerge evms

[edit] Patching Debian EVMS

This might work for Debian Sarge (but doesn't include the zero size geometry patch mentioned below, so it might not work for Linux kernel 2.6.17, 2.6.18, and beyond with versions of EVMS 2.5.5 and before (the patch is in EVMS CVS, so should probably be in EVMS 2.5.6 and beyond)):

wget -r -c -nd ftp://ftp.sonic.net/pub/users/qm/debian/evms-dm/debs
md5sum -c MD5*

then skip to "dpkg -i ..." below.

So, we get EVMS source (which I will actually test in my Debian system):

dpkg -l evms
  if nothing there, then:
apt-get install evms

Note the version it asks to install. You can either install or not install it. Installing it can't really hurt, since it will grab everything else needed. I have:

ii  evms                2.5.2-1.sarge1      Enterprise Volume Management System (core)

apt-get build-dep evms=2.5.2-1.sarge1
cd /usr/src;mkdir evms;cd evms
apt-get source evms=2.5.2-1.sarge1
apt-get install debian-keyring gnupg
gpg --keyring /usr/share/keyrings/debian-keyring.gpg --verify *dsc

This puts source code for EVMS in /usr/src/evms. You can choose a different directory. Having gnupg, debian-keyring, and gpg checking it are not necessary if you are a trusting individual. Now, patch it:

I have three patches for EVMS for versions 2.5.3, 2.5.4, and 2.5.5: If you are using Debian Sarge, perhaps adapting the patch for 2.5.3 to 2.5.2 ought to work; I tried applying 2.5.3 patch to 2.5.2, and it seemed to work with fuzz. Working patches for their own versions (I've used them succesfully before):

   11483 Nov 25  2005 2.5.3-dev-mapper-input.patch
   12175 Dec 17  2005 2.5.4-dev-mapper-input.patch
   11533 Mar  5 12:56 2.5.5-dev-mapper-input.patch

Ok, let's see. Let's say you have evms 2.5.2 for Sarge as above:

wget ftp://ftp.sonic.net/pub/users/qm/debian/evms-dm/2.5.3-dev-mapper-input.patch
patch -d evms-2.5.2 -p1 < 2.5.3-dev-mapper-input.patch
wget ftp://ftp.sonic.net/pub/users/qm/gentoo/evms/2.5.5_fpe_localdskmgr.c.patch
patch -d evms-2.5.2 -p1 < 2.5.5_fpe_localdskmgr.c.patch

or just:

curl http://www.sonic.net/~qm/debian/evms-dm/2.5.3-dev-mapper-input.patch |
patch -p1 -d evms-2.5.2
curl http://www.sonic.net/~qm/gentoo/evms/2.5.5_fpe_localdskmgr.c.patch |
patch -p1 -d evms-2.5.2

The above includes an additional patch to deal with zero size disk geometries that might be encountered in Linux kernels 2.6.17 and above in this setup, to prevent a divide by zero error (with a floating point exception signal; you'll notice right away when you try to run evms_activate -- if it gets SIGFPE then you need this zero geometry size patch). I haven't tested the above localdskmgr.c patch in evms-2.5.2 in Debian; you might have to adapt it.

Now, build it:

cd evms-2.5.2;./debian/rules binary
cd ..

If you have other equivilent debian systems without development packages, then you can just use this binary to install on those systems. I put a copy of this deb at the link I specified above before we talked about obtaining source.

And finally install the resulting deb files:

dpkg -i evms*.deb lib*.deb

[edit] Configuring EVMS

Now, in the file /etc/evms.conf, do the following modifications:

Add (there should be templates for this at the bottom of the file):

   devmapper {
       dm_user = yes
       dm_user_targets = [ HDB1 ]
   }

Note that the target must be the name of the cryptsetup device in /dev/mapper, like the above LVM examples. Do "ls /dev/mapper" to see what cryptsetup devices you created. Ahh, the example evms.conf file has a more canonical way of knowing them: "dmsetup ls" (probably works even when /dev/mapper is messed up, as long as the devices actually are actually in the kernel at the moment (opened with cryptsetup)). These are the names at the end of the "cryptsetup luksOpen /some/device xxx" you made, xxx in that example. I'm using the standard of making them all capital letters. You do it any way you like, so you can keep them distinct from the non-encrypted ones, so LVM/EVMS can know which ones to look at by your specifications in the config files (/etc/lvm/lvm.conf and /etc/evms.conf, respectively). I'm still using the HDB1 example I used above. You cannot use wildcards in dm_user_targets.

Modify:
Make sure activate and sysfs_devices have the include line:
 include [ dm* ] (perhaps just [ * ]).

Make sure exclude is there so it excludes stuff you don't want EVMS to touch in sections activate, legacy_devices, and sysfs_devices, in this example, hdb1.

Here is one working evms.conf:

engine {
        mode            = readwrite
        debug_level     = default
        log_file        = /var/log/evms-engine.log
        metadata_backup_dir     = /var/evms/metadata_backups
}
daemon {
        debug_level     = default       # Same settings as available for
        log_file        = /var/log/evms-daemon.log
}
clustering {
        membership_timeout = 10
}
activate {
        include = [ * ]
        exclude = [ hdc3 hdc6 ]
}
legacy_devices {
        scan = /dev
        directories = [ ide scsi dasd ]
        include = [ hd? sd? dasd? disc ]
        exclude = [ hdc3 hdc6 ]
}
sysfs_devices {
        include = [ * ]
        exclude = [ hdc3 hdc6 ]
}
csm {
}
multipath {
}
lvm2 {
        device_size_prompt = yes
}
devmapper {
        dm_user = yes
        dm_user_targets = [ HDB1 ]
}

Here is another working evms.conf:

engine {
    mode            = readwrite
    debug_level     = default
    log_file        = /var/log/evms-engine.log
    metadata_backup_dir     = /var/evms/metadata_backups
}
daemon {
    debug_level     = default       # Same settings as available for
    log_file        = /var/log/evms-daemon.log
}
clustering {
    membership_timeout = 10
}
activate {
    include = [ * dm* ]
    exclude = [ loop* hda* ]
}
legacy_devices {
    scan = /dev
    directories = [ ide scsi dasd ]
    include = [ * dm* ]
    exclude = [ loop* hda* ]
}
sysfs_devices {
    include = [ * dm* ]
    exclude = [ hda* ]
}
csm {
}
multipath {
}
lvm2 {
        device_size_prompt = yes
}
devmapper {
    dm_user = yes
    dm_user_targets = [ LUKS1 LUKS2 LUKS3 HDC BIGDISK ]
}

Here's /etc/evms.conf from my Debian system that I got working with this EVMS setup:

engine {
        mode            = readwrite
        debug_level     = default
        log_file        = /var/log/evms-engine.log
        metadata_backup_dir     = /var/evms/metadata_backups
}
daemon {
        debug_level     = default       # Same settings as available for
        log_file        = /var/log/evms-daemon.log
}
clustering {
        membership_timeout = 10
}
activate {
        include = [ * ]
        exclude = [ hda[2-9] hd[b-c]* sda* ]
}
legacy_devices {
        scan = /dev
        directories = [ ide scsi dasd ]
        include = [ hd? sd? dasd? disc ]
        exclude = [ hd[b-c] sda ]
}
sysfs_devices {
        include = [ * ]
        exclude = [ hd[b-c] sda ]
}
csm {
}
multipath {
}
lvm2 {
        device_size_prompt = yes
}
devmapper {
        dm_user = yes
        dm_user_targets = [ HDB1 LUKS1 LUKS2 LUKS3 HDC BIGDISK ]
}

In that example, there's HDB1, perhaps LUKS1 through LUKS3 are 3 partitions of hda but there's hda1 which is a boot partition that I want to read, and HDC is a whole disk being LUKSed, and then BIGDISK is some enourmous scsi disk that's being luxed.

[edit] Running, activating and configuring EVMS

Now, EVMS should see the unencrypted cryptsetup dm-crypt output devices in /dev/mapper properly. I'm going to try it right now on my Debian Sarge system that I compiled those packages on:

evmsgui

if no X, use:

evmsn

if no curses (!!!!), use:

evms

I usually use evmsn, but I think evmsgui is nicer.

If you haven't yet run that, notice that the volumes EVMS hasn't touched yet are marked inactive and aren't mounted, even if they are active and/or are mounted.

Anyway, the first time EVMS starts, it changes the volumes a little (it won't save this if you use evmsn or evmsgui until you tell it to). Probably before having, letting, or risking EVMS doing this you should unmount all those volumes. It puts in metadata. It can destroy your system. It can take it over. It can make hell for you. Be very careful. One way to do this is just to run "evms_activate", I'm pretty sure. Another way is within the program you just ran above, notice that the objects have been automatically marked "Modified" by the EVMS program you ran (but those modifications have not yet been saved), and you go to SAVE. You can just SAVE, or go to QUIT and it will ask you whether you want to cancel quitting, quit without saving, or save then quit. Choose carefully. "SAVE THEN QUIT" is the one that does the big deed. "Quit without saving" simply discards the planned modifications and doesn't do anything to your system. Anyway, if you quit the program after saving, restart one of the evms programs (evmsgui, evmsn, evms), and then the volumes should now be active if they were before, but not mounted unless you remounted the LVM2 partitions as EVMS partitions. If you did anything with mounted partitions, please quit and unmount and "lvm vgchange -an" and evms_activate and filesystem check all of the objects you find in all of the directories and subdirectories of /dev/evms to make certain you didn't fry your system just now.

Something like (substitute lvm2 for lvm1 systems -- look to see what's there to make sure):

umount -a   # (yeah right, like this will actually work)
cd /dev/evms/lvm2
for i in $(find -type b); do echo "Should I check object ${i}?";
read answer; if [ "x$answer" = "xy" ];then
reiserfsck --fix-fixable "$i";fi;done

Then update /etc/fstab to use equivilent /dev/evms entries and mount those instead.

Now, you can do all the wonderful things with EVMS to encrypted disk objects that you used to do or dreamed of doing before with it, like having a GUI to grow disk volumes at will, and renaming stuff, and moving it around and shrinking it after unmounting it, all without having to use command line options if you so choose. All you have to do now is do at boot, whether or not in initrd or a root partition with rc scripts:

dmsetup targets
dmsetup mknodes
evms_activate

Then, /dev/evms has all your volumes.

Anyway, after that, you can convert the LVM parts to "EVMS compatibility" volumes by selecting them and chosing "Convert to EVMS volume". This then SHRINKS THE VOLUME and puts metadata in it. You have to make sure either it is mounted as what EVMS thinks it is, or make sure it is not mounted. This changes the device name from /dev/evms/lvm2/usr to /dev/evms/usr. I'm not sure why you want to do this, but I do it to some of my systems. It makes it fun to use EVMS.

All of the metadata, partition sizes, partition names, etc., except for the stuff you didn't put in your cryptsetup dm-crypt partitions, is now encrypted.

[edit] Installing initrd

If you're using Genkernel, this is easy: Genkernel supports volumes over LUKS natively (at least in 3.4.8+) - the startLUKS function in initrd.scripts, which is called if you specify a crypt_root=X flag at boot, scans for logical volumes after luksOpen'ing it if there is also a real_root=X flag. So, all you need to do is patch EVMS as above (and not even that if you use lvm), tack on 'crypt_root=/dev/evms/hdXX real_root=/dev/evms/whatever_label_you_have' to your kernel commandline, and reboot for lots of root-on-(LVM/EVMS)-on-LUKS goodness! ~XAvAX

UPDATE2: Forget my previous statement labeled "UPDATE", as the device nodes would have already existed if I had used the stage3 tarball and the first call to evms_activate just means that you need /dev/evms/hdXX, which I fixed above. Also, I would NOT recomment 4k stacks, because I got some odd corruptions that didn't occur on lower layers. ~XAvAX

Please see other documents in the Security section for how to create and install an initrd. Be sure to make certain the correct invocations of commands mentioned here which need to be during boot are in that initrd, and test it thoroughly.

For instance, you'll need bin files (such as for evms, /sbin/evms_activate), the libraries for them (e.g., ls -al /lib/libevms* /lib/evms/*/*), conf files (e.g., /etc/evms.conf), perhaps some device files, etc.. If you followed all the steps above, you'll probably realize that you'll need/want all of the LVM stuff in the initrd too, and put it all in there. If you follow the procedures above for booting in the initrd, then many if not all of the devices will be created by the tools above as they are run in an acceptable sequence.

Be sure the version of files in the initrd works. Make sure it boots with the initrd before taking away the non-initrd booting method. (TEST!)

[edit] Todo

make a more complete initrd section.
explain putting NBD into kernel and between various layers (such as "nbd-server 500 /dev/hdb1 -l /etc/nbd_accept" (with remote ip# in /etc/nbd_accept) and "nbd-client serverip 500 /dev/nb0" then "cryptsetup luksOpen /dev/nb0 ARFMEOW" or whatver).
Update Debian section with fully tested EVMS version for later kernels that deals with zero size geometry patch, and put that additional patch in the deb files I have on my FTP site.
Grammar, etc.

Retrieved from "http://www.gentoo-wiki.com/SECURITY_LVM_and_EVMS_over_cryptsetup-LUKS"