HOWTO User Mode Linux

From Gentoo Linux Wiki

[edit] Introduction

User mode Linux allows you to run an almost completely self contained Linux system as a process inside a host Linux system.

[edit] Prerequisite Knowledge

This articles assumes you already have knowledge of working from the command line, networking, manual kernel configuration and iptables (in particular, setting up NAT). Experience with patch and resolving failed patches may also be useful.

[edit] Setting up a directory layout

This article will use the following directory setup to keep the UML setup organized.

• uml/ - The top level directory. All our UML related files will be kept under here.
  • bin/ - This is used for binaries such as the guest kernel.
  • system/ - All our UML guests files will be kept under here. Each guest will have its own subdirectory.
  • files/ - Miscellaneous files (kernel patches).
  • scripts/ - Scripts for performing common tasks.

To create this directory structure, as a non-root user, run: mkdir -p uml/{bin,system,files,scripts}

[edit] Kernel Setup

Depending on how important security and performance are to you, there's a number of different methods of setting up user mode Linux. If you're not using UML in a production environment, you can skip a lot of the kernel patching as the vanilla Linux kernel, on which all other kernels are based, includes enough of the UML code to work "out of the box".

Alternatively, you can download pre-patched binary kernels. Obviously these come with the downside that you can't configure these kernels how you want them.

[edit] General notes on patching a kernel

The first question when wanting to patch a kernel is "Which kernel sources do you patch against?" The answer to this varies depending on how much effort you're prepared to put in and your level of knowledge about patching.

You basically have two options:

1. Use the exact version of vanilla-sources on which the patch is based. So, if the patch is skas-2.6.19-v9-pre9.patch.bz2 then you'd choose the latest 2.6.19 sources (if a VERY exact version isn't given like this, then the patch was usually done against the very latest sources at the time of release, so use the most current version you can), or if the patch is called uml-2.6.18.1-bb2.patch.bz2 then you'd choose the 2.6.18.1 sources.
2. Try to patch against the latest sources or some sources which have already been patched with other patches (for example, gentoo-sources. In this case you may come up against a high number of patch failure, but you may not and the patch might go fine. Even if you do have a high number of failures, these may be easy to fix yourself.

You may also want to read the article on Kernel/Patching, the patch man page and the GNU diffutils maual section on patch before proceeding.

[edit] Host Kernel

Unless performance is important to you, you can skip the host kernel patching. A standard kernel can happily run a UML system at a level of performance that's good enough for most needs.

[edit] Using a pre-compiled host kernel

Pre-compiled host kernels are much harder to come by. The main reason for this is because host kernel configurations are so different, plus you need the sources anyway for building kernel modules. You may have some luck checking the links at the bottom of this page.

[edit] Patching the host kernel

For the host kernel, you'll want to use the SKAS3 patch from the skas-2.6 directory on BlaisorBlade's patches (or if that's down, use the mirror of BlaisorBlade's patches).

For this section it is suggested you place the files in the uml/files/ directory you created above.

From the skas-2.6 directory, locate the directory for the latest sources - this article will use skas-2.6.19-v9-pre9 as an example. Now download the .patch.bz2 file, for example skas-2.6-2.6.19-v9-pre9.patch.bz2, and the .sha1 file, for example skas-2.6-2.6.19-v9-pre9.sha1.

After downloading both files, verify that the downloaded file is the same as what was uploaded to the server by running sha1sum -c on the .sha1 file, for example: sha1sum -c skas-2.6.19-v9-pre9.sha1

skas-2.6.19-v9-pre9-broken-out.tar.bz2: FAILED open or read
skas-2.6.19-v9-pre9.patch.bz2: OK
sha1sum: WARNING: 1 of 2 listed files could not be read

If you do not see the output shown above, you should download both files and try again. If the verification fails twice, it is suggested that you contact the site owner, in this case see Blaisorblade's contacts and links page.

You can safely ignore the broken-out files because these contain the same content as the single patch file you just downloaded, but as individual patches for each fix or feature.

Now install the kernel sources to patch against. This article will use gentoo-sources as an example as patching against these has been successfully done, but you may prefer to choose the vanilla-sources.

Emerge the chosen sources with, for example: emerge =gentoo-sources-2.6.19*

You may find that for very recent kernel versions, the sources are still masked as testing (AKA keyworded). You have two choices here:

1. Add the sources package to /etc/portage/package.keywords and use the latest sources. Do note that this route may require you to keyword some other packages, usually kernel modules (and their related packages) such as alsa.
2. Go back and download the patch for an earlier kernel version.

Now that the selected kernel sources package is installed, copy it to a new directory before patching. This provides a fallback incase patching fails. It also avoids overwriting any kernel configuration in case the selected sources were already used. Note that this may take a couple of minutes. For example: cd /usr/src && cp -r linux-gentoo-2.6.19-r3 linux-gentoo-2.6.19-r3-skas-v9-pre9

Switch to the new kernel sources directory with, for example: cd linux-gentoo-2.6.19-r3-skas-v9-pre9

In case the kernel sources were used for a previous kernel, clean them up with: make distclean

Apply the patch to the kernel sources with, for example: bzcat /home/example/uml/files/skas-2.6-2.6.19-v9-pre9.patch.bz2

Check the output of patch carefully to make sure that everything was patched correctly.

Before actually configuring and compiling this kernel, create the /usr/src/linux symlink with, for example: cd /usr/src && rm linux && ln -s linux-gentoo-2.6.19-r3-skas-v9-pre9 linux

[edit] Configuring the host kernel

Now load your previous kernel configuration with: cd linux && zcat /proc/config.gz > .config

If this fails, the currently running kernel may not have /proc/config.gz support. In this case you'll want to copy the .config file from a previous sources directory.

Now configure your kernel as normal by running: make menuconfig

There are a few features you'll want to make sure you have support for. These are explained in the following sub-sections.

[edit] Loopback device support

Loopback device support allows you to mount a file containing a kernel supported filesystem as if it were a block device (such as a hard disk partition). If you're going to store your UML filesystem as a single file on disk (as this article does), you'll want to include this feature so you can mount that file.

 Device drivers --->
   Block devices --->
     <*> Loopback device support

[edit] Filesystems

You'll want to include support for the filesystems you're going to use on your guest.

[edit] Networking: Tuntap support

Probably the easiest method of reliably setting up UML guests with networking is the tuntap driver.

  -> Device Drivers
     -> Network device support
        <*> Universal TUN/TAP device driver support

Now you can setup bridging as described in Bridging .

modules=( "iproute2" )

tuntap_tap0="tun"

config_eth0=( "null" )
config_tap0=( "null" )

config_umlbridge=(
        "192.168.1.17/24"
)

routes_umlbridge=(
        "default via 192.168.1.1"
)

bridge_umlbridge="eth0"
# dynamically add them when the interface comes up
bridge_add_tap0="umlbridge"
 

[edit] Networking: IPTables support

If you intend to set up the UML guest as if it were a computer on your network, you'll need to configure IPTables support. You can find help on IPTables in the HOWTO Iptables for newbies article.

For the purposes of this article, you'll need atleast masquerade target support enabled.

 IP: Netfilter Configuration -->
   IP tables support -->
     Full NAT -->
       <*> MASQUERADE target support

[edit] Other Suggested Features

While not covered by this article, it is also suggested you include support for NFS, particularly if your guest is also running Gentoo, as this will allow you to easily share things between your guest and your host.

Related article: HOWTO Using a shared portage via NFS

[edit] Compiling the Host Kernel

Compile and install the host kernel as normal. It is suggested you reboot to the new kernel now to make sure you have all the features required to complete the rest of this article (loopback support in particular).

[edit] Changes to /etc/fstab

In the host, the 'noexec' option must be removed from the shm entry in /etc/fstab.

shm       /dev/shm        tmpfs       nodev,nosuid            0 0
#shm       /dev/shm        tmpfs       nodev,nosuid,noexec     0 0

[edit] Guest Kernel

Unlike for the host kernel, you'll need to install a patched kernel for the guest kernel. However, Gentoo being Gentoo provides us with a handy extra option, usermode-sources, a pre-patched kernel. The only downside is that usermode-sources can be quite old, so instructions for the other options are also provided.

And because compiling a UML Guest kernel isn't the same as compiling a normal / host kernel, you'll still need to follow the instructions for configuring and compiling the guest kernel below.

[edit] Downloading a pre-compiled guest kernel

Where UML isn't being used in a production environment, security and optimisation isn't a concern. In these cases it can be far easier to use a pre-compiled guest kernel. These kernels often include all the features you'll need with the added benefit that they're known to be working.

Because there are far less options for guest kernel configuration than for the host kernel, there are a number of sites such as stearns.org offering pre-compiled kernels.

Whereever you download your kernel from, it's suggested you put the downloaded files into the uml/files directory and then extract or copy the kernel to uml/bin.

If you choose to use a pre-compiled guest kernel, you can skip the rest of the guest kernel section of this article.

[edit] Using usermode-sources from Portage

While older, these kernels are again general tried and tested, but with the added benefit that they can be specifically configured.

The usermode-sources are generally only available in testing packages, so add the package to package.keywords: echo "sys-kernel/usermode-sources" >> /etc/portage/package.keywords

Now install the usermode-sources package: emerge -avt usermode-sources

You can now go to the kernel configuration section.

[edit] Patching a guest kernel

This route is really only necessary when wanting to apply other patches to the guest kernel or using UML in a production environment where security, performance and configuration are a concern.

As with the host kernel, you've got the same choice about which kernel to use. In this case it's suggested you go for the exact version of the vanilla-sources package mentioned by the patch file, as this will definitely work.

On BlaisorBlade's patches site (or mirror), go into the guest directory and find the directory for the latest version. This article will use uml-2.6.18.1-bb2 as an example.

Download the .patch.bz2 and .sha1 files to uml/files as with the host kernel and verify them with, for example: sha1sum -c uml-2.6.18.1-bb2.sha1

As with the host kernel, install the relevent sources file, copy and patch the kernel, ensuring the patch applies without any failures:

emerge =vanilla-sources-2.6.18.1
cd /usr/src
cp -r linux-2.6.18.1 linux-2.16.18-1-bb2
cd linux-2.6.18.1-bb2
make distclean
bzcat /home/example/uml/files/uml-2.6.18.1-bb2.patch.bz2 | patch -p 1

[edit] Configuring the guest kernel

Unless you're running on a 64-bit architecture, to configure the guest kernel, run the following command: make menuconfig ARCH=um

If you're running on a 64-bit architecture, because UML currently has no support for 64-bit, it'll need to be forced into a 32-bit mode by running: make menuconfig ARCH=um SUBARCH=i386

Most of the UML specific entries are in UML-specific options, however for first time users it is suggested to leave these settings at their default values.

Except for the options mentioned below, set up the guest kernel as desired, making sure to remember to add support for all the features the guest will use, in particular the filesystems options.

[edit] Loadable Module Support

Unless you have a specific need for this, it is suggested you do not use modules. The easiest way to do this is to disable loadable module support completely.

 -> Loadable module support
    [ ] Enable loadable module support

[edit] Block Devices

UML uses its own special type of block devices. You'll want to make sure these are enabled.

  -> Block devices
     [*] Virtual Block Device
     [*]   Always do synchronous disk IO for UBD

[edit] Character Devices

Character devices are used for spawning consoles on. UML has a range of options for this. You can either enable all of the options so you can use them whenever you want, or read the help descriptions for each and select only the ones you'll need. If you're not sure, select all of them.

  -> Character Devices
    [*] stderr console
    [*] Virtual serial line
    [*] null channel support
    [*] port channel support
    [*] pty channel support
    [*] tty channel support
    [*] xterm channel support
    (fd:0,fd:1) Default main console channel initialization
    (xterm) Default console channel initialization
    (pty) Default serial line channel initialization
    [*] Unix98 PTY support
    [*] Legacy (BSD) PTY support
    (256) Maximum number of legacy PTY in use
    [ ] Watchdog Timer Support
    [*] Sound support
    [*] Hardware random number generator
    [ ] iomem emulation driver 

[edit] UML Network Devices

UML guests support a range of networking options. The method this article uses is TUN/TAP, so make sure that's selected along with Ethertap support. The other methods can be left unselected.

You will also need to select the Universal TUN/TAP device driver under the Network device support options.

  -> UML Network Devices
     [*] Virtual network device
     [*]   Ethertap transport
     [*]   TUN/TAP transport
     [ ]   SLIP transport
     [ ]   Daemon transport
     [ ]   Multicast transport
     [ ]   pcap transport
     [ ]   SLiRP transport
  -> Network device support
     [*] Universal TUN/TAP device driver support

[edit] Compiling the Guest Kernel

32-bit users will need to compile the kernel by running: make linux ARCH=um

64-bit users will need to compile the kernel in 32-bit mode by running: make linux ARCH=um SUBARCH=i386

Now copy the kernel to the uml/bin/ directory with, for example: cp linux /home/example/uml/bin/linux-2.6.18.1-b2

[edit] Creating a Gentoo Guest

[edit] Preparing the filesystems

This example will create a guest system called 'egguest', stored in the uml/system/egguest directory.

Create the guest directory and change to it:

 mkdir -p /home/example/uml/systems/egguest
 cd /home/example/uml/systems/egguest

Create a 512MB swap file called swapfs and format it with the swap filesystem:

 dd if=/dev/zero of=swapfs bs=1M count=1 seek=512
 mkswap swapfs

Create an 8GB filesystem formatted to ext3. This will be the guests root filesystem. The -F flag on mkfs tells it to that it is going to format a file based filesystem instead of a partition based one.

 dd if=/dev/zero of=rootfs bs=1M count=1 seek=8192
 mkfs.ext3 -F rootfs

Note: I had issues with using the filename 'rootfs'. I think something got confused that it was my root filesystem. Change it to basefs and it was fine. (amd64, reiserfs, 2.6.22-gentoo-r5) coldflame

Mount the root filesystem using the loopback option with: mount -o loop rootfs /mnt/gentoo

[edit] Installing the Gentoo guest system

To install the Gentoo guest, follow the Gentoo Handbook install method starting with 5. Installing the Gentoo Installation Files

Skip the section on Setting the Date/Time Right - because the guest runs on top of the host it will already be using the host clock.

Except for 7.a. Timezone, skip all of 7. Configuring the Kernel. The guest system uses the guest kernel compiled earlier.

Skip the first part of 8. Configuring your system on configuring /etc/fstab. This is shown below because the device setup differs from a normal system.

Stop at the end of 9. Installing Necessary System Tools, skipping 10. Configuring the Bootloader because UML guests don't need a bootloader.

Sometimes problems may be encountered using a 64-bit host kernel. This can be solved by disabling NPTL on the guest system, by setting USE="-nptl" when compiling glibc in the guest. Since NPTL is not optional in glibc-2.7, only glibc-2.5 or earlier can be used by the guest if this problem occurs.

Don't exit the chroot or unmount the guest filesystem yet. There's a few more things to do.

[edit] Removing unnecessary / troublesome scripts

The guest UML system does not use custom tuned keymaps or console fonts and the clock on the host system is directly connected to the clock on the client system. Additionally, the system created by this article does not use loadable modules. Thus these scripts can be disabled to speed up the UML guest and avoid problems:

 rc-update del keymaps
 rc-update del consolefont
 rc-update del clock
 rc-update del modules

[edit] Configure /etc/fstab

The guest UML system does not have a /boot partition or a cdrom drive, so these entries are removed from the /etc/fstab file.

Because the guest system is using UBD based block devices instead of the normal hard disk or scsi(-like) disk based devices, the root and swap filesystems are referred to by ubd* instead of hd* or sd*. The devices start at 0 and increment by 1 for each device.

Set up your /etc/fstab as below. The article will later show how to specify which filesystem appears as which device when starting the UML system.

  /dev/ubd/0  /         ext3      noatime  0 0
  /dev/ubd/1  none      swap      sw       0 0
  none        /proc     proc      defaults 0 0
  none        /dev/shm  tmpfs     defaults 0 0

[edit] Setting up the consoles

To simplify things, cut the guest system down to a single console (instead of the normal 6). To do this, open /etc/inittab and find the section that starts something like:

# TERMINALS
c1:12345:respawn:/sbin/agetty 38400 tty1 linux

Comment out all but the first entry in this section:

# TERMINALS
c1:12345:respawn:/sbin/agetty 38400 tty1 linux
#c2:12345:respawn:/sbin/agetty 38400 tty2 linux
#c3:12345:respawn:/sbin/agetty 38400 tty3 linux
#c4:12345:respawn:/sbin/agetty 38400 tty4 linux
#c5:12345:respawn:/sbin/agetty 38400 tty5 linux
#c6:12345:respawn:/sbin/agetty 38400 tty6 linux

(I could only get it working by changing tty1 to tty0)

[edit] Network Configuration

Since the setup from this article does not involved a DHCP server, the guests networking will need to be configured. Configure /etc/conf.d/net as below.

config_eth0=( "192.168.0.1 netmask 255.255.255.0" )
routes_eth0=( "default gw 192.168.0.254" )

[edit] Finishing the installation

The installation of the guest system is now complete. Exit the chroot and unmount the guests' root filesystem:

exit
umount /mnt/gentoo/dev /mnt/gentoo/proc /mnt/gentoo

[edit] Setting up the Host system

[edit] iptables

As well as the in-kernel options, the userland tools to configure iptables will be needed. Install them with: emerge iptables

If iptables needs to be started every time the system boots, enable the init script for it with: rc-update add iptables default

Start iptables running now with: /etc/init.d/iptables start

To add NAT masquerade support to the iptables rules, run: iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

Enable IPv4 IP forwarding in the kernel options: echo 1 > /proc/sys/net/ipv4/ip_forward

To have IPv4 forwarding enabled after a reboot, edit /etc/sysctl.conf and set: net.ipv4.ip_forward = 1

[edit] Setting tunnel device permissions

The following sets permissions on the tun device, allowing the guest UML's write access to this device file to change their own IP addresses. This is not required if everything is started as root.

Add this line to /etc/udev/rules.d/40-custom.rules (create it if it doesn't exist):

 KERNEL="tun", NAME="net/%k", GROUP="uml", MODE="0660"

Create the "uml" group with: groupadd uml

The user that the guest is started by will need to be added to the uml group with, for example: usermod -a -G uml example

If the user is logged in, they will need to log out and log back in for the group change to take effect.

[edit] Guest kernel symlink

To make life easy (and commands short), create a symlink to the guest kernel called uml: cd /home/example/uml/bin && ln -s linux-2.6.18.1-bb2 uml

[edit] Install usermode-utilities

Install usermode-utilities: emerge usermode-utilities

[edit] Starting the UML guest

The following command will start the guest UML with 512MB of memory allocated to it. The TUN/TAP device connected to the guests eth0 will have an IP on 192.168.0.254 on the host system. Once started, tty1 of the guest will be attached to the standard input / output on the host (ie. it will appear on the console where the guest is launched).

  cd /home/example/uml/system/egguest/
  /uml/bin/uml mem=512M ubd0=rootfs ubd1=swapfs eth0=tuntap,,,192.168.0.254 umid=egguest con=null con0=null,fd:1 con1=fd:0,fd:1

Quick solution is to look on /etc/udev/rules.d/70-persistent-net.rules. There should be something like:

# net device (uml-netdev)<br/ >
SUBSYSTEM=="net", DRIVERS=="?*", ATTRS{address}=="'''e2:e8:b6:4a:29:1e'''", NAME="eth0"

Use ATTRS{address} value for modyfing uml starting parameter:

eth0=tuntap,tap0,'''e2:e8:b6:4a:29:1e''',192.168.0.254

[edit] Stopping the UML guest

You should normally shutdown your guest system properly (ie. using normal shutdown commands such as shutdown -h now. However your guest may become unresponsive, in which case you need to use the UML management console to resolve the problem.

Start uml_mconsole with, for example: uml_mconsole egguest

Once in the management utility, run halt to force the guest to halt.

To exit the management utility, run: quit

[edit] Notes on UML

[edit] Time Keeping

Because UML runs on top of your existing system, as long as the host clock is up-to-date, the guest clock will be also. This means there's no need for using techniques like NTP for timekeeping.

[edit] Accessing files on the host

There are 2 options for this. The following will discuss mounting folders using hostfs.

First, enable hostfs support on the guest kernel:

    [x] Host filesystem

Next, once the kernel is compiled and run,

mount none /host -t hostfs

Which will mount the host's root folder / on /host on the guest. A specific folder can also be specified:

mount none /host -t hostfs -o /path/to/hostfolder

Files will be created, modified or deleted according to the privileges of the user who spawned the UML instance.

[edit] Resources

[edit] Related Articles

• UML
• HOWTO setup DHCP
• HOWTO Using a shared portage via NFS - Currently also includes general NFS setup information.

[edit] Patches

• BlaisorBlade's site
  • Mirror of BlaisorBlade's site

[edit] Pre-compiled kernels

• stearns.org
• UML Wiki: Precompiled Kernels
• Nagafix.co.uk UML Resources