HOWTO Install VMware Tools in Gentoo

From Gentoo Linux Wiki

Jump to: navigation, search
This article is part of the HOWTO series.
Installation • Kernel & Hardware • Networks • Portage • Software • System • X Server • Gaming • Non-x86 • Emulators • Misc

Contents

[edit] Introduction

VMWare Tools is noticeably not made for Gentoo, nor is it made for X.org 7.0 (modular). As such, you will need to perform a number of modifications in order to allow it to work. This document is to serve as a place to provide information on how to install VMware Tools in Gentoo in various versions of VMware products.

If you are interested in installing Gentoo in VMware, consult Install Gentoo on VMware article.

[edit] Configuring Your X.org Window System

Emerge x11-base/xorg-x11 and it will install the vmware video driver by default. Additionally, emerge x11-drivers/xf86-input-vmmouse in order to provide smooth mouse input between your virtual machine.

Run a quick Xorg -configure as root to generate a temporary X11 server config file in /root/xorg.conf.new. Then, modify the configuration file as follows:

  1. Change the line under Section InputDevice that says Option "Device" "/dev/mouse" to Option "Device" "/dev/input/mouse0".
  2. Change Driver "mouse" to Driver "vmmouse"

Run X -config /root/xorg.conf.new to check to make sure the configuration file works without problems. If you get a window that displays a cursor X, you are good to go. Copy the configuration file to its proper location with the following command: 

# cp /root/xorg.conf.new /etc/X11/xorg.conf
Note: If you have the latest sys-apps/portage (version 2.1 or later), you can set the following variables in /etc/make.conf to emerge the appropriate packages rather than emerging xorg-x11 and xf86-input-vmmouse packages seperately. 
INPUT_DEVICES="keyboard vmmouse"
VIDEO_CARDS="vmware"
As a precaution, you can also add fbdev and vesa to the VIDEO_CARDS variable as fallback video drivers.

[edit] Getting the VMware Tools

Now that X works, you are going to install app-emulation/vmware-workstation-tools for increased performance.

Note: VMWare has created an open-source version of their tools which have been added to portage (currently masked for x86 and amd64) under app-emulation/open-vm-tools. For details, see HOWTO_Install_Gentoo_on_VMware

[edit] Getting the CD-ROM that has VMware Tools

[edit] From within VMware Workstation

Click the VM menu and select Install VMware tools. This will insert the necessary CD-ROM for you to install.

[edit] Extraction from VMware Workstation Installation Files

First, download the latest "Archived Version" of VMware Workstation in .tar.gz format from VMware's VMware Workstation download page. You do not need to be registered nor have a VMware Workstation license key to download this version. 

# wget http://download3.vmware.com/software/wkst/VMware-workstation-5.5.0-18463.tar.gz

Next, locate and extract the appropriate VMware Tools image from the tarball. The below example shows several ways of how to get the Windows ISO file. 

# tar ztvf VMware-workstation-5.5.0-18463.tar.gz | grep windows.iso
vmware-distrib/lib/isoimages/windows.iso

# tar zxvf VMware-workstation-5.5.0-18463.tar.gz vmware-distrib/lib/isoimages/windows.iso
Note: You can extract an ISO file without creating the vmware-distrib/lib/isoimages directory structure by using app-arch/pax: 
# pax -zrvf VMware-workstation-5.5.0-18463.tar.gz -s,vmware-distrib/lib/isoimages/,, vmware-distrib/lib/isoimages/windows.iso

[edit] Mounting VMware Tools CD-ROM

VMware Tools is installed via a virtual CD-ROM that is packaged within the program. In order to install it, you will need to mount the CD-ROM in Gentoo.

In a console, you will need to mount the CD-ROM as follows so that you can access it in Gentoo. 

# mount -t iso9660 -o loop /dev/cdrom /mnt/cdrom
Note: If you are unable to mount the CD-ROM, you may have not configured your CD-ROM drive in the kernel correctly. Thus, you will have to recompile the kernel with relevant drivers enabled.

[edit] Actually Installing VMware Tools

You can install VMware Tools using portage or you can do it manually.

[edit] Using Portage

From here, as root, type emerge vmware-workstation-tools and wait a bit while it installs in every location.

Afterwards, you can type etc-update to merge VMWare's xorg.conf. Make sure you do not merge and disregard the updated file because it has things that will cause X to fail.

Note: If emerging vmware-workstation-tools fails with a "failed patch" error, the following guide may be helpful.

[edit] Manual Steps

[edit] Unpacking Source Files

You will find vmware-linux-tools.tar.gz under /mnt/cdrom. Please unpack that file into a working directory, say /tmp.

Code: unpack VMware Tools 
# cd /tmp
# tar zxvf /mnt/cdrom/vmware-linux-tools.tar.gz

A new directory named vmware-tools-distrib should be created in /tmp. It contains all the VMware Tools installation files.

[USER NOTE: The following tar command fails but only because the name of the file is improper; in our ESX rig the name of the file is currently VMwareTools-3.5.0-64607.tar.gz--69.38.125.241 16:51, 13 February 2008 (UTC)]

[edit] Patching Installation Script

Because this VMWare Tools was not designed with Gentoo Linux in mind, you won't get a clean installation without some preparation works.

You will need to retrieve a patch file that modifies vmware-uninstall-tools.pl and vmware-config-tools.pl scripts. The patch below was created against the VMware Tools 3.1.0 build-9089 that comes with VMware GSX Server 3.1.0 build-9089.

Code: retrieve VMware Tools patch 
# cd /tmp/vmware-tools-distrib
# wget http://www.exonsoft.com/~kochin/vmware-tools-distrib.patch
Warning: Changes were made to let those scripts recognize kernel 2.6.x, Gentoo Linux and Xorg. I took great care not to change the functionalities of the original scripts. They should work smoothly on your virtual Gentoo Linux and maybe other Linux distributions as well. However, you are using this patch at your own risk.

Before you apply the patch, you might want to see what the patch will do.

Code: test the patch 
# patch -Np1 --dry-run --verbose < vmware-tools-distrib.patch

[USER NOTE: The following dry run of the patch has several hunks fail on our ESX rig - #6-8 and #10-14 for bin/vmware-config-tools.pl and #5 and #10 for bin/vmware-uninstall-tools.pl--69.38.125.241 16:51, 13 February 2008 (UTC)]

If no problems shown in the output, you can apply the patch for real.

Code: apply the patch 
# patch -Np1 --verbose < vmware-tools-distrib.patch
# chmod +x installer/services-gentoo.sh

The services-gentoo.sh will be copied into /etc/init.d by the installation script and used as the initscript for VMware Tools service. It needs the execution permission.

[edit] Running the Installation Script

With the patch in place, it is now a simple matter of running the installation script.

Warning: Every time you want to run the VMware Tools configuration script, you have to make sure the DISPLAY variable is not set. To make matters worse, X Window on Gentoo does not remove the environment variable DISPLAY when it terminates. The VMware Tools configuration script uses that variable to determine whether it is running under X Window. The script will refuse to run if it sees DISPLAY is set. To unset the variable, execute the following command: 
# unset DISPLAY

If you have been following the steps, you should be in /tmp/vmware-tools-distrib folder to run the script.

Code: run installation script 
# ./vmware-install.pl

The script should automatically recognize your system configurations and present you with proper defaults. Review the defaults prompted on screen, and press Enter key to accept defaults. The only place really requires your input is to set your X Window screen resolution if X Window System is installed.

Note: You may encounter the error, $all_found requires explicit package name. To get around this, add the line my $all_found = 0; near the other variable declarations in the vmware-install.pl perl script. You could also remove the line that says use strict;.
Note: If you are running on VMware Workstation 5.0 and you have an error of (EE)No Input driver matching "Keyboard", please modify VMware Tools file vmare-tools-distlib/lib/configurator/XFree86-4/XF86Config-4 or /usr/lib/vmware-tools/configurator/XFree86-4/XF86Config-4 so that the keyboard InputDevice section shows the following:
File: vmware-tools-distlib/lib/configurator/XFree86-4/XF86Config-4 
Section InputDevice
#  Driver    "Keyboard"
  Driver     "keyboard"
As you can see, the error is just from the capitalization of the K in "Keyboard".

There is a long pause when the script is executing modules-update. Please wait patiently until it finishes.

[edit] Setting up Resolution Change

I have enbled the "Quick Switch" option, so that I see all my VMWare machines in the top of my screen and simply switch to another by pressing the mouse button. A nice feature of VMWare is that you can select the "Fit Guest Now" option in the "View" menu. VMWare changes your resolution of your guest machine so that it looks like full screen, but you don't have slidebars at the botton and right of the screen. This works very nice with windows machines, but my first conclusion with Gentoo was that the option is disabled. You can turn on the "Autofit Guest" option, but that doesn't seems to do anything at all.
The next steps will provide the information for changing the resolution correctly if your host machine is at 1024x768.

  • edit /etc/X11/xorg.conf and go to the monitor Section.
File: /etc/X11/xorg.conf 
Section "Monitor"
  ...
  ModeLine "1024x768" 100 1024 1100 1200 1300 768 800 900 1000
  ...
EndSection
  • You see something like this. Replace that line with the following:
ModeLine "1023x740" 100 1016 1100 1200 1300 740 800 900 1000
  • Then go to the Screen Section and replace all 1024x768 with 1023x740.
  • Save the file and restart X. Now you have a nice filled up window without slicebars.

[edit] Verifying Installation

  • During the installation and configuration, a module named vmhgfs was created and loaded. You can verify it with the lsmod command.
Code: verify vmhgfs module 
# lsmod
Module                Size  Used by
vmhgfs               38838  0
  • The sure sign of VMware Tools running on your system is vmware-guestd shown in the process list. For example,
Code: verify VMware guest service
# ps ax
  • If you have X Window System (Xorg or XFree86) installed, try to start X Window.
Code: start X Window System 
# startx

Once the X Window System is up, try moving the mouse pointer out and in your virtual machine. You should be able to move the pointer around without having to press Ctrl-Alt to leave virtual machine.

Next, open a terminal window and run VMware Tools' GUI program vmware-toolbox.

Code: start VMware Tools GUI 
# vmware-toolbox &

You can view and change your virtual machine settings with it.

In case of receiving a "No Input driver matching `Keyboard'" error when starting X Window, please modify your X Window configuration file (/etc/X11/xorg.conf or /etc/X11/XF86Config) so that the keyboard InputDevice section shows correct keyboard InputDevice section.

File: /etc/X11/xorg.conf or /etc/X11/XF86Config 
Section "InputDevice"
...
        Driver      "keyboard"
...
EndSection

Note the error is just from the capitalization of the K in "Keyboard", the section exists just needs the K changing to a k.

Under my configuration, I use the kbd driver. I believe Keyboard is deprecated with newer builds.

[edit] Uninstalling VMware Tools

The vmware-uninstall-tools.pl was patched to work on Gentoo Linux. Just run that script to uninstall VMware Tools from your virtual Gentoo Linux.

Code: uninstall VMware Tools 
# vmware-uninstall-tools.pl

Again, there will be a long pause when the script is running modules-update. Please let it finish without interruptions.

[edit] Allowing VMware-tools to configure

The latest vmware-tools-config.pl does not function well with the latest kernel sources. In order to do this, you have to download a script that will allow the config script to pick up the kernel sources by using the following commands. 

# wget http://ftp.cvut.cz/vmware/vmware-any-any-update109.tar.gz 
# tar zxvf vmware-any-any-update109.tar.gz.

There is one more problem to counter, and that is that Gentoo keeps the directory structure of vmware-tools a little oddly. To remedy this, go into the untarred directory, edit runme.pl as follows:

  1. Replace /etc/vmware with /etc/vmware-tools.
  2. Replace any occurrence of vmware-config.pl with vmware-config-tools.pl.

Execute the runme.pl file, but do not have it run the vmware-config-tools.pl script.

Note: The vmware script will fail when it tries to configure Xorg since it was not designed for version 7. You can get around the error by editing the /opt/vmware/tools/bin/vmware-config-tools.pl and commenting out the following line by adding '#' to the start of the line.
File: /opt/vmware/tools/bin/vmware-config-tools.pl 
error ('Problem extracting version of X.org' . "\n\n");
Which should be located on or near line 4159. Save the changes and now execute the vmware-config-tools.pl script. It should allow you select your desired default resolution and have the script configure Xorg for you.


Note: An error message may print out after X loads when it tries to start the vmware tools. To allow vmware to start despite the script failing, you must remove the not_configured file by typing the following into a console: 
# rm /etc/vmware_tools/not_configured

[edit] Running VMware-tools

Now, you can run VMware-tools at startup by running the following commands.

Code: start vmware-tools and make it auto start on boot 
rc-update add vmware-tools default
/etc/init.d/vmware-tools start

If vmware-tools starts successfully, start X.org with "startx" and enjoy your quick response system!

[edit] References

[edit] Installing VMware Tools in VMWare Workstation v5.5 and XFree86

The steps below have been specifically tested with the following combinations of VMware and Kernels:

  • VMware Workstation v5.5.1 and Kernel 2.6.16-r3 -- this combination used XFree86 and NOT xorg-X11
  • VMware Workstation v5.5.2-build-29772 and Kernel 2.6.18-r1
Warning: This does not use the vmware-linux-tools ebuild as it is broken for this release.

[edit] Copy over the VMwareTools install files

From the VMware Workstation console, select VM-> Install VMware Tools .

Code: Mount the drives and copy the files from the cdrom 
# mount -t iso9660 -r /dev/cdrom /mnt/cdrom
# cp /mnt/cdrom/*.tar.* /tmp
# cd /tmp
# tar -xzvf VMwareTools-5.5.1-19175.tar.gz

[edit] Install VMwareTools

  1. Setup the empty directories the install will expect
  2. Kick off the install script
    • Follow the prompts below
    • Answer NO when it asks if you want it to invoke /usr/bin/vmware-config-tools.pl
Code: Prepare the VMware-linux-tools ebuild 
# touch /etc/init.d/network
# mkdir /etc/init.d/rc{0,1,2,3,4,5,6}.d
# cd /tmp/vmware-tools-distrib && ./vmware-install.pl

If /etc/init.d/network does not exist, create a symbolic link to your eth0-interface:

Code: Create symbolic link to eth0 
# ln -s /etc/init.d/net.eth0 /etc/init.d/network

# ./vmware-install.pl                
Creating a new installer database using the tar3 format.

Installing the content of the package.

In which directory do you want to install the binary files? 
[/usr/bin] 

What is the directory that contains the init directories (rc0.d/ to rc6.d/)? 
[/etc/init.d] 

What is the directory that contains the init scripts? 
[/etc/init.d] 

In which directory do you want to install the daemon files? 
[/usr/sbin] 

In which directory do you want to install the library files? 
[/usr/lib/vmware-tools] 

The path "/usr/lib/vmware-tools" does not exist currently. This program is going
to create it, including needed parent directories. Is this what you want? 
[yes] 

In which directory do you want to install the documentation files? 
[/usr/share/doc/vmware-tools] 

The path "/usr/share/doc/vmware-tools" does not exist currently. This program is
going to create it, including needed parent directories. Is this what you want? 
[yes] 

The installation of VMware Tools 5.5.1 build-19175 for Linux completed 
successfully. You can decide to remove this software from your system at any 
time by invoking the following command: "/usr/bin/vmware-uninstall-tools.pl".

Before running VMware Tools for the first time, you need to configure it by 
invoking the following command: "/usr/bin/vmware-config-tools.pl". Do you want 
this program to invoke the command for you now? [yes] no

[edit] Prepare to execute vmware-config-tools.pl

  1. Create a symbolic link for the mouse
  2. Edit /usr/lib/vmware-tools/configurator/XFree86-4/XF86Config-4 to fix the keyboard
  3. Edit /usr/bin/vmware-config-tools.pl to comment out the return statement (in the end of the function check_answer_headerdir)
Code: Prepare for the Install Script 
# ln -sf /dev/input/mice /dev/mouse
# nano -w /usr/lib/vmware-tools/configurator/XFree86-4/XF86Config-4
     # Driver     "Keyboard"
     Driver      "keyboard"
# nano -w /usr/bin/vmware-config-tools.pl
     line 1982: #return '';

[edit] Execute the Script

  1. Execute the configuration script
Code: Execute the configuraiton Script 
#/usr/bin/vmware-config-tools.pl

[edit] Create a script to start on boot

  1. Create /etc/init.d/vmware to start VMwareTools at boot
    • /etc/init.d/vmware-linuxs scipt doesn't work with rc-update, thus this script is required to work properly.
Code: /etc/init.d/vmware 
# nano -w /etc/init.d/vmware

#!/sbin/runscript
start() {
        ebegin "Starting VMWare Subsystem"
        /etc/init.d/vmware-tools start
        eend $?
}
 
stop() {
        ebegin "Stopping VMWare Subsystem"
        /etc/init.d/vmware-tools stop
        eend $?
}

[edit] Finish up

Code: Start up VMwareTools 
# chmod 755 /etc/init.d/vmware
# /etc/init.d/vmware start
Code: Add vmware to start at boot 
# rc-update add vmware default
Retrieved from "http://www.gentoo-wiki.com/HOWTO_Install_VMware_Tools_in_Gentoo"