Ccache
From Gentoo Linux Wiki
| Terminals / Shells • Network • X Window System • Portage • System • Filesystems • Kernel • Other |
Contents |
[edit] Introduction
ccache is a compiler cache. It uses the GCC -E switch and a hash to detect when a compilation can be satisfied from cache. The effect is that packages frequently compile 5-10 times faster than they would otherwise.
The first time that you emerge a package after setting up ccache, there will be a very slight increase in its compilation time, but thereafter (when the compilation is cached), you will notice significant reductions.
[edit] Installation
Usually after you emerged ccache and added it to your make.conf (see later in this article), ccache should work. You can emerge something and check 'CCACHE_DIR="/var/tmp/ccache" ccache -s' if the stats update. If not, you can try to install ccache manually:
- Make sure your portage version is higher than 2.0.46-r11: emerge --version
- Emerge ccache: emerge -av dev-util/ccache
- Find your CHOST (see below): emerge --info | grep CHOST
- Copy the CHOST value (in double quotes) into this command: ccache-config --install-links YOUR-CHOST-HERE
| Code: Installing and configuring ccache (/bin/bash) |
# emerge --version Portage 2.1.1-r2 (default-linux/x86/2006.1, gcc-4.1.1, glib-2.4-r3, 2.6.18-gentoo-r5 i686) # emerge -av dev-util/ccache ... Would you like to merge these packages? [Yes/No] y ... # emerge --info | grep CHOST CHOST="i686-pc-linux-gnu" # ccache-config --install-links i686-pc-linux-gnu |
[edit] File Placement
When emerging ccache as root the following ccache files and folders are installed:
- /usr/lib/ccache - houses files that portage will run instead of the original compilers
- /usr/bin/ccache - the ccache executable that the files mentioned directly above link to
- /root/.ccache - stores the cache
When emerging ccache as a user, another folder is additionally created:
- /var/tmp/ccache - stores the cache
Solid information on this structure has proven difficult to come by and anyone who can provide an authorative answer should edit this page to do so.
However it seems that /var/tmp/ccache is unique to Gentoo, and should be chosen over /root/.ccache to store the cache.
mv /root/.ccache /root/snafu.ccache
ln -s /var/tmp/ccache /root/.ccache
After that everything worked. I have been using it for awhile now (including emerge world) with no problems. (codeslinger.comspalot)
[edit] Configuration
| Code: /bin/bash |
$ ls -l /usr/lib/ccache/bin ... lrwxrwxrwx 1 root root 15 Dec 27 c++ -> /usr/bin/ccache lrwxrwxrwx 1 root root 15 Dec 27 cc -> /usr/bin/ccache lrwxrwxrwx 1 root root 15 Dec 27 g++ -> /usr/bin/ccache lrwxrwxrwx 1 root root 15 Dec 27 gcc -> /usr/bin/ccache lrwxrwxrwx 1 root root 15 Dec 27 i686-pc-linux-gnu-c++ -> /usr/bin/ccache lrwxrwxrwx 1 root root 15 Dec 27 i686-pc-linux-gnu-g++ -> /usr/bin/ccache lrwxrwxrwx 1 root root 15 Dec 27 i686-pc-linux-gnu-gcc -> /usr/bin/ccache ... |
All the files are links pointing to: /usr/bin/ccache
When portage seeks to compile a package, it of course looks for the relevant compiler. However we don't want portage to go direct to the compiler, rather we want the compiler accessed through ccache. Thus we have a number of symlinks in the ccache directory which bear names that correspond to the various c and c++ compilers that portage looks for. With ccache enabled, Portage follows the path to /usr/lib/ccache/bin, and thus one of these files is invoked instead of one of the actual compilers. As you will see from the box above, all of these files are merely links to the ccache executable, (/usr/bin/ccache), which itself will call the genuine compilers.
Configuration takes place within /etc/make.conf
| File: /etc/make.conf |
... FEATURES="ccache" CCACHE_DIR="/var/tmp/ccache" CCACHE_SIZE="2G" ... |
- Most importantly we need /etc/make.conf to inform portage to look for and use ccache. This is achieved by the first of the three lines above. If you already have some features set, naturally just add "ccache" to the list.
- If you also use DistCC, be sure you add the "ccache" option first and "distcc" second in your features list to take full advantage of both.
- The CCACHE_DIR environment variable specifies where ccache will keep its cached compiler output.
- The 3rd line sets the maximum space that is allocated to caching compilations. At the time of writing the default maximum size is 976.6M. If you can afford the space you may well want to allow for more as in the example.
- The ccache stats won't update to show your chosen CACHE_SIZE until after the emerge that follows your change in specification.
- You can also set the cache size by employing the -M option described on ccache's man page. However, if you have a CCACHE_SIZE value set in /etc/make.conf, it will overwite previously set values from as soon as you next emerge.
[edit] Execution
Now, for the purpose of testing, emerge something small...
To ensure that ccache was running, pull up the ccache stats afterwards with: ccache -s
If you're using the CCACHE_DIR in /var/tmp/ccache you need to add that variable to the command string as shown below.
| Code: /bin/bash |
CCACHE_DIR="/var/tmp/ccache" ccache -s /* The output will look something like this if it's working */ cache hit 189 cache miss 1983 called for link 159 multiple source files 4 compile failed 52 preprocessor error 4 bad compiler arguments 3 not a C/C++ file 141 autoconf compile/link 357 unsupported compiler option 30 no input file 145 files in cache 3966 cache size 30.1 Mbytes max cache size 2.0 Gbytes |
If no space shows as used, your cache directory is still empty and it might be the case that ccache (when run direct rather than by Portage) is looking for the cache store in the wrong place. You can overcome this by declaring the location of the cache store system-wide. Update your profile to include the second line that we added to /etc/make.conf as follows:
| Code: /bin/bash |
echo "CCACHE_DIR=\"/var/tmp/ccache\"" >> /etc/env.d/99local env-update && source /etc/profile |
[edit] Troubleshooting
[edit] ccache and Genkernel
Genkernel doesn't use ccache by default, even when you've enabled it for root. You can easily fix this by appending the --kernel-cc option to your genkernel call:
| Code: /bin/bash |
$ genkernel --kernel-cc=/usr/lib/ccache/bin/gcc --menuconfig all |
You can add following alias (see TIP_alias) to your profiles, and later use genkernel as usually:
| Code: /bin/bash |
$ alias genkernel='genkernel --kernel-cc=/usr/lib/ccache/bin/gcc' |
[edit] ccache and non-portage compiling
(This section has been updated in the documentation. Old method of editing /etc/env.d/ does not work anymore, probably never has: you can rm /etc/env.d/02ccache)
To ensure all compilation uses ccache, the directory /usr/lib/ccache/bin has to precede /usr/bin in your PATH/ROOTPATH environment variable. This has to be done after /etc/profile.env gets called, so changing anything in /etc/env.d will not work.
Note that this does not affect genkernel; to force genkernel to use ccache it is still necessary to specify the --kernel-cc option to your genkernel call as above.
[edit] Method A
You can accomplish this for a given user by adding this line to his .bash_profile or .bashrc:
PATH="/usr/lib/ccache/bin:${PATH}"
or for all users including root by appending to the end of your /etc/profile:
PATH="/usr/lib/ccache/bin:${PATH}"
ROOTPATH="/usr/lib/ccache/bin:${PATH}"
As always after updating environment variables, run env-update if you changed /etc/env.d entries to refresh your /etc/profile.env, source /etc/profile to apply the changes to your current shell.
[edit] Method B
You can put a file called ccache.sh within /etc/profile.d, the content of the file should be
if [ -d /usr/lib/ccache/bin ] ; then
PATH="/usr/lib/ccache/bin:${PATH}"
export PATH
fi
because there is a line at the end of /etc/profile,
for sh in /etc/profile.d/*.sh ; do
if [ -f "$sh" ] ; then
. "$sh"
fi
done
unset sh
so this should do the trick.
[edit] ccache adjusts directory permissions
Apparently ccache will sometimes try to set the permissions of the ccache_dir after and before each emerge. Type this to fix that:
| Code: /bin/bash |
# chmod g+X -R /usr/lib/ccache |
Maybe you can automate this by setting the variable CCACHE_UMASK to 777 (be carefull!).
[edit] ccache & userpriv
With earlier versions of portage, when the userpriv FEATURE is enabled, ccache is only called during the configure stage, but not during the actual make process. This bug was fixed as of sys-apps/portage-2.0.53_r3
[edit] Links
From Gentoo Linux Documentation:
Others:
