Talk:TIP Speeding up portage with tmpfs
From Gentoo Linux Wiki
additional note: CTRL+C will kill emerge, further on temerge stopps and umounts tmpfs! please sum up your results here!
Contents |
[edit] Speedup
What this would speedup is the extraction of the tarball, reading the source off the tmpfs and moving the compiled data to it's propper location.
I don't see compilation time being effected by this.
[edit] Speedup!
well, using emerge is like.. hdd->ram->cpu->ram->hdd and on finishing hdd->hdd, or quite something like that.
using temerge kills the hdd-borders of that example. only ram->cpu->ram , later on ram->hdd. If you cannot see some afforts, go test it ;)
[edit] Speedup Not Always Possible
The reason for not seeing much speed up in one or two emerge test is that linux kernel, being smart, always uses as much ram as possible. So, to make the ram available to tmpfs, you have to throw certain apps out of the memory. This means the overhead of swapping them out or relinquishing the buffer cache.
try a 'emerge -e system' with a tmpfs of 1.5G. You should see a huge difference. If you don't want to be that drastic, do at least 5 emerges one after the other. Now compare the time for last emerged package with the time for its emerge before you went to tmpfs. keep tmpfs mounted i.e. don't use script. disable ccache for testing purposes. -devsk
[edit] Speedup?
I have tested it (before making my last comment) and found it to take the same amount of time and in some cases longer. I also requested a few guys on irc with more ram than I have to test it, and with a xorg-x11 compile, very little change was shown, and not always positive change.
A problem is you also limit the amount of ram that can be used to actually compile, since a lot of it is being used by /var/tmp/portage/
Might this be a pointless optimization if you're already using CFLAGS="-pipe" code>? Per the GCC docs, -pipe code> causes the compiler to forego using temp files in favor of pipes (thus the higher memory utilization when compiling with pipe rather than without.) If you're going to benchmark this tip, try it with and without -pipe code>, I imagine the "without pipe, with tmpfs" scenario is about equal to "with pipe, without tmpfs". Nlopez
No, there are other overheads associated with creating temporary files that are avoided with pipes. dirtyepic 00:19, 30 March 2008 (UTC)
[edit] Newbie Question
- just a newbie question:
if there is not enough ram, shouldn't linux use the swap space as backup? so after all avaible ram is used emerge uses hd space which is the default. wouldn't this be a speedup even if there is not enough ram?
- If you're falling back to disk (swap is on disk) then what's the difference from the normal method? This tip is only good if you can do everything in RAM (which is at least an order of magnitude faster than disk). -Tro
[edit] RAM Speedup
I agree, that using the RAM *should* speed up the compiling process, but i disagree with your testing methods. It would be much better to use the "time" function as a way of testing. Another test that might be good, is the Read/Write times of the tmpfs, and the users hdd.
The speed of the ram varies, and so does the hdd. Is the hdd currently in heavy use? Is your ram fast? Lots of questions that will take factor into this.
MikeyB. Ps Id love to hear ppls results of time temerge <package>; and some way of testing rw of there ram.
[edit] More Speedup
great tip; one thing i've noticed is that you're not properly squeezing out all the performance you possibly could. in your temerge script you may want to do something like: FEATURES="-ccache" emerge blah blah code> as this will prevent emerge from trying to save information which will just get wiped when you unmount anyway. nice one, mate!
jieryn
ccache doesn't lose its usefulness when using this tip. As long as CCACHE_DIR code> isn't beneath /var/tmp/portage, it makes sense to leave FEATURES="ccache" code> as-is. Nlopez
the point of this technique is to remove the disk I/O bottleneck. writing large amounts of cache data to disk while compiling kind of defeats that. unsetting PORT_LOGDIR should also (minimally) help by avoiding teeing stdout to disk. dirtyepic 00:15, 30 March 2008 (UTC)
I've just tested this out, emerging gaim. turned ccache off, ran "time emerge gaim" and "time ./cemerge gaim". I have 1.8GB of memory, of which the 850 was on tmpfs. I did some light web browsing while they were compiling, but nothing that should have really affected the results.
localhost ~ # time emerge gaim ... real 6m49.148s user 3m55.751s sys 1m43.662s
localhost ~ # time ./temerge gaim ... real 6m27.470s user 3m55.031s sys 1m40.722s </code>
No noticible change or benefit. Murph
Another case study. I have 3 GB of RAM (mounted 6 GB to /var/tmp/portage for temerge), an AMD Athlon 64 X2 3800+, and a 3-disk RAID-5 array. Using cpufreq_performance with nothing but gnome-session running:
localhost ~ # emerge -feq system >/dev/null 2>&1 \
&& time emerge -eq system >/dev/null 2>&1 \
&& time temerge -eq system >/dev/null 2>&1
real 261m55.696s
user 217m16.653s
sys 96m29.649s
real 262m1.578s
user 219m59.623s
sys 98m48.327s
I really wanted temerge to be faster (I just went to the trouble of adding bash completion to it), but it doesn't look like it is. I suppose it still might be faster if you're not using a RAID setup. BrandonMintern
[edit] !!! CATEGORY info missing from info chunk, aborting…
After using this tip I began getting this error while compiling xorg-x11-6.8.2-r1 and attempted the fix documented here without avail. I noticed that compiling xorg came very close to consuming all 850M of my portage tmpfs mount and upped it to 960, after which the emerge proceeded without a hitch. It's worth noting, though, that it never took more than 87% of the 960M mount per df -h | grep tmpfs code> -- so while it doesn't actually use more than 850M, just barely meeting this value can cause problems.
Nlopez
[edit] some answers :p =
well, finaly i read this page, hum :)
i also noticed some emerging errors while using tmpfs, no idea what they are about.. no time to look it up. same on causing longer emerging-times, its mosty while using swap.. i guess
my testingmethod is really bad, using time insteam of genlop would be nicer. feel free to submit testresults :) greetz, inbreed
[edit] some test results
So, it looks like running it in RAM does seam to help, but not xorg in under 8 minutes or anything. I wrote the following file, testrun to run the different compiles for me, and umm ignore my bash, I'm still learning and didn't feel like putting a lot of enery into it.
| File: testrun |
#!/bin/bash
TIME=`which time`
FILE=results_${1}
emerge -f ${1}
echo "compiling tests with ${1}" > $FILE;
echo "Time results for \"emerge ${*}\"" >> $FILE &&
$TIME -a -o $FILE -p -- emerge ${*} ;
echo "Time results for \"./temerge1 ${*}\"" >> $FILE &&
$TIME -a -o $FILE -p -- ./temerge1 ${*} ;
echo "Time results for \"FEATURES=\"-ccache\"./temerge1 ${*}\"" >> $FILE &&
FEATURES="-ccache" $TIME -a -o $FILE -p -- ./temerge1 ${*} ;
echo "Time results for \"./temerge2 ${*}\"" >> $FILE &&
$TIME -a -o $FILE -p -- ./temerge2 ${*} ;
echo "Time results for \"FEATURES=\"-ccache\"./temerge2 ${*}\"" >> $FILE &&
FEATURES="-ccache" $TIME -a -o $FILE -p -- ./temerge2 ${*}
more $FILE
|
Where temerge1 is the file original file provided and temerge2 just mounts the tmpfs with the extra option: nr_inodes=1M I ran the tests in command line with no X running. oh my system has 1.5GB RAM. I just wonder, would having more compiler optimizations reduce the differance between emerge and temerge? I assume with more compilter optimizations a greater % of the time is spent in the cpu cycle rather than accessing hdd/ram. Anyways, here are the results.
| File: results_mozilla-firefox |
compiling tests with mozilla-firefox Time results for "emerge mozilla-firefox" real 686.55 user 487.02 sys 404.97 Time results for "./temerge1 mozilla-firefox" Command exited with non-zero status 1 real 550.46 user 488.11 sys 398.39 Time results for "FEATURES="-ccache"./temerge1 mozilla-firefox" Command exited with non-zero status 1 real 1655.29 user 2418.64 sys 449.29 Time results for "./temerge2 mozilla-firefox" Command exited with non-zero status 1 real 552.51 user 488.06 sys 400.66 Time results for "FEATURES="-ccache"./temerge2 mozilla-firefox" Command exited with non-zero status 1 real 1631.52 user 2398.63 sys 451.98 |
--Aperion
[edit] script mod
I find it's useful to keep the tmpfs mounted in case of a emerge failure so I don't lose the whole source tree when the package is 90% compiled. Especially useful on big ebuilds.
| File: modified temerge script |
#!/bin/bash
MEMSIZE=850M
mounted=false
. /etc/init.d/functions.sh
mounttmpfs() {
mount -t tmpfs tmpfs -o size=$MEMSIZE /var/tmp/portage
mounted="true"
}
compile() {
einfo "emerging ${*}"
emerge ${*}
ret=$?
}
unmount() {
ebegin "unmounting tmpfs"
umount -f /var/tmp/portage
eend $?
}
ebegin "Mounting $MEMSIZE of memory to /var/tmp/portage"
if [ -z "$(mount | grep /var/tmp/portage)" ]
then
mounttmpfs
else
eerror "tmpfs already mounted!"
exit 0
fi
eend $?
compile ${*}
if [[ -n "$mounted" && ! $ret ]]
then
unmount
fi
|
[edit] Other Benchmarks
Maybe for the testing, rather than just an end result, use genlop to show the results of an emerge and a temerge? For example a test of QT I just did.
| Code: For QT (from a 1GB Pentuim 4 @ 2200MHz) |
# genlop -t x11-libs/qt-4.1.2
Without tmpfs: x11-libs/qt-4.1.2
merge time: 2 hours, 28 minutes and 37 seconds.
With tmpfs: x11-libs/qt-4.1.2
merge time: 34 minutes and 23 seconds. |
Taking a small package and running it through time.
emerge 3.55.540 vs 2.55.249
1.24.295 vs 1.24.272
1.29.718 vs 1.29.810
[edit] TMPFS and Block Device merged as one device?
"I don't have a huge amount of memory, (512M) but the tmpfs method works great on the packages that do fit. What I was wondering is it possible to make a linear RAID with tmpfs and a hard drive, or perhaps modify unionfs to work with tmpfs and a drive directory in which it will switch to the drive directory when tmpfs is full. - Gary"
"If you create a swap file/partition, and have your system use that, then set the tmpfs to bigger than your ramsize, but smaller than your ramsize+swap, then when your ram is full because of tmpfs, the computer will use swap space as extra ram."
-Dantarion
Thanks, I figured that out a while later. Been working like a charm for a long time. I'm trying /tmp and /var/log today as well, especially for my new laptop.
-Gary
[edit] Updated location of functions.sh
My current version of portage (2.1.5_rc4) doesn't have /sbin/functions.sh, so the script won't run properly. The new equivalent of this file is /usr/lib/portage/bin/isolated-functions.sh.
- TeknoHog
Nope. According to this post the correct file should be
/etc/init.d/functions.sh
I've corrected the script.
[edit] Top tip for us using flash drives.
Less writes = longer life.
