Talk:TIP Speeding up portage
From Gentoo Linux Wiki
'== Authors Notes ==' I did this on my server ( http://gentoo-portage.com ) because I knew it would be emerge syncing every 6 hours, and it would need to access the filesystem in this area alot, i was plesently suprised when it actually made emerge work faster. Thrasher 18:51, 10 Aug 2004 (GMT)
Contents |
[edit] my stats for the speed-up
usr # cd portage portage # time du -sh . 490M . real 0m27.310s user 0m0.284s sys 0m2.424s portage # time find . > /dev/null real 0m2.720s user 0m0.152s sys 0m0.740s portage # cd ../portage.old/ portage.old # time du -sh . 577M . real 0m32.684s user 0m0.276s sys 0m2.492s portage.old # time find . > /dev/null real 0m7.200s user 0m0.156s sys 0m0.984s
5 seconds faster :)
[edit] Oh yeah, here are MY stats
This is on a 300 MHz dell with 192 MB of Ram (router box/webserver)
[edit] du -sh .
- portage.old
527M . real 2m23.236s user 0m2.241s sys 0m12.845s
- portage
238M . real 0m52.866s user 0m2.113s sys 0m7.061s
[edit] time find . > /dev/null
- portage.old
real 1m40.341s user 0m1.555s sys 0m6.643s
- portage
real 1m26.705s user 0m1.373s sys 0m4.077s
Not really sure why the size changed so much, I did this directly after the copy...it kinda makes the nearly 1.5 minute improvement seem kinda questionable.
Troycurtisjr 11:18, 10 August 2006 (UTC)
[edit] Portage got bigger
Maybe it's time to update the size of portage... I just synced and portage seems to be about 550mb and that's without any kde-* packages. I can imagine that maybe it's time to update the sizes to something bigger.
- That's just the space used on your disk. Remember that a filesystem block is fully occupied even if it holds just one byte of data and since the most common blocksize is 4Kb and a lot of files inside /usr/portage are just a few bytes long - you get this discrepancy betwen file size and disk usage. In conclusion, the portage size is about 215 Mb now, regardless what packages you have installed (you can only make it smaller by rsync exclusion).
Stefantalpalaru 18:32, 6 April 2006 (GMT)
[edit] And compress the tree
It is possible to use squashfs and unionfs to do that trick + compress the tree, please can someone integrate the following in the wiki, that is alot of edition to remain consistent and I do not want to do it. Check the forums here http://forums.gentoo.org/viewtopic-p-3292868.html#3292868 as I find my solution using tmpfs, squashfs, unionfs and a simple initscript that just mounts the stuff and either make a new image if emerge --sync has been issued or just unmount the fluff to be the most elegant. Saves time, place and sweat! Synss 21:19, 2 May 2006 (UTC)
[edit] Sparse file...
I'm afraid that using sparse file for the loop will result in this file being fragmented, thus slowing down, not?
[edit] Experiences with ReiserFS on ReiserFS
I used a ReiserFS image on a ReiserFS partition. The filesystem got corrupted for several times, even leading to system reboots or halts because of kernel panic. reiserfsck --rebuild-tree could repair the filesystem in one case, but in other cases it could not. --Langec 08:02, 1 July 2006 (UTC)
[edit] noatime
Should this use the noatime parameter? On my box, this prevents me from rsyncing.
- I'm using the noatime,notail opts, and everything seems to be working perfectly---I've emerge --synced without problems. Are you using a partition or a sparse file? Jebler 11:38, 21 November 2006 (UTC)
[edit] 3:10 Hum...
Old
pikachu portage.old # time du -sh . 699M . real 3m31.455s user 0m1.209s sys 0m4.717s pikachu portage.old # time find . > /dev/null real 3m32.660s user 0m0.705s sys 0m3.476s
New
pikachu portage # time du -sh . 604M . real 0m21.641s user 0m1.173s sys 0m5.340s pikachu portage # time find . > /dev/null real 0m18.735s user 0m0.737s sys 0m3.817s
I'm on an old-ish dell laptop. The older slower hdd might have something to do with the 3 minutes 10 seconds difference. (; Also, I'm using a reiserfs partition, not a sparse file.
Jebler 11:32, 21 November 2006 (UTC)
I got significant speedups for /var/cache/edb too, which can definitely be felt during 'Updating portage cache'.
matrix edb # time du -hs . 35M . real 0m0.881s user 0m0.016s sys 0m0.140s
matrix edb.old # time du -hs . 102M . real 0m16.812s user 0m0.052s sys 0m0.644s
--Cybersol 08:26, 21 September 2007 (UTC)
