/usr/obj is 11GB huge on FreeBSD 12-current

[Wolfram Schneider]

Hi,

I upgraded a machine from 11-stable to 12-current. The /usr/obj tree
is now 11GB huge:

FreeBSD 12-current
$ du -hs /usr/obj
11G /usr/obj

on FreeBSD 11-stable it was less the size:
$ du -hs /usr/obj
5.6G /usr/obj

this is a problem when you have a small VM with 20GB disk space or less.

Is there a way to use less /usr/obj disk space during build? I know
that we have to do some bootstrapping for newer compiler tools, but
does we need to keep all temp files during the build?


# FreeBSD 12-current
$ du -ks * | sort -n
4 compiler-metadata.mk
4 host-osreldate.h
112 etc
216 include
9972 tests
15324 libexec
17188 bin
34964 stand
57424 rescue
65280 share
80312 sbin
118616 cddl
146792 kerberos5
175244 secure
192340 gnu
251784 usr.sbin
1269916 obj-lib32
1737908 usr.bin
1863716 sys
2528160 lib
2892776 tmp

# FreeBSD 11-stable
$ du -ks * |sort -n
4 compiler-metadata.mk
116 etc
216 include
8860 tests
14212 libexec
16260 bin
36276 rescue
63300 sbin
67224 share
85268 cddl
86868 kerberos5
107672 gnu
110360 secure
172352 lib32
222200 usr.sbin
518908 world32
668756 tmp
979040 lib
989640 usr.bin
1721096 sys

-Wolfram

[Wolfram Schneider]

Post by Wolfram Schneider
Hi,
I upgraded a machine from 11-stable to 12-current. The /usr/obj tree
FreeBSD 12-current
$ du -hs /usr/obj
11G /usr/obj
$ du -hs /usr/obj
5.6G /usr/obj
this is a problem when you have a small VM with 20GB disk space or less.
Is there a way to use less /usr/obj disk space during build? I know
that we have to do some bootstrapping for newer compiler tools, but
does we need to keep all temp files during the build?

There was a change near the beginning of November; please see UPDATING
entry 20171101 -- you probably have several no-longer-used
subdirectories under /usr/obj/usr/src/.
Once those are cleared out, my experience (tracking stable/11 & head in
different slices on the same machines) is that stbale/11 is using about
5.0G, while head uses about 6.1G.

I think the suspect directories are "tmp" and "obj-lib32", together
they are 4.1GB huge.

I will run a build of current again with a clean obj tree (-current on
a recent -current). Let's see.

Can we agree that the obj tree should not grow from 5GB to 10GB for
the next release?

-Wolfram

[Wolfram Schneider]

Post by Wolfram Schneider

Post by Wolfram Schneider
Hi,
I upgraded a machine from 11-stable to 12-current. The /usr/obj tree
FreeBSD 12-current
$ du -hs /usr/obj
11G /usr/obj
$ du -hs /usr/obj
5.6G /usr/obj
this is a problem when you have a small VM with 20GB disk space or less.
Is there a way to use less /usr/obj disk space during build? I know
that we have to do some bootstrapping for newer compiler tools, but
does we need to keep all temp files during the build?

There was a change near the beginning of November; please see UPDATING
entry 20171101 -- you probably have several no-longer-used
subdirectories under /usr/obj/usr/src/.
Once those are cleared out, my experience (tracking stable/11 & head in
different slices on the same machines) is that stbale/11 is using about
5.0G, while head uses about 6.1G.

I think the suspect directories are "tmp" and "obj-lib32", together
they are 4.1GB huge.
I will run a build of current again with a clean obj tree (-current on
a recent -current). Let's see.

I run a test on universe12b (FreeBSD 12.0-CURRENT #0 r325426: Sun Nov
5) with an empty obj directory.

`make buildworld' creates 9.7GB of obj data. After running `make
buildkernel' it will grow to 12GB. This is on a ZFS filesystem (my
original report was on UFS)

-Wolfram

Post by Wolfram Schneider
Can we agree that the obj tree should not grow from 5GB to 10GB for
the next release?
-Wolfram
--

[Konstantin Belousov]

Post by Wolfram Schneider

Post by Wolfram Schneider

Post by Wolfram Schneider
Hi,
I upgraded a machine from 11-stable to 12-current. The /usr/obj tree
FreeBSD 12-current
$ du -hs /usr/obj
11G /usr/obj
$ du -hs /usr/obj
5.6G /usr/obj
this is a problem when you have a small VM with 20GB disk space or less.
Is there a way to use less /usr/obj disk space during build? I know
that we have to do some bootstrapping for newer compiler tools, but
does we need to keep all temp files during the build?

There was a change near the beginning of November; please see UPDATING
entry 20171101 -- you probably have several no-longer-used
subdirectories under /usr/obj/usr/src/.
Once those are cleared out, my experience (tracking stable/11 & head in
different slices on the same machines) is that stbale/11 is using about
5.0G, while head uses about 6.1G.

I think the suspect directories are "tmp" and "obj-lib32", together
they are 4.1GB huge.
I will run a build of current again with a clean obj tree (-current on
a recent -current). Let's see.

I run a test on universe12b (FreeBSD 12.0-CURRENT #0 r325426: Sun Nov
5) with an empty obj directory.
`make buildworld' creates 9.7GB of obj data. After running `make
buildkernel' it will grow to 12GB. This is on a ZFS filesystem (my
original report was on UFS)

Most likely reason of the bump is generation of debugging data, turned on
for 12. Another not usable thing to disable are tests and profile libraries.
Put the following into /etc/src.conf:
WITHOUT_PROFILE=yes
WITHOUT_DEBUG_FILES=yes
WITHOUT_TESTS=yes

[Julian Elischer]

This brings up two questions:
when to use make.conf and when to use src.conf,

and..

Post by Konstantin Belousov
WITHOUT_PROFILE=yes
WITHOUT_DEBUG_FILES=yes
WITHOUT_TESTS=yes

which of the following is correct and why?

WITH_DEBUG_FILES=no
WITHOUT_DEBUG_FILES=yes

Post by Konstantin Belousov
_______________________________________________
https://lists.freebsd.org/mailman/listinfo/freebsd-current

[Dimitry Andric]

Post by Julian Elischer
when to use make.conf and when to use src.conf,

make.conf is for building everything, so ports, or your own programs.

src.conf is for building and installing /usr/src. WITH_, WITHOUT_ or
MK_ settings go in here.

Post by Julian Elischer
and..

Post by Konstantin Belousov
WITHOUT_PROFILE=yes
WITHOUT_DEBUG_FILES=yes
WITHOUT_TESTS=yes

which of the following is correct and why?
WITH_DEBUG_FILES=no
WITHOUT_DEBUG_FILES=yes

Since r265399, the WITHOUT_ setting wins. Note that the value of the
WITH_ or WITHOUT_ setting does not matter, so WITH_DEBUG_FILES=no still
means that DEBUG_FILES is turned on.

If you want less ambiguity, spell your settings in src.conf like:

MK_FOO=yes
MK_BAR=no

-Dimitry

[Larry Rosenman]

Post by Wolfram Schneider
I upgraded a machine from 11-stable to 12-current. The /usr/obj tree
FreeBSD 12-current
$ du -hs /usr/obj
11G /usr/obj
$ du -hs /usr/obj
5.6G /usr/obj

Mine - also 12-current - reports 7.6G.
May we see your kernel config file, src.conf, and make.conf?


Respectfully,


Robert Huff


For the record:
borg.lerctr.org /usr/obj $ du -shA
30G .
borg.lerctr.org /usr/obj $

borg.lerctr.org /usr/obj $ cat /etc/src.conf
WITH_CLANG_EXTRAS=yes
WITH_CLANG_FULL=yes
WITH_LLDB=yes
WITH_GCC=yes
WITH_GNUCXX=yes
WITH_CCACHE_BUILD=yes
CCACHE_DIR=/var/cache/ccache
borg.lerctr.org /usr/obj $ cat /etc/make.conf
DEVELOPER=yes
SVN=/usr/local/bin/svn
SVN_UPDATE=yes
# DTRACE KERNEL, Stripped of unnecessary devices
#KERNCONF=BORG-DTRACE
KERNCONF=VT-LER

# USERLAND DTRACE
#STRIP=
#CFLAGS+=-fno-omit-frame-pointer
#WITH_CTF=1
#
#__EXIM__
LOG_FILE_PATH="syslog:${LOGDIR}/%slog"
LOGDIR=/var/log/exim
#

WITH_POSTGRES=yes

#WITH_APACHE2=yes
#CUPS is the default
WITH_CUPS=YES
CUPS_OVERWRITE_BASE=YES
WITHOUT_LPR=YES

WITH_JADETEX=yes
WITH_PKGNG=yes
#PORTS_MODULES+=emulators/virtualbox-ose-kmod
#PORTS_MODULES+=x11/nvidia-driver
#MALLOC_PRODUCTION=yes
DEFAULT_VERSIONS=pgsql=9.6 apache=2.4 php=7.0 linux=c7
#####
borg.lerctr.org /usr/obj $

Doesn't bother me much.
(this might also be because of bdrewery@'s changes with meta-mode, and not clearing /usr/obj in a while, so there may be stale/old directories).



_______________________________________________
freebsd-***@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-***@freebsd.org"

[Wolfram Schneider]

Post by Larry Rosenman

Post by Wolfram Schneider
I upgraded a machine from 11-stable to 12-current. The /usr/obj tree
FreeBSD 12-current
$ du -hs /usr/obj
11G /usr/obj
$ du -hs /usr/obj
5.6G /usr/obj

Mine - also 12-current - reports 7.6G.
May we see your kernel config file, src.conf, and make.conf?

I'm running a fresh installed FreeBSD without modifications. There is
no src.conf or make.conf on the machine.

-Wolfram

[Masachika ISHIZUKA]

Post by Wolfram Schneider
I upgraded a machine from 11-stable to 12-current. The /usr/obj tree

[snip]

There was a change near the beginning of November; please see UPDATING
entry 20171101 -- you probably have several no-longer-used
subdirectories under /usr/obj/usr/src/.
Once those are cleared out, my experience (tracking stable/11 & head in
different slices on the same machines) is that stbale/11 is using about
5.0G, while head uses about 6.1G.

Hi David.

Thank you very much for good information.
I was in trouble and removed no-longer-used subdirectories under
/usr/obj/usr/src/.
Now, I have enought space to 'make -j4 buildworld && make -j4 kernel'.

[Chris H]

Post by Wolfram Schneider
Hi,
I upgraded a machine from 11-stable to 12-current. The /usr/obj tree
FreeBSD 12-current
$ du -hs /usr/obj
11G /usr/obj
$ du -hs /usr/obj
5.6G /usr/obj

FWIW on a fresh:
FreeBSD dev-box 12.0-CURRENT FreeBSD 12.0-CURRENT #0: Wed Dec 13 06:07:59 PST 2017
***@dev-box:/usr/obj/usr/src/amd64.amd64/sys/DEVBOX amd64 (r326056)

I show 4.5Gb on /usr/obj

--Chris