[oi-dev] illumos-gate build time

[Gordon Ross]

Time for a new topic :)

On Sun, Jan 1, 2017 at 10:11 AM, Jim Klimov <jimklimov at cos.ru> wrote:
[...]
> By the way, regarding builds of illumos-gate, I estimate that most of the time is spent walking the makefiles and directories, several rounds while nightly.sh works. On lesser computers this alone adds up to tens of minutes if not hours.
>
> Are there any hints or efforts on optimising this? Maybe somehow precompile a single big makefile to reduce walks and forks, or something?

As with any serious optimization effort, the best recommendation is to
first try to measure where the time is being spent.  The last time I
heard of anyone doing that for the build, it was (in general) I/O
bound.  I vaguely recall that a large factor of the I/O time was
dominated by "stat" calls, and I thing those were mostly for files or
directories that do not exist.

Trying to make some new measurements would be a great place to start.

One reason for excessive stat calls in our build is the widespread use
of shared makefiles that have lots of pattern rules (i.e.
uts/*/Makefile.rules), and where only one or a couple of those pattern
rules are in use.  In such cases, make has to do stats of many
(typically nonexistent) objects to find out that a pattern rule does
not apply.  But before anyone goes trying to "fix" that, we'll need
measurements.