[oi-dev] Issues pertaining to gcc (for tomorrow's Developer Meeting)

[Alex Viskovatoff]

Hello all,

gcc is going to be discussed at tomorrow's developer meeting, so I
thought I would go over what I find to be the main issues, based on
discussion at the last two informal meetings.  The main "controversy"
seems to be whether OI/Illumian should provide only one gcc, the one
used to build illumos-gate (not counting gcc-3, which will be with us
for a long while yet), or whether it should provide that one as well
as a recent one.  I have not attempted to be impartial between those
two alternatives in what follows.


1. Should gcc be the preferred compiler for illumos-userlannd?

2. Which versions of gcc should OI/Illumian ship?

3. Which version of gcc should be preferred for building things other
   than illumos-gate?

4. Where should the preferred gcc version reside?

5. Should symlinks to gcc libs be placed in /usr/lib?

6. C++ library ABI incompatibility issues and how to solve them


1. Should gcc be the preferred compiler for illumos-userlannd?

This issue was raised at the informal discussion on #oi-meeting last
week by Milan Jurik, who has recently become more active in OI
discussions.  (To others, it may be a revisiting of this issue.)
There are two main considerations here.

(1) The upstream of userland is still Oracle, and Oracle's preferred
compiler is Sun Studio.  Thus, more work would likely be involved in
maintaining illumos-userland if the preferred compiler for it was gcc.

(2) Although illumos-gate can be built with Rich Lowe's gcc, as far as
I know the decision has not been made to actually use gcc to build it
for the first stable release of OI.  I would suggest that doing so
would be a very risky move, because we have so little experience with
a gcc-built Illumos.  One should be cautious here, and treat a
gcc-built Illumos as experimental for an extended period of time,
before actually releasing it to the public.  But if Sun Studio is used
to build Illumos, what is the advantage of building illumos-userland
with gcc?  As noted in (1), there is a distinct disadvantage -- that
it would be harder to keep up with upstream.


2. Which versions of gcc should OI/Illumian ship?

It is a given that illumos-gate must be built with gcc 4.4, because
porting the patches required to get it to build to later versions of
gcc is a non-trivial task.  But 4.4 is an old version of gcc.  The
experience of the SFE project is that an unpatched gcc 4.6.2 (the
current stable release) is perfectly capable of building a wide range
of software (Qt 4.7, Boost, Inkscape, Gnote).  In addition, there is
already some GNU software which _requires_ gcc not older than 4.6 to
build.  (MKVToolnix is an example; there are probably others.)
Finally, people coming to OI from the GNU/Linux world would be taken
aback if OI does not ship gcc 4.6.

One last consideration is that the default gcc in Solaris 11 is gcc
4.5.  OI/Illumos choosing 4.4 would thus create the impression that
Illumos is behind Oracle.  Choosing 4.6 would create the opposite
impression.

Userland Makefiles for gcc 4.5 have already been modified to build gcc
4.6.2.  If it is decided not to include gcc 4.6.2 in illumos-userland,
oi-sfe could continue to provide it.  But would it really make sense
to delegate the current stable release of the compiler most commonly
used in the Unix/Linux world to an extras repository?

In case Rich is not able to attend tomorrow's meeting, I'll quote what
he said on #illumos a few minutes ago: "basically, my only view on gcc
is 'Leave the ones used to build illumos alone, then do whatever you
want'."


3. Which version of gcc should be preferred for building things other
   than illumos-gate?

Since Rich Lowe's patches are not required to build illumos-gate, if
illumos-userland provided gcc 4.6, would there be any good reason to
use gcc 4.4 and not gcc 4.6 to build illumos-userland (provided that
gcc is the preferred compiler for this, as discussed in (1) above)?
I am not aware of any.  For everything else, gcc 4.6 should clearly be
the preferred gcc version, since that is what Linux distributions are
already at or at least moving toward.  To repeat, it is the current
stable release, and it has a proven track record on OI.


4. Where should the preferred gcc version reside?

The obvious place is in /usr/bin.  Solaris 11 places symlinks to gcc
4.5 there.  (OI still places symlinks to gcc 3.4.3 there; it should
stop doing so as quickly as possible.)

There is the additional issue of how the symlinks should be
implemented.  The corresponding IPS actions can go in the gcc-4.4/4.6
package itself, or there could be separate packages which only deliver
the symlinks.  The best solution would probably be pkg(1) mediators,
but that would require further investigation.


5. Should symlinks to gcc libs be placed in /usr/lib?

gcc 3.4.3 did not place its libraries libgcc_s.so.1 and libstdc++.so.6
into a standard location searched by the runtime linker.  Instead, it
patched gcc to add the location of its libraries to a shared object's
or an executable's RUNPATH.  This patch has become deprecated.  It is
natural for OI/Illumian to place symlinks to the runtime libraries of
the preferred gcc version in a standard location, namely /usr/lib.
The gcc-45-runtime package of Solaris 11 already does this.

An issue that has been discussed repeatedly on IRC is that placing
symlinks to the runtime of gcc 4.6 breaks VirtualBox.  (For some
reason, this does not happen with gcc 4.5.)  No one has been able to
figure out as of yet what that happens.  Since disabling direct
binding at runtime (see LD_NODIRECT in ld.so.1(1)) makes the problem
go away, it is possible that the problem is produced by VirtualBox's
use of direct binding.  In any case, there have been no reports of gcc
4.6 runtime residing in /usr/lib breaking anything other than
VirtualBox.

The gcc build of Illumos is done in such a way that Illumos should not
be affected by the gcc 4.6 runtime residing in /usr/lib.


6. C++ library ABI incompatibility issues and how to solve them

C has a standard ABI; that is why applications built with gcc can link
to libraries built with Sun Studio.  C++ on the other hand has no
standard ABI, and this raises various complications.  (For a
discussion of this at greater length, Google "SFE C++ libraries
revisited".)  OI at present has at least one package affected by this:
library/icu (International Components for Unicode).  (This is not
currently part of userland, and is built with pkgbuild.)  Applications
or libraries built with g++ cannot link to the system library/icu,
because it uses Sun Studio (and libCstd, specifically).

Thus the question arises of what to do about the C++ ABI
incompatibility between compilers, given that the "native" compiler of
OI is still Sun Studio.  One solution would be to say that g++ will
now be the preferred C++ compiler for OI, and place libraries built
with g++ in /usr/lib.  There are at least three problems with this
approach.  (1) It would create avoidable transitioning and
coordination problems.  (2) Sun Studio is still available for OI for
free, and releases continue to come out about once a year, so some
users will want to use it.  Placing g++-built libraries in /usr/lib
would complicate things for these users, because they would have to
decide where to place their own custom-built libraries built with
Studio.  (Sun Studio is not a bad compiler.  The problems with it are
that is closed source, its maintainers have no interest in fixing bugs
reported by non-paying users, and it is controlled by a corporation
not interested in cooperating with the open source community.)  (3)
Switching completely to g++ and making it the standard C++ compiler
would unnecessarily break compatibility with Solaris.  But it is of
benefit to OI users to be able to switch as easily as possible back
and forth between OI and Solaris.  Most significantly, if a user has
hardware that OI cannot handle but Solaris can, that user can
temporarily switch to Solaris (provided it does support that hardware)
until OI supports that hardware.

The SFE project has developed a scheme whereby C++ libraries built
with different compilers can coexist on the same system.  I suggest
that OI/Illumian should adopt this scheme.  Under it, C++ libraries
build with Sun Studio and against libCstd go in /usr, those built with
g++ go in /usr/g++, and those built with Sun Studio but against
stdcxx4 go in /usr/stdcxx.  Under this scheme, there is a duplication
of header files, but it was decided it was worth paying that cost to
avoid the constraint that all builds (with g++, Sun Studio, or
whatever) of a given library must be at the same version.  The
installation of header files on non-development machines can be
avoided by packaging development files separately or by using facets.

I believe that this approach keeps to the Solaris tradition of
introducing as little change as possible, in order to maintain as much
compatibility as possible and introduce as little breakage as
possible.


I look forward to chatting with you at tomorrow's meeting.

Best regards,

Alex