[oi-dev] illumos-userland update

[Alasdair Lumsden]

Hi All,

Earlier this month at FOSDEM we had some very positive discussions about moving OI and Illumian forward with our joint effort on illumos-userland. Further discussions have been held between the key people driving this, which I'd like to outline in this email.

Here are the key points:

1. Master repo at ssh://hg@hg.illumos.org/illumos-userland
2. Web browsable http mirror at http://hg.openindiana.org/illumos-userland/
3. Review process to be largely the same as for oi-build
4. Mirrors on BitBucket and Github to follow shortly

Igor (igork) of Nexenta has been working hard on getting illumos-userland into shape for Illumian, including adding Debian dpkg support, making things build with gcc 4.4, and adding software.

Igor's commits weren't community reviewed and don't consist of discrete changesets with a bug ID, so Bayard (buffyg) has backed these out for now. Igor and others are going to work together on getting these changes broken up into reviewable changesets so they can be re-committed.

We're also going to be working on processing the backlog of outstanding reviews on the mailing lists to get them committed to illumos-userland. For anyone who has contributed a changeset for review, thanks for your patience so far, hopefully we'll get them in soon.

We'll need to adapt the OI continuous integration to build from the new illumos-userland, and we'll also need to train others on Jenkins. If anyone is interested in helping with the continuous integration please let me know.

Bayard is working on getting gcc 4.4 into shape, which is going to be our default compiler for now as we transition software over to it. We're making gcc 4.4 the default and GCC's libraries such as libgcc and friends will be symlinks from /usr/lib to /usr/gcc/4.4/lib, and GCC won't have any RPATH hackery in it.

We'll look at supplying GCC 4.6 for end users to use should they wish, which may ship with RPATH hackery so if someone uses GCC 4.6 to build software, it adds an RPATH of /usr/gcc/4.6/lib so such software can find the right libgcc version. We're also looking into adding Clang support at a later date, with a view to transitioning C software over to this in the future, but this is very much on the back burner as there are much more important things at the moment.

Andy (andy_js) is looking at adding desktop components into illumos-userland, such as Gnome. How this will tie in with OI and JDS updates is unclear but there's a lot of scope for collaboration. Andrzej (aszeszo)

In terms of our roadmap for getting the contents of illumos-userland into a release, our plan is to get illumos-userland building on the current OI Jenkins build box, which auto-publishes to /experimental.

To get /experimental into shape, we intend to clone /dev to /experimental, and publish all the illumos-userland software to it. IPS/pkg5 will be rebuilt and published with branding fixes (the /experimental one is based on the S11->S11X backport). We will relax the incorporations in /experimental by removing version data, to allow incremental software updates during development. The package list in the incorporations can be updated as software is added or moved from the legacy OpenSolaris consolidations (Such as JDS) to illumos-userland, and potentially they can be auto-generated by a script based on packaging metadata if sufficiently complete (to be investigated).

Jon Tibble (Meths) is working on getting /dev published to /stable with his pre-stable builds. This has been paired back in scope to covering critical remote root exploit fixes. Maintaining /stable is hard due to the legacy Sun build systems and the age of the software. The key driver is that once /stable has shipped we can start doing regular publishes from /experimental to /dev, the process of which should simply involve adding version data to the incorporations.

Once we have regular /dev builds we can start moving towards our next stable. Some of the key goals I'd like to see for the next stable are:

1. Primary compiler will be GCC 4.4, with 4.5 and 4.6 support (4.5 for compat with S11)
2. OpenSSL 1.0
3. OpenJDK
4. Updated Firefox & Thunderbird
5. Updated Python & Ruby
6. Latest Apache, lighttpd, nginx, PHP, MySQL and Postgresql packages
7. Chromium
8. VirtualBox open source edition

Andrzej has had success in building the latest JDS, which now spits out IPS packages rather than SVR4 ones. Rebranding work is required however hopefully we can catch up with Solaris 11 here.

I think that covers the most important points. Exciting developments ahead!

Cheers,

Alasdair