[oi-dev] Distribution build system

[Alasdair Lumsden]

On 03/16/11 11:35 AM, Andrzej Szeszo wrote:
> Hi All
>
> The project is moving forward very slowly. I my opinion one of the main
> reasons behind it is lack of unified distribution build system.

I absolutely agree - right now it's far too hard for people to build the 
OS; not only does this hinder new developers getting involved, it 
hinders everybody.

> Previous releases were put together manually. Probably one or two core
> contributors would be be able to repeat the whole process at this point
> without investing significant amount of time into learning how things
> fit together. We need distribution release and publishing process to be
> automated and repeatable.

Absolutely agree.

> There is a significant amount of bug reports in OpenIndiana bug tracker.
> Many of bugs require very simple changes to get fixed. URL updates or
> branding updates for example. Many contributors are more than capable of
> fixing such bugs. Because it is not clear what goes where, and also how
> to build and then test things many contributors simply don't bother
> looking at bugs at all.
>
> I am proposing creating a unified distribution build system. A system
> that would build the whole distribution after issuing a single "make
> publish" or similar command.

This is exactly what we need.

> Having such system will let us to release early, release often. It
> should improve the development progress in general. Bugfixes and
> security updates would get integrated in no time. New users would have
> an easy start. We could point new contributors at the build system and
> simply ask them to start hacking. Having all consolidations referenced
> from a single build system would make it clear to them where things go.
> Base system changes, etc. - core consolidations, new software - add-on
> consolidation directory, and so on.

*nods*

That way we can have a standard for how things are laid out on the 
filesystem, for how patches are applied, what environment is used, etc.

This will also ensure everyone is building things in the same way, which 
will reduce time wasted diagnosing build issues.

> Continuous build system could be implemented using the same tools on the
> OpenIndiana build machines, including the SPARC ones.

Continuous integration would be an enormous win for the project - it 
would mean that as soon as Oracle commit some code that breaks our patch 
set, the responsible people can be notified and immediately work on the 
issue.

This would ensure that when we come to build a new /dev release, that 
there is a lot less work to do. It would also let us spread the work out 
more easily.

> Many people will say that this is not possible and that even Oracle is
> not doing it. I say such system is possible but will require significant
> amount of work and significant amount of time to prepare. It will be
> worth the effort if done right. All major open source OS distributions
> have the release process automated in one way or another. I think this
> is the key to our success. Splitting the build and release engineering
> process between consolidation maintainers/owners based on Oracle model
> proved to not to work well for us.

Anything is possible when it comes to code, and this is no exception. 
Anyone who says it can't or shouldn't be done is wrong.

If Oracle aren't doing it, its because they are behind the times. We 
should work "smarter, not harder". Working smarter is definitely the key 
to our success.

We are "stuck in the mud" until we do this.

> I would like to start a dialog between the core contributors about such
> build system. Discuss whether it is needed or not. And if the decision
> is made that it is needed - discuss requirements, technical details and
> then actually implement it!

It is needed. The decision is a no brainer, so it gets my vote. The 
question is how we implement it.

Would anyone be interested in a Hackathon to get this work completed 
over a weekend in the near future?


Alasdair.