[oi-dev] Release engineering // planing

Wed Jul 10 23:12:50 UTC 2013

Hi Erol,

On Jul 10, 2013, at 11:50 PM, Erol Zavidic <erolms at gmail.com> wrote:

> Good evening folks,
> 
> thanks for your feedbacks so far, here's the summary clustered in some way:
> 
> 1.0 - Release Engineering:
> 1.1    - should not be bureaucratic, i.e. rather an internal agreement (Alex)
I support this type for now.
> 1.2    - the process of pushing updates to /dev or /stable repos is undefined (Alex)
> 1.3    - safeguarding /stable repo (Jon)
> 1.4    - streamline code review and integration process LGTMs (Adam)
> 1.5    - build of many desktop packages impossible due to missing Manifests (David)
> 1.6    - creation of development, release and stable branches within hipster repository (Erol)
> 1.7    - implementing feature requests (as Ken did), assigning tasks to available developers (Erol)
> 
> 2.0 - Build toolset / infrastructure
> 2.1    - package dependencies influencing build/installation of other packages (Alex)
> 2.2    - build tools outdated and need refresh (David)
> 
> 3.0 - General topics
> 3.1    - decide on direction of /hipster effort (David)
> 
> Here some my thoughts about the topics above, please share your opinion.
> 
> ad 3.1: hipster is in my opinion a bleeding edge / unstable type of repo. It should not be limited to core or server packages, ideally a source of all packages for stable/dev repos. Building a server or desktop ISO should be left then for the people creating these. I personally use desktop version of OI, but that was my choice to use it.
I am interested in server aspects of OI and will work towards that direction (simple illumos-based server with available security fixes and packages in the repository, similar to the common linux model). I am not sure that it is the best to concentrate on all development branches. OpenIndiana hasn't made any stable release so far (at least none I am aware off) and this kills it. People are afraid of everything that is called /dev, /experimental or /prestable.  I and others have been using prestable builds of OpenIndiana for some time now and I declared that it is rock-solid and suitable for /stable release. Therefore, if we want to create some branches, I would definitely go for bleeding edge (/hipster) and make /stable out of current /dev. The questionn here is if we have enough man power to support /stable.

> ad 1.1, 1.4: agree here; LGTMs process failed and caused people to stop contributing. Using branches in git we can easily setup quick review process and contributors can send their pull requests towards hipster repository. Integration is then easily done by people having rights to do this (Alex, David, Adam, etc).
Unless there is greater number of contributors, I don't see point in using branches. I like Andrzej's idea of forking a oi-userland repository and posting link to a changeset for a review. Do you think that using branches will help us at the moment somehow? I think that it will complicate some things at this stage. However, I am open to any ideas, which will bring simple, non-bureacratic reviews. 

> ad 1.3: clever release engineering is definitely required. It requires integration testing. Do we have resources and/or testing environments for this?
hipster jenkins and repository, nothing else I am aware of. I checked OI boxes and there are old development zones after people, who left. i can create zones if needed, but keep in mind that dev01 is running 161a7 and dev02 151a8 from /dev-test (this was needed in order to support MIlan's work on JDS) and running /hipster zone on older version is not possible due to libc changes afaik (I had to fully upgrade my 151a7 build server to /hipster if I wanted to run /hipster build zones).

> ad 1.5: it is an issue and I would label it with prio 2 at this very moment. Can we exactly identify which packages are missing?
Almost every dynamic language, we have in the repository. If we could get to the point, that the users could install some common tools for every language (pip, gem, pecl, cpan, npm..) and up-to-date language version (python 2.7.3, ruby 1.9.x or 2.x for DTrace goodness, perl 5.14 or 5.16, lua etc), this would simplify porting other stuff. Next thing, I find important is absorbing as much as possible from ec-userland, because common server (and even desktop software) is there. ec-userland contains updated nginx, php, mysql, postgresql, apache and multimedia things like ffmpeg, vlc and their dependencies (which I am grateful for because I will have to deal with this in a week or so for my job. Thanks EC guys!). 

> ad 1.6: would it make sense to create stable, dev, release and edge branches within hipster repository to stabilize releases a bit? Propagation from edge to dev and stable would be easy-peasy with git. As a rough Idea how it could look alike have a look at Vincent Driessen blog entry.
> 
> ad 1.7: consider Ken's email with request for LibreOffice as a good example where we could implement scrum method, delivering sprints and assigning tasks to available volunteers. Which tool to use I cannot tell. JIRA is obviously very common. Using milestones and issues in github one could mimic the behavior.
I wouldn't complicate things with JIRA and just use Github. It is simple and we have it already. We have to just start creating issues and somebody should keep an overview. However, this can be very time consuming and the best thing we can do at the moment is to try coordination, so we don't duplicate our efforts and possibly concentrate our work on packages I mentioned above. For example, if few of us start to deal with one language, this should be done pretty quickly. 

> ad 2.1: package dependencies is indeed a prio 1 issue. I like the Alex's idea of having BUILD and INSTALL dependencies listed. Would be manual effort unless we can automate this. Any scripts to do this?
I am not aware any scripts to do this. Everytime I create new development environment for myself I just simply install everything because it is the easiest thing to do. I propose that packages have build.deps file, which contain FMRIs of dependencies. If a contributor is committing a new package, he should be aware of package build dependencies. We should think about build dependency tracking, once we have those files. I find having up-to-date dynamic languages as the highest priority for now.

> ad 2.2: prio 1* - needs immediate attention. David, you listed a few, can we get a full list and split amongst us who is taking which tool to update?
In the second half of 2012, I was working on making oi-build compilable by gcc. I started by updating the build infrastructure. Those pieces can be reused. I am aware of autoconf (updated in my WIP repo), binutils, libtool (update is already in userland-gate, but it is delivering older libltdl) and automake (updated in my WIP repo). At the moment, I am working on build-essential meta package, which will be based on ec-userland and openindiana wiki page. Having this single package will simplify our life. My old WIP repository can be found at https://hg.openindiana.org/users/xenol/oi-build/

Seeing open discussion about release engineering and organization is great and motivating for moving things forward. However, let's not kill most of our efforts by talking and taking no action. Let's just keep things KISS and do some small coordination.

Cheers,
Adam 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://openindiana.org/pipermail/oi-dev/attachments/20130711/04cf55a1/attachment-0005.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4804 bytes
Desc: not available
URL: <http://openindiana.org/pipermail/oi-dev/attachments/20130711/04cf55a1/attachment-0005.bin>