[oi-dev] OpenIndiana /hipster build number

Thu Jan 9 22:06:24 UTC 2014

Good evening, Andrzej.
I'm glad that you brought up this question again and proposed a rational 
sollution.

However, I have a lot of questions. Sorry, more questions than 
suggestions.

Andrzej Szeszo писал 06.01.2014 17:55:
> I propose a new scheme similar to the following:
> 
> Current version strings:
> 
> pkg://openindiana.org/SUNWcs@0.5.11,5.11-0.151.1.8.1
> pkg://openindiana.org/shell/bash@4.2.45,5.11-0.151.1.8.1
> pkg://openindiana.org/package/pkg@0.5.11,5.11-0.151.1.8.1
> 
> Proposed new version strings:
> 
> pkg://openindiana.org/SUNWcs@0.5.11,5.11-0.2014.0.0.14302.2476
> pkg://openindiana.org/shell/bash@4.2.45,5.11-0.2014.0.0.0.2476
> pkg://openindiana.org/package/pkg@0.5.11,5.11-0.2014.0.0.5055.2476
> 
> The numbers would have the following meaning:
> 
> PKGVERS_BRANCH =
> 0.$(IPSREPO_YEAR).$(IPSREPO_NUM).$(IPSREPO_SUBNUM).$(PKGREV_COMPONENT).$(USERLAND_ID)
> PKGVERS = $(PKGVERS_COMPONENT),$(PKGVERS_BUILTON)-$(PKGVERS_BRANCH)
> 
> Currently, /hipster IPS repository is only growing, there is no way to
> obsolete packages in it, updates and all operations are becoming
> slower and slower. I propose importing latest installable packages
> from /hispter into /hipster-2014.0 and continuing work in a new
> smaller repo. If it becomes too big, we could switch over to
> /hipster-2014.1, etc.

As I understand, you propose two things
1) create multiple repositories to simplify their maintenance and to 
speed up work with them
2) new versioning scheme.

So, I'll group my questions in two parts.

First one concerns multiple repositories.

1) When new repository is created? Once a quarter? When old one is too 
big?

2) What packages are migrated to the new repository?  The latest 
versions of every package in the last active repository?

3) How do we migrate user systems to new repository, let's say from
pkg.openindiana.org/hipster to pkg.openindiana.org/hipster-2014.0 and 
further to pkg.openindiana.org/hipster-2014.1 ?
Some self-destructing SMF? We clearly have to force new BE creation on 
such shift.

As for spped of operations on current repository, is it possible to 
change "pkgrepo rebuild" to "pkgrepo refresh" in Jenkins build jobs? Is 
it safe? Is speedup significant?

> 
> I don't think "$(PKGVERS_COMPONENT),$(PKGVERS_BUILTON)-" part of the
> proposer version string needs much explanation. For illumos-gate or
> pkg5 packages it will always be 0.5.11,5.11-, for other packages it
> will be package-version,5.11- (4.2.45,5.11- in case of bash).
> 
> The -0.2014.0 part would say which pkg.openindiana.org/hipster-20xx.x
> IPS repo package is coming from, for example:
> 
> -0.2014.0 = http://pkg.openindiana.org/hipster-2014.0
> -0.2014.1 = http://pkg.openindiana.org/hipster-2014.1
> 
> IPSREPO_SUBNUM we could bump when there is a "flag day", or when
> multiple packages have to be rebuilt for some reason and updated as
> the same time.
> 
> PKGREV_COMPONENT could be used to specify revision on the component in
> the oi-userland, or in case of illumos-gate or pkg5 could be set to
> git commit number of illumos-gate or pkg-gate.
> 
> USERLAND_ID could be set to oi-userland git commit number that was
> used at the time of building the package.
> 

Now, about versioning scheme.
1) I still don't see a clean way to downgrade package with proposed 
versioning scheme.

2) What is a policy for changing PKGREV_COMPONENT? I propose to set it 
to 0 by default and force people to increment it with every change in a 
component which affects its functionality (i.e., not just code cleanup). 
I would reset it to 0 with every package version bump (e.g. php 5.21=> 
php 5.22), this will allow us to keep it sane. And I'm not sure that it 
is a good idea to use it just to force package rebuild.

3) What will we do with illumos-gate updates? I mean, proposed scheme 
allows us to do it in a safe way. However, when build machine is 
updated, this will force illumos-gate packages update (and BE creation) 
on a user system even if the user just wants to update, let's say, vim. 
And when you want just install security update for php on a server it's 
a problem if doing so requires reboot (for BE activation).
I see several oppotunities.
a) We say, that /hipster is a developer's OI version and it's enough if 
it just will not allow user to update package to non-functioning state 
(i.e. updating php 5.21 to 5.22 is not possible in the same BE if build 
server updated illumos-gate-produced packages between times when php 
5.21 and 5.22 versions were built). It's not a perfect scenario, but I'm 
OK with it. It's much better than allowing user to install php 5.22 in 
the same BE and to find out that  it is not functioning because of libc 
version bump.
b) We update illumos-gate on the server once a (month/week/ ??? ) to 
allow some period of stability. Let's say, each Saturday, so for more 
important servers I can schedule update if necessary for Sunday.
c) We update it by hand on "as strictly necessary" basis (perhaps, 
combining with once a month/quarter).
Related question...
If I have system/library version A installed on build machine and 
version A+N available in server's default repository, will dependent 
packages depend on version A or on A+N? Can we publish new illumos-gate 
versions, but hold build server's version?

4) The same question about IPS. As I understand, on each IPS update pkg 
will require new BE (and IIRC it refuses to install any new package 
until pkg is updated).
I expect that pkg5 repository will not change as often as illumos-gate 
(at least after some time of initial bug-fixing). However, we should 
have some consistent update policy for it.

5) Just a question of taste. Don't you think that 
"4.2.45,5.11-0.2014.0.0.0.2476" is too long? I know, 5.11 is a sacred 
number in Solaris world, but I don't understand why should we have this 
magic in version string (for successful migration? Or IPS just will not 
like anything else?) and not very sure about 0 before year number. (Why 
not just  4.2.45-2014.0.0.0.2476  or MAGIC-2014.0.0.0.2476 for system 
packages?).
I'm just whining and don't have strong opinion on this point :)

Thanks again for looking at this.
---
System Administrator of Southern Federal University Computer Center