[OpenIndiana-discuss] New OI /dev release, release structure

[Nikola M.]

On 10/ 3/14 03:23 PM, Alexander Pyhalov wrote:
> Hello.
>
> On 10/03/2014 16:26, Nikola M. wrote:
>>> First statement is unfortunately true. To make /dev => /hipster update
>>> possible we need
>>> 1) to republish packages which were not rebuilt since /dev a8 with
>>> higher numbers (2014.x)
>>> 2) republish incorporations so that they either are empty or depend on
>>> actual packages
>> I still do not understand this one. Why killing incorporations is so
>> important,
>> it seems that incorporations definitions could be updated to reflect
>> version changes and then go through the testing to make sure everything
>> under updated incorporation works together.
> 1) I haven't said about killing incorporations here. I said making 
> them empty OR dependent on actual packages.
OK, That's non issue then.
I think they should depend on current versions of actual packages in 
Hipster, providing one that update a package that is part of 
incorporation says that updated package actually works with other 
packages inside incorporation.
That way incorporation serves its purpose even during Hipster lifetime, 
before next /dev, when incorporations could be fully and thoughtfully 
tested.
Updating package have sense only when it does not break incorporation as 
a whole and that HAVE sense.

I don't know what would be use case of empty incorporations...
> 2) But I honestly don't think that incorporations are necessary in 
> Hipster, perhaps, except entire. 
I just provided answer for this. No updated package server it's purpose 
if it breaks other packages in incorporation.
Can't update package X because of other dependencies locked by 
incorporation that needs other parts to make them work.
> They were aimed to control version numbers of software from different 
> consolidations. But we want to have only one consolidation . 
And why is that? Why would we want to have just one consolidation?
That makes separate testing and working on parts of the system hard
and not possible to enforce teams work for parts of the distribution.

When I focus on testing and building only one consolidation of packages, 
I can focus on it, instead of building _everything_ every time. Building 
everything is terrible waste of time, too.
> If you want to stick to one package version just do pkg fix entire 
> after Hipster install. We could avoid publishing empty entire after 
> ISO release and so make Hipster behave more like /dev. But I'm 
> inclined to publish empty entire to encourage wider testing.
>
>>> 3) publish neccessary obsoletion / renaming packages
>> This also needs to be checked and documented, both for recommendations
>> what to use instead if some package is removed, how to move current
>> configuration to a new one etc.
>> Packages can not just be removed with no documentation about what to use
>> instead in next release.
>
> I don't think so. We have a lot of outdated desktop-related software. 
> It has sometimes strange dependencies, brings in more outdated 
> software and so on. I think that it's good trend to remove something 
> old, unused and unmaintained. 
You generally remove something when you consider other people thoughts 
on it and how that affects them and distribution usability. (That is 
what choosing what is inside incorporation is also about)
You can't just destroy packages without having a RELEASE that people 
could go to check them when packages were last time available (or inside 
branded zone for previous release.

It might look like from Hipster's perspective (in between /dev's) that 
it is OK,
But some DOCUMENTATION must be written describing what and why and how 
is removed in Hipster and what people using or depending on those 
packages till now should do to compatible things.

That is what is called distribution mantaining, as I see it. You also 
need to have documentation of what  you do, (So that also WIKIs and 
MANUALS and DOCUMENTATION should be updated accordingly).

If one person just go around deleting packages, without trace and 
explanation and consulting it leads to unmaintainable, undocumented, 
unstable, unusable (no docs) MESS.
So maintaining LOG with explanations of the actions, especially in 
Hipster is a MUST to have other in-distribution people specializations 
possible to get their job done.

 From hat LOG, someone doing TESTING before releasing next /dev out of 
it, could make checking list testing plans according to freeze dates and 
together with Hipster's entire and incorporation version changes, makes 
good environment for effective testing that produce high quality bug 
reports (and fixes).

> If someone wants it back, just fix the package to build in oi-userland 
> and work on Hipster.
Firstly removing something randomly and then requiring extra effort to 
get it back sounds like very bad practice to me. Things don't get 
removed "just because" without explanation ,
that is how I see it.
Such practice could lead to tons of bug reports before releasing /dev.
>
>> (And there should be releases and not only package updates)
>
> We are publishing snapshots once per several months.
That is not good enough.
We and a whole users of OI distribution need Hipster blending into next 
/dev ,
something people should actually USE and report serious bugs that could 
surface during longer use.

Hipster and OI can not survive and maintain being something usable to 
people if efforts are not made toward making new updatable /dev releases.
(With Hipster living and evolving in between /dev's. (!) )

That way, what you call Hipster 'snapshots', should actually be /dev 
releases and not something that is 'not /dev'.
That way we get all benefits from both models, only next Hipster 'snapshots'
should follow what /dev is and be updatable from /dev to it,
not against /dev  or avoiding what it is.
>
>>  From that basic documentation, later before publishing new /dev
>> release, documentation for change could be made wider before release,
>> telling people what to do administering it.
>
> Agree, documentation can be enhanced.
Great. :)

>
>>> 4) test that at least typical text / GUI install can be updated from
>>> latest /dev to /hipster.
>>> Noone volunteered to do it yet.
>> It was possible before and updating from a8 was almost smooth. And I
>> think I reported the moment I figured it is not possible.
>> Response was not to fix it but to ignore it.
>
> It is not possible now. At least until all our packages has greater 
> versions then /dev. /dev now uses  X-0.151.1.9 versions. We have a lot 
> of X-0.151.1.8 versions. IPS can't handle this. 
As I see it, since 151a8 was latest ISO /dev release, for now, it could 
be enough to have one Hipster snapshot that could be updated to it with 
certainty.
Update from a9 could be added after that in next Hipster snapshot, when 
said versions could go to a10 and That could be new a10.
Does that sounds reasonable? Agreed?
> Also, a lot of obsoletion packages were removed during migration from 
> /hipster to /hipster-2014.1 repository. Part of them should be 
> republished to allow /dev => /hipster update.
That's the problem that is made as I said without taking into account
the need to have Hipster snapshot that is updatable from a8.

They surely should get back then, before Hipster snapshot, that is 
updatable to from a8.

Also I have never heard of ANY discussion on removing and obsoletion 
packages, before they are removed and that is also unacceptable as 
collaboration practice
in a living distribution of many people and not one man's project...

Maybe we can open separate Hipster mailing list,
that could be used as discussion and documentation hub for package 
adding, removal etc or some other interface where it could be published, 
announced, discussed etc.

>
>> Hipster with it's always changing contents of packages and no revisions
>> to jump between different states (versions, like entire etc)
>> makes quality control and testing almost impossible (figuring out WHEN
>> and after what change what problem arise)
>
> We have shipped entire with last ISO snapshot. We are going to ship 
> one more with new ISO snapshot. However, I agree, there's issue with 
> testing - you can't update/downgrade to the state of some particular  
> date. But it's true also for other OSes and distributions. Do you 
> propose to update entire on each package update?
YES. YES.
Hipster's entire should go up on every update of package in it. (If you 
follow rolling-release model for Hipster in between /dev's)

( And why ALL packages are rebuild on Every package update anyway? Could 
that be avoided by Hipster release engineering and compile and publish 
only that updated package and not copmile everything, anyway? )

And maybe it ideally should be different thing from 'entire' in /dev.

That way Hipster testing would have sense and tester could always 
upgrade to Exact point in time when certain bug started to appear and 
point to change that invoked bug easily.
That could be great.
>
>> Hipster proved as not suitable for any use but testing and using on
>> one's laptop or separate test server.
>
> I successfully use it for my labs. Using something illumos-based on 
> our servers is impossible due to hardware issues. But I don't think I 
> would persuade my collegues to migrate from Debian to something else 
> even if illumos supported our hardware :)
Yes, exactly. maybe they could run Debian in VM, But I would not go in 
much details about it, since you mentioned special hardware, etc.
>
>> Problem with Hipster is that it is not meant to be productiion quality
>> from the ground up and all philosophy of "let's update a package and see
>> will someone compain and report a bug"
>> works quite well for internal releases/strictly "in between" working
>> updates before next /dev release.
>
> Yes, it was created to speed up OI development. I think it partly 
> achieved this goal.
Yes, I think Hipster managed to speed things up very well.
It broke somethings in a rush, but as we all see, everything could be 
make in/line,
without loosing freedom of experimenting on Hipster between /dev releases.

Both should live from each other, yet we just need some more docs, 
consultations of community during enhancing it, a bit of tiny procedure 
here and there and clear goal
that next /dev is always a next  goal :)
>
>> Hipster spent too much time going away from Openindiana /dev and it has
>> to stop doing so.
>
> It's easy to go away from someone standing ;) Seriously, I think that 
> moving away from legacy consolidations and build systems is a 
> necessary task to ease OI development and to lower barrier for new devs.
Ii don't think so.
Breaking consolidations have little to do with lower barrier for new 
people ,
because, As you said, if updating package includes/could include 
updating consolidation definition and that's it. (maybe i am mixing 
incorporations and consolidations, help me here, both
are important to survive as I understand)

If consolidations also have have upstream sources (from Oracle and 
others) why breaking them if they make life more easy?
It is not what updating one package is important, it is important that 
such update does not break other things and that is where Maintainers 
and maintaining consolidations and incorporations goes.


Ok, after writing so much text, I could take a vacation :P ...
Seriously, I think we are making great progress here,
  it's not jus talking - it is understanding what we could do next as a 
whole ,
and I think final goal is to have next /dev for wide audience with more 
precise and clear 'modus operandi' for Hipster in between /dev releases.

If we make operational what we were talking about here and I help making 
it work,
we will have something much more powerful,
We can have one powerful distribution , easy to include new people to 
change and enhance what they need, under guidance and few simple rules, 
everyone can understand.

But some rules need to be set up and one of them should be done right now:
Goal of Hipster shoud be making new /dev release and not living by 
itself in limbo without updating users from /dev forever.  :P
We benefit a great deal if more people can use updates from Hipster in 
their next /dev update.

Nikolam