[OpenIndiana-discuss] Latest update problems should be fixed now

[Andreas Wacknitz]

Am 30.06.22 um 23:00 schrieb Joshua M. Clulow via openindiana-discuss:
> On Thu, 30 Jun 2022 at 12:32, Andreas Wacknitz <A.Wacknitz at gmx.de> wrote:
>> Am 30.06.22 um 20:56 schrieb Joshua M. Clulow via openindiana-discuss:
>>> Is there some way that we can gate future publishing to the public
>>> repository on a successful test install/update?
>> We need more volunteers as I already stated several times in the past.
>> I am doing most of everything with only very few helpers. I try to test
>> all PR's but my time and test environment is limited.
>> Besides me it is quite seldom that anybody is testing anything. The
>> situation is similar in all areas of the project you can imagine.
>> In the past I have been waiting for others to do tests but that happened
>> only very rarely and typically with days to weeks of delays.
> So, to be clear, my question is not about looking for more manual
> testing.  I know we're all limited in time and resources.
>
> I was more trying to ask about automated testing we could have done.
> Presumably there's already automation (Jenkins?) that does the build,
> and then publishes the packages.
Agnar and toasterson are working in that area for some time already. But
it already took months to get where we are now and will probably take
some more months to be usable.
The first results you can already find in the PR process - there is an
automated integration test. Alas the results are not yet helpful because
of too many limitations.
The test builds already take place in zones but there is no notion of
dependencies yet so only single packages will be build. And there are no
automatic updates like they happen on the official build system as part
of each jenkins job (with osnet-incorporation frozen to avoid problems).
>
> It seems like it would be good if it was also able to create an
> ephemeral VM, then log in and use pkg to install the built bits (to
> make sure they can be installed), then reboot and make sure the
> machine at least comes up (as a basic smoke test).  If that all
> succeeds, _then_ push the packages to the public repository.
The problem of testing is more complex for some packages, eg. we have
dependencies to consider when we update Mate packages.
This is one of the areas where we need improvements for the already
available automatic integration tests.
The python eco system is a nightmare with baselining almost impossible
because some packages have contradicting requirements, everything is in
constant flux, ...

>
> Hopefully having that automated would not then require a bunch of
> extra hands-on manual work for each PR?
As I wrote before: we are lacking volunteers who help. With only one
active admin it is hard to keep everything in a working condition.
With only one active tester it's hard (almost impossible) to test
thoroughly.
I get enough recommendations for things that should be improved. But
with too few helping hands it's hard keep the status quo let alone to
improve things.

Andreas