[oi-dev] heads up: zones not available after pkg update in global zone

[Stephan Althaus]

On 1/27/24 10:05, Stephan Althaus wrote:
> On 1/26/24 23:14, Bill Sommerfeld via oi-dev wrote:
>> On 1/26/24 13:26, Stephan Althaus wrote:
>>> On 1/26/24 18:42, Bill Sommerfeld via oi-dev wrote:
>>>> On 1/26/24 01:17, Stephan Althaus wrote:
>>>>> Maybe its only here and a local problem, but who knows ...
>>>>>
>>>>>
>>>>> I did a pkg update this morning, and the zones are not available.
>>>>>
>>>>> # zoneadm list -cv
>>>>>    ID NAME             STATUS     PATH BRAND    IP
>>>>>     0 global           running    / ipkg     shared
>>>>>     1 mail             running    /zones/mail nlipkg excl
>>>>>     2 www              running    /zones/www nlipkg   excl
>>>>>
>>>>> The zones are booted, but zlogin is not possible, nor are the 
>>>>> zones reachable via ssh.
>>>>
>>>> Looks like execing the shell failed.
>>>>
>>>> Upstream illumos-gate just made an incompatible change to the 
>>>> private interface between libc and the kernel used by the exec*() 
>>>> family of functions.
>>>>
>>>> commit 2b395c3c2a39cdc00f9fe7ac497795bd112f7663
>>>> Author: Andy Fiddaman <illumos at fiddaman.net>
>>>> Date:   Sat Jan 20 10:47:38 2024 +0000
>>>>
>>>>     5798 fexecve() needed per POSIX 2008
>>>>     Portions contributed by: Garrett D'Amore <garrett at damore.org>
>>>>     Reviewed by: Dan Cross <cross at oxidecomputer.com>
>>>>     Reviewed by: Bill Sommerfeld <sommerfeld at hamachi.org>
>>>>     Approved by: Gordon Ross <gordon.w.ross at gmail.com>
>>>>
>>>> Changes to the libc-kernel interface are relatively uncommon but 
>>>> should be expected to happen every so often.
>>>>
>>>> I see that you're using nlipkg zones, which aren't kept in sync 
>>>> with the global zone.   Did you update the zones as well?   If not, 
>>>> I'd expect most exec*() calls to fail with an EINVAL unless the new 
>>>> fourth argument to the underlying syscall is zero by chance.
>>>>
>>>> Update the zones along with the global zone and they should start 
>>>> working again.  Just updating all the variants of libc.so.1 to 
>>>> match the global zone should be enough to get you up & limping again.
>>>>
>>>>                     - Bill
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> oi-dev mailing list
>>>> oi-dev at openindiana.org
>>>> https://openindiana.org/mailman/listinfo/oi-dev
>>>
>>> Hello!
>>>
>>>
>>> Thanks for the explanation.
>>>
>>> The zone that did not work was updated a day before the global zone, 
>>> BE is "openindiana-2024:01:24" within the zone,
>>> so the exec* stuff should be in..
>>> But wait the merge was at timestamp  2024-01-23T19:29:54.000Z so 
>>> maybe not..
>>>
>>> Current BE of the global zone is "openindiana-2024:01:02" so the 
>>> commit is not in..
>>>
>>> I will try to update the ng-zones first and after that the global zone,
>>> or maybe better change the zones to ipkg first,
>>> or just update the zones now and reboot the global zone to the 
>>> (currently unused) existing BE  "openindiana-2024:01:26" for which 
>>> the problem exists..
>>
>> what would be helpful for confirming what happened would be the 
>> illumos-gate.info.git-rev property from the /system/library package 
>> in each BE of the global zone and the impacted zone.
>>
>> Try running:
>>
>> $ pkg contents -H -o value -a name=illumos-gate.info.git-rev 
>> /system/library
>>
>> for the global zone, and
>>
>> $ pkg -R <alternate-root> contents -H -o value -a 
>> name=illumos-gate.info.git-rev /system/library
>>
>> for each relevant alternate BE and zone root.
>>
>>                         - Bill
>>
>>
>>
>> _______________________________________________
>> oi-dev mailing list
>> oi-dev at openindiana.org
>> https://openindiana.org/mailman/listinfo/oi-dev
>
> Hello!
>
> This is the running combination:
>
> Global zone:
>
> $ pkg contents -H -o value -a name=illumos-gate.info.git-rev 
> /system/library
> 8361acf58a302751348aac091ab09484f3ecfb8c
>
> ng zone
>
> # pkg contents -H -o value -a name=illumos-gate.info.git-rev 
> /system/library
> 32c2a899b6e4eab014fd5243af6c011a84dc103f
>
>
> When i activate the newer BE on the global zone,
> where i cant't zlogin to the ng zone, we have this value:
>
> $ pkg contents -H -o value -a name=illumos-gate.info.git-rev 
> /system/library
> d5a816590ba1bb4affbf62f0113d75baf7f83177
>
>
> Regards,
> Stephan
>
>
>
> _______________________________________________
> oi-dev mailing list
> oi-dev at openindiana.org
> https://openindiana.org/mailman/listinfo/oi-dev

Hello!

Sorry for the late reply, covid is greeting:-/


To sum up it is mandatory to have the same git-rva within the ng zones 
and the global zones.

pkg update-r  does not work with my ng zones of branch nlipkg, but we 
always can do a pkg update within in each ng zone - "pkg update && init 
5" maybe, and afterwards within the global zone "pkg update && init 6"

This was the 'solution' for my system.


Thanks for all your help!!


Regards,

Stephan