[oi-dev] heads up: zones not available after pkg update in global zone

[Stephan Althaus]

On 1/26/24 23:14, Bill Sommerfeld via oi-dev wrote:
> On 1/26/24 13:26, Stephan Althaus wrote:
>> On 1/26/24 18:42, Bill Sommerfeld via oi-dev wrote:
>>> On 1/26/24 01:17, Stephan Althaus wrote:
>>>> Maybe its only here and a local problem, but who knows ...
>>>>
>>>>
>>>> I did a pkg update this morning, and the zones are not available.
>>>>
>>>> # zoneadm list -cv
>>>>    ID NAME             STATUS     PATH BRAND    IP
>>>>     0 global           running    / ipkg     shared
>>>>     1 mail             running    /zones/mail nlipkg   excl
>>>>     2 www              running    /zones/www nlipkg   excl
>>>>
>>>> The zones are booted, but zlogin is not possible, nor are the zones 
>>>> reachable via ssh.
>>>
>>> Looks like execing the shell failed.
>>>
>>> Upstream illumos-gate just made an incompatible change to the 
>>> private interface between libc and the kernel used by the exec*() 
>>> family of functions.
>>>
>>> commit 2b395c3c2a39cdc00f9fe7ac497795bd112f7663
>>> Author: Andy Fiddaman <illumos at fiddaman.net>
>>> Date:   Sat Jan 20 10:47:38 2024 +0000
>>>
>>>     5798 fexecve() needed per POSIX 2008
>>>     Portions contributed by: Garrett D'Amore <garrett at damore.org>
>>>     Reviewed by: Dan Cross <cross at oxidecomputer.com>
>>>     Reviewed by: Bill Sommerfeld <sommerfeld at hamachi.org>
>>>     Approved by: Gordon Ross <gordon.w.ross at gmail.com>
>>>
>>> Changes to the libc-kernel interface are relatively uncommon but 
>>> should be expected to happen every so often.
>>>
>>> I see that you're using nlipkg zones, which aren't kept in sync with 
>>> the global zone.   Did you update the zones as well?   If not, I'd 
>>> expect most exec*() calls to fail with an EINVAL unless the new 
>>> fourth argument to the underlying syscall is zero by chance.
>>>
>>> Update the zones along with the global zone and they should start 
>>> working again.  Just updating all the variants of libc.so.1 to match 
>>> the global zone should be enough to get you up & limping again.
>>>
>>>                     - Bill
>>>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> oi-dev mailing list
>>> oi-dev at openindiana.org
>>> https://openindiana.org/mailman/listinfo/oi-dev
>>
>> Hello!
>>
>>
>> Thanks for the explanation.
>>
>> The zone that did not work was updated a day before the global zone, 
>> BE is "openindiana-2024:01:24" within the zone,
>> so the exec* stuff should be in..
>> But wait the merge was at timestamp  2024-01-23T19:29:54.000Z so 
>> maybe not..
>>
>> Current BE of the global zone is "openindiana-2024:01:02" so the 
>> commit is not in..
>>
>> I will try to update the ng-zones first and after that the global zone,
>> or maybe better change the zones to ipkg first,
>> or just update the zones now and reboot the global zone to the 
>> (currently unused) existing BE  "openindiana-2024:01:26" for which 
>> the problem exists..
>
> what would be helpful for confirming what happened would be the 
> illumos-gate.info.git-rev property from the /system/library package in 
> each BE of the global zone and the impacted zone.
>
> Try running:
>
> $ pkg contents -H -o value -a name=illumos-gate.info.git-rev 
> /system/library
>
> for the global zone, and
>
> $ pkg -R <alternate-root> contents -H -o value -a 
> name=illumos-gate.info.git-rev /system/library
>
> for each relevant alternate BE and zone root.
>
>                         - Bill
>
>
>
> _______________________________________________
> oi-dev mailing list
> oi-dev at openindiana.org
> https://openindiana.org/mailman/listinfo/oi-dev

Hello!

This is the running combination:

Global zone:

$ pkg contents -H -o value -a name=illumos-gate.info.git-rev 
/system/library
8361acf58a302751348aac091ab09484f3ecfb8c

ng zone

# pkg contents -H -o value -a name=illumos-gate.info.git-rev 
/system/library
32c2a899b6e4eab014fd5243af6c011a84dc103f


When i activate the newer BE on the global zone,
where i cant't zlogin to the ng zone, we have this value:

$ pkg contents -H -o value -a name=illumos-gate.info.git-rev 
/system/library
d5a816590ba1bb4affbf62f0113d75baf7f83177


Regards,
Stephan