[oi-dev] heads up: zones not available after pkg update in global zone

[Bill Sommerfeld]

On 1/26/24 13:26, Stephan Althaus wrote:
> On 1/26/24 18:42, Bill Sommerfeld via oi-dev wrote:
>> On 1/26/24 01:17, Stephan Althaus wrote:
>>> Maybe its only here and a local problem, but who knows ...
>>>
>>>
>>> I did a pkg update this morning, and the zones are not available.
>>>
>>> # zoneadm list -cv
>>>    ID NAME             STATUS     PATH BRAND    IP
>>>     0 global           running    / ipkg     shared
>>>     1 mail             running    /zones/mail nlipkg   excl
>>>     2 www              running    /zones/www nlipkg   excl
>>>
>>> The zones are booted, but zlogin is not possible, nor are the zones 
>>> reachable via ssh.
>>
>> Looks like execing the shell failed.
>>
>> Upstream illumos-gate just made an incompatible change to the private 
>> interface between libc and the kernel used by the exec*() family of 
>> functions.
>>
>> commit 2b395c3c2a39cdc00f9fe7ac497795bd112f7663
>> Author: Andy Fiddaman <illumos at fiddaman.net>
>> Date:   Sat Jan 20 10:47:38 2024 +0000
>>
>>     5798 fexecve() needed per POSIX 2008
>>     Portions contributed by: Garrett D'Amore <garrett at damore.org>
>>     Reviewed by: Dan Cross <cross at oxidecomputer.com>
>>     Reviewed by: Bill Sommerfeld <sommerfeld at hamachi.org>
>>     Approved by: Gordon Ross <gordon.w.ross at gmail.com>
>>
>> Changes to the libc-kernel interface are relatively uncommon but 
>> should be expected to happen every so often.
>>
>> I see that you're using nlipkg zones, which aren't kept in sync with 
>> the global zone.   Did you update the zones as well?   If not, I'd 
>> expect most exec*() calls to fail with an EINVAL unless the new fourth 
>> argument to the underlying syscall is zero by chance.
>>
>> Update the zones along with the global zone and they should start 
>> working again.  Just updating all the variants of libc.so.1 to match 
>> the global zone should be enough to get you up & limping again.
>>
>>                     - Bill
>>
>>
>>
>>
>>
>> _______________________________________________
>> oi-dev mailing list
>> oi-dev at openindiana.org
>> https://openindiana.org/mailman/listinfo/oi-dev
> 
> Hello!
> 
> 
> Thanks for the explanation.
> 
> The zone that did not work was updated a day before the global zone, BE 
> is "openindiana-2024:01:24" within the zone,
> so the exec* stuff should be in..
> But wait the merge was at timestamp  2024-01-23T19:29:54.000Z so maybe 
> not..
> 
> Current BE of the global zone is "openindiana-2024:01:02" so the commit 
> is not in..
> 
> I will try to update the ng-zones first and after that the global zone,
> or maybe better change the zones to ipkg first,
> or just update the zones now and reboot the global zone to the 
> (currently unused) existing BE  "openindiana-2024:01:26" for which the 
> problem exists..

what would be helpful for confirming what happened would be the 
illumos-gate.info.git-rev property from the /system/library package in 
each BE of the global zone and the impacted zone.

Try running:

$ pkg contents -H -o value -a name=illumos-gate.info.git-rev /system/library

for the global zone, and

$ pkg -R <alternate-root> contents -H -o value -a 
name=illumos-gate.info.git-rev /system/library

for each relevant alternate BE and zone root.

						- Bill