[OpenIndiana-discuss] Split-root installations

[Jim Klimov]

Another piece of research on the split-root subject: the
svc:/system/device/mpxio-upgrade:default service seemingly
takes care of mounting (non-ZFS) root and /usr filesystems
read-only (and only with devices specified in /etc/vfstab)
in order to update metadevice and /etc/vfstab configs, and
then disables itself.

This service, as well as filesystem/root, depends on service
svc:/system/metainit:default with mode "optional_all", so an
installation can "svcadm disable metainit" for ZFS-rooted
systems, and then networking and filesystems can race to
execute. This is of course not a solution...

It may happen so (but as with any race - is not guaranteed)
that /usr will be available by the time other methods make
calls into it. Also, if SVM is used on a particular system
(i.e. for accessing legacy datasets, or non-zfs dump, or swap),
and the "metainit" service is reenabled to start at boot,
this would restore the current SMF dependency chain - of
filesystems-after-network.

1) However, a particular system with known local-storage-based
roots can probably safely remove the dependencies on metainit
and mpxio-upgrade from filesystem/root, and then establish
dependencies from several networking services on anywhere
from fs-root to fs-minimal (for NWAM), as applicable. Then
the nodename and metadevice stuff can run after the root
hierarchy got mounted, if the admin knows it to be valid
(no networked or metadeviced root-fs components).

2) I think one more valid approach to unroll these dependencies
via SMF in a packageable manner has emerged, and a rather apparent
one: to move (or duplicate, or invoke) the code from fs-root which
mounts a zfs-based /usr filesystem into a service of its own, on
which consumers of the /usr namespace would depend (optional_all).

At start this service would check if current root is zfs, and
if a child dataset or legacy-mounted ZFS /usr are known and
available - it would mount the dataset if yes. Otherwise it
would exit without an error. As a result, the networking
scripts in my split-zfs-based-root cause would be guaranteed
to have a /usr before they run.

It would (should) have no impact on systems that use monoroots
on ZFS, or that use other roots (networked, metadevice, etc.) -
these would work or fail the same as they do today.

3) Similarly, such a service can mount ZFS-based datasets of
the rest of the root hierarchy if available (/var, children
of bootfs, SHARED/*) and as a result of this, even the NWAM
method on systems with local storage would have a complete
environment to work in (for its LDAP/NIS interaction), all
without major rehaul of SMF dependencies and method code.

But in this extended case there is a possible though improbable
loophole: if some parts of the operating environment including
the rootfs are mounted from ZFS, but some major components
like /var work from nfs/cachefs/ufs/... and then some datasets
like /var/adm would be mounted on top of that. A script that
only mounts a ZFS hierarchy in order to avoid dependencies
on networking and metadevices would apparently ignore these
other options; at most it can detect them in /etc/vfstab and
stop mounting stuff under the involved mountpoint (this would
come in later via filesystem service chains that exist today).

And the current filesystem service methods should need to check
that they don't mount the same (zfs) filesystem twice, so as
to not bail out on "zfs mount" errors due to this.



Does any of this - (1) or (2)-and-maybe-(3) - also sound like a
bad solution? If so - why? If it is acceptable, then it is at
least a solution that I can script up myself (unlike plugging
gzip-9 in GRUB for example) :)


//Jim Klimov