[Caiman] [OpenIndiana Distribution - Feature #829] Support installing OpenIndiana into a hierarchy of datasets

[illumos project]

Issue #829 has been updated by Jim Klimov.


Much of the research done by me on this subject matter has culminated in a Wiki publication http://wiki.openindiana.org/oi/Advanced+-+Split-root+installation which describes the copy-pastable steps to transform a standard Solaris-like installation (with OI as the example and testbed) into a split-root one. This article includes all the missing pieces needed to complete this quest manually, which (and more) I posted as bugs, and hope that the solutions will make it upstream so that the setup can be done and maintained out of the box:
* #4352 - the relevant patches for fs-* SMF methods to mount such roots more reliably than currently existing ones, including a *limited* solution to #997 to enforce overlay-mounting for components of the root filesystem hierarchy,
* #4351 - the procedure to bring ksh93 and its dependency libraries into the root filesystem as /sbin/sh (so that /usr can be safely split off),
* #3569 - the procedure to replicate custom ZFS attributes after a "beadm clone" (such as compression for child datasets of the rootfs), including #4355 - to do this in particular when "pkg" creates the clones.

Issues #4353 and #4354 track proposed improvements to the Caiman installer with steps needed to support such installations as a hands-off automated procedure, and are not part of that article.
----------------------------------------
Feature #829: Support installing OpenIndiana into a hierarchy of datasets
https://www.illumos.org/issues/829

Author: Jim Klimov
Status: Feedback
Priority: Normal
Assignee: OI Caiman
Category: 
Target version: 
Difficulty: Medium
Tags: needs-triage


Many of SXCE and Solaris 10 systems I maintain were installed using a hierarchy of datasets for the root ZFS pool.

This approach separates the "/" root filesystem dataset from "/opt", "/var" and "/usr" (and optionally "/usrlocal") so that these 3 or 4 datasets can be kept as root's children and compressed on-disk while the "/" filesystem is not-compressed and thus well-supported by GRUB to boot.

Also some of the user/log data from "/var" is shared between boot environments by being dedicated (compressed) datasets. This allows to switch back and forth between BEs while upgrading or testing updates, and keep the same track of system historical data, etc.

NOTE that sharing "/var/tmp" like this proved to be a bad idea for some certain reasons (mountpoint may be written to by i.e. ipfilter service, which forbidds mounting the shared FS later in the boot sequence).

Granted, a separate "/usr" system is sometimes a hassle during single-user mode repairs, but when we're tight with space on smallish boot media, the compressed "/usr" is more of a bonus than a problem - workarounds are documented internally. And with latest releases the automounting worked properly enough...

While not quite officially supported, this approach was discussed on opensolaris forum and even worked transparently with LiveUpgrade - LU properly cloned the root's children when making a new BE, and attached existing shared datasets.

Here's a layout example for my current testbed system, which will hopefully work after the reboot (as older systems did well):

root at openindiana:~# df -k | egrep 'ROOT|SHARED|zfsroot|/a'
rpool/ROOT/oi_148a    14088223    190948  13897275   2% /zfsroot
rpool/ROOT/oi_148a/usr
                      14006968    109693  13897275   1% /zfsroot/usr
rpool/ROOT/oi_148a/opt
                      13899058      1783  13897275   1% /zfsroot/opt
rpool/ROOT/oi_148a/var
                      13942656     45381  13897275   1% /zfsroot/var
rpool/ROOT/oi_148a/usrlocal
                      13897306        31  13897275   1% /zfsroot/usrlocal
rpool/SHARED/var/adm    512000        66    511934   1% /zfsroot/var/adm
rpool/SHARED/var/log  13897327        52  13897275   1% /zfsroot/var/log
rpool/SHARED/var/mail
                      13948475        32  13948443   1% /zfsroot/var/mail
rpool/SHARED/var/crash
                      13897306        31  13897275   1% /zfsroot/var/crash
rpool/SHARED/var/cores
                        102400        31    102369   1% /zfsroot/var/cores

I propose that OpenIndiana installer and other tools at least expect the possibility of non-monolithic root FS. 

Better yet - the interactive and automated installers should offer to create such hierarchy if the user desires (i.e. later releases of Solaris 10 did allow to create a separate /var dataset, although not shared between BEs, which was a step in this direction).

Coupled with my bug 828 (letting OI install into existing rpools), it would be nice if the installer proposed to update such split roots as well.

Thanks,
//Jim Klimov



-- 
You have received this notification because you have either subscribed to it, or are involved in it.
To change your notification preferences, please click here: http://www.illumos.org/my/account