[OpenIndiana-discuss] Split-root installations

[Jim Klimov]

On 2013-11-26 17:25, Jim Klimov wrote:
> On 2013-11-17 03:50, Jim Klimov wrote:
>>    For years I've mentioned "split-root" installations of Solaris-like
>> systems in such a way that the root filesystem (the BE) is represented
>> by several datasets, such as a split-off /usr dataset. Also there may
>> be some datasets shared between boot environments, such as the sinks
>> for logs and crashdumps, and not all of these are required to live on
>> the rpool at all. There are cases when all such tweaks may be desirable.
>
> WARNING
>
> As discussed in another thread, it was discovered that the SMF methods
> for network/physical (both :default and :nwam) use many programs from
> /usr, and are executed before the /usr filesystem is actually mounted
> in case of a split-root installation. This tanks the NWAM setups, but
> the default ones (based on static files in /etc) succeeds for both DHCP
> and completely static addressing.
>
> I hope to fix this somehow, but a head-on approach failed: the "root"
> filesystem and other FS services depend on svc:/system/identity:node
> (indirectly via svc:/system/metainit:default) and that depends on
> svc:/network/physical... loop and maintenance... Disabling the metainit
> service does not help fix the dependency_cycle condition :\
>
> One approach that would work - a hacky one - is to add an init-script
> into /etc/rc2.d to "svcadm restart physical" for the currently enabled
> (and maybe even "online") instance of this service.

So far the best shot, both compact and effective, was fixing the
filesystem/root service like in the patch below. Are there any
fundamental objections to this sort of fix beside possible lack
of style? Getting the hands wet in the guts of the system anyhow,
can't keep them clean... ;)

--- /lib/svc/method/fs-root.orig   2013-11-17 03:21:47.000000000 +0100
+++ /lib/svc/method/fs-root     2013-11-27 00:35:29.119717438 +0100
@@ -26,6 +26,12 @@
  # Make sure that the libraries essential to this stage of booting can 
be found.
  LD_LIBRARY_PATH=/lib; export LD_LIBRARY_PATH

+NET_PHYS_FMRI="svc:/network/physical:default"
+NET_NWAM_FMRI="svc:/network/physical:nwam"
+NET_IPTUN_FMRI="svc:/network/iptun:default"
+SVCADM=/usr/sbin/svcadm
+SVCS=/usr/bin/svcs
+
  libc_mount() {
         #
         # If there is an optimized libc available in /usr that fits this
@@ -239,6 +245,19 @@
  #
  /usr/sbin/devfsadm -I -P

+#
+# After possibly finding new drivers, and after getting an /usr, redo 
network
+#
+for _SVC in $NET_PHYS_FMRI $NET_NWAM_FMRI $NET_IPTUN_FMRI; do
+       _STATE="`$SVCS -H -o state $_SVC`"
+       echo "$_SVC: $_STATE" > /dev/msglog
+       case "$_STATE" in
+           online*)    $SVCADM restart $_SVC ;;
+           maintenance)$SVCADM clear $_SVC ;;
+       esac
+done
+unset _SVC _STATE
+
  libc_mount

  exit $SMF_EXIT_OK



While the patches for the networking methods themselves did work
somewhat, they are large and may be not so apparent for review.
And this one just plain works nicely and predictably.

Possibly, the "physical:default" patch to untie it from the /usr
filesystem should also make it into the distro, so that a broken
server's root-only environment can have its static-file-defined
networking, how-ever that might help in repairs. And it is trivial.


--- /lib/svc/method/net-physical.orig-oi151a8   2013-07-21 
16:35:53.000000000 +0200
+++ /lib/svc/method/net-physical        2013-11-26 19:55:04.811003487 +0100
@@ -38,10 +38,13 @@
  #
  smf_configure_ip || exit $SMF_EXIT_OK

-
  # Make sure that the libraries essential to this stage of booting can 
be found.
  LD_LIBRARY_PATH=/lib; export LD_LIBRARY_PATH

+# Even if the program is not found explicitly or as a builtin, use some
+# valid value in the variable, to have reasonable errors at least...
+[ -x /usr/bin/cut ] && CUT="/usr/bin/cut" || { CUT=cut; builtin cut; }
+
  smf_netstrategy

  if smf_is_globalzone; then
@@ -314,9 +317,9 @@
  # enforced on the interface by the global zone.
  #
  for showif_output in `/sbin/ipadm show-if -p -o ifname,state,current`; do
-       intf=`echo $showif_output | /usr/bin/cut -f1 -d:`
-       state=`echo $showif_output | /usr/bin/cut -f2 -d:`
-       current=`echo $showif_output | /usr/bin/cut -f3 -d:`
+       intf=`echo $showif_output | $CUT -f1 -d:`
+       state=`echo $showif_output | $CUT -f2 -d:`
+       current=`echo $showif_output | $CUT -f3 -d:`
         if  [[ "$state" != "disabled" && $current != *Z* ]]; then
                 #
                 # skip if not a persistent interface, or if it should 
get IP


The bulk of the patch (including fixes to net-nwam and net-iptun, which
I don't very much insist on if the above patches are acceptable) are
here:
http://wiki.openindiana.org/download/attachments/27230229/net-physical-usr.patch

//Jim