[OpenIndiana-discuss] Split-root installations
Peter Tribble
peter.tribble at gmail.com
Wed Nov 27 10:03:49 UTC 2013
On Wed, Nov 27, 2013 at 12:51 AM, Jim Klimov <jimklimov at cos.ru> wrote:
> On 2013-11-26 17:25, Jim Klimov wrote:
>
>> On 2013-11-17 03:50, Jim Klimov wrote:
>>
>>> For years I've mentioned "split-root" installations of Solaris-like
>>> systems in such a way that the root filesystem (the BE) is represented
>>> by several datasets, such as a split-off /usr dataset. Also there may
>>> be some datasets shared between boot environments, such as the sinks
>>> for logs and crashdumps, and not all of these are required to live on
>>> the rpool at all. There are cases when all such tweaks may be desirable.
>>>
>>
>> WARNING
>>
>> As discussed in another thread, it was discovered that the SMF methods
>> for network/physical (both :default and :nwam) use many programs from
>> /usr, and are executed before the /usr filesystem is actually mounted
>> in case of a split-root installation. This tanks the NWAM setups, but
>> the default ones (based on static files in /etc) succeeds for both DHCP
>> and completely static addressing.
>>
>> I hope to fix this somehow, but a head-on approach failed: the "root"
>> filesystem and other FS services depend on svc:/system/identity:node
>> (indirectly via svc:/system/metainit:default) and that depends on
>> svc:/network/physical... loop and maintenance... Disabling the metainit
>> service does not help fix the dependency_cycle condition :\
>>
>> One approach that would work - a hacky one - is to add an init-script
>> into /etc/rc2.d to "svcadm restart physical" for the currently enabled
>> (and maybe even "online") instance of this service.
>>
>
> So far the best shot, both compact and effective, was fixing the
> filesystem/root service like in the patch below. Are there any
> fundamental objections to this sort of fix beside possible lack
> of style? Getting the hands wet in the guts of the system anyhow,
> can't keep them clean... ;)
>
General comment - attempting to hack workarounds to broken
systems at the wrong place in the startup sequence seems like
a really bad idea.
> --- /lib/svc/method/fs-root.orig 2013-11-17 03:21:47.000000000 +0100
> +++ /lib/svc/method/fs-root 2013-11-27 00:35:29.119717438 +0100
> @@ -26,6 +26,12 @@
> # Make sure that the libraries essential to this stage of booting can be
> found.
> LD_LIBRARY_PATH=/lib; export LD_LIBRARY_PATH
>
> +NET_PHYS_FMRI="svc:/network/physical:default"
> +NET_NWAM_FMRI="svc:/network/physical:nwam"
> +NET_IPTUN_FMRI="svc:/network/iptun:default"
> +SVCADM=/usr/sbin/svcadm
> +SVCS=/usr/bin/svcs
> +
> libc_mount() {
> #
> # If there is an optimized libc available in /usr that fits this
> @@ -239,6 +245,19 @@
> #
> /usr/sbin/devfsadm -I -P
>
> +#
> +# After possibly finding new drivers, and after getting an /usr, redo
> network
> +#
>
How can adding /usr change the validity of the network
configuration? There shouldn't be any new drivers (or
network booting wouldn't work at all), so there should be
nothing to do here.
If networking is broken at this stage (and it shouldn't be) then
that's a bug elsewhere - fixing fs-root is the wrong place.
> +for _SVC in $NET_PHYS_FMRI $NET_NWAM_FMRI $NET_IPTUN_FMRI; do
> + _STATE="`$SVCS -H -o state $_SVC`"
> + echo "$_SVC: $_STATE" > /dev/msglog
> + case "$_STATE" in
> + online*) $SVCADM restart $_SVC ;;
>
restart = disable and enable. If your /usr is nfs mounted,
what happens when you disable networking?
> + maintenance)$SVCADM clear $_SVC ;;
>
Again, if something is in maintenance then you need to fix the
underlying cause. (And you don't check to see whether the
clear was successful, nor do you wait for it to come online
to avoid possible race conditions.)
> + esac
> +done
> +unset _SVC _STATE
> +
>
libc_mount
>
> exit $SMF_EXIT_OK
>
>
>
> While the patches for the networking methods themselves did work
> somewhat, they are large and may be not so apparent for review.
> And this one just plain works nicely and predictably.
>
> Possibly, the "physical:default" patch to untie it from the /usr
> filesystem should also make it into the distro, so that a broken
> server's root-only environment can have its static-file-defined
> networking, how-ever that might help in repairs. And it is trivial.
>
>
> --- /lib/svc/method/net-physical.orig-oi151a8 2013-07-21
> 16:35:53.000000000 +0200
> +++ /lib/svc/method/net-physical 2013-11-26 19:55:04.811003487 +0100
> @@ -38,10 +38,13 @@
> #
> smf_configure_ip || exit $SMF_EXIT_OK
>
> -
> # Make sure that the libraries essential to this stage of booting can be
> found.
> LD_LIBRARY_PATH=/lib; export LD_LIBRARY_PATH
>
> +# Even if the program is not found explicitly or as a builtin, use some
> +# valid value in the variable, to have reasonable errors at least...
> +[ -x /usr/bin/cut ] && CUT="/usr/bin/cut" || { CUT=cut; builtin cut; }
> +
> smf_netstrategy
>
> if smf_is_globalzone; then
> @@ -314,9 +317,9 @@
> # enforced on the interface by the global zone.
> #
> for showif_output in `/sbin/ipadm show-if -p -o ifname,state,current`; do
> - intf=`echo $showif_output | /usr/bin/cut -f1 -d:`
> - state=`echo $showif_output | /usr/bin/cut -f2 -d:`
> - current=`echo $showif_output | /usr/bin/cut -f3 -d:`
> + intf=`echo $showif_output | $CUT -f1 -d:`
> + state=`echo $showif_output | $CUT -f2 -d:`
> + current=`echo $showif_output | $CUT -f3 -d:`
> if [[ "$state" != "disabled" && $current != *Z* ]]; then
> #
> # skip if not a persistent interface, or if it should get
> IP
>
>
> The bulk of the patch (including fixes to net-nwam and net-iptun, which
> I don't very much insist on if the above patches are acceptable) are
> here:
> http://wiki.openindiana.org/download/attachments/27230229/
> net-physical-usr.patch
>
>
> //Jim
>
>
> _______________________________________________
> OpenIndiana-discuss mailing list
> OpenIndiana-discuss at openindiana.org
> http://openindiana.org/mailman/listinfo/openindiana-discuss
>
--
-Peter Tribble
http://www.petertribble.co.uk/ - http://ptribble.blogspot.com/
More information about the OpenIndiana-discuss
mailing list