[OpenIndiana-discuss] Split-root installations

[James Carlson]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 11/27/2013 6:12 PM, Jim Klimov wrote:
> Hello James,
> 
> There are so many well-phrased sentences that I just can't snip
> out the few I'd respond to ;) And thanks for the historical
> insights and rationales, that is much appreciated too :)
> 
>> You can't buy disk drives small enough to make sense out of a 
>> split /usr.
> 
> Yes you can =)
> 
> Regarding "small disks" - yes, you can get a, say, 20GB Intel SLC 
> drive as a nice reliable media for your rpool. Why waste 3Gb of it 
> on a default installation (and close to that on each upgrade) when 
> with gzip-9 it takes just 1Gb?

20GB is pretty big compared to the actual needs of / and /usr on most
systems.  I think you're proving the point I was trying to make: in
the days of megabyte-sized drives that cost thousands (or tens of
thousands) of dollars, figuring out what would go on the root disk and
what would go on the usr and var and other disks was a worthwhile use
of someone's limited time and effort.

Today it's really not.  That 20GB hard drive is maybe $50 to $100.  So
what if you waste some of it by storing only a few GB of stuff?  Use
the "wasted" space for multiple BEs if you like.  Or just chalk it up
to the money required to run a real system.  Life's way too short to
be optimizing a $50 component.

> Even with larger SSDs, I'd rather spend the 2GB saved by
> compression off of each BE on something more useful than the
> bloated compressable files - be it a fast scratch area for coding,
> compilations, uploading, or just bigger L2ARC partition. So far the
> SSDs do cost a considerable amount of money ($/Gb), especially the
> good ones, and I'd like the expensive investment to work off every
> penny overpaid for it and not slack around storing files 3 times
> larger than they need be ;)

I guess we've got different optimization points at work.  Even at
(say) $5/GB, I'm not interested in optimization if it means higher
complexity that will undoubtedly result in a higher rate of failures,
and higher cost of other goods -- all the extra testing and
engineering required to support it won't be "free."

For bulk data, I can get storage that's pennies per GB, and that does
the job.

As for L2ARC, I wouldn't put that on the same device as the system
boot.  I'd rather run without an L2ARC if that's the choice.  Is that
what you're doing?

> Maybe I chimed in early enough, that the system did not decay very
> far from the state where it supported separate /usr filesystems,
> and there are only a few exceptional cases, and there is still
> support for it in the "trunk" codebase.

I was referring to the work we were doing at Sun when I was there,
from 2000 to 2009.

>> having a supported split /usr is a huge tax on every single 
>> project that integrates
> 
> There is really not much code, to my knowledge, that *should* work 
> before all the local filesystems have been mounted. This is a
> class of pretty low-level routines which must initialize the
> operating environment. If we could put the network core-filesystem
> support aside, networking could certainly start after the local
> filesystem mounting.

If I recall correctly, a fair bit of the mess was there to support PXE
boot and the absurdly complex Solaris upgrade process.  But it's been
a long time since those days.

You're right that if /usr is local, then there's not much to do before
it mounts.  Rather than moving random stuff into root or rewriting a
broken service into ksh93-isms, I think it makes a lot more sense to
look at why the service is starting too early and putting an end to that.

>>> My worry here is that you're adding complexity to an already 
>>> complex system; instead, we need significant simplification.
>> 
>> Another big +1 on that.
> 
> I am sorry to hear this... I guess these would be two major nails
> in the coffin for an attempt to RTI into the illumos-gate and
> distros?

I don't read the discussion that way.  I'm just confused about why the
changes are a good idea and why some other analysis of the problem
isn't needed.  I presume that someone filing that RTI would have good
answers, but I'm not that person.

In particular, I don't think it's at all wise to have services that
start or restart other services as part of the normal boot sequence.
That path leads to havoc.  There are special points during the install
process when trickery like that is needed, but they're by nature
"special."  If the svcadm insertions you've proposed are really
required to make it work, then I think some higher-level rethink is
needed.

> Alternately, much (not all) of this hassle could become irrelevant 
> if gzip-9 support just came for rootfs support :)

Well, gee, that sounds like a more practical target to me!  The nice
thing about doing that is that the work would have much narrower
impact than a rototilling of the system services and /-vs-/usr
contents, and might even be applicable to other projects.

- -- 
James Carlson         42.703N 71.076W         <carlsonj at workingcode.com>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (MingW32)

iEYEARECAAYFAlKWjNEACgkQ6TIHA1NkzvBHFwCfRnU5q6qsoypPr7ppbj/xCUJR
/f0An0Fb8XSzlESnUoxJEUXZsbyYI1up
=jO7C
-----END PGP SIGNATURE-----