[OpenIndiana-discuss] OI rpool on USB flash in N40L?

[Jim Klimov]

On 2013-01-27 15:08, Reginald Beardsley wrote:
> Several people have suggested FreeBSD for the OS for my N40L based ZFS server and using the internal USB socket w/ a flash drive for the OS  However, after trying FreeBSD 9.0 installed to an HDD, I'm not enthused about having to relearn FreeBSD admin. And I'm not GUI admin inclined.  I'd prefer to stick w/ what I'm most familiar with.
>
> However, the idea has considerable attraction.  Has anyone done such a thing w/ OI?  A bunch of stuff would need to move out of rpool so that it was almost R/O, but I don't see any major obstacles.  I assume the LiveCD uses a memory filesystem for these, so I'd expect it to be pretty straight forward.
>
> This still leaves the single point of failure for the root pool, but copies=2+ might give enough warning to avoid a crisis.  Creating a replacement from a disk image using dd would be pretty painless.
>
> Any reports of success or failure?  I've seen rather poor USB flash performance, but that might be hardware specific.

You might successfully start with the OI LiveUSB - perhaps ultimately 
making your similar media with the distro constructor, including all
the installed software you need. What I did do successfully is saving
the "jack" home directory on the same LiveUSB flash, so that it can
be used as recovery media for whenever my rpool fails, and I have all
needed binaries and scripts handy.

You can also learn some methodological matters from EON Storage distro,
although that was based on OpenSolaris SXCE (SVR4).

In general, if you use a flash (USB, CF, some SSDs with limited wear
resilience) media you do want to keep unnecessary writes out of it.
This involves relocating likely /var/tmp, /var/adm, /var/log, maybe
/var/svc/log, much of /var/spool and /var/mail, the dump+swap volumes,
disabling atime... maybe dynamic pieces of /etc... Then track writes
with zpool iostat and dtrace or lsof, as needed.

OI may not play nicely by default with relocated (split-off) rootfs
portions, see my notes and workarounds in bugtracker:

https://www.illumos.org/issues/829

While my notes and patch (https://www.illumos.org/issues/829#note-15)
are inclined towards storing the split-off datasets in the same rpool,
nothing IMHO fundamentally precludes you from storing them on datasets
in a different pool, as long as you can import it during boot (perhaps,
forcing the import regardless of zpool.cache by the means of a helper
SMF service as a dependency to satisfy svc:/filesystem/minimal), if
you relocate some FSes too important for the full boot - i.e. system
logs. If there is a problem with that pool, however, then you might
want to use some tmpfs-satisfied mounts for the logs/temps, or even
let the flash be (ab)used for a while.

HTH,
//Jim Klimov