[oi-dev] Desktop Illumos Still Matters

Tue Sep 4 21:38:19 UTC 2012

On Tue, Sep 4, 2012 at 1:25 PM, Andrew M. Hettinger
<AHettinger at prominic.net> wrote:
> My thoughts. Remember, they are probably only worth what you paid for them!
> ;p
>
> Nick Zivkovic <zivkovic.nick at gmail.com> wrote on 09/01/2012 10:42:14 AM:
>
>
>>
>> Yes. I am more interested in contributing drivers and the like. As far
>> as packages go, to be honest, I've experienced torture at the hands of
>> IPS (though that could very easily be my fault), and am reluctant go
>> near it. For example I tried an image-update and it failed. So I am
>> stuck on OI-147 until I backup-reinstall-import to OI-151a.
>>
>> I think packages are a high priority, but not as high as making sure
>> the latest illumos-gate can build and run on a modern desktop. For
>> example, I can't get SmartOS running on a thinkpad or my desktop
>> computer. Somewhere in June 2012, a bug was introduced that prevents
>> the illumos kernel from booting. If I had been building and testing
>> the latest source, that bug could probably have been caught before it
>> got buried in a mountain of commits. Now, I image, it is like finding
>> a needle in a haystack.
>>
>> I am willing to assist with packages, but my time is limited, and I
>> think it is more important to direct my effort to building
>> illumos-gate and writing drivers. Also, making packages is still a
>> black art to me, and wouldn't know where to start.
>
> One of the biggest issues here isn't that packages are particularly HARD to
> make with IPS (they aren't). It's that there are about three different
> approaches to it, and we need to get that standardized. Many of the packages
> are tied up in the consolidations, which DO seem to have a high barrier to
> entry. I considered putting together a source-juicer-like self-service
> system for building packages. If I can get the time, I'll revisit that. It
> would make my (and everyone else's, I think) life easier.

Ok this sounds very useful. I will investigate consolidations, and see
what can be done to lower that barrier.

>
>
>> But since we are already on the topic of packages, Adam, do you think
>> there is a way to make it less painful, more consistent? I'm _not_
>> talking about extreme measures like changing from IPS to
>> [DEB/RPM/SVR/etc]. I'm wondering if we could 1) make IPS easier to use
>> by documenting stuff in an easily accessible way [the man pages aren't
>> very helpful]
>
> The wiki would be an ideal place for this to happen. Frankly, I haven't see
> much trouble with the man pages from a user perspective, but from the
> developer's side, it could definitely use some work. Much of this was
> documented in the OS.o site, but we need to not depend on that.
>
>
>> 2) document every single IPS failure and either fix the
>> packages or the IPS code (depend on what caused the failure), and
>
> First thought here is that it needs to be in the bug tracker, but that may
> not be easily accessible either. Maybe a sub-page on the wiki?

Either should be fine. FreeBSD records their ports build failures on
their wiki. I think Gentoo recorded this on a bug tracker. Wiki is
probably easiest.

>
>
>> 3)
>> have IPS install all userland libs to a zfs dataset named rpool/ips or
>> rpool/pkgs; this way, we can zfs-send these datasets, and snap-shot
>> them, and clone them, without pulling in the rest of the file-system
>> heirarchy. This would make my bitterness toward IPS reduce
>> significantly. This way, you can migrate different user-land configs
>> between systems. Also, an easy way to do updates across a multitude of
>> systems. One can share their binaries and packages via zfs-send,
>> because they won't destroy an existing system's /usr /bin and so
>> forth. Also, OI would benefit tremendously from offering pre-made NG
>> zones on the web-site, available for downloading and running. In fact,
>> we could use Zones as a delivery mechanism for things like an Illumos
>> build-environment. An NG zone can contain a working and sandboxed
>> version of firefox. Zones are a great technology that can make the
>> system more attractive amateur power users who may become programmers
>> some day (like I did). Multiple ways of sharing pre-compiled binaries
>> can only help OI and Illumos. In fact I can see people sharing
>> datasets with packages via bit-torrent. Plus, incremental send/recv is
>> a huge benefit.
>
> Back in the bad-old-days, (if memory serves) a basic copy of userland was
> kept in /bin and /sbin that did not depend on anything. This was done to
> allow you to NFS mount everything else. IIRC the decision was made to go
> ahead and make them dynamicly linked because noone NFS mounts them anymore
> anyway, and it meant we had to keep both a simplified and full version of
> the programs. I think this will encounter many similar issues as that. If we
> are going back down this road, we could consider just delivering a /bin and
> /sbin we can use as you propose. It would have the advantage of bringing
> back this lost (albit rarely used) functionality.
>
> That said, there is nothing stopping anyone from delivering a basic userland
> in /rpool/pkgs, although I would suggest using an alternate mount point in
> /usr (/usr/zdu or some-such? solaris has a long history of delivering
> alternate userlands in filesystems off of /usr).

If dynamic linking works on binaries that are on different data sets I
see no problem, as long as those datasets' paths are in PATH. But I
could be wrong, I should check.

>
>
>> We might even be able to integrate a window manager (like i3 or dwm)
>> so that switching virtual desktop, actually switches to another zone.
>
> Can you do this in zones? Frankly, all my other zones have always been on
> the commandline for build environments and such, so I don't know. Zones are
> already pretty sweet in my book, but if something like this is already
> there, that's a whole other level.

Well, you can use Xorg on computer A to view an X11 client on computer
B over a network. The same should apply to Zones. Most of the changes
need to happen in the window manager to automatically connect
"remotely" to a zone's Xorg server/client.

It's not a zones feature, as much as it is a WM feature.

>
>
>> What kind of changes to IPS are OI willing to accept? I am willing to
>> test and improve a lot of code. As I said, I dislike IPS. But I am
>> willing to help make it better and more usable.
>
> It might be good if you could say what you have in mind. I don't believe we
> are particularly wedded to what Oracle does anymore, so if you have ideas
> speak up!

Well, I intend to fix errors as I run into them. And I am still
thinking of some things related to NGZ's that I'll test out.

>
>
>> Also, a major problem with IPS is that Sun encouraged people to use it
>> to _consume_ packages, but they didn't encourage people to _create_
>> packages. We need a self-fueling ecosystem of packages.
>
> That's not completely true. Sun did encourage package creation, that's what
> Source Juicer was about. It was marginally effective, there where a number
> of us outsiders who did contribute packages that way. It was just soo damn
> easy, it was almost harder to NOT contribute! I strongly think we need
> something like that back. I'll see what I can hack up, and see if we are
> interested in officially supporting something like that when I am done.
>
> OTOH the way they handled consolidations is a nightmare that we inherited,
> but I'm not sure I know how to fix it. Many wiser then I have tried, and it
> seems to always end in tears.

Please be specific. What was nightmarish about consolidations. Or
point me to an archive discussion or something. I've not been involved
with IPS before.

>
>
>> I also think that SmartOS's diskless boot model is great. I think that
>> booting from disk is great too. Shouldn't OI support both? I'm willing
>> to contribute to this.
>
> Personlly, I'm not interested in a diskless boot model. I think it's highly
> niche, and I think SmartOS does a fine job there. In purposing the work for
> that, I have to ask, what compelling element do we bring to the table?
>
>> I know these ideas come from SmartOS to some extent, but they are
>> great ideas that could make OI better! Making a new distribution is
>> one way to try to make things better. But I think a metamorphosis in
>> the OI distro will be more effective. I want the many Illumos distros
>> to be held up as an example of triumphant collaboration, 5 years from
>> now. But that will happen only if we avoid going down the path of
>> NIH-inspired suicide.
>
> I don't want an NIH-fest either, but I also think we need to be pragmatic. I
> don't want to compete with SmartOS on diskless, unless we have something to
> really offer. They are doing one thing and doing it well. As much as I don't
> want to die to NIH, I also don't want us to be a jack-of-all-trades, and a
> master of none.

SmartOS is a great hypervisor. But it's not adequate on the desktop,
yet. OI is (more) adequate on the desktop, but its not a great
hypervisor. I'd like to have a hypervisor with graphical capabilities,
but I'd also like to give SmartOS the chance to become all it can be.
A hypervisor is not a priority for OI at the moment. Besides we _do_
have kvm and qemu.

>
>
>> So, in short I am willing to contribute, to OpenIndiana and Illumos. I
>> will get OI-151 installed today or tomorrow.
>>
>> I will try to build illumos-gate, and will report back with any problems.
>>
>> I would appreciate any pointers on making new packages.
>>
>> Is it possible to make a new zone without an internet connection?
>
> The short answer is no. The long answer is kinda.
>
> If you have a local IPS mirror, you can point it at that. There are
> instructions on mirroring the repo here
>
> http://wiki.openindiana.org/oi/Mirroring+OpenIndiana#MirroringOpenIndiana-Oldcontentfollows
>
> I know it's not ideal, but it is what it is.

I'm working on creating pre-packaged NGZ's that can be cloned (via
`zoneadm clone`) to create new NGZ's (and augmented). They can
probably even be sent as zfs datasets/snapshots, and `zoneadm
attached/detached`. This way a user can install a zone without a
network connection or IPS. I just have to use IPS to create the
initial zone.

I'm not doing this to avoid or marginalize IPS. I am doing this
because I can create zones on one machine, but not on a machine with
no network capabilities. And it feels like I should be able to :)

>
>
>> Where can I find the OI plans for future IPS features and improvements.
>
> There hasn't been a lot of discussion about future improvements. I think
> there has been someone looking at improving performance, but I can't recall
> who right now.
>
>
>> Also, I don't know if this is available in your repos, but if not, I
>> am going to port and package the i3 window manager for OI, if I have
>> trouble I'll let you guys know.
>>
>> I am going to see what I can do about pre-built NG zones.
>>
>> I will try to find resources about NG zones, making new brands,
>> modifying existing brands, etc.
>
> Sweet! Although, I'm not sure what a new brand would be required for,
> although I'll admit I've only been a user of zones, not looked at how it is
> written in-depth.

Mostly to resurrect the lx-brand, and implement other such brands.

>
>
>> Also, I recommend updating the mission statement on the web page. It
>> is "coming soon", and not very inspiring.
>>
>> I recommend something along the lines of "making cutting edge
>> technology available to power users on the desktop..." and then
>> advertise the technologies. Trust me, it is the power users, not the
>> simple desktop users that you want. Basically an incubator for future
>> illumos devs, and a platform for those who like to play with cool tech
>> they won't get anywhere else.
>
> I have the required permission to do this, but I am loathe to make changes
> without some kind of meeting or at least an on-list vote to approve the
> specifics. Maybe if you have something in mind you could toss it up, and
> someone could approve it (AFAICT, I don't even have the authority to vote,
> let alone make a unilateral decision). Otherwise I'll bang something out.

Just a suggestion. That's my personal goal. I hope it fits with the OI
project's goals.

>
>> Thanks.
>>
>
> *snipped stuff I wasn't responding to in the interests of brevity*
>
> Andrew Hettinger
> http://Prominic.NET || AHettinger at Prominic.NET
> Tel: 866.339.3169 (toll free) -or- +1.217.356.2888 x.110 (int'l)
> Fax: 866.372.3356 (toll free) -or- +1.217.356.3356 (int'l)
>
>
> _______________________________________________
> oi-dev mailing list
> oi-dev at openindiana.org
> http://openindiana.org/mailman/listinfo/oi-dev