[OpenIndiana-discuss] Enhancement to IPS (pkg)??

Sunay Tripathi tripathi.sunay at gmail.com
Wed Oct 27 02:26:19 UTC 2010

Hi Alasdair,

On 10/25/10 04:43 PM, Alasdair Lumsden wrote:
> On 25 Oct 2010, at 05:20, Sunay Tripathi wrote:
>>> I also don't think that the only "commercial viability" of OpenIndiana
>>> will come from being to upgrade existing Solaris 10 installs.  Surely
>>> we want it to be commercially viable for free-standing new deployments
>>> as well -- a goal which is not, as far as I can tell, insurmountably
>>> distant.
>> I am all for new installed base. All the power to you. I am being
>> sincere here and not sarcastic. But can you explain what you are
>> doing to achieve that? New features, new deployment model, or
>> something else? Would appreciate a meaningful response instead of
>> more chest thumping.
> Hi Sunay,
> Sorry for joining this discussion late - I've been somewhat distracted due to
 > emergency home decorating and the $dayjob.

Thats life.

> The goal with OpenIndiana is to provide a community driven free open source
> alternative to Solaris 11, with free bug fixes and security updates. With the
> recent change in the Solaris 10 licensing terms and demise of OpenSolaris,
> Solaris has essentially now become a proprietary legacy unix platform for
> the elite that can afford it.

No comments. But my actions (in leaving Soracle) kind of tells you how I
feel about it ;^)

> This puts OpenIndiana in the unique and fantastic position (from our
> perspective) of being the de-facto distribution people will turn to when
> looking to use Solaris for situations where they cannot justify the cost
> of Solaris 11 (or where they would prefer to spend their money on things
> like hardware).

I think that is a good starting point. You will get some uptake because
of that. In general, my philosophy has been to offer differentiation as
well. This is how we managed to make a comeback with Solaris 10. The
differentiation can be in many forms: stability, ease of use, good
community support, new features, etc.

> Solaris has some amazing technologies that are light years ahead of
> Linux, and quite frankly IMHO if we get this right, there should be no
> argument on earth for people to continue to use Linux. Our goal is to
> remove all the barriers stopping widespread adoption of OpenIndiana.

I can't argue with you on that at all ;^) But I strongly think that
choice is good and competition is good because that helps both to
grow and not get stale.

> OpenSolaris was never capable of doing this so long as it failed to
> provide free security/bug fixes, and this is something we are aiming
> to address in Q1 2010 with our first stable release. Obviously to
> maintain this we'll need a dedicated team of packagers, security
> officers, etc, so we will need to continue to grow our developer base.
> We're very keen to get people on board and we're very positive
> about the future - OpenIndiana has a fantastic opportunity here and
> we'd all love for it to be a great success.

And I would like to congratulate you and other community members on
making the move and I will try to help you guys as much as I can.

> In terms of what we're doing here to achieve those goals, I can summarise it with:
> 1. Provide a stable branch with bug and security fixes
> 2. Massively increase the availability of free and open source software
> available to install out of the box (Why on earth are packages like Postfix,
> Exim, Proftpd etc still missing from the base OS?). This includes providing
> thing like ffmpeg, vlc, mplayer, etc. End users should not have to learn how
> to use pkgbuild/SFE/blastwave/etc to install useful software.
> 3. Excellent easy to use documentation; something like the FreeBSD handbook
> springs to mind. I always found it to be well written and very concise.
> 4. Address as many "paper cuts" as possible (https://wiki.ubuntu.com/PaperCut) -
> this means fixing things like out of date termcaps, ensuring the backspace key
> works instead of printing out ^H on the console, making sure lots of things
> compile out of the box, providing sane inputrc/bashrc/vimrc files, etc. It's
> the stupid little niggley things that put people off switching. OpenSolaris
> has already gone a huge way towards addressing this, such as adding much
> needed libc functions like vasprintf, putting /usr/gnu/bin at the head of
> the path, adding flags to "ps", etc. But we must continue this effort to
> ensure existing and new users are not needlessly frustrated by stupid
> silly things.

You guys have the right philosophy for all the above points. We as in
Solaris org were limited by resources in porting packages and making
them available. This is where community help can be outstanding.

> 5. Ensure the OS is slick, sexy, and build up an excellent reputation.
> We want our users to think "Wow, this is *amazing*", so much so they will
> evangelise the OS and become fanatical about it, tell their friends about
> it, so their friends start using it. And with that, will come the small
>  businesses, then the SMEs, and eventually enterprises.

The idea is great but it might be helpful to have some discussions
between yourselves and end users to define sexy and what will it
take to get that Wow feeling.

Making it easier like Ubuntu will definitely get some Wows.
You can bring Xen back to life or integrate Vbox within a zone to
support live migration and that would bring Wows and excitement from
a totally different set of people.

> The goal is to have more installs of OpenIndiana than Solaris 11. I am
> almost certain CentOS have achieved this vs RHEL, so I'm sure we can too.
>>>> I don't think you understand how mirrors work.
>>> Respectfully I believe that I do understand how mirrors work.  I've
>>> been creating them one-way-or-another since before Oracle bought Sun.
>>> Before OI you could create a mirror by downloading all of the
>>> manifests and package files (using ips-mirror.py or some other script)
>>> and run pkg.depotd out of the resultant tree.  Now that OI is here and
>>> is actually interested in folks being able to mirror there are
>>> official tarball drops and an rsync service available serving the
>>> various pkg repos that you need to install OI.
>> The mirror code that we wrote actually still depends on the origin
>> server being available. It was a mirror for data only. Maybe something
>> changed in last few months but quick look at the putbacks don't
>> show any explicit putback addressing this. Is this something you
>> fixed personally or is fixed in OI? I would love to try this out.
> You're right - the "mirror" functionality is completely pointless, the
> person that designed it seemed to miss the point that latency is responsible
> for the slow pkg experience, not bandwidth. Metadata operations are the
> bit that are painfully slow.

It was designed this way to manage the package dependencies a bit
better and provide people with better experience. The problem was that
it was still work in progress when Sun became Soracle. The pkg team is
still working on the pkg subsystem but Oracle is not going to run the
repos so it has become stale. Not sure what is the charter of pkg
team is at this point but the goal should be to make all the repo and
dependency management as distributed and customizable as possible.

> We have "solved" this by simply providing people with a tarfile of our
> entire pkg.depotd repo for them to use. I have no idea why Sun never did
> this. This way you get a fully blown authoritative pkg server with metadata.
> Spinning up a pkg.depotd server is trivial, and no harder to do than
> setting up a mirror via the other method.
> However you're absolutely right, it should be much easier to clone a
> pkg server, and hopefully the pkg5 team will introduce functionality like
> that. If they don't, then we have the option of doing so, and providing our
> additions to them to use if they wish. Such is the beauty of being free from
> Sun/Oracle - things become a meritocracy and the best idea wins (in theory!).
> The problem, as ever, is finding the time and/or the people to do the work.

Believe me when I say it that entire Solaris culture was built on
meritocracy. Its just that we tried to solve problems on a longer
horizon and thats why Solaris has lot of harder problems solved. People
enjoy Crossbow/Zones but kind of forget the pain we caused when we
made GLDv3 driver incompatible with GLDv2. Sometimes things don't
make sense at a given point of time but start making sense when the
thing is complete. So pick the short term things that make sense but
keep an eye on the long term architecture as well.

>>>> And the kernel is small part of all you need to have a operating environment.
>>> Absolutely, I was merely highlighting the fact that there is
>>> absolutely a present consumer of the file:// URI style and it clearly
>>> functions against a local file-based repository layout.
>>>> Do an experiment
>>>> for me at your home. Disconnect the internet, just try your local
>>>> machines as publisher and see if you can bring up a new system which
>>>> has all that you need (to start with, just build and compile the
>>>> ON bits).
>>> You can absolutely do this today.  OpenIndiana provides a full copy of
>>> both "/dev" (current OI packages) and "/legacy" (a mirror of the
>>> OpenSolaris "/dev" repo).  If you download these and run a local
>>> pkg.depotd then you can install without the Internet.  It also turns
>>> out that you can download both of these and merely run an Apache
>>> server with a few ksh scripts (as I described in my blog entry) to
>>> provide the same functionality.
>> So you would expect people to write their own script to achieve this.
>> Why would you not allow the pkg system to do it for you with a simple
>> command? Running pkg.depotd is for developer and not end users in my
>> opinion.
> I guess the answer here is that what you're asking for can be worked around,
> but a clean easy to use end-user implementation is missing. It would be good
> to have this, but until the IPS folk do the work, or until we can find a
> developer willing to do the work, it might not happen quite so quickly.
> There are a lot of higher priority things on the todo list at the moment,
> so a recipe/documentation people can follow to achieve the goal should
> hopefully be sufficient until such functionality is added.

I understand the resource issue.

>>>> I do appreciate your enthusiasm in using things and that is the right
>>>> approach. Things are by far no where near complete to be palatable in
>>>> enterprise but let people like me worry about that.
>>> I'm glad that people like you are worrying about it, but I think I'll
>>> continue to worry about it as well if that's perfectly alright.
>> Sorry about the last comment. I think every community member should
>> worry about it. And seems like you are quite capable. I am not trying
>> to imply that you don't know what you are doing. I was just trying
>> to steer you towards making it easy for people who don't have your
>> skill set to actually be able to run Solaris.
> Absolutely - we should all be thinking about OpenIndiana from an end
> user experience. This the point behind the "papercuts" thing above. We
> should make no assumptions about our users, other than that they have
> high standards, and expect things to work, for things to be intuitive
> and easy to use, and for things to be sane.
> At the end of the day, we want newcomers to feel that this operating
> system is slick, well built, easy to use, technically excellent, and
> going places. So much so, that if they normally deploy Solaris 10,
> Linux, or FreeBSD, that after spending time with OpenIndiana, they
> would consider deploying it instead.
> Hopefully this is a goal we can all agree on, and hopefully all
> contribute towards.
> The more organised we are, the more positive and hopeful we are,
> and the more work we put in, the more likely we are to achieve our goals :)
> So I would invite you to get stuck in and join our development
> effort if you're interested, we're primarily communicating in #oi-dev
> on irc.freenode.net. We also have a development mailing list.

and I did join the dev mailing list as well. I would definitely like
to help out where I can. Including trying to help you guys learn
from our mistakes :)

> You may
> also want to look into getting involved with Illumos if you're interested
> in working on the OS/Net side of things - they're ripping subsystems out
> left right and centre at the moment and things are moving forward; it's an
> exciting time to be involved.

I know most of the Illumos crowd. But I think making you guys
successful is more important for the future of Solaris. If you guys
can get more deployments, then the rest falls in place.


More information about the OpenIndiana-discuss mailing list