[OpenIndiana-discuss] [developer] Re: Memory usage concern

[David Halko]

On Sat, Oct 20, 2012 at 1:57 PM, Cedric Blancher <
cedric.blancher at googlemail.com> wrote:

> On 16 October 2012 01:22, Jason Matthews <jason at broken.net> wrote:
> > -----Original Message-----
> > From: Cedric Blancher [mailto:cedric.blancher at googlemail.com]
> >
> >> IMO you blame the wrong people. You can have the same kind of
> >> problems with any Illumos-based distribution if you activate a
> >> zone and let the machine just sit there for a week or so or have
> >> a lot filesystem activity using mmap(). Either way the machines
> >> will choke themselves to memory starvation. The only workaround
> >> we found are regular reboots (every 24h), or limit the
> >> ZFS ARC to an absolute minimum.
> >
> > I don't think you understand. My proxy tier does almost no reads from the
> > file system. There is no content on the server.
>
> OK, sorry then. Same symptoms, different cause, albeit it's so bad
> that it makes the OS virtually unusable for any serious work.
>
> Ced
>

This whole discussion sounds bizarre to me. I have a Solaris 10 Update 1
system with over a dozen zones with UFS with 8 Gig RAM and don't experience
these types of issues. We reboot once a year, whether we need to or not.

This is my limited understanding...
- UFS basically consumes all unused memory for paging, without ever telling
the OS, but releases it to OS processes when memory is needed. UFS does not
tell the OS that it is using the unused memory as buffer cache, so you
never know it when you check for your memory usage.
- ZFS basically consumes all unused memory for ARC, but tells the OS when
it is taking RAM. The ZFS ARC is supposed to give back memory when there is
pressure to do so. You always know how much memory is really being used
when you check your memory usage.

I had issues in the past where disk I/O started to go through the roof, on
UFS filesystems, when the free memory was getting below 4 Gig of RAM... a
sure sign the apps were starving the "invisible" UFS buffer cache.

I don't really understand how applications can be starved by ZFS unless ZFS
can not give back the buffers (lots of constant full-table scans?) Has
Illumos developers really confirmed such a ZFS bug, where it is not
returning memory to the OS?

This just does not sound right, to me.

Thanks - Dave H
http://netmgt.blogspot.com/