[OpenIndiana-discuss] [developer] Re: Memory usage concern

[James Carlson]

Daniel Kjar wrote:
> I have this problem with any VM running on either Sol10 Nevada,
> Opensolaris, openindiana.  I have the ARC restricted now but for some
> reason, and 'people at sun that know these things' have mentioned it
> before when sun still existed, when something needs a big devoted chunk
> of ram, zfs fails miserably at giving up its cache.  If I don't limit
> the ARC, a few days of even a single VM running causes the system to
> start stalling.  With the ARC limit, everything is peaches.

My understanding is that the two mechanisms are only superficially similar.

UFS was (and is) in bed with the VM implementation, so it's using memory
pages as the cache in the same way that the VM keeps paged-in data in
memory.  UFS "giving up" memory that's needed for applications is really
the same thing as saying that the VM reallocates memory pages when it
needs to.

ZFS, though, is different.  It's not as much in bed with the VM.  The
ARC is allocated memory, done in the same way that any other kernel
service can allocate memory.  There is a back-pressure "help! I need
more space" mechanism on normal kernel allocations, and this is what the
ARC relies on to give up memory when necessary, but it occurs much
further down the line than ordinary VM page reallocation.  Thus, when it
does happen, the damage has already been done -- the VM has already been
exhausted of easy-to-find pages, and we're already in a bad state.

So, yes, limiting the ARC is (unfortunately) a good thing.

-- 
James Carlson         42.703N 71.076W         <carlsonj at workingcode.com>