[OpenIndiana-discuss] [developer] Re: Memory usage concern

[Jim Klimov]

2012-10-22 18:24, Daniel Kjar wrote:
> I have this problem with any VM running on either Sol10 Nevada,
> Opensolaris, openindiana.  I have the ARC restricted now but for some
> reason, and 'people at sun that know these things' have mentioned it
> before when sun still existed, when something needs a big devoted chunk
> of ram, zfs fails miserably at giving up its cache.  If I don't limit
> the ARC, a few days of even a single VM running causes the system to
> start stalling.  With the ARC limit, everything is peaches.


I think limiting the ARC was the recommended practice for VirtualBox
in either ZFS Evil Tuning Guide or VirtualBox manuals, or both.

Also remember that VBox requires disk-based swap to be available at
the time of start of VMs in amounts comparable to VMs' RAM size, so
that if needed, these processes can be guaranteed to be able to swap.

On a different note, there are programs which check system available
memory and fail themselves if there's not enough - without asking the
OS to give them some memory. In this case ARC gets no signals to
shrink because some process needs the RAM. Rule of thumb for admins
was to estimate or measure the "stable" RAM requirements of running
processes, add 1GB for good measure, and limit ARC to the rest.

As for lags and failures in relation to ARC, I'd guess that getting
and releasing RAM in vast amounts and small chunks would fragment
the memory, possibly to the extent that while the needed total
free RAM is available, no contiguous block can be allocated of the
size needed by a process (or that there's too much random access
and virtual memory mapping involved, requiring much CPU for each
memory access from userland processes). But that's just an educated
guess.

HTH,
//Jim