[OpenIndiana-discuss] [developer] Re: Memory usage concern

[Daniel Kjar]

I have this problem with any VM running on either Sol10 Nevada, 
Opensolaris, openindiana.  I have the ARC restricted now but for some 
reason, and 'people at sun that know these things' have mentioned it 
before when sun still existed, when something needs a big devoted chunk 
of ram, zfs fails miserably at giving up its cache.  If I don't limit 
the ARC, a few days of even a single VM running causes the system to 
start stalling.  With the ARC limit, everything is peaches.

This is on a system with 4 dual core processors and 32 gigs of ram.

On 10/22/12 01:25 AM, David Halko wrote:
> On Sat, Oct 20, 2012 at 1:57 PM, Cedric Blancher <
> cedric.blancher at googlemail.com> wrote:
>
>> On 16 October 2012 01:22, Jason Matthews <jason at broken.net> wrote:
>>> -----Original Message-----
>>> From: Cedric Blancher [mailto:cedric.blancher at googlemail.com]
>>>
>>>> IMO you blame the wrong people. You can have the same kind of
>>>> problems with any Illumos-based distribution if you activate a
>>>> zone and let the machine just sit there for a week or so or have
>>>> a lot filesystem activity using mmap(). Either way the machines
>>>> will choke themselves to memory starvation. The only workaround
>>>> we found are regular reboots (every 24h), or limit the
>>>> ZFS ARC to an absolute minimum.
>>> I don't think you understand. My proxy tier does almost no reads from the
>>> file system. There is no content on the server.
>> OK, sorry then. Same symptoms, different cause, albeit it's so bad
>> that it makes the OS virtually unusable for any serious work.
>>
>> Ced
>>
> This whole discussion sounds bizarre to me. I have a Solaris 10 Update 1
> system with over a dozen zones with UFS with 8 Gig RAM and don't experience
> these types of issues. We reboot once a year, whether we need to or not.
>
> This is my limited understanding...
> - UFS basically consumes all unused memory for paging, without ever telling
> the OS, but releases it to OS processes when memory is needed. UFS does not
> tell the OS that it is using the unused memory as buffer cache, so you
> never know it when you check for your memory usage.
> - ZFS basically consumes all unused memory for ARC, but tells the OS when
> it is taking RAM. The ZFS ARC is supposed to give back memory when there is
> pressure to do so. You always know how much memory is really being used
> when you check your memory usage.
>
> I had issues in the past where disk I/O started to go through the roof, on
> UFS filesystems, when the free memory was getting below 4 Gig of RAM... a
> sure sign the apps were starving the "invisible" UFS buffer cache.
>
> I don't really understand how applications can be starved by ZFS unless ZFS
> can not give back the buffers (lots of constant full-table scans?) Has
> Illumos developers really confirmed such a ZFS bug, where it is not
> returning memory to the OS?
>
> This just does not sound right, to me.
>
> Thanks - Dave H
> http://netmgt.blogspot.com/
> _______________________________________________
> OpenIndiana-discuss mailing list
> OpenIndiana-discuss at openindiana.org
> http://openindiana.org/mailman/listinfo/openindiana-discuss

-- 
Dr. Daniel Kjar
Assistant Professor of Biology
Division of Mathematics and Natural Sciences
Elmira College
1 Park Place
Elmira, NY 14901
607-735-1826
http://faculty.elmira.edu/dkjar

"...humans send their young men to war; ants send their old ladies"
	-E. O. Wilson