[OpenIndiana-discuss] Mitigating the performance impact of scrub

[Richard Elling]

On Sep 27, 2012, at 11:51 AM, Reginald Beardsley <pulaskite at yahoo.com> wrote:

> Where would I find information about the kernel level tunable parameters?  Are there specific parameters you have in mind?

UTSL, http://src.illumos.org/source/xref/illumos-gate/usr/src/uts/common/fs/zfs/dsl_scan.c#61

The most important tend to be:
/etc/system settings:
	zfs:zfs_scrub_delay = 4         number of ticks to delay scrub (10ms/tick by default)
	zfs:zfs_scan_idle = 50           idle window size (ticks)

to set on a running system, the kernel parameters are:
	zfs_scrub_delay
	zfs_scan_idle

The algorithm intends to check to see if there are important I/Os within
zfs_scan_idle time, and if so delay scrub I/O by zfs_scrub_delay.

> What I'm seeing is very slow loads of  OpenOffice.  It's certainly not quick under the best of circumstances, but this was particularly slow.  I just needed to read a bunch of 5-6 MB .xls files and export them as .csv files.

If you are using HDDs, then zfs_vdev_max_pending might be too high. To understand
why see:
	http://blog.richardelling.com/2012/03/iops-and-latency-are-not-related-hdd.html

 -- richard

> 
> Reg
> 
> --- On Thu, 9/27/12, Richard Elling <richard.elling at richardelling.com> wrote:
> 
>> From: Richard Elling <richard.elling at richardelling.com>
>> Subject: Re: [OpenIndiana-discuss] Mitigating the performance impact of scrub
>> To: "Discussion list for OpenIndiana" <openindiana-discuss at openindiana.org>
>> Date: Thursday, September 27, 2012, 1:15 PM
>> On Sep 27, 2012, at 8:44 AM, Reginald
>> Beardsley <pulaskite at yahoo.com>
>> wrote:
>> 
>>> The only thing google turned up was "stop the scrub if
>> it impacts performance too badly" which is not really all
>> that helpful. Or ways to speed up scrubs & resilvers.
>> 
>> On modern ZFS implementations, scrub I/O is throttled to
>> avoid impacting application I/O
>> performance. There are some kernel-level tunable parameters
>> that can be adjusted, but
>> AFAIK, nobody has done any characterization studies.
>> 
>>> 
>>> In my case, I'd like to be able to run a scrub and have
>> all the performance hits taken by the scrub process if the
>> system has other loads.  I care that the scrub runs,
>> but not how fast.
>>> 
>>> From observed behavior, it appears that the scrub is
>> consuming too large a share of DRAM (12 GB in this
>> case).  Is that correct and if so, is there a way to
>> limit the proportion of memory used by the scrub process?
>> 
>> Data is cached in the ARC, but scrub data is placed in the
>> MRU/LRU side of the ARC
>> and shouldn't impact the MFU side. In most cases, the memory
>> usage during a scrub
>> is not a problem. Have you seen a different behaviour?
>> -- richard
>> 
>>> I'd like to be able to schedule scrubs regularly, but
>> the present behavior would require predicting when I didn't
>> want to use the system for work.
>>> 
>>> Thanks,
>>> Reg
>> 
>> --
>> illumos Day & ZFS Day, Oct 1-2, 2012 San Fransisco 
>> www.zfsday.com
>> Richard.Elling at RichardElling.com
>> +1-760-896-4422
>> 
>> 
>> 
>> _______________________________________________
>> OpenIndiana-discuss mailing list
>> OpenIndiana-discuss at openindiana.org
>> http://openindiana.org/mailman/listinfo/openindiana-discuss
>> 
> 
> _______________________________________________
> OpenIndiana-discuss mailing list
> OpenIndiana-discuss at openindiana.org
> http://openindiana.org/mailman/listinfo/openindiana-discuss

--
illumos Day & ZFS Day, Oct 1-2, 2012 San Fransisco 
www.zfsday.com
Richard.Elling at RichardElling.com
+1-760-896-4422