[OpenIndiana-discuss] Mitigating the performance impact of scrub

[Reginald Beardsley]

--- On Thu, 9/27/12, Richard Elling <richard.elling at richardelling.com> wrote:

> From: Richard Elling <richard.elling at richardelling.com>
> Subject: Re: [OpenIndiana-discuss] Mitigating the performance impact of scrub
> To: "Discussion list for OpenIndiana" <openindiana-discuss at openindiana.org>
> Date: Thursday, September 27, 2012, 3:46 PM
> On Sep 27, 2012, at 11:51 AM,
> Reginald Beardsley <pulaskite at yahoo.com>
> wrote:
> 
> > Where would I find information about the kernel level
> tunable parameters?  Are there specific parameters you
> have in mind?
> 
> UTSL, http://src.illumos.org/source/xref/illumos-gate/usr/src/uts/common/fs/zfs/dsl_scan.c#61
> 
> The most important tend to be:
> /etc/system settings:
>     zfs:zfs_scrub_delay = 4   
>      number of ticks to delay scrub
> (10ms/tick by default)
>     zfs:zfs_scan_idle = 50   
>        idle window size (ticks)
> 
> to set on a running system, the kernel parameters are:
>     zfs_scrub_delay
>     zfs_scan_idle
> 
> The algorithm intends to check to see if there are important
> I/Os within
> zfs_scan_idle time, and if so delay scrub I/O by
> zfs_scrub_delay.
> 
> > What I'm seeing is very slow loads of 
> OpenOffice.  It's certainly not quick under the best of
> circumstances, but this was particularly slow.  I just
> needed to read a bunch of 5-6 MB .xls files and export them
> as .csv files.
> 
> If you are using HDDs, then zfs_vdev_max_pending might be
> too high. To understand
> why see:
>     http://blog.richardelling.com/2012/03/iops-and-latency-are-not-related-hdd.html
> 

Thanks.  Very useful.  

As I understand it, increasing zfs_scrub_delay and zfs_scan_idle would throttle back the scrub so that it had less impact if other processes were active, but would have negligible impact on scrub if the system were idle.  Is that accurate?

Does:

zfs_scrub_delay = 100
zfs_scan_idle = 1000

seem excessive?  I'd really like the scrub to only run when I don't want the system to do other work.

The goal is to be able to run scrubs from cron weekly and not have it adversely impact interactive use if I want to use the system before the scrub finishes.  If it's not interfering w/ my using the system, I'm happy to have it take a couple of days to complete.  At present it takes about 10 hours, so if I'm working late and start a scrub just before going to bed the system is slow when I get up.


Reg