[OpenIndiana-discuss] improving ZFS latency with hard disks

[Toomas Soome]

> On 16. May 2021, at 06:54, dave at loud-mouth.net wrote:
> 
> Le me know if ZFS questions should be asked to another group.
> 
> Problem: A documentation application takes several minutes to open documents containing large numbers of links to images. Aparently, the application is making many separate file attribute queries, so the sum of all the separate disk rotation latencies are adding up.
> 
> The obvious solution is to move the zfs pool over to SSDs, but since the entire documentation collection is less than 10GB, I was wondering if there was a way to address the problem with RAM. As far as I can tell from the documentation, the ARC and L2ARC are based on recently used data, so it doesn't sound like accessing attributes from all of the linked files to a document would be helped by a larger ARC when the document is first opened.
> 
> Is there a way I can get more of the proximity files in the cache with any tuning parameters, or even utilize some sort of RAM disk for the files in the directory of concern?
> 


How much RAM do you have, what is arcstat reporting while accessing those files? Are files accessed locally or via NFS/SMB (where from the actual slowness is appearing)?

First open from slow media is always depending on media speed; however, the experience also depends on block sizes and fragmentation - does read ahead have chance to help etc.

With persistent L2ARC, the less actively used data is on L2 (active data is on ARC), but the L2 wont be empty after reboot. However, pointers to L2 are also in ARC, with low RAM situation, this may complicate things even more…

ramdisk versus ssd - I’d go for ssd there.

rgds,
toomas