[OpenIndiana-discuss] [zfs] zfs + iscsi = poor sequential reding performance

[Michael Kruger]

On 04/23/2016 08:26 AM, Сисадмин wrote:
>   Hi everyone,
>
> to make the long story short - i 've spent 6 weeks trting really hard, read tons of doc pages on the internet (get lots of knowledge to be honest)
>
> so i can get very good results , however unfortunetely i cant get from iscsi+zfs good results about sequential reading.

Admittedly I have not done any testing of ZFS performance on OpenIndiana.

However, about 4 years ago I went through this very exercise and tested 
nearly every available solution (FreeNas, Open-E, Nexenta, etc.,) and in 
the end chose Openfiler as my SAN solution. Using 2 NICs, I was able to 
achieve read speeds of nearly 200MB/s. I don't recall what the write 
speeds were, but I am think they were in the same neighborhood.

Here is a thread post to vmware communities, where I posted my results: 
https://communities.vmware.com/message/1986953#1986953

Assuming the storage can keep up, using 4 NIC's in an MPIO round robin 
configuration should deliver nearly 400 MB/s.

My white box lab didn't use any kind of fault tolerance. I was only 
using 2 500G disks in a raid stripe attached to an LSI controller with 
write caching was enabled. The LSI controller managed the disks.

One thing I discovered in this process was the Realtek NIC's vastly 
outperformed the Intel NICS. Whether the issue was on the ESXI side or 
the Openfiler side, I am not sure. I swapped both sides out with 
Realtek. OpenFiler however didn't natively support the Realtek NICs so I 
had to install my own drivers.

I have read the freebsd project has done some tuning to get ZFS to 
perform better. There are also some tunable parameters in illumos which 
may possibly help. See here: 
http://www.linuxtopia.org/online_books/opensolaris_2008/SOLTUNEPARAMREF/html/soltuneparamref.html

Hope that helps,

Michael