[OpenIndiana-discuss] NAS R/W performance puzzle

[Reginald Beardsley]

Some casual testing has turned up a bit of a puzzle.

I've got an N40L w/ 4x2TB ST2000DM001 in a RAIDZ2 pool with 4 GB of ECC DRAM running oi151a7.  This is NFS mounted to a Z400 running Solaris 10 (last Sun release U7?)

On the N40L I get 125 MB/s write and 250 MB/s read for 6.5 GB using /dev/zero & /dev/null w/ bs=65536. 

On the Z400 using NFS I get 50 MB/s write but only 10-20 MB/s read for the same operations.  Both systems and the network are unloaded.  Write speed over NFS is quite consistent, but the read varies by 2x.

Can anyone suggest why the NFS read is so much slower than the NFS write?  I'm baffled by the relative speed of the R/W operations reversing like that.   The only reasons I can think of are some issue in the network stack or  NFS implementations.  

Thanks,
Reg