[OpenIndiana-discuss] oi_151 NFS write performance regression

[Tomáš Hlaváček]

Hello!

I am trying oi_151 on my NAS box and I am getting strange results - 
writes from the same client to the server are running on 1/3 speed of 
oi_148. (Namely: 78e3b50db518 is a Linux NFS client running 3.0.1 
kernel, staflik.lhc.ignum.cz is a OI NFS server.) First lets have a look 
on oi_148:

78e3b50db518:/mnt# dd if=/root/debian-live-6.0.1-amd64-gnome-desktop.iso of=debian-live-6.0.1-amd64-gnome-desktop.iso bs=4k
286208+0 records in
286208+0 records out
1172307968 bytes (1.2 GB) copied, 15.607 s, 75.1 MB/s
78e3b50db518:/mnt# cd
78e3b50db518:~# umount /mnt
78e3b50db518:~# mount -o vers=3 staflik.lhc.ignum.cz:/nfs/speedtest/mnt
78e3b50db518:~# cd /mnt
78e3b50db518:/mnt# dd if=debian-live-6.0.1-amd64-gnome-desktop.iso of=/dev/null bs=4k
286208+0 records in
286208+0 records out
1172307968 bytes (1.2 GB) copied, 12.3741 s, 94.7 MB/s
78e3b50db518:/mnt# rm debian-live-6.0.1-amd64-gnome-desktop.iso
78e3b50db518:/mnt# cd
78e3b50db518:~# umount /mnt/

Now the server is rebooted to oi_151 (by beadm activate ... && reboot):

78e3b50db518:~# mount -o vers=3 staflik.lhc.ignum.cz:/nfs/speedtest /mnt
78e3b50db518:~# cd /mnt
78e3b50db518:/mnt# dd if=/root/debian-live-6.0.1-amd64-gnome-desktop.iso of=debian-live-6.0.1-amd64-gnome-desktop.iso bs=4k
286208+0 records in
286208+0 records out
1172307968 bytes (1.2 GB) copied, 36.0548 s, 32.5 MB/s
78e3b50db518:/# umount /mnt/
78e3b50db518:/# mount -o vers=3 staflik.lhc.ignum.cz:/nfs/speedtest /mnt78e3b50db518:/# cd /mnt/
78e3b50db518:/mnt# dd if=debian-live-6.0.1-amd64-gnome-desktop.iso of=/dev/null bs=4k
286208+0 records in
286208+0 records out
1172307968 bytes (1.2 GB) copied, 11.7393 s, 99.9 MB/s


So in the case of oi_148 read and write performance is comparable but in 
case of oi_151 there is a huge gap. And writes are running only 32.5MB/s 
with oi_151, but with oi_148 it ran considerably faster over 75MB/s.

Remarks:
- I do not have any parameters tweaked in /etc/system.
- The Linux client is running out of ramdisk -> it is unlikely, that 
there is a I/O performance issue on the Linux side.
- I have got a e1000g NIC in the OI box.
- I have tried ipref and results are consistent across oi_148 and oi_151.

I would appreciate any advice how to fix this or whether it would be 
better to report this issue somewhere else as a bugreport?

My output of prtconf -v is here: 
http://manwe.elfove.cz/~brill/staflik-prtconf.txt and my zpool status 
output follows:

   pool: tank
  state: ONLINE
  scan: none requested
config:

         NAME         STATE     READ WRITE CKSUM
         tank         ONLINE       0     0     0
           raidz2-0   ONLINE       0     0     0
             c1t2d0   ONLINE       0     0     0
             c1t3d0   ONLINE       0     0     0
             c1t4d0   ONLINE       0     0     0
             c1t5d0   ONLINE       0     0     0
             c1t12d0  ONLINE       0     0     0
             c1t6d0   ONLINE       0     0     0
             c1t7d0   ONLINE       0     0     0
             c1t8d0   ONLINE       0     0     0


errors: No known data errors

Cheers,
Tomas

-- 
Tomáš Hlaváček<tmshlvck at gmail.com>

Be strict when sending and tolerant when receiving. (RFC1958)