[OpenIndiana-discuss] CIFS performance issues

[Robin Axelsson]

In the past, I used OpenSolaris b134 which I then updated to OpenIndiana 
b148 and never did I experience performance issues related to the 
network connection (and that was when using two of the "infamous" 
RTL8111DL OnBoard ports). Now that I have swapped the motherboard and 
the hard drive and later added a 2-port Intel EXPI9402PT NIC (because of 
driver issues with the Realtek NIC that wasn't there before), I 
performed a fresh install of OpenIndiana.

Since then I experience intermittent network freeze-ups that I cannot 
link to faults of the storage pool (iostat -E returns 0 errors). I have 
had this issue both with the dual port Intel controller as well as with 
a single port Intel controller (EXPI9400PT) and the Realtek 8111E 
OnBoard NIC. The storage pool is behind an LSI MegaRAID 1068e based 
controller using no port extenders.

In detail (9400PT+8111E):
-------------------------
I was running a Virtual Machine with VirtualBox 3.2.14 with (1) a 
bridged network connection and was accessed over the network using (2) 
VBox RDP connection and (3) a ZFS based CIFS share to be accessed from a 
Windows computer over the network. These applications were administrated 
both over (4) SSH (port 2244) and (5) VNC (using vncserver). A typical 
start of the VM was done with 'screen VBoxHeadless --startvm ...'

I assigned the network ports the following way:

e1000g: VBox RDP, VNC, SSH
rge0: Virtual Machine Network Connection (Bridged)

I tried various combinations but the connection froze intermittently for 
all applications. The bridged network connection was worst. When I SSHed 
over rge0, the connection was frequently severed which is was not over 
e1000.

So I pulled the plug on the rg0 and let everything go through the e1000 
connection. freeze-ups became more frequent and it seemed like the 
Bridged connection was causing this issue because the connection didn't 
freeze like that when the VM wasn't running.

Note that I didn't assign the CIFS share to any particular port but 
calls to <computername> were assigned to the e1000 port in the 
/etc/inet/hosts file.
-------------------------

In detail (9402PT):
-------------------
In this setup I run essentially the same applications but all through 
the 9402PT which has two ports (e1000g1 and e1000g2). So I assign the 
applications the following way:

e1000g1: VBox RDP, SSH, <computername> (in /etc/inet/hosts)
e1000g2: Bridged connection to the virtual machine

So while running the virtual machine on the server, having an open SSH 
connection to it and a command prompt pointing (cd x:\) at the CIFS 
share (which is mapped as a network drive, say "X:") I started a media 
player and played an audio file over the CIFS share which made the 
connection freeze.

The freezing affected the media player and the command prompt but the 
RDP connection worked and access to internet inside the VM was flawless. 
The SSH connection was frozen as well. After a few minutes it became 
responsive and iostat -E reported no errors. The command prompt and the 
media player were still frozen but "ls <path to CIFS shared contents>" 
worked fine over the SSH connection. Shortly after that the CIFS 
connection came back and things seem to run ok.

So in conclusion the freeze-ups are still there but less frequent. I 
have tried VirtualBox 4.1.8 but the ethernet connection is worse with 
that version which is why I downgraded to 3.2.14 (which was published  
_after_ 4.1.8).
-------------------

These issues occur on server grade hardware using drivers that are/were 
certified by Sun (as I understand it). Moreover, CIFS and ZFS are the 
core functionality of OpenIndiana so it is quite essential that the 
network works properly and is stable.

I'm sorely tempted to issue a bug report but I would want some advice on 
how to troubleshoot and provide relevant bug reports. There are no 
entries in the /var/adm/messages that are related to the latest  
freeze-up mentioned above and I couldn't find any when running the prior 
setups. These freeze-ups don't happen all the time so it isn't easy to 
consistently reproduce them.

Robin.