[OpenIndiana-discuss] NFS Linux client slow reads

["Hung-Sheng Tsao (Lao Tsao 老曹) Ph.D."]

hi
I just
1)OI-148
2)Ubuntu=10.4 server
3)ubuntu-10.4 desktop
on virtual box env
it turn out the test
u-c to u-s

dd if=/dev/zero of=test bs=1024 count=1M
1072741824 bytes (1.1GB) copied 237.411 s 4.5M/s


u-c to oi-x
dd if=/dev/zero of=test bs=1024 count=1M
59.8019 s 18.0 MB/s

cat /proc/mounts
show r(w)size=131072 for ubuntu server
show r(w)size=1048576 for oi server

big difference















On 5/29/2011 7:36 PM, Pete Ashdown wrote:
> Mount options are default Ubuntu, relatime etc, although I tried nfs4 
> and tcp/udp with similar results.  No jumbo frames.  I didn't test 
> different block sizes because I was trying to keep the test the same 
> for both servers.  ZFS is raidz3, but as you can see below, I get 
> 317MB/s on the server itself.  I don't think that is the bottleneck.  
> I should note that I tried Solaris Express 11 and got similar results 
> as OI.
>
>
>
> On 05/28/2011 07:17 PM, "Hung-Sheng Tsao (Lao Tsao 老曹) Ph.D." wrote:
>> what are the NFS mount options?
>> what are the tcp window size
>> is jumbo frame on or off
>> bs=1024 is very small, did you test various different bs size?
>> ZFS what is the block size? stripwidth?
>>
>>
>> On 5/28/2011 8:52 PM, Dan Swartzendruber wrote:
>>> This is kind of a mystery.  Is it the same linux client in both cases?
>>>
>>> -----Original Message-----
>>> From: Pete Ashdown [mailto:pashdown at xmission.com]
>>> Sent: Friday, May 27, 2011 7:06 PM
>>> To: Discussion list for OpenIndiana
>>> Subject: [OpenIndiana-discuss] NFS Linux client slow reads
>>>
>>> Both NFS tests were conducted on the same hardware, with 4 aggregated
>>> 802.3ad gig-e ports on the server and 2 aggregated 802.3ad gig-e 
>>> ports on
>>> the client.  The OI server has 24GB RAM, 8 2TB Seagate drives, and a 
>>> 32GB
>>> SSD for ZIL and a 160GB SSD for L2ARC.
>>>
>>> Test 1: Ubuntu Linux server&  client
>>>
>>> #  dd if=/dev/zero of=test bs=1024 count=1M
>>> 1073741824 bytes (1.1 GB) copied, 13.3347 s, 80.5 MB/s
>>> # echo 3>  /proc/sys/vm/drop_caches  (clear Linux file buffers)
>>> # dd if=test of=/dev/null
>>> 1073741824 bytes (1.1 GB) copied, 10.0029 s, 107 MB/s
>>>
>>> Test 2: OI 148 server&  Ubuntu Linux client
>>>
>>> # dd if=/dev/zero of=test bs=1024 count=1M
>>> 1073741824 bytes (1.1 GB) copied, 12.0275 s, 89.3 MB/s
>>> # echo 3>  /proc/sys/vm/drop_caches
>>> # dd if=test of=/dev/null bs=1024
>>> 1073741824 bytes (1.1 GB) copied, 112.205 s, 9.6 MB/s<--- ~10% of Linux
>>> server
>>>
>>> Test 3: On the OI server
>>>
>>> # dd if=/dev/zero of=test bs=1024 count=1M
>>> 1073741824 bytes (1.1 GB) copied, 7.79955 s, 138 MB/s
>>> # dd if=test of=/dev/null bs=1024
>>> 1073741824 bytes (1.1 GB) copied, 3.38502 s, 317 MB/s
>>>
>>> Any help is appreciated.