[OpenIndiana-discuss] NFS Linux client slow reads

["Hung-Sheng Tsao (Lao Tsao 老曹) Ph.D."]

sorry all 11.4 ubuntu

On 5/29/2011 9:59 PM, "Hung-Sheng Tsao (Lao Tsao 老曹) Ph.D." wrote:
> hi
> I just
> 1)OI-148
> 2)Ubuntu=10.4 server
> 3)ubuntu-10.4 desktop
> on virtual box env
> it turn out the test
> u-c to u-s
> dd if=/dev/zero of=test bs=1024 count=1M
> 1072741824 bytes (1.1GB) copied 237.411 s 4.5M/s
>
>
> u-c to oi-x
> dd if=/dev/zero of=test bs=1024 count=1M
> 59.8019 s 18.0 MB/s
>
> cat /proc/mounts
> show r(w)size=131072 for ubuntu server
> show r(w)size=1048576 for oi server
>
> big difference
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> On 5/29/2011 7:36 PM, Pete Ashdown wrote:
>> Mount options are default Ubuntu, relatime etc, although I tried nfs4 
>> and tcp/udp with similar results.  No jumbo frames.  I didn't test 
>> different block sizes because I was trying to keep the test the same 
>> for both servers.  ZFS is raidz3, but as you can see below, I get 
>> 317MB/s on the server itself.  I don't think that is the bottleneck.  
>> I should note that I tried Solaris Express 11 and got similar results 
>> as OI.
>>
>>
>>
>> On 05/28/2011 07:17 PM, "Hung-Sheng Tsao (Lao Tsao 老曹) Ph.D." wrote:
>>> what are the NFS mount options?
>>> what are the tcp window size
>>> is jumbo frame on or off
>>> bs=1024 is very small, did you test various different bs size?
>>> ZFS what is the block size? stripwidth?
>>>
>>>
>>> On 5/28/2011 8:52 PM, Dan Swartzendruber wrote:
>>>> This is kind of a mystery.  Is it the same linux client in both cases?
>>>>
>>>> -----Original Message-----
>>>> From: Pete Ashdown [mailto:pashdown at xmission.com]
>>>> Sent: Friday, May 27, 2011 7:06 PM
>>>> To: Discussion list for OpenIndiana
>>>> Subject: [OpenIndiana-discuss] NFS Linux client slow reads
>>>>
>>>> Both NFS tests were conducted on the same hardware, with 4 aggregated
>>>> 802.3ad gig-e ports on the server and 2 aggregated 802.3ad gig-e 
>>>> ports on
>>>> the client.  The OI server has 24GB RAM, 8 2TB Seagate drives, and 
>>>> a 32GB
>>>> SSD for ZIL and a 160GB SSD for L2ARC.
>>>>
>>>> Test 1: Ubuntu Linux server&  client
>>>>
>>>> #  dd if=/dev/zero of=test bs=1024 count=1M
>>>> 1073741824 bytes (1.1 GB) copied, 13.3347 s, 80.5 MB/s
>>>> # echo 3>  /proc/sys/vm/drop_caches  (clear Linux file buffers)
>>>> # dd if=test of=/dev/null
>>>> 1073741824 bytes (1.1 GB) copied, 10.0029 s, 107 MB/s
>>>>
>>>> Test 2: OI 148 server&  Ubuntu Linux client
>>>>
>>>> # dd if=/dev/zero of=test bs=1024 count=1M
>>>> 1073741824 bytes (1.1 GB) copied, 12.0275 s, 89.3 MB/s
>>>> # echo 3>  /proc/sys/vm/drop_caches
>>>> # dd if=test of=/dev/null bs=1024
>>>> 1073741824 bytes (1.1 GB) copied, 112.205 s, 9.6 MB/s<--- ~10% of 
>>>> Linux
>>>> server
>>>>
>>>> Test 3: On the OI server
>>>>
>>>> # dd if=/dev/zero of=test bs=1024 count=1M
>>>> 1073741824 bytes (1.1 GB) copied, 7.79955 s, 138 MB/s
>>>> # dd if=test of=/dev/null bs=1024
>>>> 1073741824 bytes (1.1 GB) copied, 3.38502 s, 317 MB/s
>>>>
>>>> Any help is appreciated.