[OpenIndiana-discuss] NFS Linux client slow reads

[Pete Ashdown]

Mount options are default Ubuntu, relatime etc, although I tried nfs4 and
tcp/udp with similar results.  No jumbo frames.  I didn't test different
block sizes because I was trying to keep the test the same for both
servers.  ZFS is raidz3, but as you can see below, I get 317MB/s on the
server itself.  I don't think that is the bottleneck.  I should note that I
tried Solaris Express 11 and got similar results as OI.



On 05/28/2011 07:17 PM, "Hung-Sheng Tsao (Lao Tsao 老曹) Ph.D." wrote:
> what are the NFS mount options?
> what are the tcp window size
> is jumbo frame on or off
> bs=1024 is very small, did you test various different bs size?
> ZFS what is the block size? stripwidth?
>
>
> On 5/28/2011 8:52 PM, Dan Swartzendruber wrote:
>> This is kind of a mystery.  Is it the same linux client in both cases?
>>
>> -----Original Message-----
>> From: Pete Ashdown [mailto:pashdown at xmission.com]
>> Sent: Friday, May 27, 2011 7:06 PM
>> To: Discussion list for OpenIndiana
>> Subject: [OpenIndiana-discuss] NFS Linux client slow reads
>>
>> Both NFS tests were conducted on the same hardware, with 4 aggregated
>> 802.3ad gig-e ports on the server and 2 aggregated 802.3ad gig-e ports on
>> the client.  The OI server has 24GB RAM, 8 2TB Seagate drives, and a 32GB
>> SSD for ZIL and a 160GB SSD for L2ARC.
>>
>> Test 1: Ubuntu Linux server&  client
>>
>> #  dd if=/dev/zero of=test bs=1024 count=1M
>> 1073741824 bytes (1.1 GB) copied, 13.3347 s, 80.5 MB/s
>> # echo 3>  /proc/sys/vm/drop_caches  (clear Linux file buffers)
>> # dd if=test of=/dev/null
>> 1073741824 bytes (1.1 GB) copied, 10.0029 s, 107 MB/s
>>
>> Test 2: OI 148 server&  Ubuntu Linux client
>>
>> # dd if=/dev/zero of=test bs=1024 count=1M
>> 1073741824 bytes (1.1 GB) copied, 12.0275 s, 89.3 MB/s
>> # echo 3>  /proc/sys/vm/drop_caches
>> # dd if=test of=/dev/null bs=1024
>> 1073741824 bytes (1.1 GB) copied, 112.205 s, 9.6 MB/s<--- ~10% of Linux
>> server
>>
>> Test 3: On the OI server
>>
>> # dd if=/dev/zero of=test bs=1024 count=1M
>> 1073741824 bytes (1.1 GB) copied, 7.79955 s, 138 MB/s
>> # dd if=test of=/dev/null bs=1024
>> 1073741824 bytes (1.1 GB) copied, 3.38502 s, 317 MB/s
>>
>> Any help is appreciated. 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 254 bytes
Desc: OpenPGP digital signature
URL: <http://openindiana.org/pipermail/openindiana-discuss/attachments/20110529/bc31170e/attachment.bin>