[OpenIndiana-discuss] dc s3700 for pool devices -- poor performance

[Jason Matthews]

Do we have anyone using the s3700 in their pools with any kind of scale?

I picked up an eval from dell using an r820 4x4650L, 576GB of RAM, and 8 dc
s3700 drives on OI151a8. Testing with filebench and the random write
personality indicates one device pushes 14k 8kb write operations/second. I
really thought it would turn out more but the news gets worse.

As I expand the number of devices in the spool (stripped), the performance
actually degrades.

The pool config alternates devices against two lsi 9207-8i cards in round
robin.

Test results look like this for 8kb random writes using 64 writer threads
and a single instance of filebench:

Spans | write ops/s uncompressed | write ops/s lzjb 
1     | 10k/s                    | 14k/s            
2     | 16k/s                    | 24k/s
3     | 18k/s                    | 24k/s
4     | 18k/s                    | 20k/s
5     | 18k/s                    | 20k/s
6     | 18k/s                    | 21k/s

Conversely, I have OI151a1 systems with Intel 910 drives that write out 56k
8k ops using the same profile. I know the mpt driver changed between a1 and
8a but if it was this slow someone would have noticed it by now.

I am clearly bottlenecked some where. In theory, this is the top of midrange
hardware. I have tried adjustments to zfs_vdev_max_pending, zfs_txg_timeout,
zfs_txg_synctime but none of that has made much of a difference.

Are there any ideas out there?



j.