[OpenIndiana-discuss] ZFS read speed(iSCSI)

[Edward Ned Harvey (openindiana)]

> From: Heinrich van Riel [mailto:heinrich.vanriel at gmail.com]
> 
> I will post my findings, but might take some time to fix the network in
> time and they will have to deal with 1Gbps for the storage. The request is
> to run ~90 VMs on 8 servers connected.

With 90 VM's on 8 servers, being served ZFS iscsi storage by 4x 1Gb ethernet in LACP, you're really not going to care about any one VM being able to go above 1Gbit.  Because it's going to be so busy all the time, that the 4 LACP bonded ports will actually be saturated.  I think your machines are going to be slow.  I normally plan for 1Gbit per VM, in order to be comparable with a simple laptop.

You're going to have a lot of random IO.  I'll strongly suggest you switch to mirrors instead of raidz.

And I'll strongly suggest adding a log device.  Your cache device might not give you benefit because 90 VM's might be too large to yield a substantial cache hit, but the log device almost certainly will benefit, as all your iscsi writes will be sync writes.