[OpenIndiana-discuss] ZFS read speed(iSCSI)

[Edward Ned Harvey (openindiana)]

> From: Jim Klimov [mailto:jimklimov at cos.ru]
> 
> > With 90 VM's on 8 servers, being served ZFS iscsi storage by 4x 1Gb
> > ethernet in LACP, you're really not going to care about any one VM being
> > able to go above 1Gbit.  Because it's going to be so busy all the time, that the
> > 4 LACP bonded ports will actually be saturated.  I think your machines are
> > going to be slow.  I normally plan for 1Gbit per VM, in order to be comparable
> > with a simple laptop.
> >
> > You're going to have a lot of random IO.  I'll strongly suggest you switch to
> > mirrors instead of raidz.
> 
> I'll leave your practical knowledge in higher regard than my theoretical
> hunches, but I believe typical PCs (including VDI desktops) don't do
> much disk IO after they've loaded the OS or a requested application.

Agreed, disk is mostly idle except when booting or launching apps.  Some apps write to disk, such as internet browsing caching stuff, and MS Office constantly hitting the PST or OST file, and Word/Excel autosave, etc.

But there are 90 of them.  So even "idle" time multiplied by 90 is no longer idle time.  And most likely, when they *do* get used, a whole bunch of them will get used at the same time.  (20 students all browsing the internet in between classes, or 20 students all doing homework between 5pm and 9pm, but they're all asleep from 4am to 6am, so all 90 instances are idle during that time...)


> And from what I read, if his 8 VM servers would contact the ZFS storage
> box with requests to many more targets, then on average all NICs will
> likely get their share of work, for one connection or another, even as
> part of LACP trunks (which may be easier to manage than VLAN-based
> MPxIO, with its separate benefits however). Right?..

Yup, with 8 VM servers, each having 11 VM guests, even if each server has a single 1Gb link to the 4x LACP storage server, I expect the 4x LACP links will see pretty heavy and well distributed usage.


> It might seem like a good idea
> to use dedup as well, 

Not if you care at all about performance, or usability.


> So here's my 2c, but they may be wrong ;)

:-)

I guess the one thing I can still think to add here is this:

If the 90 VM's all originated as clones of a single system, and the deviation *from* that original system remains minimal, then the ARC & L2ARC cache will do wonders.  Because when the first VM requests the boot blocks and the OS blocks, and all the blocks to launch Internet Explorer (or whatever app)...  Those things can get served from cache to satisfy the 89 subsequent requests to do the same actions.