[OpenIndiana-discuss] Appalling disk performance for KVM
Open Indiana
openindiana at out-side.nl
Wed Jan 4 08:45:13 UTC 2012
Errata:
You need to consider what you are doing.
You are piling a virtualiser (KVM) on an OS (OpenIndiana) and on top of the
virtualiser you are piling another OS (windows) and after creating this pile
you wonder why it won't stand stable...
Keep thing where they belong. ZFS is lighting fast when it runs on Solaris
or OpenIndiana, but its performance degrades as soon as you start piling up
other things on it, like a windows harddrivelayout.
So when it comes to store or distribute data keep the data on the layer that
is fastest, in this case the OpenIndiana OS.
-----Original Message-----
From: Open Indiana [mailto:openindiana at out-side.nl]
Sent: woensdag 4 januari 2012 9:35
To: 'Discussion list for OpenIndiana'
Subject: Re: [OpenIndiana-discuss] Appalling disk performance for KVM
What you could have done was:
1. use Virtualbox for your virtual OS's
2. create a minimal C: drive for the OS only
3. create 3 iscsi drives on OpenIndiana from a ZFS drive
4. import these drives as storage in your VM's
Apart from step 1 you could do it right now. I guess you will see improved
performance, going up to 50MB/s
-----Original Message-----
From: Russell Hansen [mailto:russhan at new-swankton.net]
Sent: woensdag 4 januari 2012 5:51
To: Discussion list for OpenIndiana
Subject: [OpenIndiana-discuss] Appalling disk performance for KVM
I'm seeing absolutely terrible disk performance for my virtual machines
running under KVM.
My ZFS pool is 6 Western Digital AV-25 500GB disks arranged in 3 mirrored
vdevs connected to an LSI 9210 (reflashed IBM M1015) in IT mode.
My KVM startup script:
/usr/bin/qemu-kvm \
-enable-kvm \
-smp 2 \
-m 1024 \
-no-hpet \
-rtc base=localtime,driftfix=slew \
-drive
file=/dev/zvol/dsk/oi_data/kvm/system/disk0,if=virtio,format=raw,cache=none,
index=0 \ -drive
file=/dev/zvol/dsk/oi_data/kvm/system/disk1,if=virtio,format=raw,cache=none,
index=1 \ -net nic,vlan=0,name=lan0,model=virtio,macaddr=$MAC0 \ -net
vnic,vlan=0,name=lan0,ifname=$ETH0 \ -net
nic,vlan=1,name=int0,model=virtio,macaddr=$MAC1 \ -net
vnic,vlan=1,name=int0,ifname=$ETH1 \ -vnc $LAN_ADDR:$1 \ -monitor
telnet:127.0.0.1:444$1,server,nowait \ -usbdevice tablet \ -daemonize
My zvol settings:
NAME PROPERTY VALUE
SOURCE
oi_data/kvm/sachiel/disk1 type volume
-
oi_data/kvm/sachiel/disk1 creation Tue Nov 8 11:56
2011 -
oi_data/kvm/sachiel/disk1 used 298G
-
oi_data/kvm/sachiel/disk1 available 844G
-
oi_data/kvm/sachiel/disk1 referenced 139G
-
oi_data/kvm/sachiel/disk1 compressratio 1.00x
-
oi_data/kvm/sachiel/disk1 reservation none
default
oi_data/kvm/sachiel/disk1 volsize 144G
local
oi_data/kvm/sachiel/disk1 volblocksize 8K
-
oi_data/kvm/sachiel/disk1 checksum on
default
oi_data/kvm/sachiel/disk1 compression off
default
oi_data/kvm/sachiel/disk1 readonly off
default
oi_data/kvm/sachiel/disk1 copies 1
default
oi_data/kvm/sachiel/disk1 refreservation 149G
local
oi_data/kvm/sachiel/disk1 primarycache all
default
oi_data/kvm/sachiel/disk1 secondarycache all
default
oi_data/kvm/sachiel/disk1 usedbysnapshots 11.8G
-
oi_data/kvm/sachiel/disk1 usedbydataset 139G
-
oi_data/kvm/sachiel/disk1 usedbychildren 0
-
oi_data/kvm/sachiel/disk1 usedbyrefreservation 147G
-
oi_data/kvm/sachiel/disk1 logbias latency
default
oi_data/kvm/sachiel/disk1 dedup off
default
oi_data/kvm/sachiel/disk1 mlslabel none
default
oi_data/kvm/sachiel/disk1 sync standard
default
oi_data/kvm/sachiel/disk1 refcompressratio 1.00x
-
Representative iostat output while moving some 4GB of digital photos (approx
2MB each):
capacity operations bandwidth
pool alloc free read write read write
------------------------- ----- ----- ----- ----- ----- -----
oi_data 373G 1019G 71 377 569K 3.98M
mirror 124G 340G 15 127 125K 1.34M
c2t50014EE2AFBDC3CBd0 - - 2 71 118K 1.34M
c2t50014EE25A683C10d0 - - 0 69 7.01K 1.34M
mirror 124G 340G 31 123 256K 1.33M
c2t50014EE20518D4F6d0 - - 4 49 132K 1.33M
c2t50014EE2AFBD87B4d0 - - 4 48 123K 1.33M
mirror 124G 340G 23 125 189K 1.32M
c2t50014EE205154E0Bd0 - - 3 52 35.1K 1.32M
c2t50014EE25A6829D6d0 - - 4 51 156K 1.32M
------------------------- ----- ----- ----- ----- ----- -----
A Bonnie result from a napp-it install:
Version 1.03c ------Sequential Output------ --Sequential Input-
--Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
--Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec
%CP
2011.12.12 32G 61586 99 151550 16 95819 17 49188 99 273833 18
1166 3
------Sequential Create------ --------Random
Create--------
-Create-- --Read--- -Delete-- -Create-- --Read---
-Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec
%CP
16 32070 99 +++++ +++ +++++ +++ 32736 100 +++++ +++ +++++
+++
2011.12.12,32G,61586,99,151550,16,95819,17,49188,99,273833,18,1166.1,3,16,32
070,99,+++++,+++,+++++,+++,32736,100,+++++,+++,+++++,+++
As you can see, it basically hovers around 4-6 MB/s average with an
approximate 10KB I/O size so it appears I'm 100% iops limited. Do I need to
recreate my zvols with a larger volblocksize to get decent performance or is
it something else I need to do on the KVM config side or within the virtual
machine?
I have one Windows Server 2003 (32-bit) VM and 2x Windows Server 2008 R2
(64-bit) VMs and they all behave the same way when it comes to disk perf.
Thanks,
-Russ
_______________________________________________
OpenIndiana-discuss mailing list
OpenIndiana-discuss at openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss
_______________________________________________
OpenIndiana-discuss mailing list
OpenIndiana-discuss at openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss
More information about the OpenIndiana-discuss
mailing list