[OpenIndiana-discuss] Inefficient zvol space usage on 4k drives

[Lionel Cons]

On 8 August 2013 17:11, Richard Elling <richard.elling at richardelling.com> wrote:
> On Aug 7, 2013, at 2:50 PM, Jason Lawrence <jjlawren at gmail.com> wrote:
>
>> This might be a better question for the Illumos group, so please let me know.
>>
>> I have a zvol for a KVM instance which I felt was taking up too much space. After doing a little research, I stumbled upon http://support.freenas.org/ticket/2383 and repeated the test on my machine. I'm running a RAIDZ2 pool with eight "advanced format" 4k sector drives. I created the pool with ashift=12 with this in mind.
>>
>> root at hostname:~# zfs create -V 20g -o volblocksize=32k storage/testbed/32k
>> root at hostname:~# zfs create -V 20g -o volblocksize=8k storage/testbed/8k
>>
>> root at hostname:~# dd if=/dev/zero of=/dev/zvol/rdsk/storage/testbed/32k bs=1048576 count=20400
>> 20400+0 records in
>> 20400+0 records out
>> 21390950400 bytes (21 GB) copied, 493.836 s, 43.3 MB/s
>>
>> root at hostname:~# dd if=/dev/zero of=/dev/zvol/rdsk/storage/testbed/8k bs=1048576 count=20400
>>
>> 20400+0 records in
>> 20400+0 records out
>> 21390950400 bytes (21 GB) copied, 548.916 s, 39.0 MB/s
>>
>> root at hostname:~# zfs list | grep testbed
>> storage/testbed              64.2G  5.79T   307K  /storage/testbed
>> storage/testbed/32k          21.3G  5.79T  21.3G  -
>> storage/testbed/8k           42.8G  5.79T  42.8G  -
>>
>>
>> The 8k blocksize zvol takes up twice the space of the 32k blocksize zvol. As 8k is the default, this must be affecting others...
>
> This is expected. For 4K sector sizes and > 9 disks per raidz2 set:
>         volblocksize = 8k, raidz2 writes 2 data (8k) + 2 parity (8k)  [like mirroring]
>         volblocksize = 32k, raidz2 writes 8 data (32k) + 2 parity (8k) [like RAID-6]
>
> What most folks miss is:
>         volblocksize = 4k, raidz2 writes 1 data (4k) + 2 parity (8k) [like triple mirroring]
>
> so, don't do 4k recordsize with 4K sector disks unless you are mirroring.

What about disks (i.e. iSCSI connected to a hardware raid; the larger
sector size is intentional to improve throughput) which have even
larger sectors, e.g. 8k sectors?

Lionel