[OpenIndiana-discuss] ashift 13?
Jason Matthews
jason at broken.net
Thu Apr 9 02:02:44 UTC 2015
Do you have any notion what if any performance penalty there is staying
with 512 byte sectors?
Given the cost of high quality SSDs I am not inclined to move to larger
sector sizes unless there is a real pay off. Right now, I dont see it.
j.
On 4/8/2015 5:41 PM, Matthew Ahrens wrote:
> On Wed, Apr 8, 2015 at 5:12 PM, jason matthews <jason at broken.net
> <mailto:jason at broken.net>> wrote:
>
>
> cross posting to zfs
>
> The data seems to indicate that something strange is happening
> between the compression algorithms and sector sizes > 512 bytes.
> ZFS reports virtually the same compression ratios for either
> ashift=9 or ashift=12 zpools,
>
>
> This is a bug. The compression ratio with sector size other than 512
> bytes (i.e. ashift != 9) is reported higher than it really is. I have
> a fix in the works.
>
> but the number of data blocks
>
>
> I think you mean amount of space allocated, e.g. as reported by the
> "referenced" or "used" properties. Number of data blocks should be
> the same.
>
> consumed varies drastically between 9 bit and 12 bit sectors.
>
>
> I think you mean 512-byte and 4KB sectors (2^9 and 2^12). 9-bit
> sectors sounds like something from my nightmares :-)
>
> Uncompressed file systems report the same disk usage so the
> problem seems tied towards the compression mechanisms. Moving to
> 13 bit, as the illumos advanced sector wiki page implies, results
> in even more amplication.
>
> why should 4k sectors consume nearly 2x storage compressed versus
> 512 byte zpools? this looks like some sort of bug to me.
>
> This is not a bug, it's an inherent effect of sector size. Take for
> example a 8KB (logical size) block that compresses down to 4.12KB.
> With 512 byte sectors, we will allocate 4.5KB (9 sectors), vs with 4KB
> sectors, we must allocate 8KB (2 sectors). This effect will be more
> pronounced with smaller blocks (e.g. recordsize=8k), and not very
> noticeable with large blocks (e.g. the default of 128KB), unless the
> compression ratio is very high (e.g. 128KB blocks that compresses to
> 4.12KB)
>
> --matt
>
>
> With Identical datasets installed on each file system we see a
> almost 2x magnification in disk usage in ashift=12 versus ashift=9.
>
>
> Do you any of you big brains on the ZFS team know why this happens?
> Is this a bug? Some sort of errant behavior in the Intel DC S3700?
> If it is not a bug, why would we want to move to 4k sectors?
> Is this a case where ZFS compression isnt tuned against 4k blocks?
>
> should I file a ticket?
>
> root at dbb005:/root# zfs list -r ashift9 ashift12 |grep postgres
> ashift12/128k/lz4/postgres 355G 40.7G 355G
> /ashift12/128k/lz4/postgres
> ashift12/128k/lzjb/postgres 403G 40.7G 403G
> /ashift12/128k/lzjb/postgres
> ashift12/128k/no-compression/postgres 640G 40.7G 640G
> /ashift12/128k/no-compression/postgres
> ashift12/8k/lz4/postgres 355G 40.7G 355G
> /ashift12/8k/lz4/postgres
> ashift12/8k/lzjb/postgres 403G 40.7G 403G
> /ashift12/8k/lzjb/postgres
>
> ashift9/128k/lz4/postgres 224G 6.32T 224G
> /ashift9/128k/lz4/postgres
> ashift9/128k/lzjb/postgres 277G 6.32T 277G
> /ashift9/128k/lzjb/postgres
> ashift9/128k/no-compression/postgres 638G 6.32T 638G
> /ashift9/128k/no-compression/postgres
> ashift9/8k/lz4/postgres 224G 6.32T 224G
> /ashift9/8k/lz4/postgres
> ashift9/8k/lzjb/postgres 224G 6.32T 224G
> /ashift9/8k/lzjb/postgres
>
> root at dbb005:/root# zfs get -r compressratio ashift9 ashift12 |
> grep postgres |grep -v @
> ashift12/128k/lz4/postgres compressratio 2.86x -
> ashift12/128k/lzjb/postgres compressratio 2.31x -
> ashift12/128k/no-compression/postgres compressratio 1.00x -
> ashift12/8k/lz4/postgres compressratio 2.86x -
> ashift12/8k/lzjb/postgres compressratio 2.31x -
>
> ashift9/128k/lz4/postgres compressratio 2.86x -
> ashift9/128k/lzjb/postgres compressratio 2.31x -
> ashift9/128k/no-compression/postgres compressratio 1.00x -
> ashift9/8k/lz4/postgres compressratio 2.86x -
> ashift9/8k/lzjb/postgres compressratio 2.86x -
>
>
>
> root at dbb005:/root# zpool status ashift12 ashift9
> pool: ashift12
> state: ONLINE
> scan: none requested
> config:
>
> NAME STATE READ WRITE CKSUM
> ashift12 ONLINE 0 0 0
> c3t55CD2E404B63FCD9d0 ONLINE 0 0 0
> c3t55CD2E404B64033Bd0 ONLINE 0 0 0
> c3t55CD2E404B63F93Cd0 ONLINE 0 0 0
>
> errors: No known data errors
>
> root at dbb005:/root# zdb ashift12 |egrep ashift:
> ashift: 12
> ashift: 12
> ashift: 12
> ashift: 12
> ashift: 12
> ashift: 12
>
>
>
> pool: ashift9
> state: ONLINE
> scan: none requested
> config:
>
> NAME STATE READ WRITE CKSUM
> ashift9 ONLINE 0 0 0
> c3t55CD2E404B417CFEd0 ONLINE 0 0 0
> c3t55CD2E404B43F0C9d0 ONLINE 0 0 0
> c3t55CD2E404B640592d0 ONLINE 0 0 0
> c3t55CD2E404B43F183d0 ONLINE 0 0 0
> c3t55CD2E404B417D1Fd0 ONLINE 0 0 0
> c3t55CD2E404B43F14Cd0 ONLINE 0 0 0
> c3t55CD2E404B43EFAAd0 ONLINE 0 0 0
> c3t55CD2E404B43EFCCd0 ONLINE 0 0 0
> c3t55CD2E404B41904Ad0 ONLINE 0 0 0
> c3t50015178F3659395d0 ONLINE 0 0 0
> c3t55CD2E404B64062Fd0 ONLINE 0 0 0
>
> errors: No known data errors
>
> root at dbb005:/root# zdb ashift9 |egrep ashift:
> ashift: 9
> ashift: 9
> ashift: 9
> ashift: 9
> ashift: 9
> ashift: 9
> ashift: 9
> ashift: 9
> ashift: 9
> ashift: 9
> ashift: 9
> ashift: 9
> ashift: 9
> ashift: 9
> ashift: 9
> ashift: 9
> ashift: 9
> ashift: 9
> ashift: 9
> ashift: 9
> ashift: 9
> ashift: 9
>
> root at dbb005:/root# iostat -En
> c3t55CD2E404B417CFEd0 Soft Errors: 0 Hard Errors: 0 Transport
> Errors: 0
> Vendor: ATA Product: INTEL SSDSC2BA80 Revision: DL04 Serial
> No: BTTV331202X3800
> Size: 800.17GB <800166076416 bytes>
> Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
> Illegal Request: 0 Predictive Failure Analysis: 0
> c3t55CD2E404B41904Ad0 Soft Errors: 0 Hard Errors: 0 Transport
> Errors: 0
> Vendor: ATA Product: INTEL SSDSC2BA80 Revision: DL04 Serial
> No: BTTV331207RE800
> Size: 800.17GB <800166076416 bytes>
> Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
> Illegal Request: 0 Predictive Failure Analysis: 0
> c3t55CD2E404B8149FAd0 Soft Errors: 0 Hard Errors: 0 Transport
> Errors: 0
> Vendor: ATA Product: INTEL SSDSC2CT18 Revision: 335u Serial
> No: CVKI338200ZT180
> Size: 180.05GB <180045766656 bytes>
> Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
> Illegal Request: 0 Predictive Failure Analysis: 0
> c3t55CD2E404B43EFAAd0 Soft Errors: 0 Hard Errors: 0 Transport
> Errors: 0
> Vendor: ATA Product: INTEL SSDSC2BA80 Revision: DL04 Serial
> No: BTTV333005CU800
> Size: 800.17GB <800166076416 bytes>
> Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
> Illegal Request: 0 Predictive Failure Analysis: 0
> c3t55CD2E404B406BBCd0 Soft Errors: 0 Hard Errors: 0 Transport
> Errors: 0
> Vendor: ATA Product: INTEL SSDSC2BA80 Revision: DL04 Serial
> No: BTTV330209W2800
> Size: 800.17GB <800166076416 bytes>
> Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
> Illegal Request: 0 Predictive Failure Analysis: 0
> c3t55CD2E404B417D1Fd0 Soft Errors: 0 Hard Errors: 0 Transport
> Errors: 0
> Vendor: ATA Product: INTEL SSDSC2BA80 Revision: DL04 Serial
> No: BTTV331202Y3800
> Size: 800.17GB <800166076416 bytes>
> Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
> Illegal Request: 0 Predictive Failure Analysis: 0
> c3t55CD2E404B43F0C9d0 Soft Errors: 0 Hard Errors: 0 Transport
> Errors: 0
> Vendor: ATA Product: INTEL SSDSC2BA80 Revision: DL04 Serial
> No: BTTV333005ME800
> Size: 800.17GB <800166076416 bytes>
> Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
> Illegal Request: 0 Predictive Failure Analysis: 0
> c3t55CD2E404B43EFCCd0 Soft Errors: 0 Hard Errors: 0 Transport
> Errors: 0
> Vendor: ATA Product: INTEL SSDSC2BA80 Revision: DL04 Serial
> No: BTTV333005DU800
> Size: 800.17GB <800166076416 bytes>
> Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
> Illegal Request: 0 Predictive Failure Analysis: 0
> c3t50015178F3659395d0 Soft Errors: 0 Hard Errors: 0 Transport
> Errors: 0
> Vendor: ATA Product: INTEL SSDSC2BA80 Revision: DL04 Serial
> No: BTTV326303UP800
> Size: 800.17GB <800166076416 bytes>
> Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
> Illegal Request: 0 Predictive Failure Analysis: 0
> c3t55CD2E404B43F14Cd0 Soft Errors: 0 Hard Errors: 0 Transport
> Errors: 0
> Vendor: ATA Product: INTEL SSDSC2BA80 Revision: DL04 Serial
> No: BTTV333005RC800
> Size: 800.17GB <800166076416 bytes>
> Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
> Illegal Request: 0 Predictive Failure Analysis: 0
> c3t55CD2E404B43F183d0 Soft Errors: 0 Hard Errors: 0 Transport
> Errors: 0
> Vendor: ATA Product: INTEL SSDSC2BA80 Revision: DL04 Serial
> No: BTTV333005T3800
> Size: 800.17GB <800166076416 bytes>
> Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
> Illegal Request: 0 Predictive Failure Analysis: 0
> c3t55CD2E404B63FCD9d0 Soft Errors: 0 Hard Errors: 0 Transport
> Errors: 0
> Vendor: ATA Product: INTEL SSDSC2BA80 Revision: 0270 Serial
> No: BTTV420600LD800
> Size: 800.17GB <800166076416 bytes>
> Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
> Illegal Request: 0 Predictive Failure Analysis: 0
> c3t55CD2E404B640592d0 Soft Errors: 0 Hard Errors: 0 Transport
> Errors: 0
> Vendor: ATA Product: INTEL SSDSC2BA80 Revision: 0270 Serial
> No: BTTV420602JK800
> Size: 800.17GB <800166076416 bytes>
> Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
> Illegal Request: 0 Predictive Failure Analysis: 0
> c3t55CD2E404B63F93Cd0 Soft Errors: 0 Hard Errors: 0 Transport
> Errors: 0
> Vendor: ATA Product: INTEL SSDSC2BA80 Revision: 0270 Serial
> No: BTTV420505XQ800
> Size: 800.17GB <800166076416 bytes>
> Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
> Illegal Request: 0 Predictive Failure Analysis: 0
> c3t55CD2E404B64062Fd0 Soft Errors: 0 Hard Errors: 0 Transport
> Errors: 0
> Vendor: ATA Product: INTEL SSDSC2BA80 Revision: 0270 Serial
> No: BTTV420602P6800
> Size: 800.17GB <800166076416 bytes>
> Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
> Illegal Request: 0 Predictive Failure Analysis: 0
> c3t55CD2E404B64033Bd0 Soft Errors: 0 Hard Errors: 0 Transport
> Errors: 0
> Vendor: ATA Product: INTEL SSDSC2BA80 Revision: 0270 Serial
> No: BTTV4206020X800
> Size: 800.17GB <800166076416 bytes>
> Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
> Illegal Request: 0 Predictive Failure Analysis: 0
>
> All Intel DC S3700 drives used in the ashift9 and ashift12 zpools
>
>
More information about the openindiana-discuss
mailing list