[OpenIndiana-discuss] Initial install of 2020.10 on Z840 for production use

[Toomas Soome]

> On 25. Apr 2021, at 23:46, Reginald Beardsley via openindiana-discuss <openindiana-discuss at openindiana.org> wrote:
> 
> I've done a fresh install using the text installer on a 14 core E5-2680 V4 system with 72 GB of ECC DRAM and a 4x 4 TB 7200 rpm RAIDZ2 array.  With reconfigure set on the post install boot it all came up fine.
> 
> The prior install of 2021.04-rc1 gave ~220 MB/s for reads and writes and 87 MB/s for a file copy on a 3 disk RAIDZ1 for a 1 TB file size.  A dev/zero write is running very much slower than 2021.04-rc1.  On that it took a little over an hour.  This write has been running for almost 4 hrs.
> 

This slowdown means something more was changed. what driver is used for disks? (prtconf -D)


> With a dd write of 1 TB of  /dev/zero to a file running, the GUI response is appallingly slow.  Minutes to bring up Firefox. 18 seconds to open a MATE terminal.  Unblanking the screen took almost 1 minute to bring up the screensaver login.

writing zero blocks should be rather fast:

tsoome at beastie:~$ gdd if=/dev/zero of=test bs=1GB count=1024
1024+0 records in
1024+0 records out
1024000000000 bytes (1.0 TB, 954 GiB) copied, 249.572 s, 4.1 GB/s

this is on raidz1 in 3+1 setup with WD Black SATA disks.

Note this throughput is not disk throughput, its memory operations, because of ZLE, only metadata is updated on disks, this is iostat -xnM 1 from the time of my dd test:

    r/s    w/s   Mr/s   Mw/s wait actv wsvc_t asvc_t  %w  %b device
    0,0  702,4    0,0    5,6  1,5  0,2    2,2    0,3   6   7 rpool
    0,0  185,8    0,0    1,4  0,0  0,1    0,0    0,4   0   6 c3t0d0
    0,0  182,8    0,0    1,4  0,0  0,1    0,0    0,3   0   5 c3t1d0
    0,0  190,8    0,0    1,4  0,0  0,0    0,0    0,1   0   1 c3t3d0
    0,0  178,8    0,0    1,4  0,0  0,1    0,0    0,3   0   5 c3t4d0


rgds,
toomas

> 
> Top shows the ARC consuming 52 GB and about 10 GB free with the CPU 95% idle.  This seems to me very strange for the response I'm getting.
> 
> Relative to my Z400s or my NL40 this thing is a complete dog. It's about what an 11/780 which is badly thrashing would do.   I assume that there are system  parameters which need to be modified.  Reducing the size of the ARC seems the most probable first step.  For my prior testing I only had 8 GB of DRAM on a single DIMM.  I installed 4x of the fastest new Micron 16 GB DIMMs as specified by the Crucial website plus the 8 GB DIMM it came with.
> 
> "mdump -e" shows no events except for 3x pci.fabric events per boot which I assume are related to missing device drivers.
> 
> After the write completes, I am going to remove the 8 GB DIMM.
> 
> Reg
> 
> _______________________________________________
> openindiana-discuss mailing list
> openindiana-discuss at openindiana.org
> https://openindiana.org/mailman/listinfo/openindiana-discuss