[OpenIndiana-discuss] Slowly increasing kernel load with oi_151a

[Chris Bünger]

I experience the same issue. For me, a simple reboot did not help, but
a reboot -p.
I have similar setups with 134 and 151a and only 151a shows this behavior,

Maybe this helps.

Chris

Am 23.12.2011 um 01:08 schrieb Mirko Kaffka <mk at mkaffka.de>:

> I recently got a new machine and installed oi_151a from scratch
> (core i7-2600, intel motherboard DH67BL, 16GB RAM).
> After a few days uptime I noticed a constant system load of about 10%
> although the desktop was idle and I had not started anything that caused
> a permanent load. There was almost no I/O activity, just a few reads
> and writes every few seconds. vmstat showed 0-1% user time but
> 10-13% system time. prstat -v output was far below 1% or 0% user and
> system time for all processes.
> Over the following days the load increased further. When I took 7 cpu
> cores off-line I got about 80% sys load on the remaining core. Where
> does it come from?
>
> When I switch from multi user to single user mode the load persists.
> When I reboot, everything is fine for a while (0-1% sys load) but the load
> slowly starts increasing again. So, I have to reboot the machine about
> every 2 days what is very unpleasant.
>
> I tried to analyze the issue using intrstat, lockstat, etc. but have not
> got very far.
>
> All following commands were run in single user mode and with only one cpu
> core on-line. (I hope it's ok to put the output here?)
>
> ~ # vmstat 5
> kthr      memory            page            disk          faults      cpu
> r b w   swap  free  re  mf pi po fr de sr s1 s2 s3 s4   in   sy   cs us sy id
> 0 0 0 9070220 2859260 2  4  0  0  0  0  0  5 -1  1 14  397  265  258  0  4 95
> 0 0 0 10392120 4142932 24 64 0 0  0  0  0  0  0  0  1  508   99  227  0 21 79
> 0 0 0 10392120 4142960 0 0  0  0  0  0  0  0  0  0 13  511   60  229  0 21 79
> 0 0 0 10392124 4142964 0 0  0  0  0  0  0  0  0  0 13  509   59  226  0 21 79
>
> ~ # ps -ef
>     UID   PID  PPID   C    STIME TTY         TIME CMD
>    root     0     0   0   Dec 20 ?           0:01 sched
>    root     4     0   0   Dec 20 ?           0:00 kcfpoold
>    root     6     0   0   Dec 20 ?           2:34 zpool-rpool
>    root     1     0   0   Dec 20 ?           0:00 /sbin/init
>    root     2     0   0   Dec 20 ?           0:00 pageout
>    root     3     0   0   Dec 20 ?           8:54 fsflush
>    root    10     1   0   Dec 20 ?           0:03 /lib/svc/bin/svc.startd
>    root    12     1   0   Dec 20 ?           0:08 /lib/svc/bin/svc.configd
>  netadm    50     1   0   Dec 20 ?           0:00 /lib/inet/ipmgmtd
>   dladm    46     1   0   Dec 20 ?           0:00 /sbin/dlmgmtd
>    root   167     0   0   Dec 20 ?           1:50 zpool-tank
>    root   232     1   0   Dec 20 ?           0:00 /usr/lib/sysevent/syseventd
>    root  9518    10   0 21:01:56 console     0:00 -bash
>    root   262     1   0   Dec 20 ?           0:02 devfsadmd
>    root   276     1   0   Dec 20 ?           0:00 /usr/lib/power/powerd
>    root 10708  9518   0 21:04:53 console     0:00 ps -ef
>    root  3222     1   0   Dec 20 ?           0:00 -bash
>
>
> ~ # intrstat
>      device |      cpu0 %tim      cpu1 %tim      cpu2 %tim      cpu3 %tim
> -------------+------------------------------------------------------------
>    e1000g#1 |         1  0.0         0  0.0         0  0.0         0  0.0
>      ehci#0 |         1  0.0         0  0.0         0  0.0         0  0.0
>      ehci#1 |         1  0.0         0  0.0         0  0.0         0  0.0
>      rtls#0 |         1  0.0         0  0.0         0  0.0         0  0.0
> (cpu4..7 are all 0.0%)
>
>
> ~ # prstat -v
>   PID USERNAME USR SYS TRP TFL DFL LCK SLP LAT VCX ICX SCL SIG PROCESS/NLWP
> 10711 root     0.0 0.0 0.0 0.0 0.0 0.0 100 0.0   1   0 254   0 prstat/1
>  3222 root     0.0 0.0 0.0 0.0 0.0 0.0 100 0.0   0   0   0   0 bash/1
> [cut]
> Total: 14 processes, 366 lwps, load averages: 0.02, 0.32, 0.67
>
>
> ~ # lockstat -kIW -D20 sleep 30
>
> Profiling interrupt: 2913 events in 30.028 seconds (97 events/sec)
>
> Count indv cuml rcnt     nsec Hottest CPU+PIL        Caller
> -------------------------------------------------------------------------------
> 2878  99%  99% 0.00      293 cpu[0]                 acpi_cpu_cstate
>   12   0%  99% 0.00      224 cpu[0]                 fsflush
>   10   0% 100% 0.00      266 cpu[0]                 i86_mwait
> [cut]
> -------------------------------------------------------------------------------
>
> Is the high count on acpi_cpu_cstate normal?
>
> The hotkernel script from the dtrace toolkit finally froze my system.
> After the reboot hotkernel run flawlessly.
>
> How can I further analyze this?
>
> Thanks,
> Mirko
>
>
> _______________________________________________
> OpenIndiana-discuss mailing list
> OpenIndiana-discuss at openindiana.org
> http://openindiana.org/mailman/listinfo/openindiana-discuss