[OpenIndiana-discuss] Problem with high cpu load (oi_151a)
Gernot Wolf
gw.inet at chello.at
Thu Oct 20 18:57:26 UTC 2011
As far as I was able to understand the output of my initial diagnostic
commands, it's indeed the kernel that causes the cpu load.
I've attached the dtrace results, as they are rather lengthy. I've run
each of them a couple of seconds. What got my attention on the first
glance (as far as I'm able to understand anything of these figures), was
that "acpi" showed up a lot in the output of the second dtrace command.
As Steve already pointed out, the lockstat results also showed acpi
debug tracing functions. Which isn't the case when I run the lockstat on
the OI box at my office.
Can the power supply have anything to do with all of this? I had to
replace it a few days ago, because the old one had a meltdown...
Regards,
Gernot Wolf
Am 20.10.11 20:07, schrieb Michael Stapleton:
> That rules out userland.
>
> Sched tells me that it is not a user process. If kernel code is
> executing on a cpu, tools will report the sched process. The count was
> how many times the process was taken off the CPU while dtrace was
> running.
>
>
>
> Lets see what kernel code is running the most:
>
> #dtrace -n 'sched:::off-cpu { @[stack()]=count()}'
>
> #dtrace -n 'profile-1001 { @[stack()] = count() }'
>
>
>
> On Thu, 2011-10-20 at 19:52 +0200, Gernot Wolf wrote:
>
>> Yeah, I've been able to run this diagnostics on another OI box (at my
>> office, so much for OI not being used in production ;)), and noticed
>> that there were several values that were quite different. I just don't
>> have any idea on the meaning of this figures...
>>
>> Anyway, here are the results of the dtrace command (I executed the
>> command twice, hence two result sets):
>>
>> gernot at tintenfass:~# dtrace -n 'sched:::off-cpu { @[execname]=count()}'
>> dtrace: description 'sched:::off-cpu ' matched 3 probes
>> ^C
>>
>> ipmgmtd 1
>> gconfd-2 2
>> gnome-settings-d 2
>> idmapd 2
>> inetd 2
>> miniserv.pl 2
>> netcfgd 2
>> nscd 2
>> ospm-applet 2
>> ssh-agent 2
>> sshd 2
>> svc.startd 2
>> intrd 3
>> afpd 4
>> mdnsd 4
>> gnome-power-mana 5
>> clock-applet 7
>> sendmail 7
>> xscreensaver 7
>> fmd 9
>> fsflush 11
>> ntpd 11
>> updatemanagernot 13
>> isapython2.6 14
>> devfsadm 20
>> gnome-terminal 20
>> dtrace 23
>> mixer_applet2 25
>> smbd 39
>> nwam-manager 60
>> svc.configd 79
>> Xorg 100
>> sched 394078
>>
>> gernot at tintenfass:~# dtrace -n 'sched:::off-cpu { @[execname]=count()}'
>> dtrace: description 'sched:::off-cpu ' matched 3 probes
>> ^C
>>
>> automountd 1
>> ipmgmtd 1
>> idmapd 2
>> in.routed 2
>> init 2
>> miniserv.pl 2
>> netcfgd 2
>> ssh-agent 2
>> sshd 2
>> svc.startd 2
>> fmd 3
>> hald 3
>> inetd 3
>> intrd 3
>> hald-addon-acpi 4
>> nscd 4
>> gnome-power-mana 5
>> sendmail 5
>> mdnsd 6
>> devfsadm 8
>> xscreensaver 9
>> fsflush 10
>> ntpd 14
>> updatemanagernot 16
>> mixer_applet2 21
>> isapython2.6 22
>> dtrace 24
>> gnome-terminal 24
>> smbd 39
>> nwam-manager 58
>> zpool-rpool 65
>> svc.configd 79
>> Xorg 82
>> sched 369939
>>
>> So, quite obviously there is one executable standing out here, "sched",
>> now what's the meaning of this figures?
>>
>> Regards,
>> Gernot Wolf
>>
>>
>> Am 20.10.11 19:22, schrieb Michael Stapleton:
>>> Hi Gernot,
>>>
>>> You have a high context switch rate.
>>>
>>> try
>>> #dtrace -n 'sched:::off-cpu { @[execname]=count()}'
>>>
>>> For a few seconds to see if you can get the name of and executable.
>>>
>>> Mike
>>> On Thu, 2011-10-20 at 18:44 +0200, Gernot Wolf wrote:
>>>
>>>> Hello all,
>>>>
>>>> I have a machine here at my home running OpenIndiana oi_151a, which
>>>> serves as a NAS on my home network. The original install was OpenSolaris
>>>> 2009.6 which was later upgraded to snv_134b, and recently to oi_151a.
>>>>
>>>> So far this OSOL (now OI) box has performed excellently, with one major
>>>> exception: Sometimes, after a reboot, the cpu load was about 50-60%,
>>>> although the system was doing nothing. Until recently, another reboot
>>>> solved the issue.
>>>>
>>>> This does not work any longer. The system has always a cpu load of
>>>> 50-60% when idle (and higher of course when there is actually some work
>>>> to do).
>>>>
>>>> I've already googled the symptoms. This didn't turn up very much useful
>>>> info, and the few things I found didn't apply to my problem. Most
>>>> noticably was this problem which could be solved by disabling cpupm in
>>>> /etc/power.conf, but trying that didn't show any effect on my system.
>>>>
>>>> So I'm finally out of my depth. I have to admit that my knowledge of
>>>> Unix is superficial at best, so I decided to try looking for help here.
>>>>
>>>> I've run several diagnostic commands like top, powertop, lockstat etc.
>>>> and attached the results to this email (I've zipped the results of kstat
>>>> because they were>1MB).
>>>>
>>>> One important thing is that when I boot into the oi_151a live dvd
>>>> instead of booting into the installed system, I also get the high cpu
>>>> load. I mention this because I have installed several things on my OI
>>>> box like vsftpd, svn, netstat etc. I first thought that this problem
>>>> might be caused by some of this extra stuff, but getting the same system
>>>> when booting the live dvd ruled that out (I think).
>>>>
>>>> The machine is a custom build medium tower:
>>>> S-775 Intel DG965WHMKR ATX mainbord
>>>> Intel Core 2 Duo E4300 CPU 1.8GHz
>>>> 1x IDE DVD recorder
>>>> 1x IDE HD 200GB (serves as system drive)
>>>> 6x SATA II 1.5TB HD (configured as zfs raidz2 array)
>>>>
>>>> I have to solve this problem. Although the system runs fine and
>>>> absolutely serves it's purpose, having the cpu at 50-60% load constantly
>>>> is a waste of energy and surely a rather unhealthy stress on the hardware.
>>>>
>>>> Anyone any ideas...?
>>>>
>>>> Regards,
>>>> Gernot Wolf
>>>> _______________________________________________
>>>> OpenIndiana-discuss mailing list
>>>> OpenIndiana-discuss at openindiana.org
>>>> http://openindiana.org/mailman/listinfo/openindiana-discuss
>>>
>>>
>>> _______________________________________________
>>> OpenIndiana-discuss mailing list
>>> OpenIndiana-discuss at openindiana.org
>>> http://openindiana.org/mailman/listinfo/openindiana-discuss
>>>
>>
>> _______________________________________________
>> OpenIndiana-discuss mailing list
>> OpenIndiana-discuss at openindiana.org
>> http://openindiana.org/mailman/listinfo/openindiana-discuss
>
>
> _______________________________________________
> OpenIndiana-discuss mailing list
> OpenIndiana-discuss at openindiana.org
> http://openindiana.org/mailman/listinfo/openindiana-discuss
>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: dtrace1.txt
URL: <http://openindiana.org/pipermail/openindiana-discuss/attachments/20111020/967b585c/attachment-0002.txt>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: dtrace2.txt
URL: <http://openindiana.org/pipermail/openindiana-discuss/attachments/20111020/967b585c/attachment-0003.txt>
More information about the OpenIndiana-discuss
mailing list