Friday, May 19, 2017

High CPU utilization due to java process on windows

Issue : java is using up most of the CPU in a windows server.

Artifacts needed : powershell

Solution : Note down the java pid causing high cpu using task manager
check high CPU using taskbar using process tab and details tab.
check pid causing high CPU

If jvm is started using service, please stop the service.

Open the cmd in admin mode and right click ta the top bar of cmd and go to properties
Under "layout" tab, of "screen buffer size", increase the Height to 5000 and click OK.
exit the cmd and open again in admin mode.

Start the jvm from cmd line in admin mode and do not close the cmd window.

wait for the CPU to peak again and take down new pid from task manager.

Now on the cmd where the jvm is running, press ctrl break atleast 3 times within an interval of 5secs each, which will generate 3 threaddump

right click on cmd top bar and edit - select all, then press ENter,

Now copy all the contents to a notepad and save, this is your threaddump.

Open powershell in admin mode and enter the below command:

(get-process -id XXXX).threads | sort-object {$_.TotalProcessorTime} > c:\1.txt

XXXX represents the high cpu java pid.

in the output 1.txt, the (bottom) last stack, the Id is the thread id and TotalProcessorTime is the most used cpu time of the pid in decimal format taking maximum processor time

Below is an example:

BasePriority            : 8
CurrentPriority         : 10
Id                      : 7860
IdealProcessor          :
PriorityBoostEnabled    : True
PriorityLevel           : Normal
PrivilegedProcessorTime : 00:20:56.5781250
StartAddress            : 140726260225200
StartTime               : 5/19/2017 9:14:18 AM
ThreadState             : Running
TotalProcessorTime      : 00:49:12.1718750
UserProcessorTime       : 00:28:15.5937500
WaitReason              :
ProcessorAffinity       :
Site                    :
Container               :

TotalProcesorTime is the sum of UserProcessorTime and PrivilagedProcessorTime.
Usertime is related to application requests and Privileged is related to kernel ops.

So 3 iterations over an period time showing the same Thread ID with highest TotalProcessorTime is the real culprit.


convert the last Id from decimal to hexadecimal add 0X at the front (if needed)
search this hex value in threaddump generated earlier.
The found stack trace is the culprit.

No comments: