Tuesday, May 30, 2017

Symantec NetBackup PureDisk change URL / port / ip address

How to get the puredisk url working on another address.

Login to the server using a putty session and enter the below command

ps -ef | grep java

The result shows us that, it is a tomcat server hosting the webpage.

HOSTNAME:~ # ps -ef | grep java
root     25012 24974  0 10:56 pts/0    00:00:00 grep java
root     29395     1  0 May29 ?        00:00:19 /opt/VRTSjre/jre1.5/bin/java -Dvrtsat.libpath=/opt/VRTSat/lib/x64 -Duuid.libpath=/etc/vx/uuid/bin -Dvrtsweb.home=/opt/VRTSweb -Dvrtsweb.runtime.dir=/var/VRTSweb -Dvrtsweb.log.dir=/var/VRTSweb/log -Dvcs.home=/opt/VRTSvcs -Dvcs.conf=/etc/VRTSvcs -Xmx1024m -Dvrtsweb.max.heap.size=1024 -server -Xrs -Djava.awt.headless=true -Dsun.java2d.noddraw=true -Xms64m -Xss512k vrts.tomcat.bootstrap.Main start

The configuration must have been under /opt/VRTSvcs.
Go to the config folder /opt/VRTSweb/conf

open the file vrtsweb.xml.

# more vrtsweb.xml

<Ports securePortRedirection="false">
                <Port port="8080" protocol="http" address="" />
                <Port port="8009" protocol="ajp"  address="" />

We see that the page can be accessed on ip & ports 8080 & 8009
However is internal to the server.

So edit the file and change it like below:

# vi vrtsweb.xml

<Ports securePortRedirection="false">
                <Port port="8080" protocol="http"                                  />
                <Port port="8009" protocol="ajp"  address="" />

Change the listen address from to all ( earlier the port 8080 was listening to only, now it listens on all ip’s)
You can also change the port here if you want.

save and exit the file and restart the tomcat. (service puredisk restart)

check the ip's the host is configured to using command <ip addr>

The result will show you all the ip's the host is listening on under <inet>

simply access the url on any of the <inet> ip address and port 8080,
for example ip:8080/PureDisk/index.do

If there is an address translation in the server, simply configure a tunnel at your putty, to access it as localhost.

Friday, May 19, 2017

High CPU utilization due to java process on windows

Issue : java is using up most of the CPU in a windows server.

Artifacts needed : powershell

Solution : Note down the java pid causing high cpu using task manager
check high CPU using taskbar using process tab and details tab.
check pid causing high CPU

If jvm is started using service, please stop the service.

Open the cmd in admin mode and right click ta the top bar of cmd and go to properties
Under "layout" tab, of "screen buffer size", increase the Height to 5000 and click OK.
exit the cmd and open again in admin mode.

Start the jvm from cmd line in admin mode and do not close the cmd window.

wait for the CPU to peak again and take down new pid from task manager.

Now on the cmd where the jvm is running, press ctrl break atleast 3 times within an interval of 5secs each, which will generate 3 threaddump

right click on cmd top bar and edit - select all, then press ENter,

Now copy all the contents to a notepad and save, this is your threaddump.

Open powershell in admin mode and enter the below command:

(get-process -id XXXX).threads | sort-object {$_.TotalProcessorTime} > c:\1.txt

XXXX represents the high cpu java pid.

in the output 1.txt, the (bottom) last stack, the Id is the thread id and TotalProcessorTime is the most used cpu time of the pid in decimal format taking maximum processor time

Below is an example:

BasePriority            : 8
CurrentPriority         : 10
Id                      : 7860
IdealProcessor          :
PriorityBoostEnabled    : True
PriorityLevel           : Normal
PrivilegedProcessorTime : 00:20:56.5781250
StartAddress            : 140726260225200
StartTime               : 5/19/2017 9:14:18 AM
ThreadState             : Running
TotalProcessorTime      : 00:49:12.1718750
UserProcessorTime       : 00:28:15.5937500
WaitReason              :
ProcessorAffinity       :
Site                    :
Container               :

TotalProcesorTime is the sum of UserProcessorTime and PrivilagedProcessorTime.
Usertime is related to application requests and Privileged is related to kernel ops.

So 3 iterations over an period time showing the same Thread ID with highest TotalProcessorTime is the real culprit.

convert the last Id from decimal to hexadecimal add 0X at the front (if needed)
search this hex value in threaddump generated earlier.
The found stack trace is the culprit.