Sunday, December 2, 2012

Memory utilization / usage of java / weblogic processes on RAM for unix / linux

Get the output of below commands

1. cat /proc/meminfo
2. ps -A --sort -rss -o comm,pmem | head -n 11
3. ps -A --sort -rss -o pid,comm,pmem,rss
4. free -m
5. while read command percent rss; do if [[ "${command}" != "COMMAND" ]]; then rss="$(bc <<< "scale=2;${rss}/1024")"; fi; printf "%-26s%-8s%s\n" "${command}" "${percent}" "${rss}"; done < <(ps -A --sort -rss -o comm,pmem,rss | head -n 11)
6. ps -aux

2. COMMAND
ps -A --sort -rss -o comm,pmem | head -n 11

RESULT
COMMAND         %MEM
java            13.4
java            11.8
java            10.7
java             5.8
java             5.1
java             5.0
emagent          0.5
java             0.3
perl             0.0
ntpd             0.0

The java / weblogic processes are using only 52.1% of the total memory

4. COMMAND
free -m

RESULT
             total       used       free     shared    buffers     cached
Mem:         56378      55716        662          0        175       3245
-/+ buffers/cache:      52295       4083
Swap:         4094          0       4094


When you run free command, look for the line which says “-/+ buffers/cache”. This line which shows numbers under used and free, is the RAM you literally have used and literally

have free because of ram allocated to cache and buffers,
this is the line you should refer to when you want to see how much free RAM you have because RAM allocated to buffers and cache will be instantly released and given to a

process whenever it needs it.
So you have 4GB RAM free now.

5. COMMAND
while read command percent rss; do if [[ "${command}" != "COMMAND" ]]; then rss="$(bc <<< "scale=2;${rss}/1024")"; fi; printf "%-26s%-8s%s\n" "${command}" "${percent}" "${rss}"; done < <(ps -A --sort -rss -o comm,pmem,rss | head -n 11)

RESULT
COMMAND                   %MEM    RSS
java                      13.4    7610.14
java                      11.8    6686.60
java                      10.7    6088.11
java                      5.8     3274.82
java                      5.1     2893.45
java                      5.0     2821.26
emagent                   0.5     312.08
java                      0.3     175.11
perl                      0.0     9.97
ntpd                      0.0     4.78

RSS is the portion of memory that a process is using that is held in pysical RAM and is calculated in kb's.

6. COMMAND
ps -aux

RESULT
Warning: bad syntax, perhaps a bogus '-'? See /usr/share/doc/procps-3.2.7/FAQ
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.0   2064   620 ?        Ss   Oct20   0:03 init [5]
root         2  0.0  0.0      0     0 ?        S<   Oct20   0:00 [migration/0]
root         3  0.0  0.0      0     0 ?        SN   Oct20   0:00 [ksoftirqd/0]
root         4  0.0  0.0      0     0 ?        S<   Oct20   0:00 [watchdog/0]

shows you the rss memory as well as the virtual memory of all the pid's, you may then look for your respective pid in the list.
Please note that here the VSZ - virtual size is in kilobytes, RSS - real memory size is in byte units.



Some useful AIX commands:
svmon -Pg -t 1 |grep Pid ; svmon -Pg -t 10 |grep "N"          top 10 processes using the most paging space

svmon -P -O sortseg=pgsp                                               shows paging space usage of processes

ps gv | head -n 1; ps gv | egrep -v "RSS" | sort +6b -7 -n -r 

Wednesday, November 14, 2012

Cruise to lakshadweep

Organized by SPORTS
Booking agent: mintvalley kochi
Islands : Kalpeni & minicoy
Ship : MV lakshadweep sea
package : coral reef

Lakshadweep is one of the best places to travel if you would like to get away from the city for a couple of days. Nice warm shallow beaches, warm hearted people, world class facilities, amazing sea creatures and super duper food.
Mintvalley tours kochi (pretty - 0484 6062011) are one of the best agencies to reach out to for booking a slot on SPORTS. Make sure you plan your vacation a month in advance and always choose coral reef package over samudram package. Samudram is a 3 island tour with 158 co-passengers which creates unnecessary competition for fewer resources, while coral reef takes you to 2 islands with 8 passengers.
Make sure you travel by MV Lakshadweep which is one of the newest ships with world class facilities.
It is better to travel by 1st class which includes individual cabin with attached rest room comfortable bed, TV, radio, study table and window to the sea and centralized AC.  You also have an agent of SPORTS accompanying you on board the ship.
Food + tea + snacks on board the ship and islands are part of the package, which was a pleasant surprise.
The ship usually leaves by around 6pm even though the agency asks you to report by 1:30pm to sports office. Make sure you take a taxi or cab to sports office on indira Gandhi road, willington island, kochi as auto’s usually demand more than double the fare once they reach the destination and it might get ugly.
Once you reach the sports office on the second floor they offer you a complimentary t-shirt and cap and provide you with boarding pass and original bill of the trip. They then take you via auto (paid by them)to the boarding point where you and your luggage are checked thoroughly many times by the CISF, who themselves have no clue, as to what are they looking for. You are then loaded onto a bus and your luggage on a truck to the harbor.  The journey from here on is nothing but pure heaven.  Porters take your luggage to your cabin and the ship takes more than an hour to start its journey. If they leave by 6pm the first island kalpeni can be reached by 6:30 in the morning. A mechanized boat takes you from ship to the island where you are greeted by a sports agent who takes you to the report in a jeep.
Kalpeni is a peaceful island with predominately sunni Muslim population, preachers from the main land have slowly turned them into an orthodox yet peaceful population. Only men are seen everywhere while women generally stay home. Everyone speaks hindi and Malayalam. Snorkeling is the major highlight of this island as you can see corals and ornamental fishes in plenty, you also have canoeing and local sightseeing via jeep. You are also taken to local cloth factory where you get quality t-shirts at throw away price.  Make sure you buy tuna pickle from kalpeni. By 6 in the evening you are dropped back to the shore where a boat takes you back to the ship.
Minicoy is the most awesome island of Lakshadweep, I was truly taken aback by its natural beauty and the warmth of the people. Minicoy is ruled by women and they still retain their cultural identity and their way of life unlike kalpeni. Apart from snorkeling canoeing and visit to the lighthouse you also get to visit a local village. These people are a stark contrast to the city dwellers. Unity, honesty, community workmanship and selflessness are every evident in all the islanders. They speak very good hindi and malyalam and are in tune with the latest bollywood news. Their waters are loaded with tuna and islands with coconut trees. No dogs or crows on the island yet.  The boat drops you back to the ship and you reach kochi by 9am. No security check on arrival to the mainland.
Make sure you take a bottle of amla drink (concentrated vitamin C) to keep you from feeling sea sick.
Also carry an extra pair of T-shirt and tracks on the island and mineral water as well. Pics available on my flickr link.


Tuesday, October 9, 2012

Error in storeUserConfig windows environment


Below is the error:

wls:/nm/formsdomain> storeUserConfig('D:\nm\nmuserconfigfile.secure','D:\nm\nmuserkeyfile.secure','true')
Currently connected to Node Manager to monitor the domain formsdomain.
Creating the key file can reduce the security of your system if it is not kept in a secured location after it is created. Do you want to create the key file? y or n  -> y
Error: Not able to write secret key file.  Check the location and priviledges of the specified file location.

Solution:
1. Set the environment by using setdomainenv before executing storeUserConfig.
2. Ensure the folders have enough permissions where the .secure files are kept. 
3. Use \\ instead of \ in storeUserconfig command for example : D:\\nm\\nmuserconfigfile.secure\\
since windows has the habit of assuming \ as a char or special symbol, like \n is referred as new line it is best to give \\ to indicate a path.

Thursday, July 5, 2012

FMW admin server unable to start with other admin users account

FMW-weblogic admin server is working fine when started by user who created it. However it is not start able by other admin account users of the box.

error message:
Caused By: oracle.security.jps.JpsRuntimeException: oracle.security.jps.JpsException: [PolicyUtil] Exception while getting default policy Provider
    at oracle.security.jps.internal.policystore.PolicyDelegationController.<init>(PolicyDelegationController.java:291)
    at oracle.security.jps.internal.policystore.PolicyDelegationController.<init>(PolicyDelegationController.java:282)
    at oracle.security.jps.internal.policystore.JavaPolicyProvider.<init>(JavaPolicyProvider.java:261)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)

MW_HOME\user_projects\domains\bifoundation_domain\config\fmwconfig\bootstrap\
file cwallet.sso, right click on it and properties - security add users to admin group (give permissions)

if its still not working, restart the machine.

This is because the installation was done by one user and run by another user who does not have the same set of permissions.

Friday, June 29, 2012

custom banner on weblogic console

If you would like to have your own customized banner on weblogic console.
Go to wlserver_10.3\server\lib\consoleapp\webapp\framework\skins\wlsconsole\images
and edit the gif file in paint to create your own banner and clean restart weblogic server.
However doing this in production systems is illegal.

Tuesday, June 26, 2012

cannot open file _WLS_ADMINSERVER000000.DAT

Error message:1

<BEA-280061> <The persistent store "XXXX" could not be deployed: weblogic.store.PersistentStoreException: [Store:280020]There was an error while reading from the log file
weblogic.store.PersistentStoreException: [Store:280020]There was an error while reading from the log file
    at weblogic.store.io.file.Heap.getNextRecoveryFile(Heap.java:796)
    at weblogic.store.io.file.Heap.open(Heap.java:182)
    at weblogic.store.io.file.FileStoreIO.open(FileStoreIO.java:88)
    at weblogic.store.internal.PersistentStoreImpl.recoverStoreConnections(PersistentStoreImpl.java:332)
    at weblogic.store.internal.PersistentStoreImpl.open(PersistentStoreImpl.java:323)

java.io.IOException: Error reading from file, No such file or directory, errno=2
    at weblogic.store.io.file.direct.DirectIONative.read(Native Method)
    at weblogic.store.io.file.direct.DirectFileChannel.read(DirectFileChannel.java:134)
    at weblogic.store.io.file.StoreFile.read(StoreFile.java:282)
    at weblogic.store.io.file.Heap.getNextRecoveryFile(Heap.java:794)
    at weblogic.store.io.file.Heap.open(Heap.java:182)
    at weblogic.store.io.file.FileStoreIO.open(FileStoreIO.java:88)
    at weblogic.store.internal.PersistentStoreImpl.recoverStoreConnections(PersistentStoreImpl.java:332)
    at weblogic.store.internal.PersistentStoreImpl.open(PersistentStoreImpl.java:323)

Error message :2 

weblogic.store.PersistentStoreFatalException: [Store:280105]The persistent file store "_WLS_AdminServer" cannot open file _WLS_ADMINSERVER000000.DAT.
    at weblogic.store.io.file.FileStoreIO.open(FileStoreIO.java:128)
    at weblogic.store.internal.PersistentStoreImpl.recoverStoreConnections(PersistentStoreImpl.java:435)
    at weblogic.store.internal.PersistentStoreImpl.open(PersistentStoreImpl.java:423)
    at weblogic.store.admin.AdminHandler.activate(AdminHandler.java:126)
    at weblogic.store.admin.FileAdminHandler.activate(FileAdminHandler.java:191)
    at weblogic.store.admin.DefaultStoreService.start(DefaultStoreService.java:60)
    at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)
    at weblogic.work.ExecuteThread.execute(ExecuteThread.java:256)
    at weblogic.work.ExecuteThread.run(ExecuteThread.java:221)
Caused By: java.io.IOException: Error from open(), Permission denied, errno=13
    at weblogic.store.io.file.direct.DirectIONative.openConsiderLock(Native Method)
    at weblogic.store.io.file.direct.DirectFileChannel.<init>(DirectFileChannel.java:54)
    at weblogic.store.io.file.direct.DirectIOManager.open(DirectIOManager.java:179)
    at weblogic.store.io.file.StoreFile.openInternal(StoreFile.java:112)
    at weblogic.store.io.file.StoreFile.openDirect(StoreFile.java:168)
    at weblogic.store.io.file.Heap.openStoreFile(Heap.java:398)
    at weblogic.store.io.file.Heap.open(Heap.java:325)
    at weblogic.store.io.file.FileStoreIO.open(FileStoreIO.java:117)
    at weblogic.store.internal.PersistentStoreImpl.recoverStoreConnections(PersistentStoreImpl.java:435)
    at weblogic.store.internal.PersistentStoreImpl.open(PersistentStoreImpl.java:423)
    at weblogic.store.admin.AdminHandler.activate(AdminHandler.java:126)
    at weblogic.store.admin.FileAdminHandler.activate(FileAdminHandler.java:191)
    at weblogic.store.admin.DefaultStoreService.start(DefaultStoreService.java:60)
    at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)
    at weblogic.work.ExecuteThread.execute(ExecuteThread.java:256)
    at weblogic.work.ExecuteThread.run(ExecuteThread.java:221)

Solution 1: Stop weblogic and delete the .Dat file from *\user_projects\domains\<base_domain>\servers\<AdminServer>\data\store\default and diagnostics
start the JVM

Solution 2:  Follow solution1 and then login to console - server - config - services - change :Synchronous Write Policy: to cache-flush. select Advanced - uncheck : - save -activate changes and restart.

Error : Caused By: java.io.IOException: Error from fcntl() for file locking

Soln:
1.) Uncheck "Enable File locking " for persistent stores

==> This option is not supported while using NFS filesystem

2.) Maintain different directories for filestores of different JMS Servers and maintain a different file store for each JMS server.

ref :
http://docs.oracle.com/cd/E14571_01/web.1111/e13814/storetune.htm#CACFDGCA
http://docs.oracle.com/cd/E24329_01/web.1211/e24390/storetune.htm#CAEGGIEG
http://docs.oracle.com/cd/E17904_01/web.1111/e13701/store.htm#i1143516
Doc ID 957377.1
Doc ID 1278828.1 

Friday, June 8, 2012

Two weblogic consoles in the same set of browser

Two weblogic consoles when opened in the same set of browser, logs out.
For example, if you open both consoles in firefox or IE, then both logs out, however if you open one console in firefox and another in IE, it does not logout.

That is because the default Console Cookie Name is ADMINCONSOLESESSION, and when two cookies are created in browser with same cookie name, there is a conflict.

To avoid this:
Go to console - domain - config - gen - adv -
edit the Console Cookie Name of ADMINCONSOLESESSION to something else - save - restart

Else use ip:port/console for one and hostname:port/console for other.

Wednesday, May 30, 2012

ANT script to deploy application

Create a file called Build.xml and below are its contents:
...............................................................................................................................................
<project name="Test Deployment" default="deploy">
<taskdef name="wldeploy" classname="weblogic.ant.taskdefs.management.WLDeploy">
<classpath>
<pathelement location="D:/work/wls/wls9.2.4/weblogic92/server/lib/weblogic.jar"/>
</classpath>
</taskdef>
<target name="deploy">
<wldeploy
        action="deploy" verbose="true" debug="true" upload="true" remote="true" stage="true" library="true"
        name="conference.shared.lib" source="C:\test\test.war"
        user="weblogic" password="weblogic"
        adminurl="t3://localhost:7001" targets="examplesServer" />
</target>
</project>
...............................................................................................................................................
# change the appropriate values to reflect your environment.

Start the weblogic server.
Run the setdomainenv.sh from the domain folder and then traverse to the file containing build.xml
and type $ ant
This will deploy the file.

Wednesday, May 23, 2012

The input line is too long - during installing windows service

Error: The input line is too long - during installing windows service.

Desc: MS Windows 32bit has 2K limitation on the length of the command line. If the classpath setting in the " installSvc.cmd " is very long, the 2K limitation could be exceeded. Consequently the set up of the MS Windows Service may fail with the error.

Soln: Start the jvm in the command prompt and copy the classpath from startup log.
Place the classpath in a seperate .txt file and provide complete access to it.
now call the file in the installsvc file.
by replacing -classpath \"%CLASSPATH%\" option with the following option:
-classpath @pathname\filename

 Ex:JAVA_OPTIONS% -classpath @C:\classpath.txt -Dweblogic.Name 

ref: http://docs.oracle.com/cd/E17904_01/web.1111/e13708/winservice.htm#i1188175


Thursday, May 17, 2012

weblogic as windows service

To CREATE SERVICE
Go to${bea_dir}\wlserver_10.3\server\bin where bea_dir is home directory of weblogic installation.
Create a text file with following values:

echo off
SETLOCAL
set DOMAIN_NAME=base_domain
set USERDOMAIN_HOME=D:\work\wls\user_projects\domains\base_domain
set SERVER_NAME=AdminServer
set WLS_USER=weblogic
set WLS_PW=weblogic1
set PRODUCTION_MODE=false
set ADMIN_URL=http://localhost:7001
set JAVA_VENDOR=Sun (or set JAVA_VM=-hotspot)
set JAVA_HOME=D:\work\wls\jdk160_18
set MEM_ARGS=-Xms256m -Xmx512m
call "D:\work\wls\wlserver_10.3\server\bin\installSvc.cmd"
ENDLOCAL

save and rename to createSvc.cmd

Setting admin url and username password is optional if your domain picks up username/password from boot.propeties (domain_home\servers\AdminServer\security), no need to mention them in the script.

For Jrockit:
set JAVA_VENDOR=BEA
set JAVA_HOME=C:\bea\jrockit_160_05


execute the createSvc.cmd from command prompt so you can check if there are any errors.

This should create service with name of "beasvc sampleDomain_Adminserver_name" in the registry under HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services

check services under (start - run - services.msc) to find your service with beasvc prefix. change it to automatic and start the service and check logs.

INCLUDING CLASSPATH
If you have variables set in classpath which need to be included, edit the installSvc.cmd and look for the line where it calls commenv and replace it with setdomainEnv path.

call "%WL_HOME%\common\bin\commEnv.cmd"

TO REMOVE SERVICE
Go to${bea_dir}\wlserver_10.3\server\bin where bea_dir is home directory of weblogic installation.
Create a text file with following values:

echo off
SETLOCAL
set DOMAIN_NAME=base_domain
set SERVER_NAME=AdminServer
call "D:\work\wls\wlserver_10.3\server\bin\uninstallSvc.cmd"
ENDLOCAL

save and rename to removeSvc.cmd
execute the createSvc.cmd from command prompt so you can check if there are any errors.
(details should be the same as that of createSvc.cmd)


TO CHANGE SERVICE NAME
take a backup of the installSvc.cmd
edit installSvc.cmd and look for beasvc and change the name you wanted to.
"%WL_HOME%\server\bin\beasvc" -install -svcname:"beasvc %DOMAIN_NAME%_%SERVER_NAME%"

TO ENABLE MANAGED SERVICE TO START AFTER ADMIN SERVICE
 make a copy of installSvc.cmd say installSvc2.cmd
edit installSvc2.cmd and make it dependent on admin server script

for example

rem *** Install the service
"%WL_HOME%\server\bin\beasvc" -install -svcname:"ROBIN_%DOMAIN_NAME%_%SERVER_NAME%" -depend:"beasvcAdminService"

so now managed service will depend on admin service before starting.
----------------------------------------------------
1> If you get the below error in the server log

javax.naming.ServiceUnavailableException [Root exception is java.rmi.NoSuchObjectException: The object identified by: '31' could not be found. Either it was has not been exported or it has been collected by the distributed garbage collector.]

remove the service by removeSvc.cmd
edit createSvc.cmd
remove line          set ADMIN_URL=http://localhost:7001
save and run.
----------------------------------------------------
2>  If you get the below error in the server log

<BEA-150000> <An error occurred while establishing a connection

javax.naming.ServiceUnavailableException [Root exception is java.rmi.NoSuchObjectException: The object identified by: '31' could not be found.  Either it was has not been exported or it has been collected by the distributed garbage collector.]

Caused By: java.rmi.NoSuchObjectException: The object identified by: '31' could not be found.  Either it was has not been exported or it has been collected by the distributed garbage collector  

ensure the user has admin rights to the box
remove the service by removeSvc.cmd
edit createSvc.cmd
remove line          set ADMIN_URL=http://localhost:7001
save and run. 
----------------------------------------------------
3>  If you get the below error
'oracle.fabric.common.classloaderurl.handler' is not recognized as an internal or external command

Double quotation marks break the command syntax.

In the installSvc.cmd,
Change the %JAVA_OPTIONS% to \"%JAVA_OPTIONS%\"
(have to add \"  \")

For example
set CMDLINE="%JAVA_VM% %MEM_ARGS% %JAVA_OPTIONS% -classpath
will be come
set CMDLINE="%JAVA_VM% %MEM_ARGS% \"%JAVA_OPTIONS%\" -classpath

Doc ID 1276229.
----------------------------------------------------
To debug:
Go to the installSvc.cmd file and right at the location where you call the beasvc.exe file, insert the following options.

-debug –log:”path to log.txt”

So the entire string would look something like this.

"%WL_HOME%\server\bin\beasvc" –install  -debug  -log:c:\path.txt -svcname:"beasvc %DOMAIN_NAME%_%SERVER_NAME%" -javahome:"%JAVA_HOME%" -execdir:"%USERDOMAIN_HOME%" -maxconnectretries:"%MAX_CONNECT_RETRIES%" -host:"%HOST%" -port:"%PORT%" -extrapath:"%EXTRAPATH%" -password:"%WLS_PW%" -cmdline:%CMDLINE%


Also you can debug by traversing to the folder containing beasvc.exe
*\wlserver_10.3\server\bin> beasvc -debug "affected windows service name" 

Windows service log rotation
http://docs.oracle.com/cd/E14571_01/web.1111/e13708/winservice.htm#i1193313

Wednesday, May 9, 2012

reset / change weblogic username password

To change the admin password in a weblogic environment, please follow the below steps:

STEP 1 :  Firstly, Stop the server if running and make sure that weblogic.jar is in the classpath. (this could be checked by running setDomainEnv and env in linux or set in windows)

STEP 2 : Open a command prompt and navigate to your Domain directory’s bin folder and run “setDomainEnv.sh “
After you run setdomain the prompt should go back to domain folder from bin folder.
if it doesn't happen run the command as . ./setdomainEnv.sh (in words- dot space dot slash).
Run echo $CLASSPATH to check if weblogic.jar is set.
Run Echo %CLASSPATH% for windows.

STEP 3 : Now run the following command :

“Java weblogic.security.utils.AdminAccount NewAdminUser NewAdminPassword . “

where ” NewAdminUser ” and ” NewAdminPassword ” are the new user and password you would like to create.

Note : The “ . “ (period) at the end of the command is very important..!! as it tells the command that this should be run in the domain directory.

STEP 4 : The above command creates a file “ DefaultAuthenticatorInit.ldift “ in the domain directory.

STEP 5 :  Now copy the newly created “DefaultAuthenticatorInit.ldift “ to “*\bea10.3\user_projects\domains\change_password\security “ (where change_password is your affected domain)

Note : You can take a backup or delete  “DefaultAuthenticatorInit.ldift “ which is already present in the security folder

STEP 6 :  Now edit the boot.properties file ( located in “ *\bea10.3\user_projects\domains\change_password\servers\AdminServer\security “ & “ *\bea10.3\user_projects\domains\change_password\servers\managedserver\security “ ) with the new user name and password that we created ( i.e NewAdminUser  and NewAdminPassword )

Delete the file “ DefaultAuthenticatormyrealmInit.initialized “ located in “C:\bea10.3\user_projects\domains\change_password\servers\AdminServer\data\ldap “

Now start the server and login with the NewAdminUser  and NewAdminPassword.

To reset the admin password in a weblogic environment, please follow the below steps:

STEP 7 : After logging into the console with the NewAdminUser  and NewAdminPassword. go to security realms - myrealm (or your respective realm) - users and groups - click on your original user - passwords - change password - save - activate changes.

STEP 8 :  Edit the boot.properties file ( located in “ *\bea10.3\user_projects\domains\change_password\servers\AdminServer\security “ & “ *\bea10.3\user_projects\domains\change_password\servers\managedserver\security “ ) with the original user and password that we just reset.

jvm freeze

The stack trace :
"[ACTIVE] ExecuteThread: for queue: 'weblogic.kernel.Default (self-tuning)'" id=16 idx=0x58 tid=* prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: netscape/ldap/LDAPSearchListener@0x0000000184932A78[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at jrockit/vm/Locks.wait(Locks.java:1973)[inlined]
at java/lang/Object.wait(Object.java:474)[optimized]
at netscape/ldap/LDAPMessageQueue.waitForMessage(LDAPMessageQueue.java:200)
^-- Lock released while waiting: netscape/ldap/LDAPSearchListener@0x0000000184932A78[recursive]
at netscape/ldap/LDAPMessageQueue.waitFirstMessage(LDAPMessageQueue.java:101)
^-- Lock released while waiting: netscape/ldap/LDAPSearchListener@0x0000000184932A78[fat lock]
at netscape/ldap/LDAPConnection.sendRequest(LDAPConnection.java:1796)
^-- Holding lock: netscape/ldap/LDAPConnection@0x0000000184924B50[biased lock]
at netscape/ldap/LDAPConnection.search(LDAPConnection.java:2566) 



 The StackoverFlow issue observed is caused by a misconfiguration of LDAP server.

Soln:
1. Open the console
2. through the menu Security realms->myRealm->Providers->Authenticaton->[provider]->Configuration->Provider Specific
3. Change the Group Membership Searching to limited
4. Change the Max Group Membership Search Level to 16
5. Save and restart WebLogic Server

Wednesday, May 2, 2012

BEA-000402 BEA-000438

Error message:
Warning> <Socket> <BEA-000402> <There are: 5 active sockets, but the maximum number of socket reader threads allowed by the configuration is: 4. You may want to alter your configuration.>
are because of the original messages of not picking up libmuxer.so

or

<Error> <Socket> <BEA-000438> <Unable to load performance pack. Using Java I/O instead. Please ensure that a native performance library is in:

Soln:
1. login to console - servers - configuration - tunning - make sure 'Enable Native IO' is clicked.
2. run the setdomainenv and then run env command.
this will tell us the current libmuxer.so being used.
3. You need to make sure the corresponding libmuxer.so is used which could be done by

take a backup and at the end of your commEnv.sh file please add the lines

LD_LIBRARY_PATH=<path of the libmuxer.so>${LD_LIBRARY_PATH}
export LD_LIBRARY_PATH


path of libmuxer.so
wlserver_10.3\server\native\<server type>\<bit version>\libmuxer.so

Ref : Doc ID 965299.1

Thursday, April 19, 2012

SSL trouble on Weblogic 12c (12.1) nodemanger

On 12.1 version of weblogic:
Trying to configure SSL on nodemanger.
After setting the CustomIdentityKeyStorePassPhase  and CustomIdentityPrivateKeyPassPhase attributes in the nodemanager.properties file in clear text and restarting the nodemanager,
the values are not being replaced with the encrypted value.

On 10.3.X series it is working fine and values are being encrypted.

Workaround:
If you take the nm_data.properties file from a 10.3.x version (wlserver_10.3\common\nodemanager) and place it in (wlserver_12.1\common\nodemanager) path and then run a fresh startnodemanager.sh
The  CustomIdentityKeyStorePassPhase  and CustomIdentityPrivateKeyPassPhase values in nodemanager.properties are getting encrypted and also it deletes the nm_data.properties file.

Wednesday, April 18, 2012

To encrypt clear text passwords

To encrypt a clear text value which could be used in weblogic file
run the setdomainenv from the domain folder.
java weblogic.security.Encrypt
It will ask you for password and you may enter the value to be encrypted, The resultant encrypted value could be used in any weblogic file.

Check JDBC driver version

To check the JDBC driver version

run . ./setdomainenv.sh (dot space dot slash)
traverse to

cd *\wlserver_10.3\server\lib
java -jar ojdbc6.jar (where ojdbc6.jar is the driver's jar file)

you would get the result as
Oracle 11.1.0.7.0-Production JDBC 4.0 compiled with JDK6

so 11.1.0.7.0 is the version.

for db2 driver
wlserver_12.1\server\lib\wldb2.jar

D:\work\wls\wls12.1.1\wlserver_12.1\server\lib>java -cp wldb2.jar weblogic.jdbc.db2.DB2Driver
[FMWGEN][DB2 JDBC Driver]Driver Version: 4.2.1.035064 (F044246.U015812) 

Wednesday, March 28, 2012

WSDL excpetion in weblogic

Error message: WARNING: Input Action on WSDL operation PreparedOperation and @Action on its associated Web Method preparedOperation did not match and will cause problems in dispatching the request

or 

Error message : WARNING: Input Action on WSDL operation RegisterOperation and @Action on its associated Web Method registerOperation did not match and will cause problems in dispatching the requests
 

Weblogic version: 12c (12.1.1)

Solution : 1. change the domain from development mode to production mode.
                This could be done by opening the console and domain name --> configuration --> general
                click production mode enabled - save -activate changes - restart.
                2. Also a patch 13606167 is available.

Wednesday, March 21, 2012

Weblogic console showing old version after upgrade

After upgrading from 10.3.3 to 10.3.5, the console page is still showing old version.

config.xml & startup logs are showing the correct version.

under the middleware directory directory where the patch for upgrade installer is kept, i found two patch directories patch_wls1033 & patch_wls1035.
I deleted patch_wls1033 as my weblogic has now been upgraded to 10.3.5
The console is showing the correct version now.

Monday, March 19, 2012

Connection has been administratively disabled.

Seeing error message :
java.sql.SQLException: Connection has been administratively disabled.

intermittent issueand connection hangs for requests

Added the following parameter and problem solved
-Dweblogic.resourcepool.max_test_wait_secs=

Wednesday, March 14, 2012

domain creation stuck

Weblogic Domain creation stuck/froze at creating domain security information.......
Have taken a backup of config.sh (wlserver_10.3\common\bin)
added the variable -Djava.security.egd=file:/dev/./urandom in JVM_ARGS

it now looks like 
JVM_ARGS="-Djava.security.egd=file:/dev/./urandom -Dprod.props.file='${WL_HOME}/.product.properties' -Dpython.cachedir=/tmp/cachedir ${JVM_D64} ${MEM_ARGS} ${CONFIG_JVM_ARGS}"

rerunning config.sh solved the issue. 

Monday, February 13, 2012

Reset password in weblogic portal

Error message

Caused By: javax.security.auth.login.FailedLoginException: [Security:090302]
Authentication Failed: User canopiadmin denied
       at weblogic.security.providers.authentication.shared.DBMSAtnLoginModuleImpl.login(DBMSAtnLoginModuleImpl.java:285)
       at com.bea.common.security.internal.service.LoginModuleWrapper$1.run(LoginModuleWrapper.java:110)
       at java.security.AccessController.doPrivileged(Native Method)
       at com.bea.common.security.internal.service.LoginModuleWrapper.login(LoginModuleWrapper.java:106)
       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
       Truncated. see log file for complete stacktrace

 
In weblogic portal the authentication happens via DB, since it uses SQL based authentication.
1. We need to enter the original username password in database
2. enable plain text password in config.xml if its not already so, since the above entered credentials are in plaintext and table will have encrypted values.

to enable plaintext in config.xml as

<wls:plaintext-passwords-enabled>true</wls:plaintext-passwords-enabled>

so it would look like below in config.xml

<realm>
     <sec:authentication-provider xsi:type="wls:sql-authenticatorType">
       <sec:control-flag>SUFFICIENT</sec:control-flag>
       <wls:data-source-name>p**nDataSource</wls:data-source-name>
       <wls:plaintext-passwords-enabled>true</wls:plaintext-passwords-enabled>
 

3. restart the servers.

plaintext password is reachable via console as
console - security realm - myrealm - providers - authentication - select the respective sql authenticator - config - provider specific - Plaintext Passwords Enabled

You may disable plain text after successful login,  

Friday, February 3, 2012

To disable WLDF diagnostics

Access the Administration Console for you domain.
Click Lock & Edit to start a new configuration session.
Select the Diagnostics->Diagnostic Modules node in the left pane of the console.
disable or stop or delete the respective module and carry out a restart.

Also please check if any of the options are enabled under console - data source - configuration - diagnostics  

Else

1. Shut down WebLogic Server.
2. Clear the <SERVER>/data/store/diagnostics directories

3. Apply the below 3 parameters at JAVA_OPTIONS of your setdomainenv  file

-D_Offline_FileDataArchive=true
To control diagnostics data collected from the server instance and the applications running on them. If true, WLDF archives the data collected (default: false).

-Dcom.bea.wlw.netui.disableInstrumentation=true
To control Workshop page flow event reporting. Set to false by default in Workshop 10.1 and earlier). Set to true by default (in Workshop 10.2 and later).

-Dweblogic.connector.ConnectionPoolProfilingEnabled=false
JDBC Connection Pool profiling data (default: false).


4. Restart WebLogic Server.
 

 Else

Take a backup of config.xml and delete the lines from and to <server-diagnostic-config>
restart

Else
 
You can adopt a retirement policy by rotating the .dat file
console -  diagnostics - archives - select the respective server - create NEW 'Data Retirement Policies' followed by a restart.