Managed server crashed due to oversized LDAP

weblogic server crashed and heap analysis shows ( as done from MAT tool) maximum memory was utilized by "com.bea.security.providers.xacml.store.ldap.LDAPAuthorizationPolicyStore" which is related to DB authentication.

In the directory ./servers/<server>/data/ldap/ldapfiles contains a very large data file.
618MB    EmbeddedLDAP.data

took a backup of the ldap directory of the affected server. (./servers/<server>/data/ldap) and place it outside the domain.
stop the affected server, remove the ldap directory and start the server again. The server will pick up required ldap files from the admin server.


Any changes done on the LDAP side will result in size increase,

we can try setting a parameter which will limit the size of the EmbeddedLDAP.data file.
stop the domain
take a backup of setDomainEnv.sh
edit setDomainEnv.sh and place
-Dweblogic.security.ldap.maxSize=mysize -Dmysize=100000000
after JAVA_OPTIONS= (usually found in the end of the file)
it will look like
set JAVA_OPTIONS=-Dweblogic.security.ldap.maxSize=mysize -Dmysize=100000000 %JAVA_OPTIONS%
we are trying to set size of EmbeddedLDAP.data as 100mb
save and restart domain

https://support.oracle.com/CSP/main/article?cmd=show&type=NOT&doctype=REFERENCE&id=972082.1

Member with Operator/deployer role cannot stop Managed Server, its a OSB setup on a weblogic domain.

The  OSB internal artifacts (for ex. data sources used by OSB) are protected by a role called "ALSBSystem". The users who belong to group "ALSBSystemGroup" have access to these artifacts.

Since operator group is a WLS group, it is not able to provide required access on ALSB resources. To use any user in "operator" role to shutdown an OSB domain,
please add this user to ALSBSystemGroup as well using WLS admin console under :
Security Realms >my realm >Users and Groups
Once added to this group you may be able to cleanly shutdown the server with this user.

Out of swap space?

Getting "java.lang.OutOfMemoryError" and the Weblogic Server gets shutdown unexpectedly 

java.lang.OutOfMemoryError: requested 97 bytes for char in /BUILD_AREA/jdk6_05/hotspot/src/share/vm/runtime/sharedRuntime.cpp. Out of swap space?

Machine is highly burdened with other applications and sessions already running.

Disable or stop all the unnecessary services in the machine.
Allocate more disk memory to virtual memory and stop other applications which run simultaneously.
decrease your usage of memory by reducing the value of the -Xmx & -Xms flag

ex:  -Xms256m -Xmx256m -XX:PermSize=256m -XX:MaxPermSize=512m 

transaction roll back & jvm's being thrown out of cluster

managed servers are getting thrown out of cluster

recieved the below error messages

<BEA-010026> <Exception occurred during commit of transaction
Caused by: weblogic.transaction.internal.TimedOutException: Transaction has timed out when making request to XAResource


<BEA-000144> <Managed server WLS1D has been suspended or shutdown.>
<BEA-000115> <Lost 2 multicast message(s).>
<BEA-000111> <Adding WLS1D with ID 5716223036452484921S:WLS1:[8371,8371,-1,-1,-1,-1,-1]:CMS:WLS1D to cluster: cmsCluster view.>
<BEA-000128> <Updating 5716223036452484921S:WLS1:[8371,8371,-1,-1,-1,-1,-1]:CMS:WLS1D in the cluster.>
<BEA-000112> <Removing WLS1C jvmid:-943465781735899304S:WLS1:[8271,8271,-1,-1,-1,-1,-1]:CMS:WLS1C from cluster view due to timeout.>



I see that many transactions are being rolled back and those exceptions are causing the memory to get filled frequently, and so the garbage collection takes time causing the multicast messages to miss.
change the value of your JTA in config.xml and restart to check if we still get the same problems.

stop the domain
take backup of config.xml
change  <jta>
<timeout-seconds>90</timeout-seconds>
<security-interop-mode>default</security-interop-mode>
</jta>

from 90 to 600 and start the servers.
Fixed