Document ID: TEC616589
Last Modified Date: 9/5/2014
Author: ChristopherClark
- Products
- CA Application Performance Management
- Components
Description:
When monitoring WebSphere BPM, every time the project deploys their snapshot to the Process Server they see an OOM core dump to occur. Analysis of this dump shows thread making request to large amount of heap, 2GB+, which causes an OOM exception. Note as memory is never allocated the object does not show on heap dumps.
Solution:
APM engineering team reviewed and determined that the APM Agent itself is not allocating any memory, it is making a call to the stub (copied stack below) and that internally routes the call to the IBM IIOP protocol, which results in the large allocation request.
1XMCURTHDINFO Current thread NULL ---------------------- 3XMTHREADINFO "WebContainer : 0" J9VMThread:0x00000000459CD700, j9thread_t:0x000001002CA522B0, java/lang/Thread:0x000000075E9EEE70, state:R,prio=5 3XMJAVALTHREAD (java/lang/Thread getId:0x111, isDaemon:true) 3XMTHREADINFO1 (native thread ID:0x4C100FB, native priority:0x5,native policy:UNKNOWN) 3XMHEAPALLOC Heap bytes allocated since last GC cycle=0 (0x0) 3XMTHREADINFO3 Java callstack: 4XESTACKTRACE at com/ibm/rmi/IOR.read(IOR.java:362(CompiledCode)) 4XESTACKTRACE at com/ibm/rmi/iiop/CDRReader.read_Object(CDRReader.java:1167) 4XESTACKTRACE at com/ibm/rmi/iiop/CDRReader.read_abstract_interface(CDRReader.java:1602) 4XESTACKTRACE at com/ibm/rmi/iiop/CDRReader.read_abstract_interface(CDRReader.java:1591) 4XESTACKTRACE at com/lombardisoftware/server/ejb/tracking/_APIServicesHome_Stub.getEJBMetaData(_APIServicesHome_Stub.java) 4XESTACKTRACE at com/wily/introscope/agent/servlethelper/EJBHelper.getEJBMetaData(EJBHelper.java:28) 4XESTACKTRACE at com/wily/introscope/agent/trace/ejb/hc2/Ejb2StubNameFormatter.findInterfaceName(Ejb2StubNameFormatter.java:40)
The problem is known problem with the IBM JVM, see:-
http://www-304.ibm.com/support/docview.wss?uid=swg1PM46698
To overcome either upgrade the JVM or disable the EJB2StubTracing and EJB3StubTracing in toggles-full.pbd or toggles-typical.pbd. Simply add a # character to the TurnOn directive:-
#TurnOn: EJB2StubTracing #TurnOn: EJB3StubTracing
Search the Entire CA APM Knowledge Base
