DX Application Performance Management

Hi,

We are seeing Out of Swap Space issue when Instrumented Weblogic Applications (Sun JVM 1.5) with APM 9.0.7, In the past we were using Introscope 8.0 agents and it was pretty stable for about 3 years and recently when we upgraded to 9.0.7, we are seeing this issue.

Would like to check if anyone else has this problem if so what is the solutions you may have.

Thanks
Regards
Murali

Are you out of Heap or Swap? There is a technical advisory out on the 9.0.7.x agents that hit me on a weblogic deployment just recently around memory leaks in the SQL Agent. This solved my issue.

http://msite.ca.com/etrack.ashx?M=1302.ebead80d-e4c7-4df8-9a46-717338179533&L=171007&URL=http%3a%2f%2fwww.ca.com%2f%7e%2fmedia%2fFiles%2fTechnical%2520Advisory%2fCA_APM_Technical_Advisory_StatementToConnectionMapTracer.pdf

Thanks for your reply. We are hitting swap not heap, I have gone through that advisory and update my environment with 9.0.8, but still we are facing this swap space problem.

we had as well recently a warning about a jvm going out of memory: out of swap space - and the java agent will complain about not haeving enough native space. and we figgured that there are different things to look at:

if such a error occurs its maybe just a symptom of the real problem as the agent is one major active component.
there are components like for example the system monitors which use the native memory space of the JVM process.
in our case we had a 32bit SUN JVM on solaris sparc and we allocated 3GB Heap and 512MB Perm Size which meant we where already very close to the system memory process limit and we where just lucky that we "never hit our head" at that max limitation of ~3.8G.
- there are different things which use native memory e.g. as soon you use drivers like JDBC OCI which uses native libraries.
- or any application which is using things like directbytebuffer (there is a module for the www.eclipse.org/mat to create a report for directbytebuffer usage)
by side that there are system based process limitations on memory consumption which can lead to out of swap issues (all depending on your OS).
- basically the system allows you not to allocate more memory to that process because of a configurable or real limitation.

what was in the end our memory consumption where related to the application doing lots of native file handlers, and the agent was just not getting its amout of native space it required to run by side what the application did.

out of intrest we where trying to find out how much native memory our agent consumes for that we should provide CA support the system agents we use and they could then come back to us with a figgure, so far we didnt have done that.

The switch to 9.0.7 did increase the memory overhead for us and caused similar issues. There were two things we did to address it:

1. We changed the way that it did thread monitoring in the toggles.pbd file. There are two possibles in there. Changing the one that we used fixed the issue in 9.0.7.

2. We tweaked stack size down to 256k. We had a lot of threads the default 512k value was using up so much native memory that we didn't have space for the heap and perm space.

I can't look at the exact details of the fix as I've changed jobs and no longer have access to the information.