DX Application Performance Management

Hello community,

I have a collector that eats memory and in 24 h goes OOM.

The GC cannot free memory also with a huge full GC, forced by me with jmap

It works with 10GB Heap and has 320 agents connected but live metrics is < 200K and historic is <390K

I have produced an heap dump and I have found that a lot of memory is eaten by java/util/concurrent/LinkedBlockingDequeue$Node (4GB and 21million instances)

These objects have as a parent object the class com/ca/apm/baseline/em/BaseLineEngine

After browsing the Internet with this calss name I have solved.

I have solved the problem moving the variance.db file and restarted EM

I hope that this thread can be useful for other people facing the same issue.

Luca

Hello Luca,

Can you post the version of the apm and where did you move the variance.db?

Thanks,

Richard

• Version of APM is 10.5.2
• variance.db was removed and I let the system recreate it after boot
• Also in the log file I got the following error:

• 11/11/18 08:44:24.979 PM CET [ERROR] [Perst Baseline insertion] [Manager]

  org.garret.perst.StorageError: Invalid object reference
  at org.garret.perst.impl.StorageImpl.getPos(StorageImpl.java:60)
  at org.garret.perst.impl.StorageImpl.loadStub(StorageImpl.java:2808)
  at org.garret.perst.impl.StorageImpl.lookupObject(StorageImpl.java:2754)
  at org.garret.perst.impl.StorageImpl.findClassDescriptor(StorageImpl.java:2771)
  at org.garret.perst.impl.StorageImpl.loadStub(StorageImpl.java:2818)
  at org.garret.perst.impl.StorageImpl.lookupObject(StorageImpl.java:2754)
  at org.garret.perst.impl.LinkImpl.loadElem(LinkImpl.java:228)
  at org.garret.perst.impl.LinkImpl.get(LinkImpl.java:21)
  at org.garret.perst.impl.AltBtree$BtreePage.find(AltBtree.java:70)
  at org.garret.perst.impl.AltBtree$BtreePage.find(AltBtree.java:70)
  at org.garret.perst.impl.AltBtree.get(AltBtree.java:850)
  at org.garret.perst.impl.PersistentMapImpl.get(PersistentMapImpl.java:115)
  at com.ca.apm.baseline.engine.core.PerstManager.bulkInsert(PerstManager.java:207)
  at com.ca.apm.baseline.engine.BaselineDataManager.saveBaselinesToPerst(BaselineDataManager.java:525)
  at com.ca.apm.baseline.engine.BaselineDataManager.access$4(BaselineDataManager.java:509)
  at com.ca.apm.baseline.engine.BaselineDataManager$PerstInserter.run(BaselineDataManager.java:619)

The above error disappeared after variance.db was removed

Converted to discussion because Luca has posted for the benefit of other community users.

Thank You Luca for contribution!

Can  you please share the location of variance.db file. I am not able to  find it in traces folder.

Thanks.

The location of the variance.db is:

<EM_installation_path>/data

I  don't have this file in my data folder. this collector is running on 9.7 version and have 16 GB of RAM. I assign 12 GB of heap but it got crash. then I reduce it to 10 then 8. Now it is running on 6 but I think collector will crash again. check screenshot

I dont'think that with release 9.7 you will find variance.db

This applies to 10.5.2

Try reading this

Introscope Enterprise Manager Troubleshooting and - CA Knowledge 

Do you have an errore like this?

ERROR] [pool-1-thread-2] [Manager] failure to add data to baseline
java.lang.ClassCastException
This is telling u the baseline is corrupted

Luca