DX Application Performance Management

Can someone explain what controls the format of the perflog.txt written by the EM? I have multiple clusters, all of them are the same OS and have the same APM install & patch level 10.7 SP2. I'm seeing various formats across my clusters between MOM & Collectors, where files will have either 28, 27 or 26 values written in perflog.txt separated by commas. Biggest issue is that some have a value the precedes the JVM allocated memory value and some start with the JVM Memory value.

All EM's have the default:

introscope.enterprisemanager.performance.compressed=true
log4j.additivity.Manager.Performance=false
log4j.logger.Manager.Performance=DEBUG, performance
log4j.appender.performance=com.wily.introscope.server.enterprise.monitor.PerformanceLogFileAppender
log4j.appender.performance.layout=com.wily.org.apache.log4j.PatternLayout
log4j.appender.performance.layout.ConversionPattern=%d{M/dd/yy hh:mm:ss a z},%m%n
log4j.appender.performance.File=logs/perflog.txt
log4j.appender.performance.MaxSize=4000000

Dear Community:

Can you help Keith?

Thanks

Hal German

Reading this, you may not have much control. Why do the perflog.txt columns change order after - CA Knowledge 

So I'm guessing if I want to use this file, then I will always need to find the last update of the column headers written in the file to figure out the format. Was thinking of doing a script with an Infrastructure Agent since we have so many APM Clusters to support, to bring the APM metrics into a central place from each cluster.

MOM:

1/08/19 06:54:10 AM EST,Performance.Heuristics.PerstInsertionTime.Per.Interval,Performance.JVM.TotalMemory,Performance.JVM.FreeMemory,Performance.JVM.GCTimePercent,Performance.Workstation.NumberOfWorkstations,Performance.Harvest.HarvestDuration,Performance.Smartstor.Duration,Performance.Agent.ConnectionThrottleCount,Performance.Agent.NumberOfMetrics,Number of Historical Metrics,Performance.Agent.NumberOfAgents,Performance.Agent.MetricDataPending,Performance.Agent.MetricDataRate,Performance.Server.NumberOfMetricGroups,Performance.Transactions.Num.Inserts.Per.Interval,Performance.Transactions.Num.Dropped.Per.Interval,Performance.Transactions.Num.Queries.Per.Interval,Performance.Transactions.Query.Time.Per.Interval,Performance.Transactions.Index.Insertion.Time.Per.Interval,Performance.Transactions.Data.Insertion.Time.Per.Interval,Performance.Transactions.Num.Traces,Performance.Transactions.TT.Queue.Size,Performance.MetricDataManager.QueryRate,Performance.MetricDataManager.QueryMemory,Performance.MOM.CollectorMetricsReceivedPerInterval,Performance.Server.NumberOfMapEntityMetricGroups,Queries per interval,Queries duration (ms),Performance.MOM.NumberOfCollectorMetrics,

COLLECTOR:

1/02/19 07:38:00 AM EST,Performance.JVM.TotalMemory,Performance.JVM.FreeMemory,Performance.JVM.GCTimePercent,Performance.Workstation.NumberOfWorkstations,Performance.Harvest.HarvestDuration,Performance.Smartstor.Duration,Performance.Agent.ConnectionThrottleCount,Performance.Agent.NumberOfMetrics,Number of Historical Metrics,Performance.Agent.NumberOfAgents,Performance.Agent.MetricDataPending,Performance.Agent.MetricDataRate,Performance.Server.NumberOfMetricGroups,Performance.Transactions.Num.Inserts.Per.Interval,Performance.Transactions.Num.Dropped.Per.Interval,Performance.Transactions.Num.Queries.Per.Interval,Performance.Transactions.Query.Time.Per.Interval,Performance.Transactions.Index.Insertion.Time.Per.Interval,Performance.Transactions.Data.Insertion.Time.Per.Interval,Performance.Transactions.Num.Traces,Performance.Transactions.TT.Queue.Size,Performance.MetricDataManager.QueryRate,Performance.MetricDataManager.QueryMemory,Performance.Heuristics.PerstInsertionTime.Per.Interval,Performance.Server.NumberOfMapEntityMetricGroups,Queries per interval,Queries duration (ms),

Hi Keith,

This is the feedback I've got from Sustaining.

I hope this helps,

Sergio

---

Hi, Sergio

I had a customer asking similar question before, so I did some investigation in this area. Basically, the order of a column is determined by the time it’s reported for the first time. Some components reporting different columns are initialized in parallel, so there is no guarantee of same order each time.

If they are doing some analysis on the perflog, they would have to reference the top row with column names

Thanks

Ting

Thank you for the explanation and clarifying the setup. I have updated my code to use the last header written into the file to determine all columns parsed.