DX Application Performance Management

Hi all folks,

we use a CDV (Cross Cluster Data Viewer) to combine some EMs behind firewalls. This works nice, the people connect to the CDV and get access to all the metrics data from the EMs behind the firewall. The CDV support Webview and Workstation access to the metrics data for all customer.

After the migration to APM 10.5.1 we start using the Teamcenter.
Unfortunately the Teamcenter is not available on CDVs on the one hand and on the other, the ETC (Enterprise Team Center) doesn't support a central Webview access to the metrics data like the CDV do.
The Teamcenter provide a simple Webview list to connect to only, but the target EMs are behind firewalls and can't reached by the customer. The Experience Views and the Maps, Dashboard are useable only.

Any suggestion how to join the two functions?
Any one else with a similar problem?

Any pros and contras to use CDV or ETC.

With regards, Lutz

Dear Lutz:

     I don't know if many people have used both CDV and ETC that can answer your question. Are there other customers or CA staff that can advise in this area?

Thanks

Hal German

Yes, we have customers running both while evaluating/transitioning from CDV to ETC.

I've not heard any complaints or issues related specifically to this, but it doesn't mean that it's not since I'm not nearly as engaged as I was in my previous role.

Hello Haruhiko,

nice to know someone else use CDVs as well.

I try to do this, but all the time the CDV connect to a Collector the MOM lost contact to the Collector.

I find the following Messages in the Collector IntroscopeEnterpriseManager.log log-file.

6/15/17 11:58:50.903 AM CEST [INFO] [PO:main Mailman 4] [Manager.SessionBean] CDV Introscope Enterprise Manager connected: Node=Workstation_126, Address=g100spuitc52p1.g100.intern/7.242.8

0.19:62873, Type=socket

6/15/17 11:58:56.929 AM CEST [INFO] [PO:main Mailman 3] [Manager] Stopping all Baseline Alert Calculators

6/15/17 11:58:56.944 AM CEST [INFO] [PO:main Mailman 4] [Manager] Starting Baseline Alert Calculator with config: BaselineAlertConfig [managementModuleName=Default, domainName=SuperDomain

, alertName=Differential Analysis Control, metricSpecifier=(<all> ((.*)\|(.*)\|(.*)) Frontends\|Apps\|[^|]+\|URLs\|[^|]+:Average Response Time \(ms\) OR <all> ((.*)\|(.*)\|(.*)) B

usiness Service\|[^|]+\|Business Transactions\|.*:Average Response Time \(ms\) OR <all> ((.*)\|(.*)\|(.*)) Servlets\|[^|]+:Average Response Time \(ms\) OR <all> ((.*)\|(.*)\|(.*))

WebServices\|Client\|[^|]+\|[^|]+:Average Response Time \(ms\) OR <all> ((.*)\|(.*)\|(.*)) By Frontend\|[^|]+\|Health:Average Response Time \(ms\) OR <all> ((.*)\|(.*)\|(.*)) By

Business Service\|[^|]+\|[^|]+\|.*:Average Response Time \(ms\) OR <all> ((.*)\|(.*)\|(.*)) ASP.NET\|[^|]+:Average Response Time \(ms\) OR <all> ((.*)\|(.*)\|(.*)) WebServices\|Se

rver\|[^|]+\|[^|]+:Average Response Time \(ms\) OR <all> ((.*)\|(.*)\|(.*)) Business Segment\|[^|]+\|[^|]+\|.*:Average Response Time \(ms\) OR <all> ((.*)\|(.*)\|(.*)) Backends\|W

ebService at [^|]+\|Paths\|[^|]+:Average Response Time \(ms\) OR <all> ((.*)\|(.*)\|(.*)) Backends\|System [^|]+ on port [^|]+:Average Response Time \(ms\) OR <all> ((.*)\|(.*)\|(.*

)) Frontends\|Messaging Services \(onMessage\)\|[^|]+\|[^|]+:Average Response Time \(ms\) OR <all> ((.*)\|(.*)\|(.*)) Frontends\|Messaging Services \(receive\)\|[^|]+\|[^|]+:Estimat

ed Message Processing Time \(ms\) OR <all> ((.*)\|(.*)\|(.*)) Backends\|Messaging Services \(outgoing\)\|[^|]+\|[^|]+:Average Response Time \(ms\) OR <all> ((.*)\|(.*)\|(.*)) Back

ends\|Messaging Services \(receive\)\|[^|]+\|[^|]+:Average Response Time \(ms\) OR <all> ((.*)\|(.*)\|(.*)) Backends\|[^|]+\.+[^|]+\.+[^|]+:Average Response Time \(ms\) OR <all> ((.

*)\|(.*)\|(.*)) Automatic Entry Points\|[^|]+\|[^|]+:Average Response Time \(ms\) OR <all> ((.*)\|(.*)\|(.*)) Backends\|[^|]*:Average Response Time \(ms\) AND com.ca.apm.baseline.al

ert.IgnoreBaselineMetricsSpecifier@58f1c082), cautionThresholdLevel=5, dangerThresholdLevel=4, weRules=[true, true, true, true], windowCellCount=20, windowCellDecay=20, ruleEngineClass=co

m.ca.apm.baseline.alert.rules.wer.CustomWERuleEngine, cautionThreshold=1650, dangerThreshold=3960, extremeDangerThreshold=10000, weRuleWeights=[500, 300, 200, 100]]

6/15/17 11:58:56.944 AM CEST [INFO] [PO:main Mailman 4] [Manager] Total baseline alert calcs running: 1

6/15/17 12:35:12.884 PM CEST [INFO] [PO Route Down Executor] [Manager.SessionBean] MOM Introscope Enterprise Manager disconnected: Node=Workstation_1, Address=D100SPUITC02P1/4.242.48.18:3

5773, Type=socket

I check this, but according to DE139261 "Differential Analysis Stops Working When CDV Connects to Collectors" the Problem should be fixed with APM 10.5.1.

Nice to know, it doesn't work,

With regards,

Lutz

What is the hotfix level on your EMs?

Sorry Haruhiko,

the used APM level is 10.5.1.6.

2/23/17 11:16:39.706 AM CET [INFO] [main] [Manager] Introscope Enterprise Manager Release 10.5.1.6 (Build 990006)
2/23/17 11:16:39.706 AM CET [INFO] [main] [Manager] Using Java VM version "IBM J9 VM 1.7.0" from IBM Corporation

I spent no time to test the hotfix level in the past.

Lutz

I would check your EM properties file header to see if the EM has been patched.

If you see nothing about that says "HFX", then it hasn't been patched.

The hotfixes are available with your downloads and is a single executable JAR that will do everything for you.

Run Hotfix#1, then Hotfix#9. The EMs will need to be stopped so you'll need to plan for an outage.

Also make sure you have enabled concurrency pooling and outgoingMessageQueueSize in your properties file.

Beyond that, I'd recommend a CSO ticket for further investigation.

Hello Haruhiko,

in the past, APM 9.5.2, with the CDV I use a modified concurrency pool.

# transport.override.isengard.high.concurrency.pool.max.size=10   ## Uncomment for MOMs
transport.override.isengard.high.concurrency.pool.max.size=30

But with APM 10.5.1 I use the default pool size on the CDV and I never changed the transport.outgoingMessageQueueSize property. I connect to some Collector only for testing purpose.

> Also make sure you have enabled concurrency pooling and outgoingMessageQueueSize in your properties file.

Is this a modification for the Collector or the CDV.

I open a ticket,

Lutz

Setting your concurrency pool to 30 is too high. Have you followed the directions in the properties file for this property to calculate what this should be set at? It's basically 1 per EM, plus 1 additional for every 10 workstation users. So start your settings at 10 and go from there. Increasing these values will affect your heap usage so keep an eye on EM to see if you need to increase your heap values.

You also need to set a min value for concurrency pooling, so just copy the property and change the "max" to "min". Make min/max the same.

Add 'transport.outgoingMessageQueueSize=6000' to set this property.

All three properties need to be applied to all EMs in a cluster. You will need to restart the EM to apply the settings.

Hello Haruhiko,

I open a ticket to find out what's goint wrong.

The concurrency pool with 30 was used in a production CDV connected to 18 Collectors, this was a sample only.

I start testing the CDV based on APM 10.5.1 with one/two Collector only, but it doesn't work.

I continue testing with

transport.override.isengard.high.concurrency.pool.min.size=10

transport.override.isengard.high.concurrency.pool.max.size=10

transport.outgoingMessageQueueSize=6000

But I thing, it doesn't fix the problem.

Have a nice weekend,

Lutz

Hi Lutz:

Thanks as always for your invaluable contributions . Please let us know the outcome of the case which may help others in an identical situation

Hal German

Please send your ticket number so I can track the work that's happening.

Also, CDV is just a role of an EM, so it has the same restrictions when it comes to the number of connections it can have with other EMs. So that means we only support up to 10 connections, so you will need to choose wisely as to which ones.

Hello everybody,

everythings works well now. I'm able to connect the CDVs up to 20 Collectors and the MOMs are still connected to the Collectors.

I increase the number of the used pools (to 10) and the queue size (to 6000) (for all MOMs and Collectors, see above).

After a CDV connects to a Collector I get messages like "Stopping all Baseline Alert Calculators" and "Starting Baseline Alert Calculator" furthermore, but the MOM is stil connected to the Collector.

Nice, thanks for all help,

Lutz

p.s.

I changed the used pools to 20 for all CDVs.

Hello Lutz,

What is the version used prior to APM 10.5.1 and what are the values for the above changed properties?

Is there any change in load agents/metrics?

--

Thanks,

Santosh Gampa