CA Service Management

Hi all,

We have SDM r12.9 conventional architecture integrated with CA SOI. Service Desk has a primary server that does not accepts user/ws connections, several secondary servers and a server fully dedicated for webservices integrations.

Due to our high volume of alerts sometimes we have dozens of tickets opened at the same minute (3 or 4 per second). During this period users experience a huge delay when working with Service Desk.

Last Thursday, and during CA Service Management Office Hours I asked if there is something we can do to prevent this behaviour. Vinod told me to try to increase tomcat memory and maxthread on SDM. He also asked me to put this question on communities so he can share a knowledge document regarding this type of configuration.

Do someone has the same problem as we do?

Thank you in advance.

Pedro

Hi Pedro,

I believe these are the two documents that you are looking for:

http://www.ca.com/us/support/ca-support-online/product-content/knowledgebase-articles/tec491277.aspx

http://www.ca.com/us/support/ca-support-online/product-content/knowledgebase-articles/tec598584.aspx

Hope this helps,

Jon I.

Hi Pedro,

On top of the articles referenced by Jon and understanding that you already have several secondary servers I will  suggest you move to Advanced Availability to take advantage to have the data not passing the background(your current primary) server to commit the data to the mdb.

I suspect knowing that you already have a dedicated secondary for your WS usage that the primary is your bottom neck here.

Have a look in the two diagrams in the link below for a quick understanding.

How to Install CA SDM - CA Service Management - 14.1 - CA Technologies Documentation

My 2 cents

/J

Thank you both for your contribute on this.

Jon_Israel, documents told me how to do that changes but not on which circunstances it should be done.

I have a secondary server dedicated to web services. Those changes should take place on that secondary or on primary server?

Do you believe that applying those configurations will decrease performance issues during WS calls from CA SOI? 

Hi Pedro,

If you are going to increase the memory and the threads, then I would recommend doing it to all tomcat instances across all servers. That will help prevent further problems in the future.

Jon I.

yes this must be your first steps prior others actions. starting with the secondary used for your Web services call

fine tuning tomcat doc can be found here too.

Fine Tuning Tomcat for better performance

/J

Hi jmayer,

I've increased maxthreads for tomcat regarding http://www.ca.com/us/support/ca-support-online/product-content/knowledgebase-articles/tec598584.aspx.

What is the best way to test the behaviour on a Quality environment so I can decide if I do the same on Production?

Thank you.

Hi,

This is not much test you can do except if you have a stress testing tool that can simulate the workload of your current production.

The most important for you is to have regression testing  making sure everything is working as expected after your change but this change is straight forward so you must be clean to go to production assuming you defined your rollout plan that is also straight forward.

In another words, you can only see positive effect  but will only realize after this is in prod.

Note that you can just recycle tomcat for this change to take effect without to have to do a full restart of SDM minimizing impact to your users.

/J

Thank you jmayer. I will change maxthreads to 300 today.

Meanwhile I found some articles regarding tomcat fine tuning (not related to Service Desk) and people say that tuning makes all difference in application performance. In my opinion CA should have  better documentation regarding this subject.

HI,

I will discourage the use of PIN authentication as tis introduce extra security risk(basic authentication with password stored in clear text).

Better use a policy key and certifcates for this may you want to avoid external authentication.

/J

We have a lot of integrations in our implementation and have experienced similar issues.   In our case, we found the session log was the culprit.   You have to keep that purged down.  It can really impact performance.    We found that the system was slowing down trying to find an unused session id.    Each login will create a session log record, with a unique session id.    If you accumulate too many rows, the system will bog down trying to establish a session.     Purging down this file immediately relieved the pain point.   

We also identified that some of our processes were written where they would establish a session, perform and action and then disconnect, then establish a session, perform an action and disconnect instead of establishing a session at the beginning of the processing, perform all actions needed and then disconnecting at the end.    This change alone saved the creation of thousands of session log entries every day. 

Hope this helps.

Hi AlexisOsborne,

Thank you for your contribution on this.

We don't have active session_log. So I believe that is not our issue. Also, CA SOI creates a unique session and uses it during all processing. Other integrations for WS creates a session, do all the processing and disconnect. I believe this is the right behaviour.

Hi,

I've increased maxthreads for tomcat but still have problems with mass web services invocation.

I'm analysing Advanced Availability but I still want to know if anyone has other guess for this problem.

Thank you.

Raghu, our major issue is regarding performance for users that established a connection to Service Desk via Gui, not Web Services.

We don't use EEM for Service Desk. Authentication is done using ldap.

Users access Service Desk via IIS.

Yes Raghu.Rudraraju, you are right. Slowness happens when web services are hitting SDM.

We have a dedicated server for web services but that Options Manager option is not installed (I believe that the domsrvr for Web Services would have to be configured on primary server and I don't want that. Am I wrong?).

I'm talking about Incidents. When created, 3 SLA events (Pre-Warning, Warning and Violation) are associated with it and an email notification is sent to Affected End User and all notifiable technicians from the group where the incident is open.

I didn't open a support ticket regarding this problem. I believe that this is an issue that happens with other clients - discovered that during last CA Service Management Office Hours - and I want to know possible solutions for that.

Also I don't consider this a bug. A nice to have solution for SDM WS might be some kind of security which prevents more than X invocations for WS from the same user.

Hi, Note that you can apply restriction of the number of call to webservice using the policy.

That say this is for all call using that policy not as per user and the threshold is hour)

Domsvr don'have to be on primary for this the webservice will connect to the domsvr you have listed and this domsvr can be  on your secondary (each one having a unique name)

/J

Hi Jerome,

Which policy are you talking about? But with the threshold measure at hour probably it will have huge side effects on our monitoring environment.

If does not have to be configured on primary server that might be something to test. Thank you for letting me know.

This can be configured under the policy table of your web service policy

administration tab => policies

-1 mean no limit

you mostly use the default one that is unrestricted.

I will recommend you to create a specific one for your monitoring tool and use a key if your tools permit.

advantage to use certificate to authenticate is that this is really more fast and less resource consuming (on top of been more secure) to use vs. going to the full authentication scheme  with OS or EEM.

Hope this help.

/J

Thank you. It's better than nothing but is far from the perfect solution.

Yesterday our team had a meeting and we decided to abandon CA's connector between CA SOI and CA SDM. We will create some PAM processes and manage the number of alerts with a queue. That way we will be able to control the number of tickets created at the same time.

Also with this approach, we will be able to pass the right group to incidents since nowadays, and using CA's connector, group is overwritten when we pass Incident Area (that has a responsible group).

What do you guys think about this?

Hi Pedro - Did Jerome's response help answer your question? If so please mark it as Correct Answer. Thanks!