DX Application Performance Management

What is the best method to setup a startup and shutdown script for the MOM and Collectors on Linux for when the server is rebooted?  Add the script to rc.local, @Reboot in cron, or add your own rc.script?

Dear Scott:

I see that you a relatively new member of the Community. Welcome! I hope you find this friendly and informative and come back often to post questions and share your experiences.

Other APM Administrators: 

Each environment is typically set up differently. What do you typically do for EM Startup on system reboot? Please share your setup! 

Thanks in advance!

Hal German 

We use a custom shell script now to start and stop the MOMs and Collectors.  We login prior to the reboot and after to execute the script manually.

If using one of the rc (run levels) to start the EM, your best bet would be to use rc.local as this executes after the multiuser run level is finished.  Also use this if you don't want to tie it down to one specific run level.

Best practice would be to create a script in a /usr location and then have rc.local call that script.  This gives you the benefit of not affecting rc.local in any other way and also if you have anything else inside rc.local, any modification by mistake won't affect anything else inside that file.

What about shutdown?  rc.local only runs on startup.

Hi Scott;

Was Matt's answer helpful? Did you change your startup/shutdown as a result.

Hal German

We are still looking at different options.  On shutdown, we want to make sure all the MOMs and Collectors stop before we stop postgres and on startup want to make sure postgres is up before we start the MOMs and Collectors.  We have postgres setup in an rc.X script.  Does the product come with a set of rc.X scripts when you install Introscope on a Linux server?

Hi Scott,

The product does not come with a set of rc scripts so you need to create one to start/stop each EM by just calling "EM_HOME/bin/EMCtrl.sh (start|stop)".

For what you want with startup & shutdown it sounds like it would be best to create S* & K* scripts  in your default run level directory e.g. /etc/rc5.d (initdefault setting in /etc/inittab). You can then use appropriate script naming to get the order of execution of postgres & EM startups correct i.e. on startup the postgres S script runs before the EM S script while on shutdown the EM K script runs before the postgres K script.

Hope that helps

Regards,

Lynn

We tried startup scripts but we found that things got started in the wrong order and we had to go in and manually recycle everything, which defeated the object of having them in the first place.

I decided that the simplest approach was to have a script that allows admin users the ability to start/stop/check status of the MOM/Collectors/postgres/WebView from a single location. It uses our infrastructure o/s monitoring agents for this, as they can accept commands and execute them. That way, an admin issues a single command and everything is stopped or started without logging in at all on any of the servers. It can also display x number of lines from each Introscope log. (The user of the script can decide how many lines to display).

I know this is specific to our environment but I thought it might be useful for others. Save us a lot of time.

rmmeli

Thank you Roger for sharing your environment setup. This is greatly appreciated! Even though this is customized approach, it may still be of interest to other APM admins/users. I hope you consider providing insights to other  present/future posts as well.

Thanks for providing the experience of not having any scripts in product itselt. It need additional shell scripting.

Thanks all the members in the discussion to share.

Regards,

Anuranjan Kumar

A bit off-topic but those same o/s agents also monitor our APM infrastructure and alert us when any component is down. That has saved us a few times as well.

Thank you for discussion an ideas. We usually use rc.local for boot process.