Service Virtualization

Hi ,

Whenever my MQ server goes down , the LISA MQ services which we deployed in all Devtest Servers will shows error and ended (this is expected).

but as soon as my MQ server comes up , the VS MQ services will not start, until manually we login to server console and start..

How can we make sure all our mq vs starts in all our devtest vse as soon as MQ server comes up ?

thanks

I believe this is expected behavior.  There is an auto restart option but I don't believe that this is a continuous retry auto-restart like you are hoping for.  There is by default a hard limit on the number of errors that can be encountered before a service stops for good.  It's controlled by the property lisa.vse.max.hard.errors=<some number>.  If you set to lisa.vse.max.hard.errors=-1 in local.properties that should disable the service from stopping (I believe).

If that doesn't work for some reason, another way to do this would be to deploy a monitor (test case) to CVS that checks the status of the MQ Server continuously.  If it finds that the MQ server has come back up you could run VSEManager command line utility to restart the services you're interested in.

Thanks Josh..

I can give a try on the above suggestions..

Do you think if any possibility from IBM MQ server/queues perspective once queues are come up, it can call LISA ?

If so from then on-wards I can write a shall script to start the error-ed MQ services

From the IBM-MQ Server side, I'm not sure that something like that would be possible, but I don't know for sure.

Pretty much the only possible way to do this is to through polling.  VSE does effectively do this by restarting the service with it fails.  However, by default VSE will only try that four times before giving up.  You can increase this number by adding a property to local.properties:

lisa.vse.max.hard.errors=-1

'-1' removes the maximum entirely.  Note that this is a global setting; it will affect all of the services on that VSE server.

This will work, but it has another problem in that VSE will continually restart and fail the VSM as fast as possible until the MQ server comes back up.  You can alleviate this by modifying the VSM itself.  The 'Listen' step (or 'Consume' step in 7.x) should have an 'on error' or 'on environment error' option.  By default it goes immediately to 'Abort'.  However, you can add another step, outside the VSM's loop, which delays for some amount of time and then aborts.  If you're allowing think time then you can do this with a No-op step that has think time.  If you typically run with 0% think time then I think your only option is a Script step that calls Thread.sleep() explicitly.  Figure out how often you want the VS to "check" whether the MQ server is up again and set the think time or sleep() argument accordingly.  Then modify your Listen/Consume step to go to your sleep step instead of directly to abort when there is an error.

That, combined with setting lisa.vse.max.hard.errors to -1, effectively sets up a "poll" sequence that happens whenever the MQ server goes down.

Thanks Kevin..

few questions..

1. by setting lisa.vse.max.hard.errors=-1 will it impact on performance/memory on VSE server ?

2. As you mentioned that VSE will try for four times and turn into 'ended'.. can this be increased number of tries ?

3. Thred.sleep() .. I need to check.. probably I need to wait for couple of hours before it abort it..

> 1. by setting lisa.vse.max.hard.errors=-1 will it impact on performance/memory on VSE server ?

If you have any other services that go down and *don't* implement some kind of delay before they abort then they will spin up and down as fast as possible until your are able to manually stop them.

> 2. As you mentioned that VSE will try for four times and turn into 'ended'.. can this be increased number of tries ?

The hard errors property can be set to a maximum value of 9223372036854775807 (2^63).  Past a certain point, though, it's practically equivalent to -1 (unlimited).

> 3. Thred.sleep() .. I need to check.. probably I need to wait for couple of hours before it abort it..

So this is not a question of the total time the service will "sleep".  It's a question of how quickly you want it to come back up after the MQ server becomes available again.  Your VSE service is effectively polling the MQ server, and this is the time between each poll.  If you set your sleep time to two hours, then your VSE service will restart very few times but it will take an average of one hour after your MQ server comes back up for your VSE service to reconnect.  If you set your sleep time to 10 minutes, then your VSE service will restart more often but it will only take an average of 5 minutes after your MQ server comes back up for your VSE service to reconnect.  It's up to you to figure out the right balance of restarts and responsiveness.