Clarity

Expand all | Collapse all

Why does timeslicing go to sleep at midnight?

  • 1.  Why does timeslicing go to sleep at midnight?

    Posted Jul 22, 2014 05:05 AM

    I come in in the morning and look at the time the timeslices were last updated.

    Many of them are updated 23:50 and some a few minutes later. Then when I poke around they start displaying more recent dates like after 8 AM. When I look at the logs there is nothing after 3 PM yesterday in the bg and app logs. When I look at the shceduled jobs processing I see the timeslicing there and if I query the database I see it has been processing for more than 8 hours. The be service is up.After having looked at various things and not having seen anything suspicious I go back to the timeslices page and all of a sudden all slices are updated something like a quarter past nine. After that they stay updated and the timeslicing is processing for a couple of second every minute. Not processing for any longer time and db query returning it only when it is processing.

    What is going on? How do I find that out as there is nothing in the logs, no errors it just takes 8 plus hours to do the slicing right after midnight. Do I have to come in at midnight and see what is going on? It is a hosted system so I can access the OS to see if the server or the db is doing some moonlighting on the side.

     



  • 2.  Re: Why does timeslicing go to sleep at midnight?

    Posted Jul 22, 2014 06:18 AM

    Could it be a "rollover" consequence?

    i.e.- that the first timeslice to run after midnight has to perform all the timeslice-rollover that is due that day and that just happens to take 8 hours?

     

    (there is an old discussion about RE: TIP : Whats going on with Time Slicing? but I needed server-access to run that; I guess you could portlet-ify the query and log on in the middle of the night to see what is going on! )



  • 3.  Re: Why does timeslicing go to sleep at midnight?

    Posted Jul 22, 2014 07:36 AM

    If it is rollover how many nights in a row can I expect that? Now it has happened twice this week. Last week the bg was restarted.



  • 4.  Re: Why does timeslicing go to sleep at midnight?

    Posted Jul 22, 2014 08:30 AM

    Well, as we know, timeslicing can get complicated!

     

    When a rollover happens is a function of the setup of the slices in the system - so you could probably setup slices that caused rollover to happen every day (but you would also probably need to be trying very hard to set it up in that way!).

     

    Another simple alternative is that the slice jobs are just not running through the night - or time-slicing has been set as incompatible with some other long-running job that fires up just after midnight (the datamart perhaps?).

     

    (all guesses obviously).



  • 5.  Re: Why does timeslicing go to sleep at midnight?

    Posted Jul 22, 2014 09:22 AM

    Not that bad guesses I should say, If you look at the last run timestamp of the second red square and the adjacent Expiration and compare those with the ones one line below you can see that rollover is in the process.

    So the information was there I was just not looking at it. Would not have expected the slices to roll over in the middle of the month. No wonder there was nothing in the logs.

     

    Thanks.

     

    Just wanna see how it goes tomorrow morning when there is no rollover.



  • 6.  Re: Why does timeslicing go to sleep at midnight?

    Posted Jul 22, 2014 09:58 AM

    So you are not going to log in at 1am just to see what is going on?



  • 7.  Re: Why does timeslicing go to sleep at midnight?

    Posted Jul 22, 2014 03:57 PM

    Not really. I was planning to ride in to the office round 8 o'clock (with my ´pedal bike as there is shortage of parking) and look at the timeslices and rerun the query on processing jobs (even if it is hosted I do have db access for queries)



  • 8.  Re: Why does timeslicing go to sleep at midnight?

    Posted Jul 22, 2014 05:51 PM

    Wouldn't the traffic be a lot better at 1am?

     

    Typically around midnight is when automated jobs kick off, such as database maintenance, virus scan, bulk import of actuals/resources/transactions... that can impact Clarity job performance if those jobs happen to have a lot of data to process.

     

    While the slicing is going, you could periodically run the query below from TEC435572 to see what the job is doing:

     

     

    Here's a link to a Tuesday Tip to help with your investigation

     

    ** Bonus Tip** Query: What Jobs/Reports Ran Over the Last n Days?

     

    I dont' think that covers processes, so check your processes as well.

     

    And if you happen to catch the stalling in action, say at 1am, check the prlock table.



  • 9.  Re: Why does timeslicing go to sleep at midnight?

    Posted Jul 23, 2014 06:55 AM

    Thanks Connie. It did not go exactly as planned. No traffic and near record time. However, after hoping of the bike I realized I had lost my pants (shorts). You know people may lose their shirts on business deals I lost my pants. The down side was that my phone and wallet were there. So I had to ride back and cancel all the credit cards, get more phones etc.

    When finally at the office 10:30 the timeslicing was about as normal as it can be and the the 3 hour run of Rate matrix extraction was close to its end.

     

    My concern in the first instance (a day before the rollover) why was timeslicing jog running for 8 hours and the last updated stamp not changing on any of the slices. Then at 9:30 they were all up to date and the job was not running any more.

     

    The next day was the rollover and I can believe that it takes timeslicing longer when there is rollover.

     

    As said the third day nothing out of the ordinary at 10:30, but at that hour on  the other days there was nothing of  the ordinary either.

     

    If there is something happening other than rollover it is before office hours and it has be caught then.

    I've got queries for checking jobs processing processes with status code not done.

     

    I have to keep watching for a while to see if there is something which happens more than once.



  • 10.  Re: Why does timeslicing go to sleep at midnight?

    Posted Jul 28, 2014 09:21 PM

    LOL that must have been embarrassing

     

    if it happens again, another idea is to get a snapshot of the database activities during the off hours, for example get a AWR report for 11pm to 9am, you can look for what queries were running and how long they were taking, etc.

     

    hopefully it was a one time database contention of some sort and that it would never ever happen again, hopefully.



  • 11.  Re: Why does timeslicing go to sleep at midnight?

    Posted Jul 29, 2014 08:59 AM

    Yesterday it happened again. This time it was only one slice that was rolling over. However, the count of slices matching that request is zero.

    Apparently timeslicing had been processing since midnight

    an hour later the situation is over

     

     

    No processes with any significant processing time that had left anything in the database.



  • 12.  Re: Why does timeslicing go to sleep at midnight?

    Posted Jul 29, 2014 09:00 AM

    The system seems to be hiding the columns where the expiry and last updated are.



  • 13.  Re: Why does timeslicing go to sleep at midnight?

    Posted Jul 29, 2014 09:02 AM

    But if you click the image it displays the full width



  • 14.  Re: Why does timeslicing go to sleep at midnight?

    Posted Jul 30, 2014 03:38 PM

    Apparently that was my eeePC only with its 9 in display. On a 24 in display iti full size.



  • 15.  Re: Why does timeslicing go to sleep at midnight?

    Posted Jul 30, 2014 01:10 PM

    The consistency of the job stalling between midnight and 9am needs to be looked at more closely. Time to open a case, and if you can include the information discussed above (AWR/Statepack, etc) it will help speed up the investigation.



  • 16.  Re: Why does timeslicing go to sleep at midnight?

    Posted Jul 30, 2014 01:31 PM

    Thanks.

    I'm in a junior position in the PMO staff.

    I'll bring it to the attention of the senior consultants.

    It is r8.1.SP3. Is that supported?



  • 17.  Re: Why does timeslicing go to sleep at midnight?

    Posted Aug 01, 2014 07:13 PM

    8.1.3 is no longer supported



  • 18.  Re: Why does timeslicing go to sleep at midnight?

    Posted Aug 04, 2014 07:42 AM

    That means that can't create a case with support.

    Today is Monday and as expected it happened again. Timeslicing was not making any progress until 9:10 AM.

    Then all slices were updated (according to the last run).

     

    Dave's query only displays non zero number for prtimeentries and prj_baseline_details_ss. on the To procees line.

    The increase  while timeslicing and when done is approximaely the number of weekly timesheets. No change in PRJ_BASELINE_DETAILS_SS.

     

    The only slice rolling over is the WEEKLY_INCIDENT_ACTUALS with no records for that in the prj_blb_slices.

     

    Because the problem disappears by itself no action will be taken for now.



  • 19.  Re: Why does timeslicing go to sleep at midnight?

    Posted Oct 14, 2014 03:46 AM

    Connie_Fu

     

    It is MS SQL 2008 R2. What can we do there to find out the cause?

     

    When I came in this Monday timeslicing was set to No scheduled.

    My understanding is that a user cannot change a schduled job to get that status. Correct me if I am wrong.

    In this case there was an error in the log which said something about not being able to reschedule the job so apparently the system had unscheduled it.

     

    Because the timeslicing was not running at 10 AM I clicked resume which started it. Being the first run it took the usual 7,5 hrs.

    This time it was during the office hours and that is considered to be a problem. So we should like to dig a little deeper to that, but being nontechies do not know where and with what. Could you please advice.



  • 20.  Re: Why does timeslicing go to sleep at midnight?

    Posted Oct 15, 2014 06:48 PM

    Jobs showing as Not Scheduled when it is a recurring job, is typically a result of unstable BG scheduler. It could be due to network outage, over burdened BG or in some cases a job log built up. Perhaps restarting services every once in a while, say once every two weeks could help reset things; another practice you can start adopting is to clear out the entries on the Scheduled Job page - remove completed, failed, cancelled entries, would help. If this problem is unique to time slicing, maybe set up a job time slicing job with a recurring schedule; if other jobs also showed as not scheduled then the root cause is on the BG. There might even be something to be done on the database level.



  • 21.  Re: Why does timeslicing go to sleep at midnight?

    Posted Oct 16, 2014 09:02 AM

    I agree this time it being a temporary thing the connections having been the likely culprit and quite agree with the house keeping. Some people just don't care.

     

    This time, however, it brought the the cat to the table: The first run of timeslicing on every Monday or the first of every month takes 8 - 10 hours while the normal run time otherwise is 3 - 5 secs (not after Timesheet posting when it is longer)

    Now that it has got more attention, how do we find out why when we are on MS SQL and not Oracle so we can't use AWR?