ca.portal.admin

INDEX Updating

Discussion created by ca.portal.admin on Apr 24, 2007
Latest reply on Apr 24, 2007 by ca.portal.admin
Hello all,

We now have a job that inserts about 350,000 records in an index every
day.
Heavy stuff. We also purge some entries on a weekly basis, so it won't
grow much more in size in the future.

That job has been running for a month and already has created a huge
number of orphans and of course performance went down the tube from
there. It was so bad we had to rebuild it.

Perhaps I could tap into your collective memory and ask how a program
can process that kind of volume without so many SR8 splits.

We know from experience that the structure of the index is fine, we have
many similar ones. The problem seems to be the massive updates against
this index. What we are looking for is a programming tip or tips.
Otherwise we'll have to rebuild this index every week.

TIA,

Claude Ferland
Contractor, NYC







************************************** See what's free at http://www.aol.com.
"
IDMS Public Discussion Forum
IDMS-L@LISTSERV.IUASSN.COM
SMTP
IDMS-L@LISTSERV.IUASSN.COM
IDMS-L@LISTSERV.IUASSN.COM
SMTP








Normal

Normal
Re: INDEX Updating
"All good replies from you all as usual,

First, the index is OM ASC.with about 8 million member records maximum. The
idea is to keep only the last month of records connected to the index.
Every week another job disconnects a number of records to limit the size of
the index.

The idea of unlinked index is something we will seriously consider.

Sort key in descending order is also a good idea

We have tried the TUNE INDEX but that would fix only the bottom level and
that utility is notoriously slow, at least according to our own tests.

One other idea: frequent index rebuilds. But that would be a weekly job.

I see we have a lot of testing in front of us this week.

Thanks again everybody,
Claude Ferland
Contractor, NYC


----- Original Message -----
From: ""Gary Bryson"" <GBryson@DELTA.ORG>
To: <IDMS-L@LISTSERV.IUASSN.COM>
Sent: Tuesday, April 24, 2007 6:06 PM
Subject: Re: [IDMS-L] INDEX Updating


Claude,

Have you tried the TUNE INDEX utility? It takes a while to run but you
can specify frequent COMMIT points, so the interference can be
minimized. I run it against a very large index in the wee hours of
Monday morning.

Gary Bryson
EDS/Delta Dental



The information contained in this email message and any attachments is
confidential and intended only for the addressee(s). If you are not an
addressee, you may not copy or disclose the information, or act upon it, and
you should delete it entirely from your email system. Please notify the
sender that you received this email in error.
-----Original Message-----
From: IDMS Public Discussion Forum [mailTo:IDMS-L@LISTSERV.IUASSN.COM]
On Behalf Of JEC
Sent: Tuesday, April 24, 2007 2:38 PM
To: IDMS-L@LISTSERV.IUASSN.COM
Subject: INDEX Updating

Hello all,

We now have a job that inserts about 350,000 records in an index every
day.
Heavy stuff. We also purge some entries on a weekly basis, so it won't
grow much more in size in the future.

That job has been running for a month and already has created a huge
number of orphans and of course performance went down the tube from
there. It was so bad we had to rebuild it.

Perhaps I could tap into your collective memory and ask how a program
can process that kind of volume without so many SR8 splits.

We know from experience that the structure of the index is fine, we have
many similar ones. The problem seems to be the massive updates against
this index. What we are looking for is a programming tip or tips.
Otherwise we'll have to rebuild this index every week.

TIA,

Claude Ferland
Contractor, NYC
"
IDMS Public Discussion Forum
IDMS-L@LISTSERV.IUASSN.COM
SMTP
IDMS-L@LISTSERV.IUASSN.COM
IDMS-L@LISTSERV.IUASSN.COM
SMTP








Normal

Normal
Re: INDEX Updating
"Claude,

Have you tried the TUNE INDEX utility? It takes a while to run but you
can specify frequent COMMIT points, so the interference can be
minimized. I run it against a very large index in the wee hours of
Monday morning.

Gary Bryson
EDS/Delta Dental



The information contained in this email message and any attachments is confidential and intended only for the addressee(s). If you are not an addressee, you may not copy or disclose the information, or act upon it, and you should delete it entirely from your email system. Please notify the sender that you received this email in error.
-----Original Message-----
From: IDMS Public Discussion Forum [mailTo:IDMS-L@LISTSERV.IUASSN.COM]
On Behalf Of JEC
Sent: Tuesday, April 24, 2007 2:38 PM
To: IDMS-L@LISTSERV.IUASSN.COM
Subject: INDEX Updating

Hello all,

We now have a job that inserts about 350,000 records in an index every
day.
Heavy stuff. We also purge some entries on a weekly basis, so it won't
grow much more in size in the future.

That job has been running for a month and already has created a huge
number of orphans and of course performance went down the tube from
there. It was so bad we had to rebuild it.

Perhaps I could tap into your collective memory and ask how a program
can process that kind of volume without so many SR8 splits.

We know from experience that the structure of the index is fine, we have
many similar ones. The problem seems to be the massive updates against
this index. What we are looking for is a programming tip or tips.
Otherwise we'll have to rebuild this index every week.

TIA,

Claude Ferland
Contractor, NYC
"
IDMS Public Discussion Forum
IDMS-L@LISTSERV.IUASSN.COM
SMTP
IDMS-L@LISTSERV.IUASSN.COM
IDMS-L@LISTSERV.IUASSN.COM
SMTP








Normal

Normal
Re: INDEX Updating
"Mark,

I have been using unlinked indices for a very long time. At one shop, I
converted all MA indices to unlinked and did not get the 0326 errors.
However, the volume there was nothing like Jones. I am guessing you are
at Jones; because, I had heard quite some time ago you had gone back. I
am a senior DBA at SAIC in San Diego.

Regards,

John Collins

Outcomes