ca.portal.admin

INDEX Updating

Discussion created by ca.portal.admin on Apr 24, 2007
Hello all,

We now have a job that inserts about 350,000 records in an index every
day.
Heavy stuff. We also purge some entries on a weekly basis, so it won't
grow
much more in size in the future.

That job has been running for a month and already has created a huge
number
of orphans and of course performance went down the tube from there. It
was
so bad we had to rebuild it.

Perhaps I could tap into your collective memory and ask how a program
can
process that kind of volume without so many SR8 splits.

We know from experience that the structure of the index is fine, we have
many similar ones. The problem seems to be the massive updates against
this
index. What we are looking for is a programming tip or tips. Otherwise
we'll have to rebuild this index every week.

TIA,

Claude Ferland
Contractor, NYC

"
IDMS Public Discussion Forum
IDMS-L@LISTSERV.IUASSN.COM
SMTP
IDMS-L@LISTSERV.IUASSN.COM
IDMS-L@LISTSERV.IUASSN.COM
SMTP








Normal

Normal
Re: INDEX Updating
"No can do on the unlinked index, it has to be MA.

Mark Grindstaff


----- Original Message ----
From: JEC <jectec@WORLDNET.ATT.NET>
To: IDMS-L@LISTSERV.IUASSN.COM
Sent: Tuesday, April 24, 2007 8:41:58 PM
Subject: Re: INDEX Updating


All good replies from you all as usual,

First, the index is OM ASC.with about 8 million member records maximum. The
idea is to keep only the last month of records connected to the index.
Every week another job disconnects a number of records to limit the size of
the index.

The idea of unlinked index is something we will seriously consider.

Sort key in descending order is also a good idea

We have tried the TUNE INDEX but that would fix only the bottom level and
that utility is notoriously slow, at least according to our own tests.

One other idea: frequent index rebuilds. But that would be a weekly job.

I see we have a lot of testing in front of us this week.

Thanks again everybody,
Claude Ferland
Contractor, NYC


----- Original Message -----
From: ""Gary Bryson"" <GBryson@DELTA.ORG>
To: <IDMS-L@LISTSERV.IUASSN.COM>
Sent: Tuesday, April 24, 2007 6:06 PM
Subject: Re: [IDMS-L] INDEX Updating


Claude,

Have you tried the TUNE INDEX utility? It takes a while to run but you
can specify frequent COMMIT points, so the interference can be
minimized. I run it against a very large index in the wee hours of
Monday morning.

Gary Bryson
EDS/Delta Dental



The information contained in this email message and any attachments is
confidential and intended only for the addressee(s). If you are not an
addressee, you may not copy or disclose the information, or act upon it, and
you should delete it entirely from your email system. Please notify the
sender that you received this email in error.
-----Original Message-----
From: IDMS Public Discussion Forum [mailTo:IDMS-L@LISTSERV.IUASSN.COM]
On Behalf Of JEC
Sent: Tuesday, April 24, 2007 2:38 PM
To: IDMS-L@LISTSERV.IUASSN.COM
Subject: INDEX Updating

Hello all,

We now have a job that inserts about 350,000 records in an index every
day.
Heavy stuff. We also purge some entries on a weekly basis, so it won't
grow much more in size in the future.

That job has been running for a month and already has created a huge
number of orphans and of course performance went down the tube from
there. It was so bad we had to rebuild it.

Perhaps I could tap into your collective memory and ask how a program
can process that kind of volume without so many SR8 splits.

We know from experience that the structure of the index is fine, we have
many similar ones. The problem seems to be the massive updates against
this index. What we are looking for is a programming tip or tips.
Otherwise we'll have to rebuild this index every week.

TIA,

Claude Ferland
Contractor, NYC
"
IDMS Public Discussion Forum
IDMS-L@LISTSERV.IUASSN.COM
SMTP
IDMS-L@LISTSERV.IUASSN.COM
IDMS-L@LISTSERV.IUASSN.COM
SMTP








Normal

Normal
Re: INDEX Updating
"Hello Claude:

You can't do unlinked indexes on a set connection of OM.

Bill Allen

In a message dated 4/24/2007 8:42:50 P.M. Eastern Daylight Time,
jectec@WORLDNET.ATT.NET writes:

All good replies from you all as usual,

First, the index is OM ASC.with about 8 million member records maximum. The
idea is to keep only the last month of records connected to the index.
Every week another job disconnects a number of records to limit the size of
the index.

The idea of unlinked index is something we will seriously consider.

Sort key in descending order is also a good idea

We have tried the TUNE INDEX but that would fix only the bottom level and
that utility is notoriously slow, at least according to our own tests.

One other idea: frequent index rebuilds. But that would be a weekly job.

I see we have a lot of testing in front of us this week.

Thanks again everybody,
Claude Ferland
Contractor, NYC


----- Original Message -----
From: ""Gary Bryson"" <GBryson@DELTA.ORG>
To: <IDMS-L@LISTSERV.IUASSN.COM>
Sent: Tuesday, April 24, 2007 6:06 PM
Subject: Re: [IDMS-L] INDEX Updating


Claude,

Have you tried the TUNE INDEX utility? It takes a while to run but you
can specify frequent COMMIT points, so the interference can be
minimized. I run it against a very large index in the wee hours of
Monday morning.

Gary Bryson
EDS/Delta Dental



The information contained in this email message and any attachments is
confidential and intended only for the addressee(s). If you are not an
addressee, you may not copy or disclose the information, or act upon it, and
you should delete it entirely from your email system. Please notify the
sender that you received this email in error.

Outcomes