ca.portal.admin

Re. Performance Degradation

Discussion created by ca.portal.admin on Feb 9, 2006
Or maybe the common error of doing an unload/reload of an area with
owner records without also unloading/reloading areas with member records
stored via
cross-area pointers from the owner area? The new home page of the owner
is no longer proportional to the offset of the member.

73 de Jim, KD1YV


Vohs, Bill wrote:

>From: Bill Vohs, SC Data Center, Monroe, Wi
>Date: February 1, 2006
>
>Subject: Enlarge/Expand IDMS Database Causes Severe Performance
>Degradation
>
>We are running:
>
> z/OS 1.4
> IDMS Release 16 SP02
> EMC DMX2000 DASD
>
>We have a need to enlarge a number of the IDMS databases.
>
>Every time we do, we experience severe performance degradation.
>Run times elongate by a factor of 3 to 5.
>Job that ran in 30 minutes now takes 3-4 hours.
>
>Our first attempt was back in November 2005.
>The approach at that time was to do an UNLOAD and RELOAD to the
enlarged
>area.
>This time we enlarged the databases by a factor of 3.
>I.e. One database went from 15,000 tracks to 45,000 tracks.
> Another database went from 7,000 tracks to 21,000
tracks.
> These all on 3390 DASD on the EMC DMX2000.
>
> Our run times were:
>
> Before UNLOAD/RELOAD 11/09/2005 74
minutes
> 11/10/2005 46 minutes
>
> After UNLOAD/RELOAD 11/16/2005 174
minutes
> 11/1/2005 198 minutes
>
> This happened with almost every one of our jobs.
>
>Our second attempt was January 29, 2006..
>The approach this time was to do a EXPAND PAGE to an enlarged area
>This time the databases were enlarged by a factor of 2.
>One database went from PAGE SIZE 5064 to 10796 from 15,000 tracks to
30,000
>tracks.
>Another database went from PAGE SIZE 5064 to 10796 from 7,000 tracks
to
>14,000 tracks.
>
> The run times are:
>
> Before EXPAND PAGE 01/26/2006
64
>minutes
>
> After EXPAND PAGE
01/31/2006 209
>minutes
> 02/01/2006 212 minutes
>
>
>We have had an IDMS consultant review our work and he says every thing
looks
>great.
>
>Our z/OS and IDMS statistics show nothing out of the ordinary.
>
>We are at a bit of a loss.
>
>Might anybody have any suggestion for us?
>
>
>

"
IDMS Public Discussion Forum
IDMS-L@LISTSERV.IUASSN.COM
SMTP
IDMS-L@LISTSERV.IUASSN.COM
IDMS-L@LISTSERV.IUASSN.COM
SMTP








Normal

Normal
Re: Performance Degradation after Database Enlarge/Expand
"I had a similar experience. After unloading and reloading a large MRP area we had a local mode update job that runs against this area and did a lot of area sweeps against it as well. With the added pages from the reload, which was substantial, it made the sweeps take a lot longer. In wallclock time I'd say the job was a good 4-5 hours longer.

We left it alone because we knew we'd be upgrading to a Z800 processor and we'd be upgrading the opsys from os/390 to z/os as well. Now, of course this batch job just flies. It runs in about half the time it did prior to the unlosd reload.

I hope this helps.

J. Wayne Doneker
BAE Systems
York Pa.
717 225 8109
Email: john.doneker@baesystems.com

vohs@SCDATACENTER.COM 2/8/2006 2:03:25 PM >>>
From: Bill Vohs, SC Data Center, Monroe, Wi
Date: February 1, 2006

Subject: Enlarge/Expand IDMS Database Causes Severe Performance
Degradation

We are running:

z/OS 1.4
IDMS Release 16 SP02
EMC DMX2000 DASD

We have a need to enlarge a number of the IDMS databases.

Every time we do, we experience severe performance degradation.
Run times elongate by a factor of 3 to 5.
Job that ran in 30 minutes now takes 3-4 hours.

Our first attempt was back in November 2005.
The approach at that time was to do an UNLOAD and RELOAD to the enlarged
area.
This time we enlarged the databases by a factor of 3.
I.e. One database went from 15,000 tracks to 45,000 tracks.
Another database went from 7,000 tracks to 21,000 tracks.
These all on 3390 DASD on the EMC DMX2000.

Our run times were:

Before UNLOAD/RELOAD 11/09/2005 74 minutes
11/10/2005 46 minutes

After UNLOAD/RELOAD 11/16/2005 174 minutes
11/1/2005 198 minutes

This happened with almost every one of our jobs.

Our second attempt was January 29, 2006..
The approach this time was to do a EXPAND PAGE to an enlarged area
This time the databases were enlarged by a factor of 2.
One database went from PAGE SIZE 5064 to 10796 from 15,000 tracks to 30,000
tracks.
Another database went from PAGE SIZE 5064 to 10796 from 7,000 tracks to
14,000 tracks.

The run times are:

Before EXPAND PAGE 01/26/2006 64
minutes

After EXPAND PAGE 01/31/2006 209
minutes
02/01/2006 212 minutes


We have had an IDMS consultant review our work and he says every thing looks
great.

Our z/OS and IDMS statistics show nothing out of the ordinary.

We are at a bit of a loss.

Might anybody have any suggestion for us?

"
IDMS Public Discussion Forum
IDMS-L@LISTSERV.IUASSN.COM
SMTP
IDMS-L@LISTSERV.IUASSN.COM
IDMS-L@LISTSERV.IUASSN.COM
SMTP








Normal

Normal
Re. Performance Degradation
"Or maybe the common error of doing an unload/reload of an area with
owner records without also unloading/reloading areas with member records
stored via
cross-area pointers from the owner area? The new home page of the owner
is no longer proportional to the offset of the member.

73 de Jim, KD1YV


Vohs, Bill wrote:

>From: Bill Vohs, SC Data Center, Monroe, Wi
>Date: February 1, 2006
>
>Subject: Enlarge/Expand IDMS Database Causes Severe Performance
>Degradation
>
>We are running:
>
> z/OS 1.4
> IDMS Release 16 SP02
> EMC DMX2000 DASD
>
>We have a need to enlarge a number of the IDMS databases.
>
>Every time we do, we experience severe performance degradation.
>Run times elongate by a factor of 3 to 5.
>Job that ran in 30 minutes now takes 3-4 hours.
>
>Our first attempt was back in November 2005.
>The approach at that time was to do an UNLOAD and RELOAD to the enlarged
>area.
>This time we enlarged the databases by a factor of 3.
>I.e. One database went from 15,000 tracks to 45,000 tracks.
> Another database went from 7,000 tracks to 21,000 tracks.
> These all on 3390 DASD on the EMC DMX2000.
>
> Our run times were:
>
> Before UNLOAD/RELOAD 11/09/2005 74 minutes
> 11/10/2005 46 minutes
>
> After UNLOAD/RELOAD 11/16/2005 174
minutes
> 11/1/2005 198 minutes
>
> This happened with almost every one of our jobs.
>
>Our second attempt was January 29, 2006..
>The approach this time was to do a EXPAND PAGE to an enlarged area
>This time the databases were enlarged by a factor of 2.
>One database went from PAGE SIZE 5064 to 10796 from 15,000 tracks to
30,000
>tracks.
>Another database went from PAGE SIZE 5064 to 10796 from 7,000 tracks to
>14,000 tracks.
>
> The run times are:
>
> Before EXPAND PAGE 01/26/2006 64
>minutes
>
> After EXPAND PAGE
01/31/2006 209
>minutes
> 02/01/2006 212 minutes
>
>
>We have had an IDMS consultant review our work and he says every thing
looks
>great.
>
>Our z/OS and IDMS statistics show nothing out of the ordinary.
>
>We are at a bit of a loss.
>
>Might anybody have any suggestion for us?
>
>
>

"
IDMS Public Discussion Forum
IDMS-L@LISTSERV.IUASSN.COM
SMTP
IDMS-L@LISTSERV.IUASSN.COM
IDMS-L@LISTSERV.IUASSN.COM
SMTP








Normal

Normal
Re: Performance Degradation after Database Enlarge/Expand
" one other thought, if you DID increase the number of buffers to now
hit the prefetch threshold, but these are not jobs suited for
prefetch, THAT in and of itself could cause delays ..

but again, we do not have enough information to know what is happening


chris

"
IDMS Public Discussion Forum
IDMS-L@LISTSERV.IUASSN.COM
SMTP
IDMS-L@LISTSERV.IUASSN.COM
IDMS-L@LISTSERV.IUASSN.COM
SMTP








Normal

Normal
Performance Problem
"Hello Bill:

I believe that by increasing the page size and the number of pages that you
may have severely impacted the I/O Subsystem, especially the caching
subsystem. How much memory is available in the DASD Caching Subsystem?

As an example I placed a large amount of buffer pages (5000) in an area for
a local mode batch job to increase it's performance, the result was that this
job ran better but everything else in the system suffered because I had
overloaded the DASD Caching Subsystem, we only had 14 GIG Total for DASD Cashing.

Theoretically you have done everything right, the only jobs that should see
an increase in elapsed run time would be area sweeps because of the increase
in the number of pages in the area.

Since you unloaded and reloaded these areas into larger areas you should
have helped the CALC algorithm (Depending on the key), you should of also helped
the VIA Clustering because of the larger page size and you would of rebuilt
the indexes as part of the unload and reload.

Things to look out for, does your Schemas restrict records to a specific
page range, percentage of the area or number of pages? What about your sets, any
page range restrictions there? Any thing special regarding your segments?
Are these jobs run Local or CV? Did you increase the number of pages in the
buffer or just the buffer page size? Beware of PREFETCH if your jobs are more
random in their processing.

Bill Allen

"
IDMS Public Discussion Forum
IDMS-L@LISTSERV.IUASSN.COM
SMTP
IDMS-L@LISTSERV.IUASSN.COM
IDMS-L@LISTSERV.IUASSN.COM
SMTP








Normal

Normal
Re: Performance Degradation after Database Enlarge/Expand
"One last thought. Prefetch, via Z/OS or IDMS, can be a very GOOD thing or a very BAD thing.
If you had prefetch on and didn't need it, it may have been ok with your smaller files and clustered data.

Since you unloaded/reloaded and spread your data out, and prefetch is on for random access, you may be loading far more data than you need to and causing yourself to thrash. (Seen it happen!)

Add another penny..............
Scott

________________________________

From: Brady, Scott (HAL)
Sent: Wed 2/8/2006 2:59 PM
To: IDMS Public Discussion Forum; IDMS-L@LISTSERV.IUASSN.COM
Subject: RE: Performance Degradation after Database Enlarge/Expand


Questions:
1. Local mode jobs? Retrieval only?
2. Has online transaction time also taken a hit?

Things to look for:
1. What is the I/O response time? Can you get RMF stats before and after and determine if the I/O respomse has changed?
2. Channel issues? Since you have larger files, you ARE pushing more data. Can the channel handle it?
3. Data placement? Despite best efforts, UNLOAD/RELOAD can't put the data back the way it was put in. I've found that as data is added and subtracted, especially highly volital data, the system seems to hum along, find a ""niche"" for the data. An unload/reload will mess this up by following ""the rules"". It may take a cycle of your business to get the data back to where it really belongs. (Yeah, pretty Zen, huh?)
4. Back to I/O. What files are you accessing? The ""source"" database (the one the CV updates) or a copy? What are you using to copy? I've found degradation in batch jobs due to the hardware ""SNAP"" not really being as fast as it says it is. Maybe all the I/O time is being used by the hardware to REALLY copy files?
Lastly, Everything else that was said is appropriate.

My nickle's worth. (and please excuse any spelling errors!)

Scott Brady
Holland America Line


________________________________

From: IDMS Public Discussion Forum on behalf of Miley, Dan L
Sent: Wed 2/8/2006 1:58 PM
To: IDMS-L@LISTSERV.IUASSN.COM
Subject: Re: Performance Degradation after Database Enlarge/Expand



You didn't mention the type of jobs that were experiencing performance
problems. But if they are doing area sweeps they will of course run
longer. I would suggest that you use prefetch with file buff parm set
to at least 3000. If the jobs are using indexes perhaps a rebuild
would help. That's about all I can say based on the limited information
supplied.

Dan Miley
Lockheed Martin

Outcomes