IDMS

I am about to attempt a resize of database area that is essentially almost 2 billion occurrences of a single record type. The resize to almost 4 billion will buy the site about 3 to 5 years growth at current rates, so not an urgent problem. However, when the next resize is required, no matter how I split the page number and number of records per page within the DBKEY Radix, I will be limited to 2**32 (i.e. 4 billion) occurrences. Are there any plans to increase the DBKEY, or any reasonable suggestions to handle this issue?

N.B. I have discussed deletion/archiving of ‘old’ occurrences, but that will barely scratch the surface for this area. I have also suggested some sort of logical segmentation, but that is not going down well with the application owners and developers (but may be the only alternative!).

Brian,

Can you possibly create partitions of data?

For instance, take your primary key and run it thru IDMSCALC with a page range of, say 1 to 10.

Take the resulting “page” and translate that into, for instance, a DBName with the single digit on the end representing a partition. (IE, ABC0, ABC1, ABC2, … ABC9).

Then take your data and split it up, yes, you’ll need a onetime program for this, into the 10 partitions.

Then, once stored with a specific key, it will always be addressable in a specific partition.

There would be more to it than this, but this is just a high level.

Your partition/area relationship could be by area name or page group or both.

Chuck

Charles (Chuck) Hardee<mailto:Chuck.Hardee@ThermoFisher.com>

Senior Systems Engineer/Database Administration

EAS Information Technology<mailto:DBA%20Engineering%20-%20DB2_IDMS>

Thermo Fisher Scientific

300 Industry Drive | Pittsburgh, PA 15275

Phone +1 (724) 517-2633 | Mobile +1 (412) 877-2809 | FAX: +1 (412) 490-9230

Chuck.Hardee@ThermoFisher.com<mailto:Chuck.Hardee@ThermoFisher.com>  | www.thermofisher.com

WORLDWIDE CONFIDENTIALITY NOTE: Dissemination, distribution or copying of this e-mail or the information herein by anyone other than the intended recipient, or an employee or agent of a system responsible for delivering the message to the intended recipient, is prohibited. If you are not the intended recipient, please inform the sender and delete all copies.

A few thoughts off the top of my head

I think multiple segments is a good solution – if there is some identifiable column upon which to get a good segmentation

Include a master table that defines the range and dbname of every segment (its own buffers – perhaps populate at cv startup)

Then every program that reads or updates this record would read the master table first to get the proper dbname and then call the user program passing the desired dbname

The size of the dbkey is not going to change in the foreseeable future.  The suggestions regarding segmentation are the way that most users address this scenario as many users have databases that exceed this size.  There is a Knowledge Document (KD) that addresses the usage of database segmentation. Here is the link to the KD site.

http://ecm.ca.com/kb/pages/Authoring/dashboard.aspx

The document number is TEC538495.

Here is a link to TEC538495 https://www.ca.com/us/services-support/ca-support/ca-support-online/knowledge-base-articles.TEC538495.html 

While segmentation and/or a partitioning methodologies are great suggestions, realize that they will work up to a point.  The restriction will be that any records that are related by set connections will all physically need to be stored in the same segment/partition.  If your database has been designed such that that can be maintained, then you are good to go.  On the other hand, if your database structure is such that there are records participating in sets that will need to be maintained **across** segment/partitions, you are essentially out of luck.   You could implement the cross segment/partition relationships non-physically by use of a foreign key that embodies a traditional foreign key value and a segment/partition id.  However, the resulting overhead of managing run units chasing "set chains" across multiple segments/partitions would be onerous at best and I don't even want to think about the performance implications.

Many years ago when I was on the Technical Advisory Committee (TAC) we had discussions about the impact of the 2**32 record limitation (exclusive of record size) and thought about having someone write an enhancement request for 64 bit DBKeys, but it was dismissed out of hand because of the deep and extensive coding implications it would cast on the IDMS systems software.  So Dick Weiland's earlier point is well taken - 2**32 is a hard and fast wall that will be with us for some time.  A careful examination of the structure of your database will yield what you can and cannot tolerate as far as implementing a segment/partition methodology and may well get you out of the woods.  If your database structure can tolerate that, then I would suggest a well crafted database procedure to manage the segment/partitioning overhead transparently to minimize changes required to the application for database access.

If not, then you will need every bit of the 3 to 5 years before you runout of space again to examine other alternatives!  As much as I love IDMS, it may not hold the solution for the data volume that you require, but walking away may not be possible either and is dependent upon your application investment and it's place in the overall business operations landscape of your company.

Here's an "out of the box" suggestion which may help.  I was faced with a similar situation (but not for space or the 2**32 record limitation) several years ago and proposed a "hybrid" solution that would permit the use of the IDMS application while storing the database assets in a externally maintained relational database !  Many of today's relational platforms permit storing numbers of records well in excess of the 2**32 limitation while serendipitously making the data available for other applications outside of traditional IDMS silos.  Just about every IDMS DML action can be translated into a specific SQL statement.  A database procedure can be written which intercepts the DML, converts it to SQL and passes the command thru to the relational engine, then after getting the request satisfied, reformats the content to meet the IDMS record format that was accessed.  Is it perfect?  No.  Are there other implications?  Yes!  There may be some additional compromises needed to make it work.  But depending on your legacy investment and the overall cost of recreating the system functionality on another platform, in another language using a different database (if that's even possible) from scratch may be cost prohibitive and make such a "hybrid" alternative work for you and your company.