TPFDF - Part II: A Consultant's Perspective
by Peter Fischer

The ten years I have been working in the TPF/ALCS database environment I have been involved in all stages of the ACPDB, TPFDB, and now TPFDF product. The product has changed, as the environment has changed, but the customer's main questions have remained the same.

• What is the future of database handling in the TPF/ALCS environment?
• What strategic role is TPFDF playing today?
• What role will it play ten years from now?
• Should we install the product now?

When asked these questions a couple of years ago, my answer was clear and I recommended to install the product because it proved at the very beginning to have the potential to become the TPF/ALCS standard database manager.

At that time the product supported some basic functions which improved application program development and included future plans to provide all the functions needed to overcome the known traditional database bottlenecks. The success of the product over the years and the proven benefit in its database organization layout and data manipulation show that customers following the recommendation a couple of years ago are now getting the benefit in the form of easy maintenance and faster integration of application requirements.

TPFDF TODAY
Today the product has become the de facto standard in the TPF/ALCS high performance transaction database environments. The concepts and functionality designed in the early 1980's for the databases in the coming decade still can be handled with the same product. The product has developed a much wider range of functions, but is still based on the initial concepts which proved to be the right design combining flexibility and performance.

But today we have to ask the same questions again. Where will the product be ten years from now, is it strategic for my installation and should I buy it now?

Today, I get this question asked many times more than before. My point of view today is different from years ago. I do not believe that TPFDF will be the database manager in the TPF/ALCS world, but a customer should acquire it today. I will try to explain the reason for this statement in the following paragraphs.

TPFDF TOMORROW
In the year 2000 the product will be 20 years old - a very long life for a software product. The database environment is changing. Especially over the last 2 years I have been confronted with so many new database aspects, trends, and directions in the TPF/ALCS environment which lead me to believe that we are now moving towards a new database philosophy and concept, even in the TPF/ALCS world.

One or two years ago my consulting services were used in defining a record layout, setting up a database structure or defining the centralised definitions. The requirements that I have to deal with today concentrate on much more complex subjects like: where should I place my data, can I distribute databases geographically and/or on different platforms? What are the performance aspects if I distribute data, which migration paths do I have to follow, what is the impact on my current operating environment, etc.? The complexity of the database issues for today's data processing needs has grown drastically over the last couple of years. TPF and ALCS which remained outside these discussions for a very long time have become more and more involved in this process as customers are reviewing their corporate data models and integrating the high performance TPF/ALCS database environment into the overall company strategy.

Another aspect is the change of the transaction profile which changed from a simple query (What is the fare from Zurich to New York?) to complex database searches (What is the cheapest fare between Europe and Australia on Fridays, considering that I would not like to fly on Airline X and Airline Y?) This new query type transactions has pushed the database manager to its limits. The requirements to change queries on an ad hoc basis have grown with new applications. Even though TPFDF provides improved flexibility in query handling (over traditional TPF/ALCS databases), new queries or changes to queries have to be re-programmed in each application.

I do not believe that TPFDF will be able to cope with the database requirements 10 years from now without major internal changes and rewrites. The modern application profile shows more and more the need for relational databases. The relational approach is a proven concept that provides the flexibility, query functions and distributed data capabilities that will be required into the future. Based on the current speed of improvement in hardware and processing power, I am sure that today's performance problems that relational database managers suffer from will be resolved in ten years time.

I have seen several design attempts to put SQL (Structure Query Language, access language to relation databases) calls on top of TPFDF to simplify query type database calls. This is not the right approach to tackle the problem because TPFDF's layout does not support the internal database organization structure required to handle a full SQL implementation. I consult for more and more customers to identify the optimal platform for their applications, and to chose database managers to support these applications based on the platform and the application profile. The trend shows that many customers are looking into moving their TPF/ALCS databases into SAA/relational environments in the future. Does that mean TPFDF has no role to play and there is no need to install it?

Absolutely not! As I mentioned before, more and more customers are very active in reviewing the corporate data model. Is the data stored in the operating environment or even in the geographical location where it really belongs and where it is best placed cost wise or access wise? Many TPF and ALCS customers are redefining their database environments and have come to the conclusion that many TPF/ALCS databases could be moved to a different platform and preferably to a relational database manager due to the nature of the application and database request profiles. Whether this relational database manager will be on a different platform or native under TPF or ALCS does not change the customers big dilemma to migrate the existing databases into the "new" environment. But how?

1. The large TPF/ALCS installations have about 700-1200 physical record types defined in the system. This means that in the case of the above mentioned migration, the same number of migration paths and procedures would have to be designed because almost every database record layout is different. The logic and the database relationship is hardcoded in the application program and requires a huge investigation effort.
   
   By defining databases under TPFDF, all databases have the same structure, and use the same access methods, and therefore provide only one, or a limited number of migration paths. A lot of information about the data relationship is defined in the TPFDF statement or centralized database definitions (DBDEF). It is obvious that it is a much easier job to migrate from such an environment to the new platform database manager.
   
2. Although not based on a relational concept, TPFDF provides many features and data record layouts which are similar to relational tables. Careful design allows a database to be defined in TPFDF's "tables" which can be converted on a one-to-one basis when the need arises. This allows the customer to be well prepared in advance.
   
3. There will be a need for TPFDF to be able to dynamically share data with relational databases ensuring the data is kept synchronized. Alternatively existing applications, with little or no application rewrite, would be able to move the data residence from TPF or ALCS to a relational database or to perform duplicate updates on a TPF or ALCS database and the relational database. TPFDF must provide this or equivalent functionality to support data residency independence.
   
4. Even so, there will remain a subset of databases in the TPF/ALCS environment, but similarity of the relational and the TPFDF structure will simplify data exchange and application communications where TPFDF structured databases are used on a TPF/ALCS platform.
   
5. TPFDF provides powerful database validation, capture and restore tools. These tools provide the basis to develop centralized, standardized powerful migration utilities to extract data from TPFDF databases and convert it into the required output format - whether from one or multiple source databases into one or multiple target databases.

In order to be the strategic tool to lead the TPF/ALCS database environment into the future, I think a couple of things have to happen. Many of TPFDF's features are driven by the large number of parameters in a TPFDF program statement. This implementation still ties the database process to the programmers knowledge of the product. I would like to see TPFDF develop into a product with less new functions, but increased internal intelligence in the database handling and database definitions. This will simplify the API, make it more compatible with modern database languages and reduce the amount of performance problems due to "bad" programming. It is outside the scope of this article to list the area of changes, but I would like to give an example.

It is not unusual that a central database file (Passenger Name File, Customer Profile, etc.) requires 10 different index files referencing the detail file. This design helps to meet the customer requirements for accessing the data in many different ways. In order to get to the detail file through the correct index, the application programmer has to code the access path and provide the index search argument. Therefore the programmer has to understand the physical data structure, and the database handling is different from a SQL based access call.

TPFDF should be able to analyze the key provided by the application and determine the optimal index path to access the data based on its internal optimiser. The centralized database definition concept will play a much bigger role than it does today. These improvements will result in more straight forward database calls and programs will be easier to migrate to a new platform.

TPFDF has always based on strong Database Administration support within the TPF/ALCS environment. The trends and directions to share data and migrate data will require a fully functional database administration to cope with the challenges to come over the next years. Because TPF and ALCS can no longer stand outside of the overall corporate database environment, the database administrator has to understand the company's data model across platforms and database concepts. I believe that the database administration will be the key to a successful migration of today's environment into the future.

A company willing to follow this path to position the applications and databases today in order to be prepared for coming migrations can only succeed by making sure that the application programmers and the database administrators are well trained and prepared for the coming challenges.

I might be too optimistic to expect all this to happen within the next ten years. Whether it will be ten or fifteen years is not important. The TPF/ALCS database environments are already changing today, and the number of old applications and old database structures require that a start is made on these changes today to position the applications and databases for the coming migrations. The only adequate tool which will help you in this process is TPFDF.