Module 01
Mainframe ground truth & IDMS anatomy
Before any database command makes sense, you need to know what is running it and where. This module maps the machinery: which process actually holds the data, how a program talks to it, and why the same database can be changed in ways that leave no trace in the change log.
The database does not run inside your program. It runs in a separate, long-lived process that owns the data files and serves every program on the machine. This module names the parts of that process and walks through what happens between your program asking for a record and getting one back.
Why it matters for copying data elsewhere: there is more than one route into this database, and only some of those routes write to the change log that replication reads. Finding the ones that do not is a week-one job, not a month-nine discovery.
New words on this page, one line each
- mainframe
- A large, extremely reliable shared computer. Not old and not slow — designed so that it never stops.
- central version (CV)
- The long-running process that owns the database files and coordinates everyone using them. Much like a PostgreSQL server process.
- local mode
- A program opening the database files directly instead of going through that shared process. Faster, exclusive, and it may write nothing to the change log.
- run unit
- One program’s session with the database, from opening it to closing it. Locks belong to this.
- area
- A named block of storage. The thing that gets opened, locked, backed up and restored as a unit.
- journal
- The change log. Every change is written here before it is written to the data itself.
- batch
- A program that runs unattended from start to finish, usually overnight. Nobody is waiting for it.
- MAXERUS
- A configured ceiling on how many outside programs may connect at once. A real limit, and one you can exhaust.
What CA IDMS actually is
The shape is familiar: a server process owns the files, clients connect to it, and it handles locking and crash recovery. Records are rows, record types are tables, and areas are roughly tablespaces.
The difference that matters is that there is no query planner. Nothing decides how to fetch your data. The program states the route step by step, so speed and correctness are properties of the application code rather than of statistics the database keeps.
IDMS — Integrated Database Management System — is a CODASYL network database. It was built in 1973 at B.F. Goodrich from an earlier IBM prototype, commercialised by Cullinane Database Systems (later Cullinet), acquired by Computer Associates in 1989, and is owned today by Broadcom. It runs on z/OS, z/VSE and z/VM. Systems built on it in 1985 are still taking transactions this morning.
Two corrections up front, because both mistakes will cost you a design decision:
-
It is not hierarchical. IMS is hierarchical. A network database allows a record to have
many owners of different types — an
EXPERTISErecord can be owned simultaneously by anEMPLOYEEand by aSKILL. That is a graph, and it is exactly how CODASYL expresses many-to-many. - It is not a flat-file system with an API. IDMS has full ACID transactions, write-ahead journalling, deadlock detection, area-level and record-level locking, online backup, and rollforward recovery. Anything you would expect of a relational engine's durability story, it has, and has had for four decades.
What it does not have, natively, is a declarative query optimiser deciding your access path. You decide the access path, in code. That single fact explains almost every other difference you will meet, including why replicating out of IDMS is more interesting than replicating out of Db2.
The product family you will see named in documents
| Name | What it is | Why you care |
|---|---|---|
| IDMS/DB | The database engine itself: storage, navigation, locking, journalling. | This is what you are replicating from. |
| IDMS/DC | The built-in teleprocessing monitor — IDMS's own CICS equivalent. Manages terminals, tasks, program pools, storage pools. | Determines how online transactions reach the database and how long they hold locks. |
| UCF | Universal Communications Facility. Lets CICS, TSO or z/VM front-end an IDMS/DC session. | Explains why some shops run "IDMS under CICS" and others do not. |
| IDD | Integrated Data Dictionary. A database about the database: schemas, subschemas, record and element definitions, programs, users. | Your best single source for generating target DDL. Module 04. |
| ADS/Online (ADS/O) | A 4GL for writing IDMS/DC applications as dialogs and processes. | Much legacy business logic lives here, not in COBOL. It is not portable. |
| LRF | Logical Record Facility. Lets a subschema expose a flattened "logical record" assembled from several physical records by path definitions. | A pre-relational view mechanism. Often already contains the joins you were about to reinvent. |
| IDMS SQL option | An SQL engine and catalog on top of the same storage, able to address both native SQL-defined tables and network records. | Module 07. It is frequently the cheapest read path for a distributed tool. |
| CA IDMS Server | JDBC/ODBC drivers plus a server component providing SQL access from off-platform. | How a Linux or Windows ETL process talks to IDMS at all. |
| ASF | Automatic System Facility. End-user-defined tables stored in IDMS without a DBA writing schema DDL. | Shadow data. Often forgotten in scope until late. |
Two ways to reach the data, and why one of them is a trap
The shared process is the warehouse with a front desk. Everybody queues, one clerk coordinates every movement, and every movement is written in the day book.
Local mode is somebody letting themselves in at night with their own key. Much faster, and nobody else can be inside at the same time — but unless they choose to write in the day book, no record exists that anything moved at all.
An IDMS database can be accessed two ways, and the difference governs concurrency, recovery and whether replication can see the change at all.
Central version (CV)
A long-running z/OS address space that owns the database files. Every application — online and batch — sends its requests to the CV, which serialises them, manages the buffer pools, holds the locks and writes the journals. This is normal production operation.
- Many concurrent run units
- Full locking and deadlock detection
- Journalling and automatic recovery
- Batch connects via a
SYSCTLdataset naming the CV
Local mode
A batch job opens the database files directly in its own address space. Fast, because there is no cross-memory hop and no lock manager, and dangerous, because nobody else can be using those areas.
- Single job, exclusive access to the areas
- No inter-job locking
- Journalling only if the job requests it (
JOURNALin SYSIDMS) - Used for bulk load, reorg, some overnight batch
What is inside that shared process
Six things live in that process: a scheduler handing out work, the database engine itself, a cache of recently used storage blocks, a buffer for the change log, memory holding loaded programs, and a lock manager.
Almost every performance question reduces to one of those six. “The database is slow” is never a useful sentence. “The cache is missing 60% of its reads on this data” is.
The CV is a single address space containing several pools of storage plus the engine components. Knowing
the pieces by name lets you read a DCMT DISPLAY output or a performance report without guessing.
The configuration files you will hear named
| Artefact | Defines | Changed by |
|---|---|---|
SYSGEN | The CV itself: tasks, programs, terminals, storage pools, limits like MAXTASKS and MAXERUS. | System DBA; requires a CV recycle for most changes. |
DMCL | Device Media Control Language. Maps logical areas onto physical files, sets page size, buffer pools, journal files, db-key radix. | DBA; some changes require unload/reload. |
DBTABLE / DBNAME | Which segments make up a named database. Lets one CV serve several logically separate databases. | DBA; a run unit picks one via DBNAME. |
SYSIDMS | A parameter file read by batch jobs: DBNAME, DICTNAME, JOURNAL, LOCAL, and dozens more. | Whoever writes the JCL — including you. |
SYSCTL | A small dataset that tells a batch job which CV to connect to. | Operations. Its presence or absence is how a job silently becomes local mode. |
//EXTRACT EXEC PGM=IDMSBCF,REGION=0M
//STEPLIB DD DSN=IDMS.LOADLIB,DISP=SHR
//SYSCTL DD DSN=IDMS.SYSCTL,DISP=SHR <-- present: run under the CV
//SYSIDMS DD *
DBNAME=EMPDB
DICTNAME=APPLDICT
JOURNAL=ON
/*
//SYSIPT DD *
...commands...
/*
Remove the SYSCTL DD and add LOCAL=ON to SYSIDMS, and the same program
now opens the files directly. Same JCL shape, completely different concurrency, recovery and replication
behaviour. This is the kind of detail that decides whether a modernisation project discovers a data gap in
week 3 or in month 9.
Six words for "unit of work", and what each really means
The run unit — one program’s session with the database. It owns the locks and it owns the position markers.
That has a consequence you meet again in Module 08: a program that opens a session and then waits for a human to type something is holding database locks for as long as that person takes to think.
Mainframe documentation uses these interchangeably in prose and precisely in reference material. Learn the precise meanings; they matter for locking and for interpreting journal records.
| Term | Precise meaning | Lifetime |
|---|---|---|
| Run unit | One binding of a program to a subschema. Owns the currency block, the locks and the journal thread. The fundamental IDMS unit. | BIND RUN-UNIT → FINISH |
| Task | A dispatchable unit inside IDMS/DC. Started by a terminal input, a DCMT command or another task. | One transaction's worth of processing |
| Session | A user's connection to DC, spanning many tasks. Holds signon identity and scratch/queue context. | Signon → signoff |
| Transaction | The recovery unit: everything between checkpoints. A run unit can contain several by issuing COMMIT. | Checkpoint → checkpoint |
| External run unit (ERU) | A run unit driven from outside the CV — batch, CICS, a JDBC client. Counted against MAXERUS. | Same as run unit |
| Job step | Pure z/OS: one EXEC PGM= in JCL. Says nothing about IDMS. | Program start → end |
Lab — trace one request end to end
Step through what actually happens between a program issuing OBTAIN CALC EMPLOYEE and getting
a record back. Each step names the component that owns it.
The commands you will see quoted in incident tickets
You will rarely type these, but you will read their output in incident tickets, and knowing what they interrogate tells you what state IDMS keeps.
DCMT DISPLAY ACTIVE TASKS *> what is running right now
DCMT DISPLAY STATISTICS SYSTEM *> task rates, waits, storage
DCMT DISPLAY BUFFER *> buffer pool sizes and hit ratios
DCMT DISPLAY AREA EMP-DEMO-REGION *> area status, lock mode, who has it
DCMT VARY AREA EMP-DEMO-REGION OFFLINE *> quiesce for a reorg
DCMT DISPLAY JOURNAL *> journal file status and switch state
DCMT DISPLAY LOCKS *> who is waiting on whom
DCUF SET DICTNAME APPLDICT *> point your session at a dictionary
DCMT DISPLAY AREA and DCMT DISPLAY JOURNAL are the two you will personally care
about during a replication cutover: the first tells you whether an area is quiesced (so your image copy is
consistent), the second tells you whether the journal your capture agent depends on has been switched or
archived out from under it.
What this means for replication
- Identify every access route into the database, not just the online ones. Online DC transactions, batch under the CV, batch in local mode, ADS dialogs, ASF tables and SQL clients can all change data. Only some of them journal.
- Journal retention is a project dependency. Your capture agent reads the journal stream. If operations archives and reuses journals on a fixed cycle sized for recovery, not for replication, a capture outage longer than that cycle forces a full refresh. Agree retention in writing, early.
- MAXERUS is a real ceiling. External run units — which is what an off-platform reader is — are capped by SYSGEN. Adding a replication reader, a validation query tool and an ETL job at once can exhaust it and start failing production batch. Ask for the current and maximum values.
- Local-mode reorgs renumber everything. An unload/reload changes physical addresses. If anything downstream keyed on a db-key, it breaks silently. Module 03 explains exactly why.
Exercise 1.1 — Interrogate your own source system
Write down the questions you would send to the IDMS DBA this week. A good list includes:
- Which segments and areas hold the data in scope, and which DBNAME groups them?
- Which jobs run in local mode against those areas, on what schedule, and do they journal?
- What is the journal file configuration — how many, what size, archived how often, retained how long?
- What are
MAXTASKSandMAXERUS, and what is the current peak usage? - Is the SQL option installed and licensed? Is CA IDMS Server available?
- When was the last unload/reload of the areas in scope, and what triggers the next one?
- Are there ASF tables or LRF logical records in scope that are not in the main schema?
Every one of these has bitten a modernisation project. Asking them in week one is the cheapest risk reduction available to you.