Transactions, journals, locks & recovery

Write the diary first, then move the card

The rule is the same one every durable database follows: the log record describing a change is hardened before the data page containing that change is written. If the system dies at any instant, the journal always contains enough to reconstruct or undo whatever state the data pages are in.

IDMS journals hold record images. A change to an occurrence produces a before-image and an after-image of that occurrence, tagged with enough identification to know exactly which occurrence it was.

How the log fills up, and what happens to the old ones

A CV has several disk journal files, declared in the DMCL, used in rotation. When one fills, IDMS switches to the next and the full one is offloaded by the archive journal process to tape or to a dataset. If every journal is full and none has been archived, the CV cannot continue — updates stop.

The four ways things get put right

Lab — read a journal stream and recover from it

Locking, and why it is about program structure

IDMS locks at two levels, and both are visible to you.

Because locks live for the run unit's lifetime rather than a statement's, IDMS lock duration is a property of application structure. A batch program that reads a million records without committing accumulates a million locks and blocks anything that touches them. The standard remedy is periodic COMMIT ALL — commit the work, release the locks, keep the currency.

Deadlock

Two run units, each holding what the other wants. IDMS detects the cycle, selects a victim, and returns a status in the 3000 family to it; the victim's work is rolled back and it is expected to retry.

*> Run unit A                    *> Run unit B
   OBTAIN CALC DEPARTMENT '2000'    OBTAIN CALC DEPARTMENT '3200'
   KEEP EXCLUSIVE CURRENT           KEEP EXCLUSIVE CURRENT
   ...                              ...
   OBTAIN CALC DEPARTMENT '3200'    OBTAIN CALC DEPARTMENT '2000'
   KEEP EXCLUSIVE CURRENT   <--- waits   KEEP EXCLUSIVE CURRENT  <--- waits

*> Fix: agree a global lock ordering. Every program that needs
*> several departments takes them in ascending DEPT-ID order.
*> Deadlock becomes impossible, not merely rare.

The other levers: shorten transactions, take the strongest lock you will need first rather than upgrading a shared lock to exclusive later, and avoid holding locks across terminal I/O in pseudo-conversational applications.

Tuning, in priority order

1. Buffer pool size. Every page found in memory is an I/O that did not happen. This is the largest single lever and the cheapest to pull.
2. Physical clustering. VIA placement (Module 03) decides how many pages a set walk touches. Fixing clustering is a reload; the payoff is permanent.
3. Access path choice. An application doing a sweep where a CALC would do is a code fix, not a database fix.
4. Commit frequency. Too rare: lock contention and enormous rollback exposure. Too frequent: journal write overhead and, for replication, a change stream full of tiny transactions.
5. Journal configuration. Buffer size and file size determine how often you pay a switch, and how much log is at risk.
6. Page size and reserve. Fewer, larger pages mean fewer I/Os for clustered access and more contention per page.

Where the numbers in a performance argument come from

DCMT DISPLAY STATISTICS SYSTEM     *> task rates, waits, storage high-water marks
DCMT DISPLAY BUFFER                *> pages, reads, writes, hit ratio per pool
DCMT DISPLAY AREA  ...             *> lock mode, in-use count, status
DCMT DISPLAY JOURNAL               *> which journal is active, archive state
DCMT DISPLAY LOCKS                 *> current holders and waiters

*> run-unit level statistics are also available to the program itself
    ACCEPT STATISTICS INTO WS-STATS-BLOCK.

ACCEPT STATISTICS lets a program read its own I/O and lock counters — useful when you are trying to prove that a particular extract is or is not the thing causing pain.

What this means for replication

• The journal is your change stream. Everything you can replicate is something the journal records; everything the journal omits is invisible. Local-mode jobs without journalling, and utility operations that bypass normal logging, are therefore replication blind spots. Enumerate them.
• Transaction boundaries come from CKPT and ENDJ. Applying changes in commit order, not arrival order, is what makes the target transactionally consistent rather than merely eventually similar.
• Before-images are a gift. They let you compute the changed-column set exactly, detect conflicting concurrent updates, and reconstruct a deleted row for an audit target.
• Journal retention is a hard dependency. Write it into the runbook: minimum retention, who to notify before a journal reconfiguration, and what the recovery procedure is if capture falls behind retention.
• Rollbacks are normal traffic. Your pipeline must not apply changes from a run unit that later rolls back. This is why capture buffers by transaction and applies on commit, and why a single enormous uncommitted transaction can stall the whole pipeline.
• Get the incident feed. Cold starts, rollforwards, area restores and reorganisations all change what the journal means. You want to hear about them from operations, not from a reconciliation report three days later.

Checkpoint