retrofit
0.15.0-wip.89
retrofit.sh

These docs are for 0.15.0-wip.89, a WIP, and may describe what no release does yet. The newest release is 0.14.0: this page for 0.14.0.

Changelog

All notable changes to retrofit are recorded here, newest first, in the categories of Keep a Changelog. Before 1.0 retrofit promises no compatibility: a change is disclosed here, not prevented.

[Unreleased] records what the builds since the last release changed; a release renames it to its version. Its fixed Log format heading records every change to the oplog’s vocabulary, parameters, headers or seal bytes: what changed and what a user does, usually regenerating the log. CI fails a commit that changes internal/oplog/catalog/*.golden.json, internal/oplog/serialize/ outside its tests, or internal/seal/testdata/leafbytes_*.golden but adds no item under that heading itself (make logformat-check); with make hooks, the commit-msg hook refuses it at git commit. A commit whose serializer edit changes none of the vocabulary, parameters, headers or seal bytes may instead end its message with a Log-format: unchanged trailer; a reader refusing a log it used to accept is a format change. A change to either set of goldens always needs the item.

[Unreleased]

Log format

  • An AddForeignKey whose referenced columns are not, as a set, the columns of a primary key, a non-deferrable unique constraint or a unique index of the referenced table (not a partial or expression index) is refused as fk-referenced-key-missing, naming the foreign key, the table and the columns, as PostgreSQL refuses it at apply. import, op add and validate accepted it, and snapshot and migrate wrote DDL that failed. A referenced partition counts its parent’s keys; a reference to a table in another schema is not checked. A log holding one is refused wherever it is replayed to be checked or rendered: remove the foreign key (op remove) or revise it to reference a key (op revise), then regenerate the log or re-seal. The log’s bytes are unchanged.
  • CreateTrigger gains function_external, a trigger function held by its schema-qualified name, and its function becomes optional beside it: an operation sets exactly one. A trigger in one schema calling a.touch() used to bind a language=internal stub in its own schema, or a same-named function there, and render EXECUTE FUNCTION touch(). It now binds a.touch itself when schema a is registered and holds it, and renders "a".touch() under that function’s current name; otherwise it holds the name as written. A log written before keeps its stubs and replays unchanged: re-import the schema to rebind such a trigger.
  • AddColumn and CastColumn take type_ref=, the ElementID of another registered schema’s type that the column’s type= names, beside the spelling. import, drift, op add and an op revise of type= write it, so a column typed by another schema’s type follows a RenameType there with no operation in its own schema, and validate refuses one whose type no registered schema holds. A log written before holds such a column by its spelling and reads as before; regenerate the log to bind it.
  • CopyColumn takes type_ref= beside a type= override that names another registered schema’s type, written by op add and an op revise of type=, so the copy follows a RenameType there in snapshot and migrate and validate refuses one whose type no registered schema holds. Before, such an override was held by its spelling. A copy that inherits its source’s type keeps the source’s binding as before; a log written before holds an override by its spelling: revise its type= to bind it.

Fixed

  • drift pairs a foreign key into another schema by its spelling: the schema, table and both column lists. An unchanged one used to plan its drop beside an add-foreign-key-cross-schema-unsupported skip at exit 0, and the next drift planned nothing, so the lost key read as converged. A paired key renames, changes DEFERRABLE or validates in place; one whose ON DELETE, ON UPDATE or MATCH changed is dropped and added again.
  • drift adds a foreign key into another schema as import records it, held by its qualified spelling. It used to skip the key as add-foreign-key-cross-schema-unsupported at exit 0, so a key the file gained, or one whose referenced table or columns changed, never reached the log; that skip kind is gone. A key whose ON DELETE SET NULL (…) or SET DEFAULT (…) names columns is still skipped, as one in its own schema is.

Known issues

  • A project with several registered schemas is not supported yet. How a reference from one schema into another, a drop that another schema depends on, and the verbs that read more than one schema’s log should behave is still being designed, and each can go wrong today.
  • Ending a column that a later RunRawScript body names is refused only when the body is bound. For a body stored opaque, op remove or op revise of the column, or a merge whose other side removed it, writes the log, and validate accepts it. migrate then emits the script against a table without the column, so the migration fails whole at apply. validate --history fails a pull request that adds such a body, unless the name sits only in a conditional branch or the body does not parse; it only advises on a body already on the target that the pull request leaves naming a removed column. Apply the migration to a scratch database in CI (retrofit migrate --from root --to last, or from the latest Seal).
  • drift asks the reversal question only when the database holds something logged after the drop. A database that has not yet run the log’s newest drop reads as intent, and the plan re-adds the item with no question. Review a drift plan’s re-adds against the log before accepting them.
  • A released transition or cleanse data record, in a project that opts in to them, is protected only by its position in the log: an edit to one behind a Seal passes validate, and migrate emits the edited bytes. Nor is a record the release shipped told apart from a branch’s backfill integrated behind it, which a database already past that point never runs.
  • Before 1.0 the oplog may gain operators and header keys. A build refuses a log with a newer retrofit-oplog-format or a key it does not know, so CI and every developer upgrade together.

[0.14.0]

Added

  • migrate warns apply-risk-lock-held-across-step when a table one op locks stays locked to the transaction’s commit while a later step runs whose length retrofit cannot bound, such as a RunRawScript, a data body or a scan: the table’s writers wait for that step too. It names the table, the op holding the lock and the step that lengthens it, and its hint says how to split the window. validate --fail-on-apply-risk names, for each op it counts, the tables the transaction already holds, and the two verbs agree on the holder, a fused cast run’s first cast included. The warning is counsel: it never fails the gate.
  • DetachPartition keeps the PRIMARY KEY, UNIQUE, EXCLUDE, index and foreign-key copies PostgreSQL leaves on a partition it detaches, each under the name PostgreSQL gave it, where earlier releases lost them (see Log format). op add takes a name it cannot derive as --kept-name <ancestor>=<name>, and a name that cannot be given refuses as detach-kept-name-taken or copy-clone-name-collision, in import and drift too. import binds the statements after a DETACH to the kept copy; drift plans a detach over kept copies, dropping those the target lacks and adding the target’s own after the DETACH; snapshot renders a re-attached copy under its name; and dropping the detached table frees the copies’ names. A copy with a key facet retrofit does not model (DEFERRABLE, INCLUDE) stays a drift skip rather than being planned without it.
  • A CopyTable clone, an ExtractTable child (one copying the source’s key, or a lookup child) and an UnjoinTable child with a key start from their source’s data records as they stood at the op, as every migrated database holds them. snapshot and migrate --from root fill such a product where they rendered it empty; drift and data capture compare a capture of it against those rows, where every capture read as new; and op seal no longer refuses it as reference-rows-uncaptured until it is captured. A source record that cannot be rewritten into the product (a populated COPY, an UPDATE, DELETE or MERGE, or a body that does not repair to the op) is refused by snapshot and migrate as stale-dml-reference under the reason it names, marked stale in data list, and listed by data check as the new routed-body-unroutable warning. snapshot and migrate warn routed-floor-key-lost when a later capture of an ExtractTable survivor drops a key the child’s rows still reference, which fails at apply.

Changed

  • validate --fail-on-apply-risk now also counts a column-type change (CastColumn) that rewrites the table, or scans it for an index or check, under a new code, apply-risk-window-table-rewrite, when the recovery retrofit can express (expand/contract for a rewrite, a drop and re-add for a scan) applies to it; a cast it cannot vouch for stays advisory. An existing user of the flag can see the gate fail on upgrade; migrate and validate without the flag are unchanged.
  • The refusal arg-binding is now arg-shape: arguments that do not fit the verb together, such as - beside other sources or an AddForeignKey target given both ways. Its messages and hints are unchanged. The reverse-policy-payload hint for a missing parameter now reads add {param}=, or revise reverse.
  • The reverse-guard note’s validation value not-valid-carrier is now not-valid-as-check, and its JSON group carrier_column is now checked_column: a NOT NULL a down returns as a NOT VALID CHECK (<column> IS NOT NULL). Its hint names that CHECK where it said carrier. No alias is kept.
  • The command retrofit data carry is now retrofit data reanchor. Its report reads reanchored:, would reanchor: and nothing to reanchor, its JSON records are type reanchor with outcome reanchored, and its stale-witness note is reanchor-witness-stale. The hints retrofit-data-carry, retrofit-data-carry-again, retrofit-data-carry-op-seal, carry-lead, make-readable-carry and then-finish-carry are now retrofit-data-reanchor, retrofit-data-reanchor-again, retrofit-data-reanchor-op-seal, reanchor-lead, make-readable-reanchor and then-finish-reanchor, and each names the new command. No alias is kept.
  • The refusal carry-failed, which op remove, op revise, op unseal and reconcile print when the sweep after their write fails, is now reanchor-failed; its hint finish-carry is now finish-reanchor. The write still stands.
  • The skip kind constraint-carrier-undecodable is now constraint-undecodable: retrofit could not decode its own stored form of a constraint. Its class (internal-defect), its drift-skip-internal-defect warning and the exit status are unchanged.
  • The refusal constraint-facet-uncarried is now constraint-facet-unmodeled, and rule-custom:constraint-facet-uncarried is now rule-custom:constraint-facet-unmodeled: a declared constraint parameter retrofit’s model has no place for (enforced=false, period=true, without_overlaps=true), which replay would drop. Its message now reads replay does not reproduce <param>=<value>; its hints are unchanged.
  • The surface export (retrofit surface --as json) keys the hint variant of a code whose refusal prints another code’s hint whole borrowed, with template {borrowed_hint} and slot borrowed_hint, in place of carried, {carried_hint} and carried_hint. The codes it appends are unchanged.
  • import --schema <name> leaves out a statement of any other schema, registered or not, and counts it as cross-schema. A schema other than public refused the whole run as schema-mismatch, and public was left out uncounted. The schema-mismatch code is gone. A run that mints nothing for its schema fails import-loss-bearing-skips when a skip lost something, exits 0 on a populated log, and otherwise refuses oplog-empty with a hint naming the schemas it left out. A pg_dump file holding the target schema still refuses on a populated log as state-source-on-populated-oplog.
  • import decides that a file’s primary-key drop and add, and the foreign keys dropped and re-added around them, are one ChangePrimaryKey from the ops it has minted, not from a scan predicting the names PostgreSQL holds later in the file. A rotation folds where nothing between the drop and the add rebuilds the old key’s index or re-checks a moved foreign key, and the folded ops replay to the same state. Any other stays a DropPrimaryKey and an AddPrimaryKey, with an import-change-primary-key-declined warning saying why. Re-importing a file can give a different op stream: more rotations fold on a table where a key name was taken, and one whose window rewrites the table stays apart. A key dropped under a foreign key still bound to it is refused by replay’s rule, as a split import refuses it. A rotation split across files stays a drop and an add, as before.

Fixed

  • drift no longer records a function whose signature names a type it set aside. A function with a parameter (of any mode), return, SETOF or TABLE column typed by a CREATE TYPE the log lacks is a function-type-unminted skip, of class gap, in place of a CreateFunction that failed at apply. A function of the same name the log holds is not dropped.
  • import of a pg_dump --schema-only file records a function whose string-literal LANGUAGE sql body, or %TYPE or %ROWTYPE declaration, reads a table, view or function the file creates later just after that statement, with a note naming the move. pg_dump writes such a function first and turns check_function_bodies off to restore it; migrate --from root kept the file’s order, and PostgreSQL refused the function under the default setting. A later LANGUAGE sql function calling such a function moves after it too, as pg_dump’s name order often writes a callee first. A function stays where it is when any other statement it would pass names it, runs code or holds data, or when its caller cannot move.
  • migrate no longer places a RunRawScript whose DO block holds COMMIT or ROLLBACK (the batched-backfill pattern) inside its transaction, where PostgreSQL failed the script with SQLSTATE 2D000 and rolled the file back. Under --envelope=own the window segments around the script and reports migrate-segmented-non-atomic; caller refuses migrate-tx-incompatible; independent emits the script with that advisory. --into-schema treats the copy the same way. migrate --down refuses migrate-non-reversible under every envelope when a window script’s reverse holds COMMIT or ROLLBACK, since the down never segments; op revert is not refused. A data body written as such a DO block now refuses migrate-dml-unscreened like a bare COMMIT;, and --allow-unscreened-dml admits it with the segment count reported as a lower bound. Only transaction control in the parsed PL/pgSQL counts, not one in a string, a comment or EXECUTE.
  • import keeps a LANGUAGE c function’s link symbol (AS '$libdir/pgcrypto', 'pg_gen_salt'), and snapshot and migrate render both parts as pg_dump prints them. Only the object file was kept, so PostgreSQL looked the function’s own name up as the symbol and refused it at apply, and drift never saw a changed symbol.
  • An authored LANGUAGE internal function, such as a wrapper over int4pl, is rendered as a function. snapshot and migrate took it for a trigger callback stub and left it out at exit 0 with no skip entry, and drift planned a ReplaceFunction for it unchanged. Only a trigger function with an empty LANGUAGE internal body is a stub.
  • snapshot writes a key equal to another of the table’s keys, such as a PRIMARY KEY and a UNIQUE over the same columns added by separate statements, as its own ALTER TABLE ... ADD after the CREATE TABLE. Inside the CREATE TABLE PostgreSQL merged the two, so the snapshot read back with one key, and drift against a project’s own snapshot planned a DropUnique.
  • op add refuses as payload-shape a comma-form --partition_bound whose datums are bracketed or quoted (from=["2026-01-01 00:00:00+00"], the oplog’s display spelling), naming the JSON record form. The punctuation was stored in the datum: PostgreSQL ignored it at apply, but drift read every bounded partition as re-bound and skipped it, and identify refused.
  • validate refuses as rule-exists-and-alive a view, policy, trigger WHEN, standard-SQL function body or partition key that reads an element a later op mints, as a hand-ordered log can hold. It passed, and snapshot at the reader’s cut left the reader out. A body PostgreSQL binds only when it runs, such as PL/pgSQL, is unaffected.
  • validate refuses as change-primary-key-dangling-fk a ChangePrimaryKey that leaves a foreign key covered only by the unique index its old key was promoted from (USING INDEX). PostgreSQL drops that index with the key, so migrate emitted a foreign-key re-add that it refused.
  • identify and drift read a column cast to its own type (lower((email)::text) on a text column) as the bare column, as PostgreSQL stores it. Once a CastColumn made such a cast a no-op, PostgreSQL dropped it from views and index keys: identify found no cut for a database retrofit had just built, and drift planned a ReplaceView, or a DropIndex and CreateIndex of an unchanged index. drift’s index compare also folds a comparison label and a whole interval value as identify does.
  • An accepted copy-table question in drift lands the whole copy in one run. The copy follows the drops and renames that free the names its clones take, with what each of those must follow, where it refused copy-clone-name-collision. The target’s own indexes, checks, unique keys and foreign keys on the copy are planned with it, and a clone the target’s copy lacks, or names otherwise, is dropped or renamed. Each landed only on a second run.
  • identify --expect last and migrate --from-dump refuse from-dump-working-copy for a test bed holding a run that reconcile --allow-sealed-rewrite demoted, after that run is sealed again. They answered from-dump-no-match with a drift hint, and following it recorded a drop of the hotfix.
  • op show --touching and op revise --dry-run resolve a foreign key a ChangePrimaryKey re-minted, by its name or id, as op add does. They listed only the original AddForeignKey, and refused no-match once the re-mint was renamed.
  • schema create refuses as name-conflict a name that a kind = "schema" [[exclude]] entry names, with a hint to remove the entry first. It wrote a [[schema]] entry that every later verb refused, and left an oplog behind.
  • op info names the lock five operators take: AddIdentity rewrites nothing (it reads the column once to seed the sequence), RenameIndex takes SHARE UPDATE EXCLUSIVE on the index, and RenameTrigger, RenameView and RenameMaterializedView take ACCESS EXCLUSIVE.
  • op add refuses as no-match a schema slot that names another registered schema (--schema a CreatePartition ... in=b), saying an op cannot yet place an element in another schema’s log. It bound an element of the target schema whose id began with those letters, at exit 0.
  • import and drift record a partition whose parent sits in another schema as a gap, so the run fails rather than losing the link. Read for the partition’s schema, CREATE TABLE b.c PARTITION OF a.p skips as partition-parent-cross-schema, its own partitions and dependents as cross-skipped-partition-cross-schema-child, and pg_dump’s ALTER TABLE ONLY a.p ATTACH PARTITION b.c records attach-partition-cross-schema-parent. Read for the parent’s schema, drift records that ATTACH as attach-partition-unresolved-child, as import does. Both minted a table with no columns or read the link as the decision cross-schema, and exited 0.
  • import lists a statement of another registered schema once in its skip manifest and summary. A CREATE POLICY, ALTER POLICY, CREATE VIEW or rename was listed and counted twice.
  • sequence-not-created-by-render’s hint names the column holding the sequence. Where another registered schema’s identity or serial column creates it, the hint says to apply that schema’s render first; it said to create the sequence, which made that render fail. A sequence another column of the same schema holds is OWNED BY that column, not the first column drawing from it.
  • op seal refuses as rule-exists-and-alive a seal over a reference into another registered schema that validate finds dangling, such as a drop under another schema’s view, where the schema being sealed holds the reference or dropped its target. It sealed over it. import --seal and drift --seal do not run this check yet.
  • import --force of a source holding nothing for its schema and no data (a SET-only prelude, or a file wholly of another schema) refuses oplog-empty and leaves the populated log as it was. It wrote the log empty first, then refused. A schema with data records gets oplog-empty rather than force-corpus-orphan. To empty a log on purpose, run schema remove and schema create.
  • drift refuses as drift-extension-moved, writing nothing, a source in which an extension sits in another schema than the log’s: one stated WITH SCHEMA elsewhere, or, in a pg_dump -n file that leaves the extension out, references respelled into its new schema. After ALTER EXTENSION ... SET SCHEMA it recorded each respelled default, cast, check, view and index as a change at exit 0, and every script it rendered failed at apply. The hint names moving the extension back with ALTER EXTENSION; retrofit cannot record a move yet.
  • Several refusals, hints and notes say what they mean, with codes, exits and JSON fields unchanged. op add says a token names a table where the slot takes a column; identify --expect names the nearest cut; the range-inverted hint of migrate --down names --from as the rollback target; CreatePartition on a plain table says it is not partitioned, not that it is not live; apply-risk-exclusive-lock’s hint no longer says the runner pauses writers; and a count of one reads singular, where several messages printed 1 tables or 1 bodies.
  • --help and the published command reference show a placeholder (<path>, <cut>, <opid>) for every flag value, where sixteen flags showed a name taken from an example. reconcile names fork-forced-order; op seal, data list and data check name which products start from their source’s rows (see Added) and which op seal still gates; data repin, op seal and migrate name data reanchor first for a revised or removed anchor; and validate --fail-on-apply-risk names a cast that scans the table for an index or check.

Log format

  • DetachPartition gains three optional params, appended after its anchor halves: kept-ids, one fresh id per key, index or outbound foreign key the detached table keeps from its parent and the parent’s ancestors; kept-anchors, the ancestor element each id copies, recorded by the producer; and kept-names, the name each copy takes, given at op add as --kept-name <ancestor>=<name>. A log written before has none, and a replay of it mints no copy, as it did. An earlier release refuses a log holding them; regenerate it with this one.
  • A plain LANGUAGE plpgsql function body written ahead of a table it reads now records that read where the table is created, or renamed into the name. pg_dump writes functions first and import keeps the file’s order. The body then behaves as it does when the table comes first. A DropTable or DropColumn of an element it reads refuses as rule-no-live-references, naming the function, and snapshot writes a renamed table’s or column’s new name in the body. Earlier releases accepted the drop and kept the old name. A body reading a column the table never gets is listed as an opaque reader, as when the table comes first. A log holding such a drop is refused wherever it is replayed to be checked or rendered. To read it, replace the function (ReplaceFunction) or drop it (DropFunction) ahead of the drop, then regenerate the log or re-seal. The log’s bytes are unchanged.
  • CopyTable, SplitTable and DetachPartition record, beside each list of IDs they mint (column-ids, adjacent-ids), the element each ID was minted over (column-anchors, adjacent-anchors). Replay binds an ID to its element by that record, never by position, so reordering two ops that cannot reorder no longer moves an ID onto the wrong element. A record that does not match its source exactly refuses as derived-anchor-unmatched, naming the op, the list and each unmatched element; the verbs that replay it surface the refusal as structural-op-invalid (validate), render-blocked-on-partial-forest (rendering, element, op revert, schema ref-tables and the data verbs that read the log), parse-error (op seal) or replay-failed (data witness), each keeping the derived-anchor-unmatched prefix. A log written before this release holds no anchors, so it refuses the same way wherever it is replayed past one of these ops: re-import the database to regenerate the log. The retired detach-id-list-arity code is this one.
  • CopyTable and SplitTable no longer require column-ids and adjacent-ids at parse: a line without one now parses, and replay refuses it as derived-anchor-unmatched, as DetachPartition does. No log needs regenerating; a line that has both lists is unchanged. The refusal names anchors in the order the line records them, each with its table and name.

Known issues

  • A project with several registered schemas is not supported yet. How a reference from one schema into another, a drop that another schema depends on, and the verbs that read more than one schema’s log should behave is still being designed, and each can go wrong today.
  • Ending a column that a later RunRawScript body names is refused only when the body is bound. For a body stored opaque, op remove or op revise of the column, or a merge whose other side removed it, writes the log, and validate accepts it. migrate then emits the script against a table without the column, so the migration fails whole at apply. validate --history fails a pull request that adds such a body, unless the name sits only in a conditional branch or the body does not parse; it only advises on a body already on the target that the pull request leaves naming a removed column. Apply the migration to a scratch database in CI (retrofit migrate --from root --to last, or from the latest Seal).
  • drift asks the reversal question only when the database holds something logged after the drop. A database that has not yet run the log’s newest drop reads as intent, and the plan re-adds the item with no question. Review a drift plan’s re-adds against the log before accepting them.
  • A released transition or cleanse data record, in a project that opts in to them, is protected only by its position in the log: an edit to one behind a Seal passes validate, and migrate emits the edited bytes. Nor is a record the release shipped told apart from a branch’s backfill integrated behind it, which a database already past that point never runs.
  • Before 1.0 the oplog may gain operators and header keys. A build refuses a log with a newer retrofit-oplog-format or a key it does not know, so CI and every developer upgrade together.

[0.13.0]

Added

  • op revert records a rollback in the log. After migrate --down has rolled a database back, it appends the reversal that down computes: the ghost’s ops after the log’s last op, the reference leg’s records, a declared reverse as one raw script, and the data-loss notes. A database migrated from the log then gets the same result, and a later down knows what was undone; migrate still never writes the log. Each appended op records reason="undoes <opid>", naming the window op it undoes, unless reason=<text> is given. Recording twice appends nothing, and recording after more ops land appends only the new reversal. The fork guard and the validate gate run before the write, and --dry-run previews what would be appended. A revert that is not at the log’s tail refuses as revert-not-at-tail, and one whose leading leg is off a Seal as revert-leading-leg-unsealed, each with a hint naming the way out. migrate --down reads what op revert recorded and advances its range past it, so a window is not undone twice.
  • A reversal that re-creates a dropped table, column or other element (the down’s ghost and op revert) keeps the element’s ElementID when the re-create passes the same-element predicate, so the element’s history continues where a fresh id orphaned it. The mint is marked same_element=reversal (see Log format), and --dry-run prints the mark. validate refuses an unmarked mint of an id the log already knows, and op revise refuses to edit a marked mint. op squash nets a drop and the mints that return its element as one group, and op seal reports a data loss a squash of its range would avoid.
  • A down over a DropTable or DropColumn says that the element it re-creates comes back without its rows, as a re-created-empty item of reverse-irreversible, beside the forward’s data-loss-on-apply line. A paired table’s stand-in names each column that comes back empty. A forward window that re-creates a dropped element says it comes back empty too.
  • schema database-at <cut> pins the cut the local database is at, in the self-ignored .retrofit/database-at; --clear removes the schema’s line, and bare it prints the pin in force and where it came from. op add, import and drift also take the pin as --database-at or RETROFIT_DATABASE_AT, and judge apply-risk from it (see Changed). A pin file that exists but cannot be read refuses as io-error rather than being rewritten.
  • op add RenameTable, op add RenameColumn, import, drift and op revert take --allow-readers-moved <old-name>, repeatable. It attests that the readers of the table or column with that qualified old name are handled by the time the rename applies, and withholds apply-risk-rename-in-use for that rename; every other rename still warns, and the warning’s hint names the flag. A value that answers no rename, including one a refused run never reached, is reported as having no effect.
  • op add CastColumn raises apply-risk-table-rewrite when the cast makes PostgreSQL rewrite the table, naming the dependents rebuilt with it, a partition’s own and a name-less index included.
  • op add and op revise refuse as body-resolve a name in a function, trigger, view or raw script body that neither the log nor the project’s declarations hold where the body binds it: a top-level script statement, an always-reached DO statement, a BEGIN ATOMIC body, a %TYPE or %ROWTYPE declaration, or a string-literal SQL body with no polymorphic argument. Where PostgreSQL binds the name only when the body runs (a PL/pgSQL statement, a polymorphic SQL body, a conditional branch), they warn body-name-absent instead. A name an operator took off the schema (MoveColumn, ExtractTable, a partition operator, DropTable and the like) is named in the refusal with that operator, and a held body citing it raises body-name-removed. import judges every body at the end of its source, and validate --history holds a range to the same check: a certain site fails, a conditional one is advisory. Objects the log cannot see, such as an extension’s functions, are declared in [[external]] entries in retrofit.toml.
  • op add warns schema-partition-tree-indexed for a CreateIndex on a partition root or middle, and for a PartitionTable of a table that already holds an index: identify skips an indexed partition tree, so it cannot place a database over one. Where that tree is the only difference left, the from-dump-no-match, from-dump-behind and --expect outcomes say so and that drift finds nothing else differing, and outside --expect say to name the cut with migrate --from. Outcomes, codes and exit statuses are unchanged.

Changed

  • op add, import and drift judge apply-risk over the window from the database’s starting state, as validate --fail-on-apply-risk does, rather than one op in isolation. The starting state is the schema database-at pin, else the latest Seal at or before the op, else root. A table created inside that window holds no rows when the database applies the op, so the empty-table codes (apply-risk-add-unique, apply-risk-add-not-null and the like) are withheld on it, and apply-risk-rename-in-use is withheld on an old name the starting state does not hold. A line saying a code was withheld on an assumed starting state prints once per run. A schema in a fork keeps the per-op verdict.
  • A repeated --schema on any verb refuses as flag-invalid-value, where the last value won: op seal --schema a --schema b sealed only b. --from, --to and --at still take the last value.
  • validate and every verb that folds the registered schemas (import, drift, snapshot, identify, element list, element history) refuse two [[schema]] entries rooted at one SQL schema as name-conflict, naming both. Identical imports had bound a cross-schema view on some runs and refused it on others.
  • identify refuses a log that breaks a replay rule with the rule’s own code and hint, as validate, migrate and snapshot do: a rule-no-live-references log gets the replace-or-drop-function recourse where it was refused render-blocked-on-partial-forest with no hint. A replay-classed entry this build cannot check refuses as unverifiable.
  • op add CreatePartition payment public p1 reads public as the schema, as CreateTable does, where it bound name=public and partition_by=p1. A partition named like the schema spells name=.
  • op revise of an op an unsealed Seal once covered says the op was sealed under that Seal, since unsealed, so a database placed there is a working copy. It said the op was never sealed and a database at the Seal was unaffected.
  • A RunRawScript body kept opaque by a cast to a managed type gets a hint for the cast’s shape: drop a 'literal'::app.kind cast where the literal meets a column of that type, or write one statement per arm of a CASE of string literals.
  • schema create of a name already taken refuses with a hint naming retrofit init --config <dir>/retrofit.toml for a second project, and init below a project it leaves alone notes that project on stderr.
  • validate no longer reports schema-stray-oplog for an oplog inside a nested project, a directory with its own retrofit.toml.
  • Wording: a scalar param spelled as a flag (--referenced_table t) refuses unknown-key with a hint spelling it referenced_table=<value>; the missing-required hint gives the param’s help; the SplitTable and MergeTable examples spell --condition=; the RunRawScript summary says when a body is bound; data scaffold --purpose says both values are experimental; and the run-once RunRawScript notes of snapshot and identify read in the singular for one op.

Fixed

  • op unseal no longer refuses a log for a verdict the log already held before the unseal, such as a sealed drop a newer replay rule refuses. It reopens the tail so the op can be repaired and the log re-sealed; the other verbs still refuse the log until then.
  • op unseal, op remove and op revise under --dry-run no longer say the write stands when the post-write sweep refuses: the hint says nothing was written.
  • A function whose argument, OUT, table or return type is a managed type or a relation’s row type, stored unqualified (f(mood)), is emitted qualified in CREATE, DROP, ALTER ... RENAME, COMMENT ON and the restored grants. snapshot and migrate rendered it bare, so the CREATE failed or bound a same-named type on the search_path. Built-in and unmanaged types stay as written, and identify resolves a dump’s qualified spelling against such a function.
  • drift no longer plans a DropFunction and a CreateFunction for an unchanged function whose signature pg_dump prints differently from the log: a managed type schema-qualified (SETOF, arrays, row types and OUT columns included), a long-form type (timestamp without time zone, int), or a typmod PostgreSQL discards (varchar(10), numeric(10,2)). New CreateFunction records store the folded spelling; sealed records keep their bytes and still pair.
  • op add refuses as rule-unique-name-in a CreateFunction PostgreSQL would read as an existing function: one that differs only by a discarded typmod (f(varchar(20)) beside f(varchar(10))) or by qualifying its own schema’s managed type (f(mood) beside f(public.mood)). Both were stored, and migrate then issued a second CREATE FUNCTION that failed.
  • drift and identify no longer plan a ReplaceFunction, or answer from-dump-no-match, for a LANGUAGE sql body the log rendered whose string literal sits under a CASE, an array subscript or an array_to_string separator. The CASE and subscript fix reaches CHECK, index predicate, view and policy bodies too.
  • identify places a database whose skipped run holds a function body that reads nothing (BEGIN RETURN 2; END) ahead of a later ChangePrimaryKey, CastColumn, AddEnumValue or RenameTable, and a body whose only mention of a renamed table is an EXECUTE string. It answered from-dump-no-match and routed to drift.
  • identify on a dump that matches no cut no longer runs the full placement search over a log holding RenameIndex, where one such op took nearly a minute.
  • identify no longer refuses the project’s own snapshot when [[exclude]] names a schema an extension is placed in, and no longer scans a large snapshot in quadratic time per candidate cut.
  • A log whose first op names a prev= the file does not hold no longer panics migrate, snapshot or identify: they replay it alone and judge its replay rules. validate and op seal still refuse it as structural-prev-unknown. With several registered schemas, snapshot, element list and migrate no longer refuse such a log as unverifiable with a hint naming a build that does not exist.
  • import --force that re-mints a schema’s ids warns for each other registered schema whose bodies still name the old ids, and validate then fails rule-exists-and-alive. snapshot refuses such a body as invalid-cut, naming its holder, where it wrote an <elem:...> placeholder into the SQL.
  • DropColumn of a column ExtractTable derived (a rejoin column, or a survivor’s foreign key column) refuses under its live dependents as on any table, where it was admitted and left a snapshot with a PRIMARY KEY on a missing column; drop the foreign key and primary key first. AttachPartition over such keys no longer fails snapshot with invalid-cut.
  • op revise refuses a changed RunRawScript body that op add would refuse (a bare statement, two statements, an empty body), in every form. It wrote them at exit 0, for migrate to emit outside the one-DO contract.
  • schema remove removes the schema’s block wherever it sits in retrofit.toml, in every spelling the loader accepts. It deleted the oplog at exit 0 but left the block when another [[schema]] followed. A removal that cannot take every registration refuses before the oplog is deleted.
  • schema remove and schema move honor an absolute oplog_root: remove deleted a path that did not exist and left the real oplog, and move refused as move-failed. schema move and schema engine rewrite the config line the loader reads, where a quoted or repeated key reported success and kept the old value.
  • The apply-risk-recorded-unsealed note advises retrofit validate --fail-on-apply-risk --from <cut>; the command it printed passed --schema, which validate does not take, and exited 2.
  • op info no longer says DropGeneratedExpression ends an element: the column and its values stay.

Log format

  • The 15 operators that create an element (CreateTable, AddColumn, AddPrimaryKey, AddForeignKey, AddUnique, AddExclude, AddCheck, CreateIndex, CreateView, CreateMaterializedView, CreatePolicy, CreateType, CreateFunction, CreateTrigger, CreateExtension) take one optional derived key, same_element=reversal, on a mint that returns an element a drop ended under its own ElementID. A line without the key is unchanged; no log needs regenerating. A build that predates the key refuses a line that has it, by name.
  • A value outside the declared set of a derived enum (same_element, and MoveColumn’s arm) now reads as an unknown key, so validate reports the line unverifiable and the verbs that render or write refuse it by key, where it was a kind-mismatch parse error. An authored enum still refuses.

Known issues

  • Ending a column that a later RunRawScript body names is refused only when the body is bound. For a body stored opaque, op remove or op revise of the column, or a merge whose other side removed it, writes the log, and validate accepts it. migrate then emits the script against a table without the column, so the migration fails whole at apply. validate --history fails a pull request that adds such a body, unless the name sits only in a conditional branch or the body does not parse; it only advises on a body already on the target that the pull request leaves naming a removed column. Apply the migration to a scratch database in CI (retrofit migrate --from root --to last, or from the latest Seal).
  • identify on a dump that matches no cut can take minutes on a log holding a MergeTable, JoinTable, UnjoinTable or partition operator, where a dump that matches answers in seconds.
  • drift asks the reversal question only when the database holds something logged after the drop. A database that has not yet run the log’s newest drop reads as intent, and the plan re-adds the item with no question. Review a drift plan’s re-adds against the log before accepting them.
  • In a project with several registered schemas, drift plans a DropForeignKey for an unchanged cross-schema foreign key that import recorded, skips the file’s key as add-foreign-key-cross-schema-unsupported, and exits 0. Review each DropForeignKey in such a plan with --dry-run before recording it.
  • op add, op seal and migrate read only their own schema’s log, so a drop in one registered schema is accepted, sealed and emitted while another registered schema depends on its target. validate fails such a drop when the dependent is a view, policy, function or other body in another schema, but accepts it when the dependent is a foreign key or a column typed by the dropped type. Apply the migration to a scratch database holding every schema before it lands.
  • migrate refuses as emit-render-failed a body that names an element of another registered schema, though snapshot renders it. The detail reads as if the element were dropped; it is live in the other schema’s log. No cut renders it. Use snapshot for a fresh database.
  • A trigger whose EXECUTE FUNCTION names a function in another registered schema imports with a function-synthesized-stub finding and renders the call without its schema, which fails at apply or binds a same-named function on the search path. Treat each such finding as a broken trigger.
  • In a project with several registered schemas, validate --fail-on-apply-risk --from <opid> resolves the opid in every schema, so the schemas that do not hold it refuse as opid-not-found. Name the cut with an anchor, such as a Seal, instead.
  • drift records a CreateFunction whose signature names a type it set aside, so the function fails at apply under --allow-capability-gaps. Record the type before running drift.
  • migrate --from root of a log imported from a pg_dump --schema-only file can create a function ahead of the table its string-literal LANGUAGE sql body or %TYPE declaration reads, which PostgreSQL refuses under the default check_function_bodies. snapshot of the same log applies. Use snapshot for a fresh database.
  • A released transition or cleanse data record, in a project that opts in to them, is protected only by its position in the log: an edit to one behind a Seal passes validate, and migrate emits the edited bytes. Nor is a record the release shipped told apart from a branch’s backfill integrated behind it, which a database already past that point never runs.
  • Before 1.0 the oplog may gain operators and header keys. A build refuses a log with a newer retrofit-oplog-format or a key it does not know, so CI and every developer upgrade together.

[0.12.0]

Added

  • drift over a database that skipped a logged drop, as an out-of-order deploy does, asks a reversal question where it re-added the item with no question. The plan line names the op it reverses and that op’s Seal. The answers are keep, which an unanswered question takes and which leaves the log as it is, and accept, which records the restore. A table, a column, a key, a constraint, a trigger and a rename back all take the question.
  • A RunRawScript DO body that names only elements its binder can follow is now recorded bound to them. A later rename renders through the body in migrate, op show, the down and reconcile’s diff, and op remove of an element the body names refuses as rule-exists-and-alive. A body the binder cannot read stays opaque.
  • The script migrate emits reads the privileges held on a table, view or column before a re-create drops it and restores them on the replacement, so a grant made outside the log stays. Columns moved by MoveColumn, ExtractTable, JoinTable or InlineTable are named as not restored.
  • drift and import without --seal print one apply-risk-recorded-unsealed note per run, naming the tables, columns and renames they recorded: apply-risk reads what the unsealed tail creates as new and empty, and a table recorded from a populated database is neither.
  • The docs gain a refusal codes page, generated from the binary’s code registry, with each code’s kind, its exit statuses and the commands that can print it.

Changed

  • migrate renders a ReplaceView the old way, one CREATE OR REPLACE VIEW, only when the view’s column names at the op’s input cut are a name prefix of the new list and every prefix column’s type is proven unchanged. Any other definition is a rebuild: it drops the view’s tracked readers and the view, creates the view with the new definition, restores its grants, and re-creates its triggers and readers, with a -- WARNING: line naming each reader that may fail and a -- post-deploy: REFRESH for each materialized view re-created empty. The rebuild takes ACCESS EXCLUSIVE on the view and each reader, and op add raises apply-risk-view-rebuild for it. A view whose prefix column reads another view’s or a materialized view’s output column now rebuilds even for a tail-column add, since the type cannot be proven. In the lendlog fixture the two ops b56bc83a6caf and 1ed10791196f render as rebuilds where they rendered one CREATE OR REPLACE VIEW before; drift and the down emit one ReplaceView, no longer a DropView and a CreateView, so the view keeps its ElementID.

  • validate --history gains the opaque-body check: it fails when the pull request adds a body retrofit stores opaque (a RunRawScript with no bound_params=, a string-literal or unresolved LANGUAGE sql function body, or a view, materialized view or expression slot whose text names no element), since a later rename or drop leaves what it names stale and only a database apply checks it. Each finding names the line, its operator, slot and cause, and the rewrite. A pipeline that keeps applying the range to a database before it lands adds --allow-database-apply: the check then lists its findings as advisory and does not fail. op add, import and op revise refuse nothing new; the cause finding they print for such a body names the check and the consent, and import now prints it for a string-literal function and a view whose query falls back.

  • reconcile no longer applies a forced order that the run did not ask for. It refuses as fork-forced-order, exit 1, naming the flag that asks for the order and what the order overrides, and the JSON error gains forced_orders. A bare reconcile reports each forced order.

  • migrate over a RenameColumn, RenameTable, RenameView or RenameMaterializedView that a plain LANGUAGE plpgsql function reads follows the ALTER with a CREATE OR REPLACE FUNCTION under the new name, since PostgreSQL never rewrites such a body. Where the same migration also renames a type in the function’s signature, it refuses and names the drop-before, create-after sequence.

  • drift no longer records a column whose type it set aside. A column typed by a CREATE TYPE the log lacks is a column-type-unminted skip, of class gap, in place of an AddColumn or CastColumn that failed at apply.

  • drift raises a merge question only between versioned or snapshot names, so a table prefix such as wp_ no longer makes every table one family. It asks an index rename only between equal definitions, and pairs rename questions the same way on every import of one file.

  • import and data capture print preamble: N statements before their summary or verdict, so the summary is the last line of stdout. The text, the JSON placement and the counts are unchanged.

  • validate --fail-on-apply-risk and migrate’s counsel count a table created in the window as populated once a MergeTable, a MoveColumn down, a splice-band data body or a staged load gives it rows. A range that passed may now exit non-zero.

  • spec_ref in op info and the SPEC line of op add <Op> --help print the Operators page URL, not a repository path. Help text names docs pages by title and says operator, not SMO.

Fixed

  • A rename now follows, and a drop now counts, the references in a function body that earlier builds left stale or did not see: quoted, three-part and qualifier names in LANGUAGE plpgsql, %type declarations, reads through a CTE or an aliased subquery, THEN results, IN lists, VALUES rows, row-SET sources and subscripts. RETURN QUERY and RETURN NEXT bodies and assignments to a parameter now bind. A read the binder cannot resolve makes the body opaque, where it had bound a shadowing variable or nothing.
  • A BEGIN ATOMIC body’s INSERT, UPDATE, DELETE and MERGE statements now bind, with each MERGE clause scoped as PostgreSQL scopes it. drift no longer proposes a ReplaceFunction for an unchanged one on every run.
  • A trigger’s UPDATE OF list and a bare WHEN or parameter DEFAULT now follow renames. The UPDATE OF columns are quoted, and drift orders a column drop after the triggers that list it.
  • CreateView and ReplaceView expand a * inside a CTE, a FROM subquery and every arm of a set operation, not only the top-level leftmost one. A rename that makes a derived relation’s read ambiguous now skips the view in snapshot, named, rather than writing one PostgreSQL rejects.
  • migrate --down now re-creates a dropped LANGUAGE plpgsql function and a dropped trigger under the target’s names, orders a re-created function at its own position, drops a window-created reader ahead of the column it reads, and renders a bound script’s bare relations under the declared search path or refuses.
  • A reference floor crossing its declaration’s removal now applies as a counted row delta when no key identifies it, writes a body=none floor as its empty state’s delta, deletes a dropped NULL key NULL-safely, and folds a partition child’s columns by its root’s ids. A keyless floor had duplicated its rows at every apply, at exit 0.
  • CopyTable and SplitTable clones of keys and indexes now render named. PostgreSQL had picked a numbered name when the derived one was held, and a later DropIndex or RenameIndex then hit the wrong object.
  • import keeps %TYPE on a function’s argument or return type and keeps a managed type named like a built-in’s internal name (public.bool). A function’s SET and RESET clauses fold as PostgreSQL’s proconfig does, and each element of its search_path list renders as its own literal. drift resolves SET search_path FROM CURRENT as import does.
  • migrate parenthesizes a column default the DEFAULT slot cannot read bare, such as now() AT TIME ZONE 'utc'.
  • import moves a drop past a later statement of the same file that clears the LANGUAGE plpgsql reader blocking it, as PostgreSQL applies it.
  • ops upgrade-sql-bodies keeps a function’s volatility, parallel safety, SECURITY DEFINER, SET, COST, STRICT, ROWS and LEAKPROOF, which it had reset to their defaults while reporting success. A body given to op revise now binds at the line’s cut, as one given to op add does.
  • drift orders a merge or join question after a surviving foreign key’s drop, names the operator on a structural question’s text line where it printed <?>, and reports an earlier op that replay refuses under that op’s own code.
  • validate --history no longer fails fate-kept on a revise of a revision the base holds or on a removal whose replacer stays on the chain. Its collapse-parent finding names the revert and a recovery that completes after a hand edit rewires a successor. validate reports its other findings beside a seal digest mismatch.
  • reconcile names a remedy for a displaced identity generation, and describes a partition bound as the DDL spells it and an owned sequence by the name its column renders.
  • identify on a dump matching no cut no longer searches every placement on a log holding a CopyTable, SplitTable or ExtractTable, or on a dump of an indexed partition tree. One such dump had run past 30 minutes.
  • op show notes the sealed ops its default scope hides, so a touching query over sealed history no longer reads as no history. data list marks an orphaned record as data check refuses it.
  • A refusal or hint no longer names a verb that refuses in turn, a flag the user did not pass, or a generic cause. op add and op revise list the accepted keys for an unknown key=value, import of only CREATE SCHEMA names schema create, a stream source prints as a bare - in recipes, a malformed foreign key target refuses under a registered code, and -v folds identical repair sites into one line with a count.
  • import --help says sources are never re-sorted and a glob expands lexically (V10 before V2), with --from-file for any other order.
  • drift on a 4,651-op dry run takes 31 s, where it took 3,619 s. validate on a log whose tables churn columns grows about 8x at 4x the operations, where it grew 16.2x. A three-file reconcile over a 2,311-op log peaks at 262 MB, where it peaked at 1.82 GB.

Log format

  • ReplaceView accepts any new column list: replay no longer refuses a narrowing or a reorder, and the view-replace-superset code leaves the catalog, the registry and the rule census. A sealed ReplaceView writes the same bytes and replays as before; the render now rebuilds the view where CREATE OR REPLACE VIEW cannot take the change. An older build refuses such a line as view-replace-superset; upgrade it to read the log.
  • A RunRawScript whose bound body names a quoted relation or column (app."Account", "isActive") is now bound, where earlier builds kept it opaque. Its bound sentinel writes each quote of the name doubled, and the line gains a new producer-only bound_quoted=[body] beside bound_params=, listing the body params that hold such a sentinel. An older build refuses the line by that unknown key, and its validate refuses it as body-bound-edited; upgrade it to read the log. A rename of such a name to one that needs quoting now renders it quoted instead of refusing body-render-unquotable. A line without a quoted name is unchanged, so a log needs no regeneration.
  • A bound body whose sentinel was hand-edited malformed now loads whether or not bound_quoted= lists it, and replay refuses it as body-bound-edited; earlier in this release it refused at load as body-bound-unmarked.
  • A plain LANGUAGE plpgsql function body now records the tables, views and columns it reads, as an ansi-sql body does, so snapshot writes it with a renamed element’s new name. A DropTable, DropView or DropColumn of an element such a body still reads refuses as rule-no-live-references, naming the function, at op add, where earlier releases accepted it. A log holding one is refused by that code wherever it is replayed to be checked or rendered (validate, snapshot, migrate, data check, data list and data witness among them); op show and schema engine still read it. data check still reads the sealed part of a log whose only such drop sits after its last Seal. The log’s bytes are unchanged. To read such a log, replace the function with a body that no longer reads the element (ReplaceFunction) or drop it (DropFunction) ahead of the drop, then regenerate the log or re-seal.
  • A trigger’s WHEN and a function parameter’s DEFAULT now record the managed functions they call, as PostgreSQL does. A DropFunction of a function either still calls refuses as rule-no-live-references, naming the trigger or the function, at op add, where earlier releases accepted it and the migration failed at apply. A log holding one is refused by that code wherever it is replayed to be checked or rendered. The log’s bytes are unchanged. To read such a log, drop the trigger (DropTrigger), or replace the calling function without the call (ReplaceFunction) or drop it (DropFunction), ahead of the drop, then regenerate the log or re-seal. A call to one of two overloads of a name, or an unqualified call whose name a built-in shares, still records nothing.
  • RunRawScript gains the optional, producer-written bound_params=[body, reverse_body] param, written after the reversal params and before reason. A build without it refuses a line that holds it as an unknown key; no log written today holds it, so nothing needs regenerating.
  • A body param whose text holds the bound-script form "<bound:…>" but that bound_params= does not list refuses at load as body-bound-unmarked, whatever the operator. A log written by retrofit never holds one. Restore the marker, or replace each "<bound:…>" with the name it stands for.
  • Replay refuses more drops PostgreSQL refuses, as rule-no-live-references, at op add and wherever a log holding one is replayed: a DropFunction while a partition key calls the function; a DropColumn or MoveColumn of a column a trigger’s UPDATE OF list names; a DropTable while a column is typed by its row type or a body names that type; and a drop of what a BEGIN ATOMIC body’s DML statement, a subscript or a %type declaration reads. The log’s bytes are unchanged. To read such a log, clear the reference ahead of the drop, then regenerate the log or re-seal.
  • A body that names a dropped element refuses as rule-exists-and-alive, for a view, materialized view, function, policy, trigger WHEN, column default, generation expression or partition key expression. Only a merge that orders the drop first reaches this, and reconcile --clean now refuses that order as fork-no-clean.
  • An index, or a PRIMARY KEY, UNIQUE or EXCLUDE constraint, named as another relation or index of its schema refuses as rule-unique-name-in. It failed at apply with relation ... already exists.
  • validate and op seal refuse a prev= cycle as structural-prev-cycle and a dangling prev= as structural-prev-unknown, exit 1, wherever in the file they stand. A merge resolution or a rebase can leave either.
  • A CopyTable or SplitTable that clones a key or index whose name cannot be derived refuses as copy-clone-name-collision, naming the workaround of dropping it first and re-adding it after. The EXCLUDE case had refused as rule-exists-and-alive.

Known issues

  • Ending a column that a later RunRawScript body names is refused only when the body is bound. For a body stored opaque, op remove or op revise of the column, or a merge whose other side removed it, writes the log, and validate accepts it. migrate then emits the script against a table without the column, so the migration fails whole at apply. validate --history fails a pull request that adds such a body. A pipeline that passes --allow-database-apply should apply the migration to a scratch database (retrofit migrate --from root --to last, or from the latest Seal).
  • identify on a dump that matches no cut can take minutes on a log holding a MergeTable, JoinTable, UnjoinTable or partition operator, where a dump that matches answers in seconds.
  • drift asks the reversal question only when the database holds something logged after the drop. A database that has not yet run the log’s newest drop reads as intent, and the plan re-adds the item with no question. Review a drift plan’s re-adds against the log before accepting them.
  • In a project with several registered schemas, drift plans a DropForeignKey for an unchanged cross-schema foreign key that import recorded, skips the file’s key as add-foreign-key-cross-schema-unsupported, and exits 0. Review each DropForeignKey in such a plan with --dry-run before recording it.
  • op add, validate, op seal and migrate read only their own schema’s log, so a drop in one registered schema is accepted, sealed and emitted while another registered schema depends on its target. Apply the migration to a scratch database holding every schema before it lands.
  • migrate refuses as emit-render-failed a body that names an element of another registered schema, though snapshot renders it. The detail reads as if the element were dropped; it is live in the other schema’s log. No cut renders it. Use snapshot for a fresh database.
  • A trigger whose EXECUTE FUNCTION names a function in another registered schema imports with a function-synthesized-stub finding and renders the call without its schema, which fails at apply or binds a same-named function on the search path. Treat each such finding as a broken trigger.
  • drift records a CreateFunction whose signature names a type it set aside, so the function fails at apply under --allow-capability-gaps. Record the type before running drift.
  • A released transition or cleanse data record, in a project that opts in to them, is protected only by its position in the log: an edit to one behind a Seal passes validate, and migrate emits the edited bytes. Nor is a record the release shipped told apart from a branch’s backfill integrated behind it, which a database already past that point never runs.
  • Before 1.0 the oplog may gain operators and header keys. A build refuses a log with a newer retrofit-oplog-format or a key it does not know, so CI and every developer upgrade together.

[0.11.0]

Added

  • op info --as json, for one operator or the --dump, gives each operator a verbs object: for op_add, import, drift, snapshot and migrate, whether that verb ships the operator. Each value is available, unavailable, or syntax-unavailable for an operator this build cannot spell.

Changed

  • A fresh text-mode snapshot loads only the rows a record states as its table’s whole content (a replace section, or a reference restate) between the tables and the constraints, as before, and fires no trigger on them. Every other record (an append or delta section, a seed restate) now runs in a second -- retrofit:snapshot-data v1 band after the whole schema, before COMMIT, with constraints, indexes and triggers live, as migrate --from root runs it, so its rows fire the enabled triggers and meet the foreign keys in authored order. Each band re-seats its own tables’ sequences. A note, snapshot-state-under-trigger, names each state table and trigger migrate would fire; a warning, data-statement-order-fk, names a foreign key the order meets (a state ahead of its parent’s statements, or a child’s statements ahead of its parent’s) with the recourse. --as=json, --data-only and the banner opid are unchanged.
  • migrate warns data-statement-order-fk for the same two orders in its trailing seed band, where every foreign key is live: a staged replace table whose key reaches a parent only the band’s later records supply, and a child’s records ahead of its parent’s. The warning names the table, its parent, the key and the recourse; a key declared INITIALLY DEFERRED raises neither, except under --envelope=independent. The script is unchanged.
  • The drift plan: header counts changes, between removals and questions: drift plan: N additions, M removals, C changes, Q questions. A change is any planned op the header does not count as an addition or a removal, such as a rename, a cast or a NOT NULL flip, so a plan made only of those no longer reads 0 additions, 0 removals. The JSON plan-header record gains a changes field.
  • transition and cleanse data records are experimental. Every verb refuses them unless retrofit.toml opts in ([experimental] purposes = ["transition", "cleanse"]), and warns on every use once it does: a released one is protected only by its position in the log. Write a backfill as a DO block to keep it under a Seal’s digest. Without the key, import --purpose and data scaffold --purpose refuse to mint either, and validate, data check, data list, data witness, data repin, migrate and snapshot refuse a corpus holding one, as experimental-not-enabled, naming each file and its purpose. With it, each verb reads and mints as before and warns experimental-purpose once per such file. The table is project-wide; an entry naming anything else refuses as config-invalid. A retrofit that predates the table refuses the whole file (unsupported table [experimental]).
  • migrate --down reports a row loss its render chose (data-loss-on-apply-render-chosen) as a warning, as data-loss-on-apply is, not as a note after the other warnings.
  • migrate --allow-unreached-dml, removed in 0.10.0, refuses flag-retired and says to drop the flag, instead of failing as an unknown flag.
  • op show --touching resolves a function, trigger, policy, extension or materialized view, by name, signature or element id, as element history does.
  • Refusals that pointed at generic advice now name their own recovery: a bare token in a payload record names the record’s fields (and ExtractTable’s --into); a body calling an overloaded function says no spelling binds it; a data file refused only for its record marker line leads with the marker grammar; reconcile --clean over a demotion whose other order replays clean names that order’s flags before --accept-dirty; schema engine <name> <version> names --schema.
  • A skipped sequence names itself on drift’s skip line and skipped event, and identify’s residual skip lines no longer end in a space.
  • after-traffic-outcome on a RunRawScript, whose declared reverse is opaque, says the down may refuse or may drop the rows written since, rather than that it does.
  • snapshot --help-all and docs/snapshot.md say that triggers and policies follow the data band, so a snapshot build runs no trigger on its seed rows while migrate --from root --label does.

Fixed

  • The drift plan: header counts the creation of a policy, function, trigger or extension as an addition and its drop as a removal, as it does for a table or a view; it had counted them as changes. --show=removals now hides those creations and --show=additions those drops.
  • migrate no longer emits a script PostgreSQL refuses with 55006 (pending trigger events) when an op writes a table holding a DEFERRABLE INITIALLY DEFERRED constraint trigger and the same transaction then alters that table, as AddNotNull … fill= does. Under --envelope own it refuses as migrate-deferred-trigger-pending, naming the trigger, the table and the ops; under caller the script and a deferred-trigger-owed warning name the SET CONSTRAINTS <schema>.<trigger> IMMEDIATE; the enclosing transaction can run first. A --down is treated the same way.
  • migrate now warns, as deferred-trigger-pending, where a data record writes a table holding a DEFERRABLE INITIALLY DEFERRED constraint trigger and a later op of the same transaction alters that table, which PostgreSQL refuses with 55006. The script has a -- retrofit: line above that op, and the warning names the SET CONSTRAINTS <schema>.<trigger> IMMEDIATE; the record can end with.
  • op seal, import --seal and drift --seal now refuse, as data-file-anchor-unresolved (exit 2), while the schema’s data corpus holds a record whose anchor-op= no longer names an op, as data check does, with the same detail and data repin’s recourse. Before, the seal was written, migrate refused the release it cut, and repairing the record needed op unseal first, since data repin refuses a sealed target. Regenerating a log under a committed data file, or op revise or op remove of the op a record anchors on, leaves such a record.
  • data check now refuses an oplog whose retrofit-oplog-format line is older or newer than this build reads, as oplog-format-too-old or oplog-format-too-new, exit 1. Before, it reported no findings and exited 0 on a log every other verb refuses, so a check run alone passed it. On a sealed log, validate now refuses the same line by the same code and the parser’s own hint, not sealed-oplog-unparsable with advice to repair the line.
  • op add PartitionTable, CreatePartition and AttachPartition now warn schema-partition-bound-timestamptz-zoneless when a range or list bound on a timestamptz key has no UTC offset (from=2024-01-01). PostgreSQL reads such a bound in the applying session’s time zone, so identify and drift cannot compare it with pg_dump’s output. The op still appends; the warning names the zoned spelling (2024-01-01 00:00:00+00). identify’s refusal detail now gives each skip’s hint under it, so a zoneless bound reads as needing an offset rather than as unsupported partitioning.
  • import --dry-run naming a schema that is not registered no longer creates it. The run still previews the import, then leaves the project as it found it: no <schema>.oplog, no [[schema]] row, no default_schema pin, and no new oplog_root directory. Previewing such an import no longer needs a retrofit schema create first.
  • snapshot --include-drop=scoped now drops the views and materialized views it writes, before the tables, so it re-applies to a database that already holds a schema with a view over one of its tables. Before, the script stopped at DROP TABLE because the view depended on the table, and rolled back. An object the prologue does not drop, such as a function or a view outside the snapshot, still fails the apply when it depends on a table or view the prologue drops.
  • import of DROP MATERIALIZED VIEW IF EXISTS naming a materialized view the import does not hold is now a counted no-op that exits 0, as DROP VIEW IF EXISTS is. Before, it was a capability gap and exited 2. IF EXISTS naming a table, view or index still exits 2, since PostgreSQL refuses it as not a materialized view.
  • snapshot --include-drop=scoped of a schema with a foreign key now applies to an empty database. The prologue’s foreign key drop reads ALTER TABLE IF EXISTS … DROP CONSTRAINT IF EXISTS …, so it does nothing when the table or its schema is absent. Before, it failed with schema "<name>" does not exist or relation "<name>" does not exist and the script rolled back.

Log format

  • A data file whose header says purpose=transition or purpose=cleanse refuses as experimental-not-enabled on every verb that reads it, unless retrofit.toml opts in with [experimental] purposes = ["transition", "cleanse"]. The file’s bytes and the oplog are unchanged; add the key to read such a log as 0.10.0 did, with a warning per file.

Known issues

  • Ending a column that a later RunRawScript body names is not flagged: op remove or op revise of the column, or a merge whose other side removed it, writes the log, and validate accepts it. migrate then emits the script against a table without the column, so the migration fails whole at apply. To catch it before landing, have CI apply the migration to a scratch database (retrofit migrate --from root --to last, or from the latest Seal).
  • identify on a dump that matches no cut can take minutes, where a dump that matches answers in seconds. In one measured case, a history of about 120 operations, a dump matching no cut took 137 s against 0.57 s for one that matched.
  • drift over a database that skipped a migration the log records as a drop, as an out-of-order deploy or a migration-folder tool running an older pending migration does, re-adds the dropped column as an AddColumn, with no question and no note. Review a drift plan’s re-adds against the log before accepting them.
  • A released transition or cleanse data record, in a project that opts in to them, is protected only by its position in the log: an edit to one behind a Seal passes validate, and migrate emits the edited bytes. Nor is a record the release shipped told apart from a branch’s backfill integrated behind it, which a database already past that point never runs.
  • Before 1.0 the oplog may gain operators and header keys. A build refuses a log with a newer retrofit-oplog-format or a key it does not know, so CI and every developer upgrade together.

[0.10.0]

Added

  • This change log. CI requires a “Log format” entry for any change to the oplog catalog golden, the serializer or the seal’s leaf-byte goldens. A serializer edit that leaves the format unchanged may say so with a Log-format: unchanged commit trailer instead.
  • Each release archive carries a man page per command and the bash, zsh and fish completion scripts, and the Homebrew cask installs them, so man retrofit-migrate works after brew install.
  • migrate and snapshot take --allow-unnamed-label <label> (repeatable). It attests that a labelled dataset was left unnamed on purpose, such as a test-only fixtures label on a long-lived target. The apply-risk-data-exclusions warning no longer counts that dataset, and its hint names this flag beside --label. Each value names one exact label, so a dataset added later still warns. An attestation that answers nothing prints a note that it had no effect.
  • CopyTable and SplitTable carry each EXCLUDE constraint of their source onto every table they create, under the name PostgreSQL gives it there (<table>_<keys>_excl); migrate adds it after the table’s LIKE. ExtractTable, JoinTable and UnjoinTable move an EXCLUDE under its own name onto the table that receives its key columns, and refuse one whose keys would land on two tables, naming it. MergeTable drops each source’s EXCLUDE with the source, and a merged row that breaks one on into fails the apply. A copy refuses an EXCLUDE whose name it cannot derive, or two that would take one name.
  • InlineTable carries an EXCLUDE constraint where it refused a table holding one. The survivor keeps its own, and a source’s moves onto the survivor under its own name, re-keyed onto the survivor’s key columns, as a UNIQUE does. migrate checks the rows the re-added constraint would hold by its own operators and refuses, naming it, when a padded row, or on the lookup arm two survivor rows sharing a folded row, would conflict, before anything is dropped. migrate --down of an ExtractTable over a table holding an EXCLUDE now inverts, re-adding it on the table it came from.

Log format

  • AddUnique takes an optional nulls_not_distinct=true for a UNIQUE NULLS NOT DISTINCT, which import now carries in every spelling where it skipped the key as unique-constraint, and drift where it skipped it as add-unique-unmodeled-facet. snapshot and migrate write UNIQUE NULLS NOT DISTINCT (…). drift recreates a key that differs from the target only by it (DropUnique, then AddUnique), and identify tells the two apart. Replay refuses the key beside backing_index, since USING INDEX takes the index’s NULLS treatment, and a keep over such a key refuses with reverse-policy-not-null: the NULLs it writes still collide. A UNIQUE that also carries INCLUDE, WITH (…), a tablespace or an access method still skips. A build without the key refuses a log that holds it, naming the key. A log that skipped such a key keeps the skip; re-import the statement to carry it.
  • The columns list of CreateView, ReplaceView, CreateMaterializedView and ReplaceMaterializedView percent-escapes each name’s %, , and ; (%25, %2C, %3B), so a quoted column name holding a comma, such as "revenue, usd", stays one column where it split into two. import, drift and migrate write the escaped form; op add’s columns= takes it too. A list whose names hold none of the three is unchanged, as is its content hash. A log that holds a literal %25, %2C or %3B in a view column name now reads it as %, , or ;; re-import that view. No other log needs regenerating.
  • CreateTrigger takes optional constraint=true, deferrable=true and initially_deferred=true for a CREATE CONSTRAINT TRIGGER, which import and drift now carry where they skipped it as trigger-unsupported. Its FROM form still skips as trigger-unsupported. Replay refuses a constraint trigger that is not AFTER, FOR EACH ROW, on a table and without transition tables, initially_deferred without deferrable, either one on a plain trigger, and RenameTrigger of a constraint trigger. Its name is also one of its table’s constraint names: a CHECK or key of that name refuses with rule-unique-name-in, and an unnamed one takes the next free derived name. import records ALTER TRIGGER … RENAME of one as the gap constraint-trigger-rename-unsupported, and refuses a trigger created over one’s name, or one created over a plain trigger’s (constraint-trigger-replace-invalid). snapshot and migrate write CREATE CONSTRAINT TRIGGER with its DEFERRABLE clause, and migrate counts it among the constraints a deferred key’s name must not share. A build without the keys refuses a log that holds them, naming the key. No log needs regenerating.
  • CreateFunction takes an optional procedure=true for a CREATE PROCEDURE, which import and drift now carry where they skipped it as create-procedure-unsupported. A procedure shares the functions’ names and signatures; replay refuses returns, volatility, parallel_safety, strict, cost, rows or leakproof on one, and a trigger that executes one. snapshot and migrate write CREATE OR REPLACE PROCEDURE, and name it as a PROCEDURE in DROP, ALTER … RENAME and COMMENT ON, each argument type marked IN. import reads the PROCEDURE and ROUTINE forms of DROP, ALTER and COMMENT ON; with the FUNCTION form, a name written without an argument list names the one routine of that name and kind, of any arity. It refuses what PostgreSQL refuses: an unmarked argument list two procedures answer to, or a name without a list that more than one routine answers to (routine-name-not-unique), and a statement naming a routine of the other kind, a CREATE OR REPLACE across kinds or a trigger on a procedure (routine-kind-mismatch). A build without the key refuses a log that holds it, naming the key. No log needs regenerating.
  • CreateFunction and ReplaceFunction take optional rows= (a function’s ROWS n, kept as written, like cost=) and leakproof=. import and drift carry both from CREATE FUNCTION, where they skipped the function as function-rows-unsupported or function-leakproof-unsupported, and import folds an ALTER FUNCTION … ROWS or … LEAKPROOF into a ReplaceFunction; drift still skips a target ALTER FUNCTION, as it does … COST. A build without them refuses a log that holds them, naming the key. No log needs regenerating.
  • A log holding one of the PostgreSQL 18 surfaces (AddColumn generated_kind=virtual, enforced=false, period=true, without_overlaps=true, SetConstraintEnforced, SetConstraintInherit) now fails validate with syntax-not-available, and no verb renders it. Regenerate the log without them, or revise each op to its PostgreSQL 17 form (generated_kind=stored, enforced=true, period=false, without_overlaps=false).
  • RunRawScript takes reverse_noop=<reason> and reverse_blocked=<reason> beside reverse_body, at most one per line, and a new DeclareReversal line (target=, supersedes=, one of those arms) declares a sealed script’s reversal without changing a sealed byte. A build without them refuses a log that holds them, naming the key or operator. No log needs regenerating.
  • reconcile --base --ours --theirs reads an input that has no engine-version header, and merges the key like any other: an absent key is a value. Every other command still refuses such a log, and no log written before needs regenerating.
  • reconcile --base --ours --theirs keeps a side’s own followers’ prev= bytes when the other side removes the line they follow: the removed line stays in that side’s place as its tombstone. No log needs regenerating.
  • The three-file mode’s refusal hints name the inputs by role (ours, theirs) and its own re-run, never a flag it refuses.
  • reconcile --base --ours --theirs refuses fork-write-write for a value both sides change that no block order decides (one side revising a base op in place, the other setting the same property); --keep or --swap writes the stated outcome. No log needs regenerating.
  • The conflict-marker and duplicate-opid refusals name the three-file mode (reconcile --base --ours --theirs --result) with its inputs by role, and a partition-scoped column facet set against its parent’s recursing op refuses fork-write-write naming <partition>.<column>. No log needs regenerating.
  • reconcile --base --ours --theirs writes --swap’s order with a block’s tombstones beside the block they follow, so --swap over (base, ours, theirs) writes what --keep over (base, theirs, ours) writes, and reads a line a side holds as the base does through the other side’s revision of its predecessor as unchanged. No log needs regenerating.
  • AddIdentity, DropIdentity and SetIdentityGeneration take an optional partition=, a partition beneath the column’s owner whose own identity the op adds, drops or changes in place on a PostgreSQL 15 or 16 pin. op add writes it for a <partition>.<column> token there; from 17 on the op refuses on a partition’s column. No log needs regenerating.
  • AddColumn and AddIdentity take optional identity_start, identity_increment, identity_minvalue, identity_maxvalue, identity_cache and identity_cycle=true for an identity’s sequence options, which import now carries on an inline identity column, ADD COLUMN and ADD GENERATED where it skipped them as identity-options-dropped. An option that restates PostgreSQL’s default for the column’s type and direction is dropped; AS, OWNED BY, RESTART and a partition’s own identity still skip. Replay refuses an option without an identity (identity-option-without-identity), one that is not an integer (identity-option-not-integer), and a bound outside the column’s type, MINVALUE not below MAXVALUE, a START outside them, INCREMENT 0 or CACHE below 1 (identity-option-out-of-range). snapshot and migrate write the options in the identity’s sequence list; drift carries the target’s options onto an identity it adopts and reports identity-options-differ for a paired one whose options differ. A build without the keys refuses a log that holds them, naming the key. A log that skipped such options keeps the skip; re-import the statement to carry them.
  • SetIdentityOptions changes an identity column’s sequence options (ALTER COLUMN … SET START WITH | INCREMENT BY | [NO] MINVALUE | [NO] MAXVALUE | CACHE | [NO] CYCLE). It takes identity_start, identity_increment, identity_minvalue, identity_maxvalue, identity_cache, identity_no_minvalue, identity_no_maxvalue and identity_cycle (false is NO CYCLE); an option it does not name keeps its value. import writes it for those forms, which it skipped as AlterTableSetIdentity (RESTART still skips), and drift for an identity whose options differ from the target’s, which it skipped as identity-options-differ. A build without the operator refuses a log that holds it. No log needs regenerating.
  • SetIdentityOptions takes an optional partition=, as SetIdentityGeneration does, and a partition-scoped AddIdentity takes the identity_* options it refused: on a PostgreSQL 15 or 16 pin a partition’s own identity keeps its sequence options through ATTACH and DETACH, which dropped them. import writes them for an ADD GENERATED or ALTER COLUMN … SET on an attached partition, which it skipped as identity-options-dropped or AlterTableSetIdentity, and drift plans the scoped SetIdentityOptions for a partition whose own identity’s options differ from the target’s. A log that skipped such a statement keeps the skip; re-import it to carry the options.
  • AddExclude and DropExclude add and drop an EXCLUDE constraint. AddExclude takes method (btree, gist, spgist or hash), the key list as CreateIndex spells it (columns, key_expressions, key_opclasses, column_directions, nulls_ordering), key_operators (each key’s WITH operator, && or OPERATOR(schema.op)), name, deferrable and initially. import writes them for ALTER TABLE … ADD CONSTRAINT … EXCLUDE, the table-level EXCLUDE inside CREATE TABLE and DROP CONSTRAINT of one, where it skipped the first as alter-table-other and the second as inline-exclude-constraint. An EXCLUDE with a WHERE, INCLUDE, WITH (…), USING INDEX TABLESPACE or a key COLLATE, or on a partitioned table or a partition, still skips, now as exclude-constraint in both spellings. snapshot and migrate write it, and AddExclude raises apply-risk-add-exclude. InlineTable, PartitionTable and DepartitionTable refuse a table that holds one, and AddExclude refuses a partitioned table. A build without the operators refuses a log that holds them. A log that skipped an EXCLUDE keeps the skip; re-import the statement to carry it.
  • drift reads an EXCLUDE in both spellings and pairs it with the log’s by its keys, method, operators, operator classes, sort order and deferral, not by name. A difference in any of them plans DropExclude then AddExclude, and a name-only difference plans RenameConstraint. An operator pg_dump qualifies by the search path’s first schema (OPERATOR(public.===)) compares equal to the bare ===. A target EXCLUDE with a facet AddExclude cannot carry is an exclude-constraint skip and drops nothing. identify keys the same facets, so two cuts differing only by an EXCLUDE’s operator are not one match. Before, drift reported every EXCLUDE the log held as an exclude-constraint skip and ignored the target’s.
  • AddExclude takes optional include_columns, storage_params and predicate for an EXCLUDE’s INCLUDE (…), WITH (…) and WHERE (…), each spelled as on CreateIndex. import and drift now carry the three where they skipped the constraint as exclude-constraint, and snapshot and migrate write them. An unnamed EXCLUDE with INCLUDE takes PostgreSQL’s name over its keys and its INCLUDE columns (t_x_n_excl), where import held its name over the keys alone. A WHERE that does not resolve skips the whole constraint, since without it the constraint compares every row; a WITH (…) value the log cannot spell, USING INDEX TABLESPACE and a key COLLATE still skip. drift plans a change to any of the three as DropExclude then AddExclude, and DROP COLUMN of a column only the INCLUDE or WHERE names drops the constraint with it, as PostgreSQL does. A build without the parameters refuses a log that uses them. A log that skipped such an EXCLUDE keeps the skip; re-import the statement to carry it.
  • PartitionTable takes two optional lists, partition_parents and partition_keys, one entry per partition, to declare a tree of several levels; a single-level record is unchanged and carries neither. Nothing to regenerate: an existing log replays as it did. op show --framing=add writes a partition bound or key as one JSON object, which op add takes back.
  • Every oplog names its form on a fourth header line, # retrofit-oplog-form: merge or # retrofit-oplog-form: compact. A log without the line is refused (oplog-form-header-missing), as is any other value (unknown-oplog-form). In the merge form, which schema create writes unless --form compact is given, a removed or revised operation stays in the file as a ~ tombstone or behind a replaces= list; op compact drops those traces. Every read checks each live Seal over its chain, so a Seal whose digest no longer matches is refused. For an existing log, add # retrofit-oplog-form: compact as line 4, after # engine-version:, then run retrofit schema form merge to keep the history from then on.
  • op show’s runnable framings print CreateIndex’s key_opclasses as --key_opclass, not --key_opclasse. The log’s bytes are unchanged; re-run op show for any copied --key_opclasse line.
  • A # retrofit-* header key this build does not know is refused on every verb that loads the oplog (unknown-key, writes-blocked-on-unknowns or render-blocked-on-unknowns by the verb’s role), naming the key and its line, instead of being carried through opaquely. Run a build that knows the header; do not delete it.
  • ExtractTable declares the rule extract-partitioned-source: an extract that moves every live column refuses a partitioned parent or a partition as its source. A log holding such an extract no longer replays. Regenerate the log, running DepartitionTable or DetachPartition before the extract, or leave a parent’s partition key on it so the source survives.
  • AttachPartition ends the partition’s own columns into its parent’s, and DetachPartition mints the detached table’s columns and CHECKs afresh, recording their IDs in its new optional column-ids and adjacent-ids. SetDefault, DropDefault, AddNotNull, DropNotNull and SetColumnComment take an optional partition (partition=<table>, or the on-column spelled <partition>.<column>) that scopes the op to one partition of the column’s family. New refusals: partition-not-beneath-owner, partition-not-null-inherited, partition-identity-absent, detach-id-list-arity and attach-partition-check-mismatch. A log whose ops name an attached partition’s own columns, or a detached table’s columns by their earlier IDs, no longer replays: regenerate any log that holds an AttachPartition.

Fixed

  • migrate --down over a window that drops a table holding an EXCLUDE constraint, or merges such a table into another, re-creates the table with each EXCLUDE under its name. It refused the window migrate-non-reversible where the same window over a UNIQUE inverted. An EXCLUDE a CopyTable or SplitTable cloned still refuses. The down of an ExtractTable that inverts as an InlineTable still refuses a table that holds an EXCLUDE, and now names the constraint.
  • reconcile <fork-selector> --clean on a fork nested in another fork’s block closes that fork alone. It also hung the enclosing fork’s other block onto the result, closing the enclosing fork in an order nothing checked, and when that block’s lines came first it moved the nested fork’s ops under it. The enclosing fork now stays a fork, which the next reconcile --clean writes or refuses fork-no-clean. The fork-nested refusal and the bare report’s next step name the nested fork’s selector to close first, where they said no reconcile invocation could resolve it.
  • op add ChangePrimaryKey requires name= for the new key, as AddPrimaryKey does, unless it sets backing_index, and drift and import give a rotated-in key PostgreSQL’s <table>_pkey. An unnamed key was added bare, so the name PostgreSQL gave it depended on where an earlier migrate window ended, and a later window could drop or rename a name the database did not hold. A log that already holds an unnamed ChangePrimaryKey still reads.
  • migrate reports a seed record no --label named as not-selected even when it is anchored behind --from. It used to print a dml-behind-window warning for it, and --allow-unnamed-label noted that it had no effect, so a fixtures dataset kept warning on every later window. A selected record behind the window still warns dml-behind-window.
  • The dml-chain-near-miss warning a data mint raises compares a new chain only with existing chains of its own band (reference, seed or history). data capture --label fixtures on a table that also holds a cleanse chain no longer warns that a label drift forks the chain, or suggests re-running with the other chain’s label-set: a chain of another band is never the one a capture supersedes.
  • data capture’s verdict line prints not comparable: once before the reason instead of twice, and data capture --dry-run rehearses a table it would capture as would capture:, not would captured:.
  • migrate’s data-replace-prologue and data-replace-cascade warnings say a replaced table “is replaced by its full observed state” and replacing <table>, not “cleared” and “clearing”: a keyed replace upserts the observed rows and deletes only the keys the state lacks, and never empties the table.
  • op add CreateIndex --help spells the per-key operator-class flag --key_opclass, which op add and op revise accept beside --key_opclasses. The misspelling --key_opclasse is refused.
  • init against an existing .gitattributes prints a note naming each line it appends, instead of changing the file silently.
  • A data file’s record marker line that leaves out a required token (kind=, seq= or the src= provenance) is refused naming the missing token, not as carrying an “unrecognized or mis-ordered” one. Every marker-line refusal (G-e) now spells the marker grammar as its remedy, and the shared repair hint names the data file; data check and the other corpus readers no longer append a line about a “sidecar”.
  • identify and migrate --from-dump refuse from-dump-schema-absent for a dump of a database that holds neither the (non-public) schema nor any of its objects, and point at migrate --from root. They matched such a dump to the CreateSchema cut, so migrate --from-dump emitted a script without CREATE SCHEMA that failed when applied.
  • identify and migrate --from-dump list the schema facets that did not pair (cut only: and dump only: lines in the detail) when the dump matches no cut but drift would plan nothing against the nearest one. The from-dump-no-match hint no longer sends the user to a drift --dry-run that reports 0 ops, and the from-dump-behind hint no longer says such a divergence was made outside the log; both offer the nearest cut by hand with migrate --from.
  • retrofit completion <shell> prints the shell’s completion script. The root’s help listed it, but running it was refused as an unknown command, and the scripts’ completion requests were refused the same way.
  • Every verb that loads the oplog refuses an operator this build does not know, a ~ tombstone included, naming the operator and the line, with the hint “upgrade the CLI to one that knows the operator; do not delete or edit the line”. The op verbs and validate refuse it as unknown-operator, except that validate keeps sealed-oplog-unparsable for one inside a sealed range. A renamed or split operator keeps its re-import or rename remedy.
  • validate reports an unknown key once, naming its line, the key and the rules it leaves unverifiable, instead of one unverifiable row per rule for every op of the operator. No hint says to hand-fix the op. It also reports a header key this build does not know, and a retrofit-model-version newer than the build, as unverifiable (exit 2); an unsealed log with a newer model-version no longer passes.
  • reconcile exits 2, not 1, for a file only a newer build can read: an unknown operator, a too-new retrofit-oplog-format or retrofit-model-version, an unknown header, and (as before) an unknown key. The JSON code tells them apart; a conflict marker, a repeated header or a refused fork stays exit 1.
  • reconcile, the op verbs and op revise -i refuse an oplog that exists but cannot be read (for example mode 000) with oplog-unreadable (exit 2), printing cannot read oplog and the read error, instead of oplog-not-found (“no oplog”) or a bare command-error.
  • validate refuses an oplog holding an op that needs a newer engine version than its engine-version pin, with unsupported-on-engine-version (exit 2), as migrate and snapshot do. A log whose pin was lowered by hand, as in a resolved header merge conflict, no longer passes the CI gate.
  • op seal and identify refuse the same log with the same unsupported-on-engine-version (exit 2): op seal writes no Seal over it and identify resolves no cut. op info drops the note that said they did not check the engine-version gate.
  • validate, migrate and the other commands that read the log refuse a CreatePartition carrying both partition_columns and partition_expression, as they already refused it on CreateTable and PartitionTable. Such a sub-partition used to replay, and migrate rendered its key from the expression alone, dropping the columns. A CreateTable carrying both with no partition_by is refused too, rather than read as a plain table with its key dropped.
  • op add and the commands that read the log refuse an ExtractTable that moves every live column of a partitioned parent or a partition, with extract-partitioned-source. Such an extract ended the source while a parent’s partitions stayed live, or while a partition’s parent still named it, and the rendered DROP diverged the database from the log.

Changed

  • Every own-envelope migrate file now carries an open-transaction check: a head ahead of BEGIN; and a recheck after the ON_ERROR_STOP guard. This enforces the own contract, that the file owns its transaction. Applied inside a transaction that is already open (psql --single-transaction, a runner’s own transaction, psql AUTOCOMMIT off, a single-Sync batch), the file refuses by name, own-envelope-transaction-open (SQLSTATE 25001), before any DDL. That includes cells that happened to work before, such as a lone unsegmented file under psql --single-transaction, and cells that applied but lost the file’s atomicity at exit 0. The check needs USAGE on plpgsql. Every own file’s bytes change, so a checksum recorded over an earlier own file (Flyway’s, Liquibase’s) no longer matches a re-emit.
  • PostgreSQL 18 is a supported engine version, like 15, 16 and 17, at the PostgreSQL 17 syntax level: every verb writes and reads the SQL it does at 17. The PostgreSQL 18 syntax refuses syntax-not-available (exit 2) at every engine version: AddColumn generated_kind=virtual, enforced=false on AddForeignKey and AddCheck, period=true on AddForeignKey, without_overlaps=true on AddPrimaryKey, AddUnique and ChangePrimaryKey, and the SetConstraintEnforced and SetConstraintInherit operators. op add and op revise refuse it, and validate, op seal, identify, snapshot and migrate refuse a log that holds it, and drift refuses to append to one. op info marks these values and operators as not available.
  • migrate refuses a forward window whose reference record is superseded by a later replace of its table while a step between them reads the table (dml-superseded-in-window, exit 2). Before, such a window exited 0 and the dependent step ran on rows the pruned record would have filled, so an AddNotNull could fail at apply. data capture and drift warn with the same code when the floor they mint makes such a window, the snapshot capture hint names the split and the version, and op seal refuses a range that would seal the pair.
  • migrate renders a PartitionTable of several levels as one nested PARTITION OF tree behind a LOCK TABLE ... ACCESS EXCLUSIVE and a guard that refuses, before any change, rows the declared tree does not route or that a CHECK of the window rejects. A DepartitionTable rebuild opens with the same lock, and the target of one must be the root: a leaf or middle refuses departition-target-is-partition. Its warnings now walk every descendant, naming each partition-owned index, constraint, policy and trigger (an inherited copy is not named), each partition’s stamped DEFAULT, NOT NULL and comment, and each kept serial sequence the rebuild drops; they say the down across it refuses for now. The generated-column recompute warning says the same. ExtractTable of a column that a partition-owned element or a middle’s partition key names refuses, as it did for the source’s own.
  • op info shows, for each rule, what retrofit does to an op that breaks it at two moments: OP ADD (op add and the other commands that write an op) and READING THE LOG (migrate, snapshot, validate, op seal, identify), each refuses or lets through. --as json keeps when and adds enforcement: {"append": ..., "replay": ...}, derived from it.
  • op info says READING THE LOG refuses for partition_cols_xor_expr on CreateTable, CreatePartition and PartitionTable, and for view-replace-superset on ReplaceView: the commands that read the log refuse an op that breaks them, which op info said they let through. --as json reports these rules with when: append+replay, enforcement.replay: refuses and binding: binding, no longer author-time-convenience.
  • MergeTable’s rules list MergeShape on sources: the refusal it already raised, when a source’s columns differ from into’s or a source has a nextval default or no live column, now has a declaration.
  • Advisory messages end without a period. The second sentence of data-motion-partition-not-pg-declarative and of move-column-down-refused, the advice, moved to the hint. A migrate --down script’s reverse-unrecorded line ends ; the database sits at <opid>. The five witness warnings on a record the down reverts state their cause instead of repeating their code, and data-motion-source-consumed, dml-body-partition-dependent, data-only-single-statement-autocommit and the two drift-skip warnings are reworded in plain terms.
  • data-loss-on-apply messages end without a period too: the reasons op add, migrate and migrate --down build for a drop, a cast, a join, an unjoin, a matview replace and a lookup extract or inline. Drift’s extract-recognition diagnostics read “an ExtractTable”.
  • The Advisory codes page shows every form of a message that varies with a field, each led by the value that selects it, and follows a message that is only a placeholder with its detail and hint.

Removed

  • migrate --allow-unreached-dml and its dml-ahead-of-window warning. A data record anchored after --to is outside the window and is now omitted without a refusal, a warning or a flag, as a reference or seed record past --to already was. Promoting an environment that sits behind the tail no longer needs the flag; drop it from scripts, where it is now an unknown flag. dml-anchor-unresolved now means only an anchor the oplog does not hold, fixed with retrofit data repin.
  • LayerSeparation and NotSealed no longer appear in any operator’s rules: retrofit never checked either. Layer separation is a property of the oplog’s slots (docs/concepts.md, “The oplog”); the sealed region is guarded by sealed-region-write.

Known issues

  • Ending a column that a later RunRawScript body names is not flagged: op remove or op revise of the column, or a merge whose other side removed it, writes the log, and validate accepts it. migrate then emits the script against a table without the column, so the migration fails whole at apply. To catch it before landing, have CI apply the migration to a scratch database (retrofit migrate --from root --to last, or from the latest Seal).
  • identify on a dump that matches no cut can take minutes, where a dump that matches answers in seconds. In one measured case, a history of about 120 operations, a dump matching no cut took 137 s against 0.57 s for one that matched.
  • drift over a database that skipped a migration the log records as a drop, as an out-of-order deploy or a migration-folder tool running an older pending migration does, re-adds the dropped column as an AddColumn, with no question and no note. Review a drift plan’s re-adds against the log before accepting them. A fix is planned after v0.10.0.
  • A released transition or cleanse data record is protected only by its position in the log: an edit to one behind a Seal passes validate, and migrate emits the edited bytes. Nor is a record the release shipped told apart from a branch’s backfill integrated behind it, which a database already past that point never runs.
  • Before 1.0 the oplog may gain operators and header keys. A build refuses a log with a newer retrofit-oplog-format or a key it does not know, so CI and every developer upgrade together. Planned after this release: op record; Archive, Unarchive and Purge; Annotate and Retract; erasing a released record; renaming a schema; possibly a marker for experimental operators.

Published .