retrofit
0.15.0-wip.89
retrofit.sh

These docs are for 0.15.0-wip.89, a WIP, and may describe what no release does yet. The newest release is 0.14.0: this page for 0.14.0.

retrofit import

retrofit import reads one or more .sql sources and appends an op to the oplog for each statement in retrofit’s vocabulary, in the order the sources state them (a drop a function body reads, and a function created before what its body binds, can move; see Drops read by function bodies and Functions created before what their bodies bind). It writes the oplog, the schema’s [[schema]] row in retrofit.toml when it registers one, and the data files it sets preserved DML aside in; it never runs a statement against a database. Import existing SQL and Import a migration tree in order are the task guides.

Synopsis

From retrofit import --help:

Usage:
  retrofit import <source>... [--schema <name>]... [--all] [--from-file <path>] [flags]

Description

import checks each statement against the state the oplog and the earlier statements built, and appends the ops it states: a CREATE TABLE appends a CreateTable and an AddColumn per column. A statement outside the vocabulary is listed on the skip manifest; custom DML is set aside in a data file (Preserved DML).

Sources

A source is a path with a .sql extension, - for standard input, or a stream such as <(…) or a FIFO. Any other operand (a URL, another extension, an extension-less regular file) refuses with kind-mismatch. A stream that delivers zero bytes refuses with source-empty before any parse; on a pipe that is the sign of a producer that failed upstream. A zero-byte .sql file is read as an empty source.

- may appear once, alone: combined with another source, given twice or listed in a --from-file list, it refuses with arg-shape, as does an invocation with no source at all. A parse error names the source path, or - for standard input.

List files

--from-file <path> reads a newline-delimited list of sources:

  • A path resolves against the list file’s directory. A stream (<(…), /dev/stdin) has no directory, so its paths resolve against the working directory (the -C directory, when given). An absolute path is read as written.
  • Only a line whose first non-blank character is # is a comment; a # after a path is part of the path.
  • Positional sources are read first, then the list, in its order.
  • --from-file - names a file called -; /dev/stdin reads the list from standard input. A - entry inside the list refuses with arg-shape, and so does a list that holds no path.

Schema selection

The schema is the one --schema names, or the one the resolver chain selects. With no result, the run refuses with default-schema-unset; outside a project (no retrofit.toml up from the working directory) it refuses with no-project.

--schema <name> naming an unregistered schema registers it before the sources are read. The run writes <name>.oplog under the project’s oplog_root when it declares one, else in the working directory (or at the project root when the working directory is outside it). It appends a [[schema]] row, and sets default_schema when it is the project’s first schema. The resolver chain never registers a schema.

A source that declares another schema (CREATE SCHEMA <y>) refuses with schema-mismatch when --schema names a schema not yet registered, or on a populated oplog when --force is not given. A declaration of pg_catalog, information_schema, public or a schema an [[exclude]] entry names (Project exclusions) is not counted. Otherwise CREATE SCHEMA <y>, like any statement on an object in another schema (<y>.<table>), is a cross-schema skip (Skipped statements). A batch that appends no op and skips a cross-schema statement refuses with schema-mismatch.

Output

Each appended op prints as + <opid> <Operator> <target>, followed by a summary line (imported 12 ops; skipped 2 statements (1 sequences, 1 GrantStmt)). Under --as=json each op is an {"type":"appended",…} record and the run ends with an {"type":"ok",…} envelope. The schema the run resolved is printed as note: schema-context: <name> on stderr in text mode, and as the envelope’s schema field in JSON. Output contract gives every record’s shape.

Dry run

--dry-run runs the whole pipeline and writes nothing: not the oplog, not the [[schema]] engine_version update, and no data file. It raises every refusal the real run raises before writing a data file (a page the reader would refuse, a hand-authored record a producer would overwrite, a delta beside a floor no producer joins), with the same code. The real run raises those refusals before its oplog write too, so a refused run leaves the oplog as it was. The summary line opens would import.

Against a schema --schema names for the first time, --dry-run previews the import, then leaves the project as it found it: no <name>.oplog, no [[schema]] row, no default_schema and no new oplog_root directory.

Extending and replacing

A bare import extends a populated oplog: the new ops follow its tail. --force discards the oplog’s content and writes a fresh one from the sources. Two guards hold before anything is written, and under --dry-run each prints a preview instead of refusing:

  • Sealed history. Every Seal the run would replace must be named by its full opid in --allow-sealed-rewrite; a Seal left unnamed refuses with seal-removal-refused, and the hint names each opid. A value that is not a full opid refuses with flag-invalid-value before any work, --dry-run included. An oplog that does not parse holds no Seal that can be read, and is replaced with a note saying so.
  • Orphaned data files. A data-file record anchored on an op the fresh oplog no longer has refuses with force-corpus-orphan. Pass --purge-orphaned-corpus to remove those records, or pass the DML sources among the run’s sources so it writes them again against the new ops. A hand-authored record is not written again and must be authored again.

A branch that appended to the replaced history and is merged with the fresh one holds two CreateSchema roots of one schema, which validate and reconcile refuse.

A state source on a populated oplog

A source that opens like a pg_dump of a schema (the leading SET lines and SELECT pg_catalog.set_config('search_path', '', false), ahead of its first object) is a state, not a list of changes. Offered to a populated oplog without --force, it refuses with state-source-on-populated-oplog before writing anything, and a batch with one such source refuses whole. A database changed outside the oplog is recorded by retrofit drift, retrofit import --force starts the oplog over, and the rows a dump holds go through retrofit data capture.

An oplog that holds ops but declares no CREATE SCHEMA for the resolved schema refuses with schema-not-in-oplog, whatever the source holds. --force writes a fresh oplog past it, and retrofit schema remove drops the file.

Seals

--seal appends one Seal after the run’s ops. Its reason is derived from the source names in argv order (imported V01__init.sql, V02__add_index.sql, …), so it records the order of the import. When two sources share a basename, each colliding name widens to the shortest trailing path no other source ends with, so a tree of <version>_<name>/up.sql directories (Diesel writes this shape) reads imported 001_a/up.sql, 002_b/up.sql. The widened name stays relative. --seal=<text> sets the reason to <text>.

--seal-per-file appends one Seal after each source’s ops:

  • Each Seal’s reason is that source’s name, widened against the rest of the batch when it collides, so identify names the migration a database is at.
  • --seal-per-file --seal=<text> gives every Seal the reason <text>.
  • A single-source run is a plain --seal.
  • A source that appends no op gets no Seal. The summary says so (sealed 41 ops across 82 sources (2 sources minted no ops, so were not sealed)), and the import-summary record counts them as sources_zero_op.

Every Seal the run appends passes the check op seal runs over the range before anything is written, and --dry-run raises the same refusals. A reference record the range would cover with no witness refuses with data-witness-missing, naming the record; that includes a record this run writes, when a source holds rows for a declared reference table. Its recovery is the same import without --seal, --seal-per-file or --dry-run, then retrofit data witness, then retrofit op seal. A reference replace anchored in the range with no upsert form at its anchor refuses with replace-no-upsert-form (The replace prologue). A data corpus holding a record whose anchor-op= names no op refuses with data-file-anchor-unresolved, as data check does.

What import recognizes

Each statement below appends the operators named:

  • CREATE TABLE, with inline column and table constraints (EXCLUDE among them), GENERATED ALWAYS AS (<expr>) STORED columns and identity columns (GENERATED { BY DEFAULT | ALWAYS } AS IDENTITY).
  • Declarative partitioning: CREATE TABLE … PARTITION BY appends a CreateTable with its partition key; CREATE TABLE … PARTITION OF … FOR VALUES appends CreatePartition; ALTER TABLE … ATTACH PARTITION appends AttachPartition; ALTER TABLE … DETACH PARTITION appends DetachPartition. A partition key that mixes columns and expressions, or holds more than one expression, is skipped.
  • A sequence-backed column: a column default nextval('<seq>'::regclass) naming a sequence the sources create OWNED BY that column, with no options changed from the defaults, on an integer column of a table that is neither partitioned nor a partition, is recorded as GENERATED BY DEFAULT AS IDENTITY. That is the shape pg_dump writes for a serial column. Any other nextval default stays the column’s default.
  • CREATE INDEX: btree, gin, gist, brin, hash, spgist; the predicate (WHERE) and the access method (USING) are parameters of the op.
  • ALTER TABLE … ADD CONSTRAINT { PRIMARY KEY | FOREIGN KEY | UNIQUE | CHECK | EXCLUDE }. A validated CHECK (<col> IS NOT NULL [AND …]) folds to one AddNotNull per column; a NOT VALID one stays an AddCheck. EXCLUDE, in either spelling, appends AddExclude with its INCLUDE columns, WITH (…) parameters and WHERE predicate. An EXCLUDE that AddExclude cannot hold (USING INDEX TABLESPACE, a key COLLATE or operator class parameters, an access method or operator outside the stored set, a partitioned table) is skipped. A key’s plain operator class is kept on the op.
  • UNIQUE: the inline CREATE TABLE … UNIQUE (cols), the standalone ALTER TABLE … ADD CONSTRAINT … UNIQUE (cols) and UNIQUE USING INDEX <idx> (when the index is constraint-eligible) each append AddUnique. NULLS NOT DISTINCT is the op’s nulls_not_distinct in every spelling, ADD COLUMN … UNIQUE and a partition’s own key included. A UNIQUE with a facet AddUnique cannot hold is skipped: INCLUDE (…), WITH (…), USING INDEX TABLESPACE, USING <method>, or no column list outside that eligible USING INDEX form.
  • ALTER TABLE … DROP CONSTRAINT appends the drop for the constraint’s kind (DropPrimaryKey, DropForeignKey, DropUnique, DropCheck, DropExclude). A primary key dropped and added again on one table is one ChangePrimaryKey. ALTER TABLE … VALIDATE CONSTRAINT appends ValidateConstraint.
  • ALTER TABLE … { ADD | DROP } COLUMN. A cross-table DROP COLUMN and ADD COLUMN of an unreferenced column fold to one MoveColumn (simple).
  • ALTER TABLE … ALTER COLUMN: TYPE (CastColumn, with its USING expression), SET DEFAULT, DROP DEFAULT, SET NOT NULL (AddNotNull), DROP NOT NULL (DropNotNull), DROP EXPRESSION (clears a STORED generated expression; PostgreSQL 13 and later), the identity forms ADD GENERATED, DROP GENERATED and SET GENERATED, and an identity’s sequence options (SET START WITH, SET INCREMENT BY and the rest, as SetIdentityOptions; RESTART is skipped).
  • ALTER TABLE … ALTER CONSTRAINT … { DEFERRABLE | NOT DEFERRABLE } and ALTER TABLE … RENAME CONSTRAINT.
  • ALTER TABLE … REPLICA IDENTITY ….
  • ALTER TABLE … { ENABLE | DISABLE } TRIGGER <name> (EnableTrigger, DisableTrigger); the ALL, USER, REPLICA and ALWAYS forms are skipped.
  • Renames: ALTER TABLE … RENAME TO (RenameTable), ALTER TABLE … RENAME COLUMN (RenameColumn), ALTER INDEX … RENAME TO (RenameIndex), ALTER VIEW … RENAME TO (RenameView), ALTER MATERIALIZED VIEW … RENAME TO (RenameMaterializedView), ALTER TYPE … RENAME TO (RenameType), ALTER FUNCTION … RENAME TO (RenameFunction), ALTER TRIGGER … RENAME TO (RenameTrigger) and ALTER POLICY … RENAME TO (RenamePolicy).
  • DROP TABLE, DROP INDEX and DROP TYPE, with or without IF EXISTS, and with a comma list of names: one DropTable, DropIndex or DropType per live target (see Guarded drops). DROP FUNCTION, DROP TRIGGER and DROP POLICY append DropFunction, DropTrigger and DropPolicy.
  • CREATE TYPE … AS ENUM and a composite CREATE TYPE … AS (…) append CreateType; ALTER TYPE … ADD VALUE and ALTER TYPE … RENAME VALUE append AddEnumValue and RenameEnumValue.
  • CREATE VIEW, CREATE OR REPLACE VIEW, DROP VIEW, CREATE MATERIALIZED VIEW and DROP MATERIALIZED VIEW, in their single-schema forms; a view in another schema is a cross-schema skip (Skipped statements).
  • CREATE FUNCTION and CREATE PROCEDURE (a CreateFunction with procedure=true), single-schema. CREATE OR REPLACE of a function that exists, and an ALTER FUNCTION that sets an attribute (volatility, STRICT, SECURITY DEFINER, PARALLEL, COST, ROWS, LEAKPROOF, SET), append ReplaceFunction. The PROCEDURE and ROUTINE forms of ALTER, DROP and COMMENT ON are read as the FUNCTION form is.
  • CREATE TRIGGER and CREATE CONSTRAINT TRIGGER, single-schema. A constraint trigger’s FROM form, and an ALTER TRIGGER … RENAME of a constraint trigger, are skipped.
  • CREATE POLICY, ALTER POLICY (ReplacePolicy) and the row-level-security toggles (SetRowLevelSecurity).
  • CREATE EXTENSION, DROP EXTENSION and SET search_path; SET search_path changes the registered search path.
  • COMMENT ON TABLE, COLUMN and FUNCTION. A function comment resolves its signature as ALTER FUNCTION … RENAME does, so pg_dump’s f(n integer) names f(integer); a bare COMMENT ON FUNCTION f resolves only when one overload of f is live, and any other miss is a comment-function-unresolved skip.
  • COMMENT ON CONSTRAINT … ON <table>, for a primary key, foreign key, unique or check constraint, resolved by the name the constraint has at that statement, so a later rename keeps the comment; <table>_pkey also names an inline PRIMARY KEY. Any other target (a PostgreSQL 18 named NOT NULL, an unknown name) is a comment-constraint-unresolved skip. COMMENT ON CONSTRAINT … ON DOMAIN is in the comments bucket.
  • DO $$ … $$; blocks append RunRawScript, unless a data-motion shape matches the block first.

A statement fails validation when its target is not in the state the oplog and the earlier statements built: an ALTER TABLE against a table no earlier source creates refuses with validation-error, and an op whose precondition fails (a CreateTable for a table that exists, an AddColumn for a column that exists) refuses with a rule-* code naming the first failing op. When the source looks like the database moved without the oplog (a duplicate CreateTable, a dangling foreign key on a populated oplog), the hint names retrofit drift. One run reports the first such statement only.

Unresolved trigger callbacks

A CREATE TRIGGER whose callback function neither the sources nor the current cut define is accepted: the run appends a CreateFunction with language=internal for the callback, which op show lists and the op count includes, so the CreateTrigger has an element to name. snapshot leaves that function’s DDL out and still writes the CREATE TRIGGER, so the emitted DDL applies only where the target database already has the function; elsewhere PostgreSQL rejects the trigger with function … does not exist. The run exits 0 and prints the function-synthesized-stub finding, whose text and JSON shapes are in the Output contract.

A callback qualified with another schema (EXECUTE FUNCTION <y>.<f>()) is resolved by its bare name: it binds to <f> in the resolved schema when one is defined there, and otherwise gets the stub, even when <y> defines <f>. snapshot writes the call without <y>. A trigger callback in another schema is not available.

Guarded drops

DROP TABLE, DROP INDEX and DROP TYPE with IF EXISTS are read per target, at each target’s position in the source. A live target appends its drop; an absent one appends nothing and is not listed. In a mixed list, the live targets append and the absent ones do not. The bare form is an explicit drop: a target it cannot resolve fails the run or is listed as a skip. So a pg_dump --clean --if-exists prologue is a no-op against a fresh oplog.

DROP VIEW, DROP MATERIALIZED VIEW, DROP TRIGGER, and ALTER TABLE … DROP CONSTRAINT or DROP COLUMN, with IF EXISTS naming an object that is not there, append nothing and are listed as a *-if-exists-noop skip, which loses nothing and leaves the exit at 0. A view or materialized view drop naming a table, view or index of another kind is a loss-bearing skip, as PostgreSQL refuses it.

DROP SCHEMA IF EXISTS appends nothing, since no operator drops a schema. A table name held as a deferred data-motion target (below) is not an ordinary absent table: its drop is listed as cross-skipped-data-motion-target.

Drops read by function bodies

PostgreSQL records no dependency for a plain PL/pgSQL function body, so it applies a file that runs DROP COLUMN or DROP VIEW on a target a body still reads, as long as a later statement in the same file clears that read. import records the state that file reaches, so it can append these ops out of source order:

  • When the file replaces every such reader with CREATE OR REPLACE FUNCTION, the drop is appended just after the last replacement, with the note schema-import-drop-moved-past-reader-replace.
  • When the file drops every such reader, their DropFunction ops are appended ahead of the drop, the later DROP FUNCTION appends nothing, and the note is schema-import-reader-drop-moved-ahead.

Any other case keeps the source order, and the drop refuses while a body still reads its target. Only the oplog order changes: import never re-sorts or rewrites the files.

Functions created before what their bodies bind

PostgreSQL binds a string-literal LANGUAGE sql body, and a PL/pgSQL %TYPE or %ROWTYPE declaration, when it creates the function, and with check_function_bodies on (the default) refuses the function while a table, view or function the body names does not exist. A pg_dump --schema-only file writes these functions ahead of what their bodies read and turns check_function_bodies off so it restores; import drops that SET. import records the state the file reaches, so it can append a new function out of source order:

  • A CREATE FUNCTION whose body binds a table, view or function that a later statement of the same source creates is appended just after that statement (the last of them, when it binds several), with the note schema-import-function-moved-after-bound-name. The COMMENT ON FUNCTION and ALTER FUNCTION statements right after it move with it.
  • A LANGUAGE sql function whose body calls a moved function is appended after it, with the same note.

Any source qualifies, a dump or not. Any other case keeps the source order: a function in another schema, one the source creates twice, a SQL body with a polymorphic argument (PostgreSQL binds it only when called), a name the oplog or an earlier source already holds, or a statement passed over that names the function, is a DO block, holds data, sets search_path, or drops or renames a name the body reads. Only the oplog order changes: import never re-sorts or rewrites the files.

What import cannot produce

import records each change as the source states it. Beyond the folds above, two shapes are recognized as one operator when the source holds all of their evidence: a cross-table DROP COLUMN and ADD COLUMN of an unreferenced column (MoveColumn (simple)), and a solid-match ExtractTable.

Data-motion operators

A migration script’s ExtractTable is recognized when the source table is fully consumed: every column moved into a child, no constraint or index left on the source, and the source dropped with DROP TABLE. The run appends the ExtractTable and prints import-extract-recognized.

Every other data-motion shape appends the plain operators the statements state (CreateTable, DropColumn, …) and prints import-data-motion-deferred, which names op add. That covers the column extract that keeps its source (a CREATE of the child, an INSERT … SELECT and an ALTER TABLE <source> DROP COLUMN, with no DROP TABLE), CopyTable, JoinTable, SplitTable, MergeTable and the partition converts. Fold them into the operator in the unsealed tail, or author it with op add; Record a table split or merge yourself has the recipes.

A child created by CREATE TABLE … AS SELECT in a deferred shape gets no op, so every later statement against it (ALTER TABLE, CREATE INDEX, DROP TABLE, and row statements such as an UPDATE backfill) is listed as cross-skipped-data-motion-target, until a CREATE TABLE in the schema reuses the name. Its rows are not written to a data file: author the data-motion op first, then capture them at HEAD.

import-move-column-fold-mismatch refuses a source whose ADD COLUMN, key-correlated UPDATE and DROP COLUMN read as one keyed MoveColumn that the run could not append as one. Its workaround is a DropColumn and AddColumn, with the data step authored by op add.

Skipped statements

A statement outside the vocabulary is listed on the skip manifest with its kind, name, schema, source line and raw text, and is not recorded: standalone sequences, privileges (GRANT, REVOKE, OWNER), statements on an object in another schema (cross-schema), composite-type attribute changes (ALTER TYPE … { ADD | DROP | ALTER } ATTRIBUTE; a composite is changed by a drop and a create), and any other DDL outside the vocabulary.

A cross-schema skip is a statement whose own object is in a schema other than the resolved one: under --schema <x>, CREATE TABLE <y>.<t> is one, and so is CREATE TABLE public.<t>. An object in the resolved schema that names another schema is not. A foreign key to <y>.<t> is recorded as written. A view or policy body that reads <y>.<t> is recorded when <y> is registered and holds <t>, and is otherwise a view-query-unresolved or policy-body-unresolved skip. snapshot writes each of these names with its schema. A trigger callback in another schema is the exception (Unresolved trigger callbacks).

Under --as=json each skip is a {"type":"skipped",…} record, and the terminal envelope holds only the count, as the integer skipped. Text mode prints one summary line that tallies the skips by kind, with a breakdown by disposition when they span more than one class. --skip-manifest writes each record to a file in either mode.

An inheritance child (CREATE TABLE c (…) INHERITS (p), no PARTITION OF) is skipped whole as pg-legacy-inherits-partition, a capability gap: no op models the inherited columns or the link. Every later statement against the child is listed as cross-skipped-legacy-inherits-child, until a plain CREATE TABLE reuses the name.

A skip that drops content no operator holds (the gap and integrity classes) refuses the run with import-loss-bearing-skips, and the hint counts the kinds. The ops the run recognized are still written to the oplog. --allow-capability-gaps covers capability-gap skips only in a batch that appended at least one op; an integrity skip, or a batch that appended no op, still refuses with the flag.

Skip manifest

--skip-manifest <path> writes one JSONL record per skipped statement. The file holds the same records as the stream, raw_text untruncated in both; a stream record may add a key on top (the zero-op-step advisory adds source, the input the record’s src_file names).

{"kind":"CreateSeqStmt","line":2,"name":"sq","raw_stmt_kind":"CreateSeqStmt","raw_text":"CREATE SEQUENCE sq","schema":"","src_file":"schema.sql"}

Keys are written in alphabetical order, so the bytes are stable across runs; read them by name. schema is empty for an unqualified statement, and src_file names the file line counts in. The field set is open.

The file is written at the run’s last step, in any output mode, so a run refused for loss-bearing skips still writes it. A run refused before that step (state-source-on-populated-oplog, data-file-collision, parse-error and the other early refusals) writes neither file.

Summary file

--summary-file <path> writes the import-summary record as one JSON object:

{"imported":2,"run_raw_script":0,"skipped":1,"skipped_by_class":{"decision":1},"skipped_by_kind":{"CreateSeqStmt":1}}

imported, run_raw_script, skipped and skipped_by_kind are always present; skipped_by_class, sealed and sources_zero_op are present when they apply. skipped_by_class splits the skipped total by disposition. The per-kind and per-class tallies are in this record only, not on the terminal envelope, and text mode does not print it. It is written when the skip manifest is, and skipped by the same early refusals.

The two files must be two paths: one path for both refuses with summary-file-collides-with-skip-manifest. A path that resolves to a file the run manages (the oplog, retrofit.toml wherever --config or RETROFIT_CONFIG puts it, or any name the run could write or read under the data root, existing or not) refuses with skip-manifest-clobbers-managed-file before anything is written. Any other name under the data root is allowed. The check follows .., symlinks and case variants on a case-insensitive filesystem.

Preserved DML

Custom DML (INSERT, UPDATE, DELETE, MERGE) is written verbatim to a data file, one per table, under the schema’s data_root. The data file is the format, naming, keys and re-ingest rules, and Purpose the chain that classifies each statement. A statement no step classifies is listed with its cause and written to no data file. The skipped_by_kind counts for InsertStmt, UpdateStmt, DeleteStmt and MergeStmt in the import-summary record count every such statement the run reads, whether or not it reaches a data file: a listed one (a dump data band’s included) and an unqualified DELETE that makes the records replace count too. Under --ddl-only, a declined statement counts under declined by --ddl-only instead.

  • A --purpose other than transition or cleanse refuses with flag-invalid-value.
  • Both purposes are experimental. Unless retrofit.toml opts into the purpose ([experimental] purposes), --purpose refuses with experimental-not-enabled before anything is written, --dry-run included. A run with the opt-in warns experimental-purpose once for each such file in the corpus, the one it wrote included (Experimental purposes).
  • Every run reads the schema’s data corpus first, with or without --purpose: a corpus holding a transition or cleanse record the project has not opted into refuses with experimental-not-enabled.
  • --purpose and --label together refuse with flag-conflict.
  • An empty or whitespace-only --label refuses with label-empty.
  • Without --disposition, a TRUNCATE or unqualified DELETE before INSERTs makes the records replace. A statement the source states before it, at the same anchor, is superseded by the floor that follows: nothing is written for it, and the run prints dml-superseded naming it. A --disposition other than replace or append refuses with disposition-unknown.
  • A source with no DDL leaves the oplog as it was and anchors its records at the oplog’s last op. Against an empty oplog it refuses with dml-table-not-found, since no table stands there; so does any statement whose table does not stand at the op its rows anchor on.
  • A pg_dump data band (the statements under a -- Data for Name: ...; Type: TABLE DATA header) is listed, not captured, and leaves the exit at 0. Each table prints not captured: <table> (<n>) with the cause dump data band (in JSON, a not-captured record with table, rows and cause), followed by the commands that capture it, such as retrofit data capture <source>.
  • A table’s INSERT band in one source larger than the schema’s data_band_max_bytes (or the built-in default) refuses with data-band-over-cap. The bound is checked per source before any source is parsed, so a batch with one over-cap source refuses whole, and every over-cap source and table is named in one error.

--purge-superseded removes the earlier records a re-ingest supersedes (Re-ingesting into the corpus).

Engine version

import parses with the PostgreSQL 17 grammar, so PostgreSQL 18 syntax (NOT ENFORCED, VIRTUAL generated columns, WITHOUT OVERLAPS) does not parse: the run refuses with parse-error. PostgreSQL versions gives each command’s row at each major.

--engine-version <major> sets the major the imported schema is pinned to. On the run that creates the oplog, the one resolved value is written to both the oplog’s # engine-version: header and the schema’s [[schema]] engine_version. The value resolves in this order, first hit wins:

  1. --engine-version
  2. the existing oplog’s header, on a run that extends an oplog
  3. [project] engine_version in retrofit.toml
  4. the built-in default

On a run that extends an oplog, the header is kept as it was, so the flag changes only the [[schema]] value:

  • Above the header (--engine-version 18 on a 17 oplog): the run succeeds, and the next bare import resolves from the header and writes 17 back.
  • Below the header (--engine-version 17 on an 18 oplog): the run succeeds, and the next command that loads the project refuses with engine-version-target-below-oplog, until the value is restored or the header lowered. retrofit schema engine <major> --schema <name> moves both together and is the way out.

A major this build does not support refuses with unknown-engine-version before anything is written.

Options

From retrofit import --help:

Flags:
      --all                               import into every registered schema, all or none, as with no --schema (and satisfying RETROFIT_REQUIRE_EXPLICIT_SCHEMA)
      --allow-capability-gaps             import anyway when capability-gap skips drop content: statements no operator records (exit 0 + warning instead of exit 2; the import-summary record is byte-identical either way); the one carve-out is validation-errors and rule-* refusals, which still abort
      --allow-readers-moved stringArray   attest that the readers of the table or column with this qualified old name are handled by the time the rename applies (repeatable; byte-exact, never trimmed); every other rename still warns
      --allow-sealed-rewrite <opid>       [sealed history] attest each Seal this run would demote or replace, by its full <opid> (repeatable, comma-separated); a value attests the Seal it names and selects nothing, and a Seal left unnamed refuses seal-removal-refused
      --data-root <path>                  root <path> that overrides the schema's `data_root`
      --database-at string                the cut the database these ops are written for is at, from which apply-risk is judged (else the latest Seal)
      --ddl-only                          decline every preserved-DML statement in the run (authored DML, dump data bands and synthesized floors): nothing is minted, the declined tables are listed, and the run exits on its DDL alone; orthogonal to --allow-capability-gaps, which covers DDL retrofit cannot represent
      --disposition <replace|append>      asserted <replace|append> disposition on every preserved-DML record
      --dry-run                           run the full pipeline without writing the oplog file
      --engine-version <15|16|17|18>      supported major <15|16|17|18> for the imported schema (default: project engine_version)
      --extra-seed-path <path>            root <path> that appends to the declared `seed_paths`
      --force                             overwrite an existing populated oplog (mint fresh, keeping the ids other registered schemas reference; a bare import extends instead)
      --from-file <path>                  newline-delimited <path> listing .sql files, imported in listed order (relative to the list's directory; # comments and blank lines ignored)
  -h, --help                              help for import
      --help-all                          show extended help (vocabulary, semantics, caveats)
      --label <label>                     opaque <label> stamped onto every preserved-DML record (repeatable set; key:value accepted, not required)
      --purge-orphaned-corpus             consent to remove every corpus record whose anchor no longer exists in the replaced oplog (--force only; a same-anchor re-capture replaces its version in place)
      --purge-superseded                  remove earlier same-chain data blocks superseded by this ingest (positional supersession within one continuous oplog; the anchor is alive at both positions); never a witnessed record, sealed or in the tail; each is reported kept. Default keeps them as --at render assets
      --purpose <transition|cleanse>      asserted <transition|cleanse> purpose on every preserved-DML record (one of: transition, cleanse; name a seed cohort with --label, and declare reference tables with --ref <table> or retrofit schema ref-tables add <table>)
      --ref <table>                       declare <table> reference data (repeatable): a CreateTable this run mints is written reference_data=true; a table live at the run's starting cut gets SetReferenceData(value=true) appended after the run's DDL ops
      --schema <name>                     registered schema <name> (lazy-creates when unregistered); repeat it to import into each schema named, all or none
      --seal[=<text>]                     append a Seal op after the imported prefix with reason <text> (bare flag auto-derives it from source(s)); refused as `op seal` refuses when the range holds an unwitnessed reference record, including one this import mints
      --seal-per-file                     emit one Seal op after each source that minted ops (sources minting none are not sealed; equivalent to a bare '--seal' import per file)
      --seed-path <path>                  root <path> that replaces the declared `seed_paths`
      --skip-manifest <path>              write one JSONL record per skipped statement to <path> (kind/schema.name/line/raw_stmt_kind/raw_text)
      --summary-file <path>               write the aggregate import-summary record to <path> as one JSON object (imported/run_raw_script/skipped/skipped_by_kind[/skipped_by_class][/sealed][/sources_zero_op])

The global flags are in the Output contract.

Diagnostics

A refusal writes nothing to the oplog, except import-loss-bearing-skips, which is raised after the recognized ops are written. The error’s code names it (Output contract), and Refusal codes gives the exit status of every code import refuses with.

Four records are printed on the stdout stream (import records) and leave the exit at 0: function-synthesized-stub (Unresolved trigger callbacks), import-extract-recognized and import-data-motion-deferred (Data-motion operators), and dml-superseded (Preserved DML).

Advisory codes lists every code with its class.

See also

Import existing SQL, Import a migration tree in order, FAQ, retrofit drift, retrofit data, Operators, Output contract, The oplog.

Before 1.0, any release may change behavior; the release notes list what changed.

Published .