retrofit
0.13.0
retrofit.sh

retrofit

Schema evolution for PostgreSQL: retrofit records each schema change in an append-only log of operators (the oplog) and emits canonical DDL and migration scripts from it. Start with Install, then Create a project.

Get started

InstallInstall retrofit with the install script, Homebrew or a release archive, and check that it runs
Create a projectStart a project from scratch or from existing SQL, read the oplog it writes, and emit DDL
What's nextWhere to read after the first project, from the model to each verb’s reference

Concepts

The oplogWhat retrofit records, how an op line reads, element identity, and the schema replay builds
Sealing and deployingSealed and unsealed ops, tracking where each database is, and finding and recording drift
Branches and mergesWhat a merge of two branches does to the oplog, and the two forms an oplog can take
Data filesWhere rows live beside the oplog, what each purpose does, and how a stale reference is repaired
Kinds of operatorsThe kinds of change an operator records, and when import or drift writes the ops
How a command picks its schemaSchema identity, the resolver chain, and the three ways a schema is created on first use
ConsentsThe inputs that clear a refusal, the two kinds, and how each is spelled
How a migration appliesWhat a migration script assumes, who owns its transaction, why some scripts commit in parts, and how a reverse script is built
PrinciplesThe rules retrofit upholds, and why some things are not configurable

Guides

Import existing SQLBootstrap an oplog from a schema file, a dump or migration scripts, preview it, seal it, and extend or start over
Import a migration tree in orderPass a tree of migration files in the order they applied, check the order first, and seal each migration
Exclude objects the project does not ownKeep a provider’s tables, extensions and schemas out of the oplog with one retrofit.toml entry each
Work with several schemasRegister more schemas, scope them to directories, change the default, move an oplog, and pick a schema for one command
Record a schema changed outside the oplogDump a database changed outside the oplog, preview what drift records, answer its questions, and seal the result
Emit the schema as SQLEmit the schema at a point in the oplog as SQL, with its reference and seed rows or without them
Find an element's ID and historyList a schema’s elements with their IDs, script over them, and read one element’s history, live or dropped
Emit a migration scriptEmit the migration script between two points in the oplog, from where a database is, with seed rows, or in reverse
Choose who opens the transactionEmit a script that opens its own transaction, or one your runner wraps, or one applied a statement at a time
Bound how long a migration waits for a lockMake a migration statement give up on a lock rather than queue every later query behind it
Recover after a failed apply or a mergeFinish a migration that stopped part way, and reconcile the fork a merge leaves in the oplog
Merge branches that change the oplogResolve conflict markers, bring main in with the three-file merge, settle a property both sides set, and use jj, bisect and a pre-commit squash
Ship a hotfix ahead of the next releaseShip a sealed hotfix from production’s release, take it into main ahead of a sealed release, and re-test and seal that release again
Check a pull request in CIWrite and check the history bundle, merge main’s tip with the pull request’s head, check the data files, apply the migration to a scratch database, and annotate the findings
Gate merges on GitHub and GitLabRun the pull request checks in GitHub Actions and GitLab CI, tell a conflicted pull request what to do, compact to the open pull requests’ frontier, and deploy from a Seal
Work with row dataSee which rows the emit commands apply, apply a dataset, and write a data file’s rows by hand
FAQShort questions about the order import reads a migration tree in, each with a command to check it

Command reference

retrofitSchema evolution CLI
retrofit dataEvery data subcommand and schema ref-tables, their flags, records and refusals, and the data file format
retrofit data captureCapture a dump’s observed rows onto the data corpus
retrofit data carryCarry corpus records whose anchor was retired or sealed past onto their successor
retrofit data checkCheck data-corpus files against the data-file grammar
retrofit data listList the data fold the emit verbs would apply
retrofit data repinRe-point one corpus data file’s anchor-op at an opid you name
retrofit data retireWithdraw a data chain by name, minting an empty floor at the head
retrofit data scaffoldMint an empty data-corpus file carrying declared metadata axes
retrofit data witnessWitness reference-data records: bind their bytes to the oplog
retrofit driftEvery drift flag, how it pairs a dump with the oplog, what it asks and skips, and the reference rows it captures
retrofit elementRead the live elements of a schema’s history
retrofit element historyHow the element history operand binds, which ops belong, the records it prints, and what a refusal leaves behind
retrofit element listEvery element list flag, the live set it prints, the ref each record names, and the warning it prints
retrofit identifyReport which cut a pg_dump --schema-only is at, and how it has diverged
retrofit importEvery import flag, the statements it recognizes and skips, the preserved DML, and what a refusal leaves written
retrofit initInitialize a new retrofit project
retrofit migrateEvery migrate flag, the shape of its output, the envelope, segments and lock timeout, and the way out of a refusal
retrofit opEdit the oplog
retrofit op addAppend a new operator instance to the active oplog
retrofit op compactDrop the oplog’s traces (tombstones, replaces=) to a bound
retrofit op infoPrint an operator’s reference entry, or the whole catalog with --dump
retrofit op removeDrop one or more unsealed ops from the oplog
retrofit op revertAppend the reversal a down computes to the log
retrofit op revisePatch bound parameters on an unsealed op
retrofit op sealSeal the unsealed tail of the oplog
retrofit op showRender ops from the oplog
retrofit op squashReduce a range of the unsealed tail to its net effect
retrofit op truncateRewind the oplog to a checkpoint by removing a suffix of ops
retrofit op unsealDemote the most-recent N Seal record(s), keeping their ops as unsealed tail
retrofit opsRun bulk maintenance over the oplog
retrofit ops upgrade-sql-bodiesUpgrade LANGUAGE sql opaque function bodies to BEGIN ATOMIC
retrofit reconcileHow reconcile orders a fork, its bias and strictness, Seals in a fork, and each refusal a merge produces with the command that clears it
retrofit schemaManage registered schemas
retrofit schema createCreate a new oplog and register the schema
retrofit schema database-atWrite, clear or print the cut the local database is at (the apply-risk pin)
retrofit schema engineRead or set a schema’s engine version (production pin + emit target)
retrofit schema formPrint or switch the oplog’s form line (merge or compact)
retrofit schema listList registered schemas (name, path, engine version, default marker)
retrofit schema moveMove a schema’s oplog to another directory
retrofit schema ref-tablesRead or edit a schema’s reference-data capture policy
retrofit schema ref-tables addDeclare one or more tables as the schema’s reference data
retrofit schema ref-tables removeUndeclare one or more of the schema’s reference-data tables
retrofit schema removeRemove a registered schema (deletes the oplog file by default)
retrofit schema scopeManage [directory_scope] entries
retrofit schema scope addAdd a [directory_scope] entry
retrofit schema scope removeRemove a [directory_scope] entry
retrofit schema set-defaultSet or clear default_schema in retrofit.toml
retrofit snapshotEvery snapshot flag, the order of its output, the data band, and what the output leaves out
retrofit validateValidate retrofit.toml, project layout, and the oplog

Reference

OperatorsEvery operator, its category, and which commands handle it
Output contractWhat each command prints, on which stream, in which shape, and its exit code
Advisory codesEvery advisory code, its class and its register
Refusal codesEvery refusal code, its kind, its exit status and the commands that print it
pg_dump parityHow snapshot’s output covers each pg_dump --schema-only option
PostgreSQL versionsWhich PostgreSQL majors are supported, and what each verb does at each one
Versions and compatibilityHow builds are numbered, what may change between releases before 1.0, and the header lines that decide whether a build reads a log
History checksThe history bundle’s members, each history check and what fails it, the record validate writes, and what each git and host path gives
GlossaryEvery term retrofit coins, one sentence each, and the section that explains it

Release notes

Release notesWhat each release changed

Before 1.0, any release may change behavior; the release notes list what changed.

Published .