retrofit
Schema evolution for PostgreSQL: retrofit records each schema change in an append-only log of operators (the oplog) and emits canonical DDL and migration scripts from it. Start with Install, then Create a project.
Get started
| Install | Install retrofit with the install script, Homebrew or a release archive, and check that it runs |
| Create a project | Start a project from scratch or from existing SQL, read the oplog it writes, and emit DDL |
| What's next | Where to read after the first project, from the model to each verb’s reference |
Concepts
| The oplog | What retrofit records, how an op line reads, element identity, and the schema replay builds |
| Sealing and deploying | Sealed and unsealed ops, tracking where each database is, and finding and recording drift |
| Branches and merges | What a merge of two branches does to the oplog, and the two forms an oplog can take |
| Data files | Where rows live beside the oplog, what each purpose does, and how a stale reference is repaired |
| Kinds of operators | The kinds of change an operator records, and when import or drift writes the ops |
| How a command picks its schema | Schema identity, the resolver chain, and the three ways a schema is created on first use |
| Consents | The inputs that clear a refusal, the two kinds, and how each is spelled |
| How a migration applies | What a migration script assumes, who owns its transaction, why some scripts commit in parts, and how a reverse script is built |
| Principles | The rules retrofit upholds, and why some things are not configurable |
Guides
| Import existing SQL | Bootstrap an oplog from a schema file, a dump or migration scripts, preview it, seal it, and extend or start over |
| Import a migration tree in order | Pass a tree of migration files in the order they applied, check the order first, and seal each migration |
| Exclude objects the project does not own | Keep a provider’s tables, extensions and schemas out of the oplog with one retrofit.toml entry each |
| Work with several schemas | Register more schemas, scope them to directories, change the default, move an oplog, and pick a schema for one command |
| Record a schema changed outside the oplog | Dump a database changed outside the oplog, preview what drift records, answer its questions, and seal the result |
| Emit the schema as SQL | Emit the schema at a point in the oplog as SQL, with its reference and seed rows or without them |
| Find an element's ID and history | List a schema’s elements with their IDs, script over them, and read one element’s history, live or dropped |
| Emit a migration script | Emit the migration script between two points in the oplog, from where a database is, with seed rows, or in reverse |
| Choose who opens the transaction | Emit a script that opens its own transaction, or one your runner wraps, or one applied a statement at a time |
| Bound how long a migration waits for a lock | Make a migration statement give up on a lock rather than queue every later query behind it |
| Recover after a failed apply or a merge | Finish a migration that stopped part way, and reconcile the fork a merge leaves in the oplog |
| Merge branches that change the oplog | Resolve conflict markers, bring main in with the three-file merge, settle a property both sides set, and use jj, bisect and a pre-commit squash |
| Ship a hotfix ahead of the next release | Ship a sealed hotfix from production’s release, take it into main ahead of a sealed release, and re-test and seal that release again |
| Check a pull request in CI | Write and check the history bundle, merge main’s tip with the pull request’s head, check the data files, apply the migration to a scratch database, and annotate the findings |
| Gate merges on GitHub and GitLab | Run the pull request checks in GitHub Actions and GitLab CI, tell a conflicted pull request what to do, compact to the open pull requests’ frontier, and deploy from a Seal |
| Work with row data | See which rows the emit commands apply, apply a dataset, and write a data file’s rows by hand |
| FAQ | Short questions about the order import reads a migration tree in, each with a command to check it |
Command reference
| retrofit | Schema evolution CLI |
| retrofit data | Every data subcommand and schema ref-tables, their flags, records and refusals, and the data file format |
| retrofit data capture | Capture a dump’s observed rows onto the data corpus |
| retrofit data check | Check data-corpus files against the data-file grammar |
| retrofit data list | List the data fold the emit verbs would apply |
| retrofit data reanchor | Re-anchor corpus records whose anchor was revised, removed or sealed past onto the op that now holds their place |
| retrofit data repin | Re-point one corpus data file’s anchor-op at an opid you name |
| retrofit data retire | Withdraw a data chain by name, minting an empty floor at the head |
| retrofit data scaffold | Mint an empty data-corpus file carrying declared metadata axes |
| retrofit data witness | Witness reference-data records: bind their bytes to the oplog |
| retrofit drift | Every drift flag, how it pairs a dump with the oplog, what it asks and skips, and the reference rows it captures |
| retrofit element | Read the live elements of a schema’s history |
| retrofit element history | How the element history operand binds, which ops belong, the records it prints, and what a refusal leaves behind |
| retrofit element list | Every element list flag, the live set it prints, the ref each record names, and the warning it prints |
| retrofit identify | Report which cut a pg_dump --schema-only is at, and how it has diverged |
| retrofit import | Every import flag, the statements it recognizes and skips, the preserved DML, and what a refusal leaves written |
| retrofit init | Initialize a new retrofit project |
| retrofit migrate | Every migrate flag, the shape of its output, the envelope, segments and lock timeout, and the way out of a refusal |
| retrofit op | Edit the oplog |
| retrofit op add | Append a new operator instance to the active oplog |
| retrofit op compact | Drop the oplog’s traces (tombstones, replaces=) to a bound |
| retrofit op info | Print an operator’s reference entry, or the whole catalog with --dump |
| retrofit op remove | Drop one unsealed op, or a contiguous range |
| retrofit op revert | Append the reversal a down computes to the log |
| retrofit op revise | Patch bound parameters on an unsealed op |
| retrofit op seal | Seal the unsealed tail of the oplog |
| retrofit op show | Render ops from the oplog |
| retrofit op squash | Reduce a range of the unsealed tail to its net effect |
| retrofit op truncate | Rewind the oplog to a checkpoint by removing a suffix of ops |
| retrofit op unseal | Demote the most-recent N Seal record(s), keeping their ops as unsealed tail |
| retrofit ops | Run bulk maintenance over the oplog |
| retrofit ops upgrade-sql-bodies | Upgrade LANGUAGE sql opaque function bodies to BEGIN ATOMIC |
| retrofit reconcile | How reconcile orders a fork, its bias and strictness, Seals in a fork, and each refusal a merge produces with the command that clears it |
| retrofit schema | Manage registered schemas |
| retrofit schema create | Create a new oplog and register the schema |
| retrofit schema database-at | Write, clear or print the cut the local database is at (the apply-risk pin) |
| retrofit schema engine | Read or set a schema’s engine version (production pin + emit target) |
| retrofit schema form | Print or switch the oplog’s form line (merge or compact) |
| retrofit schema list | List registered schemas (name, path, engine version, default marker) |
| retrofit schema move | Move a schema’s oplog to another directory |
| retrofit schema ref-tables | Read or edit a schema’s reference-data capture policy |
| retrofit schema ref-tables add | Declare one or more tables as the schema’s reference data |
| retrofit schema ref-tables remove | Undeclare one or more of the schema’s reference-data tables |
| retrofit schema remove | Remove a registered schema (deletes the oplog file by default) |
| retrofit schema scope | Manage [directory_scope] entries |
| retrofit schema scope add | Add a [directory_scope] entry |
| retrofit schema scope remove | Remove a [directory_scope] entry |
| retrofit schema set-default | Set or clear default_schema in retrofit.toml |
| retrofit snapshot | Every snapshot flag, the order of its output, the data band, and what the output leaves out |
| retrofit validate | Validate retrofit.toml, project layout, and the oplog |
Reference
| Operators | Every operator, its category, and which commands handle it |
| Output contract | What each command prints, on which stream, in which shape, and its exit code |
| Advisory codes | Every advisory code, its class and its register |
| Refusal codes | Every refusal code, its kind, its exit status and the commands that print it |
| pg_dump parity | How snapshot’s output covers each pg_dump --schema-only option |
| PostgreSQL versions | Which PostgreSQL majors are supported, and what each verb does at each one |
| Versions and compatibility | How builds are numbered, what may change between releases before 1.0, and the header lines that decide whether a build reads a log |
| History checks | The history bundle’s members, each history check and what fails it, the record validate writes, and what each git and host path gives |
| Glossary | Every term retrofit coins, one sentence each, and the section that explains it |
Release notes
| Release notes | What each release changed |