retrofit
0.15.0-wip.89
retrofit.sh

These docs are for 0.15.0-wip.89, a WIP, and may describe what no release does yet. The newest release is 0.14.0: this page for 0.14.0.

Ship a hotfix ahead of the next release

Production runs release 6. Release 7 is sealed on main and applied to a test database, and production needs a fix now, before release 7. This page takes the fix from production’s release to production, brings it into main ahead of release 7, then re-tests release 7 on top of it and seals it again.

The oplog is the log of operations (ops) your schema is recorded in. A Seal, written by op seal, marks the ops before it as the team’s stable range (Sealed and unsealed ops). Each op and each Seal has an opid, the 12-character hex id the commands print. The examples use one schema, sales, whose oplog is sales.oplog. Release 6’s Seal is c00a2d6a0f6a and release 7’s is 6653dfb40f55; your opids differ.

Use this page only while release 7 is on test databases. Once a production database has applied release 7, do not demote its Seal as below; keeping two release lines is not covered here.

Check where production is

pg_dump --schema-only "$PROD_URL" > prod.sql
retrofit identify prod.sql
at seal c00a2d6a0f6a: release 6

identify reads the schema dump and prints the Seal it matches, with the reason the Seal was given. Production at release 6’s Seal is where this page starts.

Branch the hotfix from production’s release

git switch -c hotfix release-6   # the commit production's release was built from

Branch from the commit production’s release came from, tagged release-6 here. The branch’s oplog then ends at release 6’s Seal.

Write the hotfix and mark it stable

retrofit op add AddColumn orders paid_at timestamptz
retrofit op seal --reason "hotfix 6.1"
git commit -am "hotfix 6.1"

Add the fix as ops, as on any branch, then seal it before production takes it. op seal prints the new Seal’s opid, c24acb91ca6b here: the hotfix Seal.

An unsealed hotfix is written after release 7 when main takes it, behind ops production never applied (When two branches change the oplog).

Ship the hotfix to production

(
  set -eo pipefail   # bash: any failure stops the subshell
  retrofit identify prod.sql --expect c00a2d6a0f6a
  retrofit migrate --from c00a2d6a0f6a --to c24acb91ca6b \
    | psql -X -v ON_ERROR_STOP=1 "$PROD_URL"
)

Name both ends of the migration. On the hotfix branch the latest Seal is the hotfix Seal, which is where migrate starts by default. The --expect line stops the subshell unless production is at release 6’s Seal. Where you keep production’s position in a file, as Where each database is shows, write the hotfix Seal’s opid back once the migration is applied.

Take the hotfix into main

git switch main
git merge --no-ff --no-commit hotfix
retrofit validate    # reports the fork the merge leaves
retrofit reconcile   # names both Seals and a command for each order; writes nothing

The merge keeps both branches’ new lines in sales.oplog, so release 7’s ops and the hotfix’s both follow release 6’s Seal: a fork. Both sides end in a Seal, so reconcile writes nothing until you say which release stands. Its report says which side each order writes first and ends with a command for each order.

Run the command that names release 7’s Seal. It writes the hotfix first:

retrofit reconcile --clean --swap --allow-sealed-rewrite=6653dfb40f55
retrofit validate
git commit -am "Take hotfix 6.1 into main ahead of release 7"

The hotfix Seal stands, and release 7’s Seal is demoted: the report prints Demoted Seal 6653dfb40f55 (release 7, range c00a..d39f): its ops are unsealed again. Release 7’s ops follow the hotfix Seal unsealed, so op revise can change them for the hotfix. Refusals after a merge has the other orders a Seal allows.

Where release 7’s ops do not apply after the hotfix, as when the hotfix adds a column release 7 also adds, --clean writes nothing and names the rule the order breaks. Write the order anyway with --accept-dirty, then revise or remove each release 7 op the report names:

retrofit reconcile --accept-dirty --swap --allow-sealed-rewrite=6653dfb40f55
retrofit op remove 22dfba5e4ba6   # the release 7 op the report names
retrofit validate

Use the three-file reconcile instead

To write the oplog from the three files of the merge, as Bring main in with the three files does, pass main as --ours, the hotfix branch as --theirs, and the same two flags:

git merge --no-ff --no-commit hotfix
t=$(mktemp -d)
git show "$(git merge-base main hotfix):sales.oplog" > "$t/base.oplog"   # release 6
git show main:sales.oplog > "$t/main.oplog"                              # release 7
git show hotfix:sales.oplog > "$t/hotfix.oplog"                          # the hotfix
retrofit reconcile --base "$t/base.oplog" --ours "$t/main.oplog" \
  --theirs "$t/hotfix.oplog" --result sales.oplog \
  --swap --allow-sealed-rewrite=6653dfb40f55
retrofit validate

Then commit the merge.

Re-test the next release and mark it stable again

A test database that applied release 7 holds its ops without the hotfix before them. Rebuild it at the hotfix Seal, from a copy of production or from snapshot into an empty database, saving any rows you need first. Then apply what production takes next:

retrofit snapshot --at c24acb91ca6b | psql -X -v ON_ERROR_STOP=1 "$UAT_URL"   # an empty database
retrofit migrate --from c24acb91ca6b | psql -X -v ON_ERROR_STOP=1 "$UAT_URL"

Test release 7 there. Revise its ops with op revise where the test calls for it, and rebuild and apply again. When it passes, seal it:

retrofit op seal --reason "release 7"
git commit -am "release 7"

op seal prints release 7’s new Seal opid, 2097568a2bce here. Here, where the hotfix adds orders.paid_at, identify on a test database that applied release 7 before the hotfix reports the release 7 ops it holds and the hotfix op it lacks. identify reads only the schema, so rebuild every such test database, even one it matches to a Seal.

Confirm production and ship the next release

pg_dump --schema-only "$PROD_URL" > prod.sql
retrofit identify prod.sql
at seal c24acb91ca6b: hotfix 6.1

On main, production is at the hotfix Seal. Its next release starts there:

retrofit migrate --from c24acb91ca6b --to 2097568a2bce \
  | psql -X -v ON_ERROR_STOP=1 "$PROD_URL"

Before 1.0, any release may change behavior; the release notes list what changed.

Published .