retrofit
0.14.0
retrofit.sh

Merge branches that change the oplog

Two branches that each change an oplog meet when one is integrated into the other. This page brings main into your branch: resolving conflict markers, the three-file merge, a property both sides set, jj, bisecting over the commits a rebase replayed, squashing your work before each commit, and a pull request GitHub reports as conflicted.

The examples use one schema, sales, whose oplog is sales.oplog at the repository root. Resolve on your branch, so main only receives an oplog validate passes. When two branches change the oplog explains the fork a merge leaves, and The two forms the oplog’s merge and compact forms. A hotfix shipped to production ahead of a sealed release has a page of its own: Ship a hotfix ahead of the next release.

Resolve conflict markers in the oplog

retrofit init writes *.oplog merge=union into .gitattributes, so wherever git merges locally (a merge, a rebase, a pull, a cherry-pick) both sides’ new lines are kept without a conflict, and validate reports the fork they leave. Conflict markers reach the oplog where the merge does not read .gitattributes: a host that merges the file as text, its web conflict editor, or jj. No command reads past a marker; each names every hunk and says how to resolve it in the file’s form.

In the merge form, keep both sides:

  1. Delete the marker lines (<<<<<<<, =======, >>>>>>>) and any base section (from ||||||| or ------- to the next marker line). Keep every operation line from both sides.
  2. Run retrofit validate. It reports the fork the two sides leave.
  3. Close the fork with main’s operations first: after a merge of main or a rebase onto main, write the oplog from the three files. Where they are not at hand, close it in the one file, as Reconcile after a merge shows.

Keeping both is the resolution in the merge form, revisions and removals included: each operation’s fate is read from its trace line. Deleting either side’s operation lines drops that side’s operations from the log. Closing the fork in the one file can write your branch’s value for a property over main’s with no report, where the three files stop on it.

One case needs an edit after keeping both: an operation both sides revised, or one revised and the other removed. Every command then stops and names both lines, and its hint names the edit that leaves the operation one fate. Make it, then run validate and close the fork as above.

In the compact form, keep both only for a hunk where both sides only appended. Where a side revised or removed a line inside the hunk, write the file from the three files instead (The two forms says why).

Bring main in with the three files

The three-file reconcile merges three versions of the oplog per opid: where your branch forked from main, main at its tip, and your branch before this integration. Use it after a merge of main and after a rebase onto main. After either, ORIG_HEAD holds your branch’s tip from before the integration, and still does while a merge stops on a conflict. The three inputs are three git show lines, and the result is the schema’s oplog:

git merge --no-ff --no-commit main   # bring main in; ORIG_HEAD is your branch before it
t=$(mktemp -d)
git show "$(git merge-base ORIG_HEAD main):sales.oplog" > "$t/base.oplog"   # where your branch forked
git show main:sales.oplog > "$t/main.oplog"                                 # main at its tip
git show ORIG_HEAD:sales.oplog > "$t/branch.oplog"                          # your branch before this integration
retrofit reconcile --base "$t/base.oplog" --ours "$t/main.oplog" \
  --theirs "$t/branch.oplog" --result sales.oplog
retrofit validate

Pass main as --ours, so main’s operations keep their places, and the schema’s oplog as --result, every time. retrofit reconcile has every flag, and the CLI output contract what each exit of the three-file merge covers and the JSON record a CI step reads.

When the merge succeeds, it has written the oplog: stage it and commit the merge. In a merge already in progress, skip the first line. When it stops, it writes nothing, so the oplog still holds git’s merge of the two, which validate may pass; resolve what it names before you commit. The usual stop is a property both sides set: Settle a property both sides set.

After a rebase, run the lines without the first, and commit the result as a new commit on top of the replayed commits, not amended into the last of them. The history checks then see what the rebase dropped. Use the three files, too, where keeping both sides cannot resolve what a rebase left: a compact-form hunk where a side revised or removed a line, jj’s diff style, or a stop whose hint names the three-file merge.

Settle a property both sides set

When main and your branch each set one property, the three-file merge stops and names, for --keep and for --swap, the value the property ends as and the operation written last. Run the merge again with --keep, which keeps main’s operations in their places. Where both sides appended, --keep writes your branch’s value after main’s: then run the remedy the message names after --keep, with -f naming the schema’s oplog, so the property takes main’s value. The remedy is most often an op revise, but can be an op remove or an op add. Where --keep already ends the property with main’s value, as when your branch revised an operation in place and main set the property after it, skip the remedy:

retrofit reconcile --base "$t/base.oplog" --ours "$t/main.oplog" \
  --theirs "$t/branch.oplog" --result sales.oplog --keep
retrofit op revise -f sales.oplog <opid> <key>=<value>   # the remedy the refusal names after --keep
retrofit validate

Where the refusal names your branch’s value after --keep and no remedy, --keep has still written your value over main’s. Before you commit, revise your operation to main’s value, or remove it where that leaves main’s value standing, and run retrofit validate. If neither validates, do not commit the merge. Under --as=json, the refusal gives that remedy as an empty string (CLI output contract).

Then commit the merge. Leave --swap aside: where both sides appended, it writes your branch’s operations before main’s, which the pull request’s history checks fail. The exception is a Seal in your branch’s block, which is written first, so --keep stops and the message names --swap alone. Run the merge with --swap then, and where it ends the property with your branch’s value, the remedy the message names for it. The history checks report that forced move and pass.

A merge whose order is forced (only one order replays, or a Seal names it) and whose written order reports overrides stops the same way, with fork-forced-order, until you name the forced order’s flag. Decline it by leaving the merge uncommitted; the refusal writes nothing.

Set up reconcile as jj’s merge tool

jj does not read .gitattributes, so every concurrent oplog change a jj user merges comes out as conflict markers, and jj’s default diff style writes one side as a diff from the base, which deleting the markers cannot keep. Set the three-file merge up as jj’s merge tool instead:

# ~/.config/jj/config.toml
[merge-tools.retrofit]
program = "retrofit"
merge-args = ["reconcile", "--base", "$base", "--ours", "$left", "--theirs", "$right", "--result", "$output"]

[merge-tools.retrofit-main-right]
program = "retrofit"
merge-args = ["reconcile", "--base", "$base", "--ours", "$right", "--theirs", "$left", "--result", "$output"]
jj new main feature                       # main first: $left is main
jj resolve --tool retrofit sales.oplog

jj passes the rebase destination as $left in a rebase, and the first parent in a merge. Name main first (jj new main feature, or a rebase onto main) and use retrofit. A merge that names your branch first (jj new feature main) passes your branch as $left and main as $right; use retrofit-main-right there, which passes $right as --ours, so main is the upstream side either way. When the merge stops, the conflict stays in place, and nothing is written. A fork-forced-order stop reads as the write-write stop does: the conflict stays in place until the tool is run with the flag that names the forced order.

Bisect over the commits a rebase replayed

A rebase replays each commit of your branch under union, and reconcile runs at the tip, so a commit between main and the tip may hold a fork or the residue of one. git bisect run meets those commits; skip every commit validate fails and test the rest:

#!/bin/sh
# git bisect run ./bisect.sh
retrofit validate >/dev/null 2>&1 || exit 125   # a commit validate refuses: skip it
exec ./run-the-test.sh

Only the commit that reaches main needs to validate, so a commit in between that validate fails is union residue the test cannot run on.

Squash your work before each commit

op squash --unshared <file> nets only the operations of the unsealed tail that the file does not hold, and compacts to the same bound. Given the last commit’s oplog, it reduces the work since that commit, wrong turns included, and leaves no trace of it in either form. It never nets, re-mints or removes an operation the file holds, so main’s unsealed operations and your committed ones stay as they are.

Run it from a pre-commit hook. The hook stands aside while git concludes a merge, cherry-pick, revert, rebase or am: the last commit’s oplog then lacks operations already committed elsewhere, which the squash would net as new. It runs only when the oplog is staged, so a commit that leaves the log alone never meets it, and stands aside when the oplog also has unstaged edits, which staging the squashed file would add to the commit:

#!/bin/sh
# .git/hooks/pre-commit
g=$(git rev-parse --git-dir)
for f in MERGE_HEAD CHERRY_PICK_HEAD REVERT_HEAD rebase-merge rebase-apply; do
  [ -e "$g/$f" ] && exit 0              # git is concluding an integration: stand aside
done
git diff --cached --quiet -- sales.oplog && exit 0   # the oplog is not staged
git diff --quiet -- sales.oplog || exit 0            # unstaged edits too: commit it as staged
[ -f sales.oplog ] || exit 0                         # the oplog was removed
tmp=$(mktemp) || exit 1
trap 'rm -f "$tmp"' EXIT
git show HEAD:sales.oplog > "$tmp" 2>/dev/null || : > "$tmp"
retrofit op squash --unshared "$tmp" && git add sales.oplog

A database that applied operations the hook then nets is a working copy, rebuilt with its rows saved first, so a developer who applies as they go may leave the hook out. On a forked oplog op squash stops, and the commit with it, until reconcile closes the fork.

Resolve a conflicted pull request on GitHub

GitHub’s pull request merges do not read .gitattributes (GitHub community discussion 9288), so merge=union does not apply there. Two pull requests that each append to an oplog conflict as text once the first merges, since both appended after the same line, and the merge button stays disabled until the branch is updated. GitLab’s merges honour merge=union: there the same two merge requests merge without a conflict and leave a fork, which validate in the pipeline fails.

Resolve it on the branch, where union applies:

  1. Bring main into the branch locally, by a rebase onto main or a merge of main, and write the oplog from the three files. After a rebase, commit the result on top of the replayed commits, not amended into the last of them.
  2. Where the three files are not at hand, close the fork in the one file, as Reconcile after a merge shows.
  3. Run retrofit validate; once it passes, commit the oplog and push the branch.
git rebase main                 # union keeps both sides' lines; ORIG_HEAD is your branch before it
t=$(mktemp -d)
git show "$(git merge-base ORIG_HEAD main):sales.oplog" > "$t/base.oplog"
git show main:sales.oplog > "$t/main.oplog"
git show ORIG_HEAD:sales.oplog > "$t/branch.oplog"
retrofit reconcile --base "$t/base.oplog" --ours "$t/main.oplog" \
  --theirs "$t/branch.oplog" --result sales.oplog
retrofit validate               # passes
git commit -m "Bring main's operations into the oplog" sales.oplog   # a new commit on top

When the merge stops, settle it as in Settle a property both sides set. Keeping both sides in GitHub’s web conflict editor leaves the fork on the branch, where validate fails it until reconcile closes it.

Before 1.0, any release may change behavior; the release notes list what changed.

Published .