Check a pull request in CI
A pull request’s check runs five steps over a project: the history
checks over the commits the pull request brings, a merge of main’s tip
with the pull request’s head, data check on every schema, the
migration applied to a scratch database, and the findings turned into
the host’s annotations. Each step below is a shell script a CI job runs
at the project root, in a full clone, with retrofit and jq on its
path. Gate merges on GitHub and GitLab wires the steps
into each host’s jobs.
The steps read each schema’s oplog. Merge branches that change the oplog shows how a pull request brings main in and closes the fork a merge leaves.
Write and check the history bundle
retrofit validate --history <dir> runs the history checks over the
commits an integration made. It runs no git, so the job first writes
those commits into a directory, the history bundle
(The bundle lists its members). The
script reads three commits from its environment:
CANDIDATE: the commit the host tests, a pull request’s merge commit or a merge queue group’s commit.TARGET: the target branch’s tip.BEFORE: the push’s previous tip, when the host gives one. Leave it empty after a force push, when the old tip may not be in the clone. Without it, one compact-form check is reported unchecked.
Save the script in the repository, as ci/oplog-bundle.sh for one. It
writes every member but the commits’ files and the optional holders,
then runs validate --history until it names no file it lacks:
#!/bin/sh
# Write the history bundle for the commit under test, then check it.
# CANDIDATE and TARGET are required; BEFORE is optional; EVERY_COMMIT=1
# checks every commit of the pull request. Writes validate.ndjson.
set -eu
bundle=${BUNDLE:-$(mktemp -d)}
mkdir -p "$bundle/dead"
case "${BEFORE:-}" in *[!0]*) ;; *) BEFORE= ;; esac # all zeros: no before
echo 1 > "$bundle/version"
git rev-parse --is-shallow-repository > "$bundle/shallow"
{
echo "candidate=$CANDIDATE"
echo "target=$TARGET"
if [ -n "$BEFORE" ]; then echo "before=$BEFORE"; fi
} > "$bundle/roles"
git rev-list --parents "$CANDIDATE" "$TARGET" $BEFORE > "$bundle/graph"
git log --format='%H%n%B%x00' "$CANDIDATE" "$TARGET" $BEFORE > "$bundle/messages"
# The dead set, per schema: the target's tip, then the opid of every
# operation line (a ~ line and a Seal included) the schema's oplog held at
# any commit reachable from the target's tip.
prefix=$(git rev-parse --show-prefix)
retrofit schema list --as json > "$bundle.schemas"
jq -r 'select(.type == "item") | "\(.name) \(.path)"' "$bundle.schemas" |
while read -r name path; do
{
echo "$TARGET"
git rev-list "$TARGET" | while read -r c; do
if git cat-file -e "$c:$prefix$path" 2>/dev/null; then
git cat-file blob "$c:$prefix$path"
fi
done | sed -n -E 's/^~?[A-Za-z]+\(([0-9a-f]{12}).*/\1/p' | sort -u
} > "$bundle/dead/$name"
done
# Write each file validate names, and run it again until it names none.
while :; do
status=0
retrofit validate --history "$bundle" --allow-database-apply ${EVERY_COMMIT:+--every-commit} --as json \
> validate.ndjson || status=$?
jq -r '.data.history.missing[]?' validate.ndjson > "$bundle.missing"
[ -s "$bundle.missing" ] || break
while IFS= read -r m; do
if [ -e "$bundle/$m" ] || [ -e "$bundle/$m.absent" ]; then
echo "validate names $m again" >&2
exit 2
fi
rest=${m#commits/}; sha=${rest%%/*}; file=${rest#*/}
mkdir -p "$(dirname "$bundle/$m")"
if git cat-file -e "$sha:$file" 2>/dev/null; then
git cat-file blob "$sha:$file" > "$bundle/$m"
git rev-parse "$sha:$file" > "$bundle/$m.blob"
else
: > "$bundle/$m.absent"
fi
done < "$bundle.missing"
done
exit "$status"
The first run names the files the checks need, and the script writes each one and runs again until none is named. It branches on that list, never on the exit, and the last run’s exit is the step’s. The step fails when a check fails, or when the bundle cannot be trusted, as from a shallow clone; History checks lists each check and what fails it.
--allow-database-apply states that CI applies the range to a database
before it lands, which
the scratch database step
does on every push. With it, opaque-body lists a body the pull request
adds that names no element, and passes. A pipeline without that step
takes the flag out of the script, and the check then fails such a pull
request.
The dead set (dead/<schema>) is the one member the script derives,
from every commit reachable from the target’s tip, at the path the
candidate’s config records. For a schema whose oplog moved, read each
earlier path into its dead set as well.
Merge main’s tip with the pull request’s head
The history checks read only the files an integration wrote, so they
cannot see a property main set and the pull request set again where no
fork decides which value stands
(What each path promises).
This step runs the three-file
merge over main’s tip
and the pull request’s head, which stops on that conflict. Save it as
ci/oplog-three-file.sh, for one:
#!/bin/sh
# Merge main's tip and the pull request's head with the three-file
# reconcile, per schema, and fail on a refusal. TARGET and PR_HEAD are
# required. Writes nothing in the project.
set -eu
t=$(mktemp -d)
base=$(git merge-base "$TARGET" "$PR_HEAD")
prefix=$(git rev-parse --show-prefix)
retrofit schema list --as json > "$t/schemas"
jq -r 'select(.type == "item") | "\(.name) \(.path)"' "$t/schemas" |
while read -r name path; do
for c in "$base" "$TARGET" "$PR_HEAD"; do # absent at this path on a side: skipped
git cat-file -e "$c:$prefix$path" 2>/dev/null || continue 2
done
git show "$base:$prefix$path" > "$t/base.oplog"
git show "$TARGET:$prefix$path" > "$t/main.oplog"
git show "$PR_HEAD:$prefix$path" > "$t/pr.oplog"
echo "$name:"
retrofit reconcile --base "$t/base.oplog" --ours "$t/main.oplog" \
--theirs "$t/pr.oplog" --result "$t/result.oplog"
done
TARGET: main’s tip, or, in a merge queue’s group or a train’s car, the commit the host merges the pull request onto.PR_HEAD: the pull request’s head, the commit its author pushed, never the merge commit the host builds.
Each schema is read at the path the checkout’s config records, and a schema absent there on any side is skipped. For a schema whose oplog moved since the merge base, read that side’s earlier path instead.
The step fails when the merge stops. The job’s log then names the
property, the operation each side set it with, and, for --keep and
for --swap, the value the property ends as and the edit that undoes
it.
A stop other than a write-write is fork-forced-order: the merged order
is forced and reports overrides nobody asked for. The log names the fork,
why the order is forced and each override; the step fails until the merge
is run with the flag that names the forced order.
Run the step on every push to the pull request, and where the host
tests the merged result, on that: a merge queue’s group, a train’s car.
Run it also in a job on main’s push, once for each open pull request
that changes an oplog, with TARGET main’s new tip and PR_HEAD that
pull request’s head
(Check open pull requests when main moves).
The step reads the pull request’s files as data and runs nothing from
it.
Once a pull request has brought main in, the merge base holds main’s change, and the step has nothing to see. So bring main in with the three files, after a rebase as well as after a merge. Where the merge stops on a property both sides set, settle it as Settle a property both sides set shows; once that merge is committed, the step passes.
Check every commit of a pull request
By default only the candidate must validate. A commit between the fork
point and the candidate may hold a fork, since a rebase replays each
commit under union and reconcile runs at the tip. A team that merges
by fast-forward puts every commit of the pull request on main. To have
each one validate, under its own form line, set EVERY_COMMIT=1 on the
bundle script, which passes --every-commit:
retrofit validate --history "$bundle" --allow-database-apply --every-commit --as json
Check the data files
data check passes a record it only lists, so the step reads the
counts in the data-check record’s listed object. Fail the pull
request on a listed record whose row shape migrate cannot apply, the
two classes the script counts, since the deploy would stop on it. A
record with no row-shape stamp is one migrate applies; never fail on
it.
set -eu
retrofit schema list --as json > schemas.ndjson
jq -r 'select(.type == "item") | .name' schemas.ndjson |
while read -r name; do
retrofit data check --schema "$name" --as json > data-check.ndjson
jq -e 'select(.type == "data-check")
| (.listed["anchor-shape-mismatch"] // 0) + (.listed["anchor-shape-unsupported"] // 0) == 0' \
data-check.ndjson > /dev/null
done
A schema without a data root is an empty corpus, which data check
passes, so the step runs on every project. A file data check cannot
read fails the step on its own. The schemas are listed on a line of
their own, so a schema list that fails over a retrofit.toml it
cannot read stops the step rather than handing the loop no schemas. See
retrofit data check for what each
finding means.
Apply the migration to a scratch database
opaque-body lists a body a pull request adds that names no element,
and no history check reads what such a body uses. One that uses a column
another change removed, revised or dropped passes them, and fails where
the migration is applied. A pull request that adds no body can break one
main already holds, so run this step on every push. Apply the migration
to a throwaway PostgreSQL in the pull request’s check (a service
container the job starts) to find that, and any other body that does not
apply. It shows the migration applies to an empty database, not that it
applies to yours.
set -eu
retrofit schema list --as json > schemas.ndjson
jq -r 'select(.type == "item") | .name' schemas.ndjson |
while read -r name; do
retrofit migrate --schema "$name" --from root --to last > "migrate-$name.sql"
psql -X -v ON_ERROR_STOP=1 -f "migrate-$name.sql" "$SCRATCH_DATABASE_URL"
done
Each file is written before psql reads it, so a migrate that fails
stops the step rather than handing psql an empty script. To rehearse
the step a database deployed at the latest Seal
takes, build the scratch database to the Seal first, then apply the
rest, in place of the loop’s two lines:
retrofit migrate --schema "$name" --from root --to seal > to-seal.sql
retrofit migrate --schema "$name" --from seal --to last > after-seal.sql
psql -X -v ON_ERROR_STOP=1 -f to-seal.sql "$SCRATCH_DATABASE_URL"
psql -X -v ON_ERROR_STOP=1 -f after-seal.sql "$SCRATCH_DATABASE_URL"
On a schema with no Seal, --to seal fails; keep the loop’s two lines
there. Applying with psql
says why each psql line passes -X -v ON_ERROR_STOP=1.
Annotate the findings on GitHub or GitLab
validate --history --as json writes each failed check’s findings with
the file and line each names
(The record lists every field). An
opaque-body finding also names the line’s operator, the body’s
slot and, where one exists, its cause, beside the line and the
rewrite. Under --allow-database-apply that check is advisory, so its
findings stay in validate.ndjson and the filters below read none of
them.
GitHub reads ::error lines from a step’s output as annotations:
jq -r '
def esc: gsub("%"; "%25") | gsub("\r"; "%0D") | gsub("\n"; "%0A");
(select(.type == "validate-history") | .data.history.checks[]
| select(.status == "failed") | .check as $check | .findings[]
| "::error file=\(.path),line=\(.line),title=\($check)::\(.message + (if .recovery then "; " + .recovery else "" end) | esc)"),
(select(.type == "failed") | "::error title=\(.code)::\(.message + " (" + .hint + ")" | esc)")
' validate.ndjson
GitLab reads a code quality report, a JSON array the job names under
artifacts:reports:codequality:
jq -s '[.[] | select(.type == "validate-history") | .data.history.checks[]
| select(.status == "failed") | .check as $check | .findings[]
| {description: .message, check_name: $check,
fingerprint: "\($check):\(.commit):\(.path):\(.line):\(.opid // "")",
severity: "major",
location: {path: .path, lines: {begin: ([.line, 1] | max)}}}]' \
validate.ndjson > gl-code-quality.json
When validate stops before any check runs, it names no line to
annotate. On GitHub it is the second ::error line; on GitLab it is the
job’s log.
Related
- History checks: the bundle’s members, each check
and what fails it, and the record
validatewrites. - Merge branches that change the oplog: bring main in and settle what the merge stops on.
- Gate merges on GitHub and GitLab: the jobs that run these steps on each host.
retrofit validateandretrofit reconcile: every flag.