Principles

The rules retrofit upholds. They explain why the tool behaves the way it does, and why a few things you might expect to be configurable aren’t.

Each is enforced in code, tests or review.

For the mental model the principles act on, see concepts.md. For a hands-on tour see getting-started.md.

1. Rows in the target database are the asset

The tool emits schema changes for live databases, whose rows have to survive the change. Four things follow.

• “Data” means rows in the target database. Not bytes in the oplog. The oplog is the change recipe you’re authoring. Friction is reserved for the boundary where retrofit’s output reaches a database.
• The tool doesn’t apply DDL itself. snapshot emits canonical SQL to stdout; whether/how it reaches your database is your pipeline’s responsibility. Defaults at the emission boundary are non-destructive: --include-drop=none (the default) emits CREATE only; destructive prologues are explicit opt-ins via --include-drop=scoped or =schema.
• Change operators preserve element identity. RenameColumn, CastColumn, MoveColumn, ExtractTable/JoinTable, CopyColumn all bind to the element’s ElementID, not its name. Foreign keys, indexes, and views referencing that ElementID don’t need updating across the change.
• Lossy authoring surfaces a data-loss-on-apply advisory. Each shipped operator carries an information-preservation classification: additive, preserving, or lossy. When op add appends a lossy op (DropTable, DropColumn, DropView, CastColumn with an explicit using= clause, …) it emits a structured note: data-loss-on-apply: … advisory naming the affected element. The op is still appended: the oplog is not the database, and the unsealed tail is freely revisable (principle #3). The advisory exists so you see what the apply will do before the apply happens.

2. Never silently ignore: fail or warn

If retrofit doesn’t understand something, you find out.

• Unknown DDL surfaces as a structured skip-manifest entry, not a best-effort guess.
• Unknown enum values error out at DDL-emit time, not silently drop the clause.
• Unknown vocabulary (or a future model version) is named, not skipped: validate fails or ends unverifiable, and never passes the log as fully validated.
• Ambiguous in-body column references error with candidates listed; ambiguous unresolved references in import route to the skip-manifest with a warning.
• Unknown flag values fail before the verb runs.

If you see retrofit do something quietly that you didn’t ask for, it’s a bug.

3. Sealed = immutable; unsealed = freely revisable

The oplog has two regions and two completely different sets of rules.

• Sealed range is locked. The seal pins a Merkle digest over the sealed ops’ canonical bytes. Editing a sealed op in place fails validate with seal-digest-mismatch. Sealing is the act that promises “this is what production saw.”
• Unsealed tail is a developer scratchpad: op revise, op remove and editor-buffer line deletion are all by design. Friction here would fight the workflow with no asset to protect.
• A seal detects edits; it does not prevent them. import --force re-bootstraps the whole oplog, sealed prefix included, with fresh opids, but only when --allow-sealed-rewrite=<seal opid> names each Seal it replaces, and the run reports each one. A hand edit inside a sealed range fails verification at the next validate. Witnessed reference data (retrofit data witness) takes the same posture: an edited sealed reference record fails the next validate with data-witness-mismatch, and snapshot and migrate refuse to emit with the same code.
• Recovery from accidental oplog edits is git, not retrofit. The oplog is a text file in your repo, so your version control is the recovery surface. There are no backup files or undo logs.

4. Identity is minted once and durable

Every addressable thing in retrofit has an immutable ID.

• Opids are minted when an op is constructed: a drawn event identity, not a function of the line. The same act performed twice is two ops with two ids, because it happened twice. An id, once minted, is never re-minted for an unchanged line: reorder it, rewrite its prev=, rebase it through reconcile, revise it back to a value it already carries, and it keeps the id it was born with.
• ElementIDs are minted at element creation and stay fixed for the element’s lifetime (across renames, casts, extracts, moves). They are not reused.
• Param order is append-only. A new optional parameter is added at the end of an operator’s signature; a shipped signature’s emission order is frozen, so the bytes a given op writes out today are the bytes it writes out after the catalog grows. Once a catalog signature ships, it’s frozen.
• Genealogy vertices are content-addressed. The replay forest is reproducible from the oplog alone.
• No producer may mint ops to shore up a partially defined operator. An operator whose outcome renders as different DDL under snapshot and migrate is still one op. The identities of the elements it derives are recorded on that op, and their shape is derived at render time. No producer appends sibling ops to complete it: a sibling can be hand-deleted, which would leave the contract half-applied. No runtime flag marks these operators; the rule binds whoever defines one.

Durable identity is what makes the change operators expressible.

5. Determinism end-to-end

The same oplog produces the same outputs forever.

• Canonical byte form for every operator. Two retrofit binaries on two machines compute identical bytes for the same op.
• Sorted iteration anywhere bytes leave the process. No map-iteration reaches stdout, the snapshot, or a digest.
• Replay is total: given the same oplog prefix and target opid, every consumer materializes the same schema.

This is what makes the snapshot a build artifact you can throw away and recreate, and what makes digests trustworthy.

6. Atomicity at every write boundary

A failed command never leaves you with a half-written file.

Every oplog append, config write, and registry edit goes through an atomic write path: tempfile → fsync → rename (the atomic commit point) → fsync directory (best-effort). If the process is killed, the file on disk is either the old version or the new version. Never partial. The old-or-new guarantee rests on the atomic rename, not the directory fsync.

7. Machine-parseable output is non-negotiable

Every byte retrofit prints is meant to survive grep, jq, awk, and screen readers.

• One output gateway. Direct prints from verbs are CI failures.
• NDJSON under --as=json: zero-or-more {"type":"<name>",...} intermediate records followed by exactly one terminal {"type":"ok"|"failed",...} record. One discriminator field (type) names every record.
• Errors land on stdout (not stderr) so one parser handles success and failure.
• No color codes. No emoji. No --no-color flag. -q/--quiet and -v/--verbose are persistent root flags that control only human text advisories on stderr: -q suppresses the Courtesy tier, stacking it (-qq) drops the whole advisory tier, and verbose reveals the Info tier. This advisory axis is orthogonal to --as=text|json; --quiet never suppresses errors, results, or structured JSON, at any level.
• Error codes are wire format. Once a clierr.CLIError code ships, renaming or removing it is a breaking change.

The full contract is in cli-output-contract.md.

8. Comments and hand-edits are preserved

When it edits retrofit.toml or .gitattributes, files you also edit by hand, retrofit changes them line by line rather than re-serializing them, so your comments and formatting survive; only runs of trailing blank lines may normalize.

9. .gitattributes merge=union is a runtime invariant

Applies when the working tree is a git checkout. Outside a git tree, the check is skipped (no merge surface to protect). Concurrent appends from two branches need the line *.oplog merge=union text eol=lf in .gitattributes at merge time, not at write time. When retrofit detects an ancestor .git directory it checks for the line on every config load; the oplog-mutating and registry verbs (op add/op seal, schema create/schema remove) surface a gitattributes-missing warning when it’s absent; re-run retrofit init to restore the line.

The warning does not change the exit code.

A VCS is never executed: retrofit relates to git only through the files it writes (.gitattributes) and passive detection of an ancestor .git directory. It never shells out to git or any other version-control binary, so no git install is needed.

10. Single-schema projects work without ceremony

The most common onboarding path (one project, one schema) should have no friction.

• The first schema create auto-pins default_schema. Every schema-context verb then resolves to it without --schema.
• Multi-schema projects resolve via implicit oplog-dir (dirname(path)) or explicit [directory_scope] entries; the user types --schema only when the resolver chain can’t decide.
• The six-step resolver chain (see concepts.md § The resolver chain) is the single contract. Every schema-context verb runs it; every verb names the resolved schema in its output (stderr in text mode, terminal envelope in JSON).
• Ambiguity is never resolved silently. When the chain has no answer, the verb refuses with default-schema-unset. When CI needs to forbid silent fall-through, set RETROFIT_REQUIRE_EXPLICIT_SCHEMA=1.

11. Validation accumulates, then reports

Validators don’t stop at the first violation. A proposed op gets a single pass; all violations come back in one envelope. One iteration, one report.

When a request bumps into a principle

Most requests the project declines run into one of these:

If you hit one of these and have a reason it shouldn’t apply, that’s worth raising. It’s a design conversation, not a flag flip.