Changelog

All notable changes to retrofit are recorded here, newest first, in the categories of Keep a Changelog. Before 1.0 retrofit promises no compatibility: a change is disclosed here, not prevented.

[Unreleased] records what the builds since the last release changed; a release renames it to its version. Its fixed Log format heading records every change to the oplog’s vocabulary, parameters, headers or seal bytes: what changed and what a user does, usually regenerating the log. CI fails a commit that changes internal/oplog/catalog/*.golden.json, internal/oplog/serialize/ outside its tests, or internal/seal/testdata/leafbytes_*.golden but adds no item under that heading itself (make logformat-check); with make hooks, the commit-msg hook refuses it at git commit. A commit whose serializer edit changes none of the vocabulary, parameters, headers or seal bytes may instead end its message with a Log-format: unchanged trailer; a reader refusing a log it used to accept is a format change. A change to either set of goldens always needs the item.

[Unreleased]

Added

• This change log. CI requires a “Log format” entry for any change to the oplog catalog golden, the serializer or the seal’s leaf-byte goldens. A serializer edit that leaves the format unchanged may say so with a Log-format: unchanged commit trailer instead.
• Each release archive carries a man page per command and the bash, zsh and fish completion scripts, and the Homebrew cask installs them, so man retrofit-migrate works after brew install.
• migrate and snapshot take --allow-unnamed-label <label> (repeatable). It attests that a labelled dataset was left unnamed on purpose, such as a test-only fixtures label on a long-lived target. The apply-risk-data-exclusions warning no longer counts that dataset, and its hint names this flag beside --label. Each value names one exact label, so a dataset added later still warns. An attestation that answers nothing prints a note that it had no effect.

Fixed

• The dml-chain-near-miss warning a data mint raises compares a new chain only with existing chains of its own band (reference, seed or history). data capture --label fixtures on a table that also holds a cleanse chain no longer warns that a label drift forks the chain, or suggests re-running with the other chain’s label-set: a chain of another band is never the one a capture supersedes.
• data capture’s verdict line prints not comparable: once before the reason instead of twice, and data capture --dry-run rehearses a table it would capture as would capture:, not would captured:.
• migrate’s data-replace-prologue and data-replace-cascade warnings say a replaced table “is replaced by its full observed state” and replacing <table>, not “cleared” and “clearing”: a keyed replace upserts the observed rows and deletes only the keys the state lacks, and never empties the table.
• op add CreateIndex --help spells the per-key operator-class flag --key_opclass, which op add and op revise accept beside --key_opclasses. The misspelling --key_opclasse is refused.
• init against an existing .gitattributes prints a note naming each line it appends, instead of changing the file silently.
• A data file’s record marker line that leaves out a required token (kind=, seq= or the src= provenance) is refused naming the missing token, not as carrying an “unrecognized or mis-ordered” one. Every marker-line refusal (G-e) now spells the marker grammar as its remedy, and the shared repair hint names the data file; data check and the other corpus readers no longer append a line about a “sidecar”.
• identify and migrate --from-dump refuse from-dump-schema-absent for a dump of a database that holds neither the (non-public) schema nor any of its objects, and point at migrate --from root. They matched such a dump to the CreateSchema cut, so migrate --from-dump emitted a script without CREATE SCHEMA that failed when applied.
• identify and migrate --from-dump list the schema facets that did not pair (cut only: and dump only: lines in the detail) when the dump matches no cut but drift would plan nothing against the nearest one. The from-dump-no-match hint no longer sends the user to a drift --dry-run that reports 0 ops, and the from-dump-behind hint no longer says such a divergence was made outside the log; both offer the nearest cut by hand with migrate --from.
• retrofit completion <shell> prints the shell’s completion script. The root’s help listed it, but running it was refused as an unknown command, and the scripts’ completion requests were refused the same way.
• Every verb that loads the oplog refuses an operator this build does not know, a ~ tombstone included, naming the operator and the line, with the hint “upgrade the CLI to one that knows the operator; do not delete or edit the line”. The op verbs and validate refuse it as unknown-operator, except that validate keeps sealed-oplog-unparsable for one inside a sealed range. A renamed or split operator keeps its re-import or rename remedy.
• validate reports an unknown key once, naming its line, the key and the rules it leaves unverifiable, instead of one unverifiable row per rule for every op of the operator. No hint says to hand-fix the op. It also reports a header key this build does not know, and a retrofit-model-version newer than the build, as unverifiable (exit 2); an unsealed log with a newer model-version no longer passes.
• reconcile exits 2, not 1, for a file only a newer build can read: an unknown operator, a too-new retrofit-oplog-format or retrofit-model-version, an unknown header, and (as before) an unknown key. The JSON code tells them apart; a conflict marker, a repeated header or a refused fork stays exit 1.
• reconcile, the op verbs and op revise -i refuse an oplog that exists but cannot be read (for example mode 000) with oplog-unreadable (exit 2), printing cannot read oplog and the read error, instead of oplog-not-found (“no oplog”) or a bare command-error.
• validate refuses an oplog holding an op that needs a newer engine version than its engine-version pin, with unsupported-on-engine-version (exit 2), as migrate and snapshot do. A log whose pin was lowered by hand, as in a resolved header merge conflict, no longer passes the CI gate.
• op seal and identify refuse the same log with the same unsupported-on-engine-version (exit 2): op seal writes no Seal over it and identify resolves no cut. op info drops the note that said they did not check the engine-version gate.
• validate, migrate and the other commands that read the log refuse a CreatePartition carrying both partition_columns and partition_expression, as they already refused it on CreateTable and PartitionTable. Such a sub-partition used to replay, and migrate rendered its key from the expression alone, dropping the columns. A CreateTable carrying both with no partition_by is refused too, rather than read as a plain table with its key dropped.
• op add and the commands that read the log refuse an ExtractTable that moves every live column of a partitioned parent or a partition, with extract-partitioned-source. Such an extract ended the source while a parent’s partitions stayed live, or while a partition’s parent still named it, and the rendered DROP diverged the database from the log.

Changed

• op info shows, for each rule, what retrofit does to an op that breaks it at two moments: OP ADD (op add and the other commands that write an op) and READING THE LOG (migrate, snapshot, validate, op seal, identify), each refuses or lets through. --as json keeps when and adds enforcement: {"append": ..., "replay": ...}, derived from it.
• op info says READING THE LOG refuses for partition_cols_xor_expr on CreateTable, CreatePartition and PartitionTable, and for view-replace-superset on ReplaceView: the commands that read the log refuse an op that breaks them, which op info said they let through. --as json reports these rules with when: append+replay, enforcement.replay: refuses and binding: binding, no longer author-time-convenience.
• MergeTable’s rules list MergeShape on sources: the refusal it already raised, when a source’s columns differ from into’s or a source has a nextval default or no live column, now has a declaration.
• Advisory messages end without a period. The second sentence of data-motion-partition-not-pg-declarative and of move-column-down-refused, the advice, moved to the hint. A migrate --down script’s reverse-unrecorded line ends ; the database sits at <opid>. The five witness warnings on a record the down reverts state their cause instead of repeating their code, and data-motion-source-consumed, dml-body-partition-dependent, data-only-single-statement-autocommit and the two drift-skip warnings are reworded in plain terms.
• data-loss-on-apply messages end without a period too: the reasons op add, migrate and migrate --down build for a drop, a cast, a join, an unjoin, a matview replace and a lookup extract or inline. Drift’s extract-recognition diagnostics read “an ExtractTable”.
• The Advisory codes page shows every form of a message that varies with a field, each led by the value that selects it, and follows a message that is only a placeholder with its detail and hint.

Removed

• LayerSeparation and NotSealed no longer appear in any operator’s rules: retrofit never checked either. Layer separation is a property of the oplog’s slots (docs/concepts.md, “The oplog”); the sealed region is guarded by sealed-region-write.

Log format

• Every oplog names its form on a fourth header line, # retrofit-oplog-form: merge or # retrofit-oplog-form: compact. A log without the line is refused (oplog-form-header-missing), as is any other value (unknown-oplog-form). In the merge form, which schema create writes unless --form compact is given, a removed or revised operation stays in the file as a ~ tombstone or behind a replaces= list; op compact drops those traces. Every read checks each live Seal over its chain, so a Seal whose digest no longer matches is refused. For an existing log, add # retrofit-oplog-form: compact as line 4, after # engine-version:, then run retrofit schema form merge to keep the history from then on.
• op show’s runnable framings print CreateIndex’s key_opclasses as --key_opclass, not --key_opclasse. The log’s bytes are unchanged; re-run op show for any copied --key_opclasse line.
• A # retrofit-* header key this build does not know is refused on every verb that loads the oplog (unknown-key, writes-blocked-on-unknowns or render-blocked-on-unknowns by the verb’s role), naming the key and its line, instead of being carried through opaquely. Run a build that knows the header; do not delete it.
• ExtractTable declares the rule extract-partitioned-source: an extract that moves every live column refuses a partitioned parent or a partition as its source. A log holding such an extract no longer replays. Regenerate the log, running DepartitionTable or DetachPartition before the extract, or leave a parent’s partition key on it so the source survives.
• AttachPartition ends the partition’s own columns into its parent’s, and DetachPartition mints the detached table’s columns and CHECKs afresh, recording their IDs in its new optional column-ids and adjacent-ids. SetDefault, DropDefault, AddNotNull, DropNotNull and SetColumnComment take an optional partition (partition=<table>, or the on-column spelled <partition>.<column>) that scopes the op to one partition of the column’s family. New refusals: partition-not-beneath-owner, partition-not-null-inherited, partition-identity-absent, detach-id-list-arity and attach-partition-check-mismatch. A log whose ops name an attached partition’s own columns, or a detached table’s columns by their earlier IDs, no longer replays: regenerate any log that holds an AttachPartition.