Changelog
All notable changes to retrofit are recorded here, newest first, in the categories of Keep a Changelog. Before 1.0 retrofit promises no compatibility: a change is disclosed here, not prevented.
[Unreleased] records what the builds since the last release changed; a
release renames it to its version. Its fixed Log format heading records
every change to the oplog’s vocabulary, parameters, headers or seal bytes:
what changed and what a user does, usually regenerating the log. CI fails a
commit that changes internal/oplog/catalog/*.golden.json,
internal/oplog/serialize/ outside its tests, or
internal/seal/testdata/leafbytes_*.golden but adds no item under that
heading itself (make logformat-check); with make hooks, the
commit-msg hook refuses it at git commit. A commit whose serializer edit
changes none of the vocabulary, parameters, headers or seal bytes may
instead end its message with a Log-format: unchanged trailer; a reader
refusing a log it used to accept is a format change. A change to either set
of goldens always needs the item.
[Unreleased]
Log format
- An
AddForeignKeywhose referenced columns are not, as a set, the columns of a primary key, a non-deferrable unique constraint or a unique index of the referenced table (not a partial or expression index) is refused asfk-referenced-key-missing, naming the foreign key, the table and the columns, as PostgreSQL refuses it at apply.import,op addandvalidateaccepted it, andsnapshotandmigratewrote DDL that failed. A referenced partition counts its parent’s keys; a reference to a table in another schema is not checked. A log holding one is refused wherever it is replayed to be checked or rendered: remove the foreign key (op remove) or revise it to reference a key (op revise), then regenerate the log or re-seal. The log’s bytes are unchanged. CreateTriggergainsfunction_external, a trigger function held by its schema-qualified name, and itsfunctionbecomes optional beside it: an operation sets exactly one. A trigger in one schema callinga.touch()used to bind alanguage=internalstub in its own schema, or a same-named function there, and renderEXECUTE FUNCTION touch(). It now bindsa.touchitself when schemaais registered and holds it, and renders"a".touch()under that function’s current name; otherwise it holds the name as written. A log written before keeps its stubs and replays unchanged: re-import the schema to rebind such a trigger.AddColumnandCastColumntaketype_ref=, the ElementID of another registered schema’s type that the column’stype=names, beside the spelling.import,drift,op addand anop reviseoftype=write it, so a column typed by another schema’s type follows aRenameTypethere with no operation in its own schema, andvalidaterefuses one whose type no registered schema holds. A log written before holds such a column by its spelling and reads as before; regenerate the log to bind it.CopyColumntakestype_ref=beside atype=override that names another registered schema’s type, written byop addand anop reviseoftype=, so the copy follows aRenameTypethere insnapshotandmigrateandvalidaterefuses one whose type no registered schema holds. Before, such an override was held by its spelling. A copy that inherits its source’s type keeps the source’s binding as before; a log written before holds an override by its spelling: revise itstype=to bind it.
Fixed
driftpairs a foreign key into another schema by its spelling: the schema, table and both column lists. An unchanged one used to plan its drop beside anadd-foreign-key-cross-schema-unsupportedskip at exit 0, and the nextdriftplanned nothing, so the lost key read as converged. A paired key renames, changesDEFERRABLEor validates in place; one whoseON DELETE,ON UPDATEorMATCHchanged is dropped and added again.driftadds a foreign key into another schema asimportrecords it, held by its qualified spelling. It used to skip the key asadd-foreign-key-cross-schema-unsupportedat exit 0, so a key the file gained, or one whose referenced table or columns changed, never reached the log; that skip kind is gone. A key whoseON DELETE SET NULL (…)orSET DEFAULT (…)names columns is still skipped, as one in its own schema is.
Known issues
- A project with several registered schemas is not supported yet. How a reference from one schema into another, a drop that another schema depends on, and the verbs that read more than one schema’s log should behave is still being designed, and each can go wrong today.
- Ending a column that a later
RunRawScriptbody names is refused only when the body is bound. For a body stored opaque,op removeorop reviseof the column, or a merge whose other side removed it, writes the log, andvalidateaccepts it.migratethen emits the script against a table without the column, so the migration fails whole at apply.validate --historyfails a pull request that adds such a body, unless the name sits only in a conditional branch or the body does not parse; it only advises on a body already on the target that the pull request leaves naming a removed column. Apply the migration to a scratch database in CI (retrofit migrate --from root --to last, or from the latest Seal). driftasks thereversalquestion only when the database holds something logged after the drop. A database that has not yet run the log’s newest drop reads as intent, and the plan re-adds the item with no question. Review a drift plan’s re-adds against the log before accepting them.- A released
transitionorcleansedata record, in a project that opts in to them, is protected only by its position in the log: an edit to one behind a Seal passesvalidate, andmigrateemits the edited bytes. Nor is a record the release shipped told apart from a branch’s backfill integrated behind it, which a database already past that point never runs. - Before 1.0 the oplog may gain operators and header keys. A build refuses
a log with a newer
retrofit-oplog-formator a key it does not know, so CI and every developer upgrade together.
[0.14.0]
Added
migratewarnsapply-risk-lock-held-across-stepwhen a table one op locks stays locked to the transaction’s commit while a later step runs whose length retrofit cannot bound, such as aRunRawScript, a data body or a scan: the table’s writers wait for that step too. It names the table, the op holding the lock and the step that lengthens it, and its hint says how to split the window.validate --fail-on-apply-risknames, for each op it counts, the tables the transaction already holds, and the two verbs agree on the holder, a fused cast run’s first cast included. The warning is counsel: it never fails the gate.DetachPartitionkeeps thePRIMARY KEY,UNIQUE,EXCLUDE, index and foreign-key copies PostgreSQL leaves on a partition it detaches, each under the name PostgreSQL gave it, where earlier releases lost them (see Log format).op addtakes a name it cannot derive as--kept-name <ancestor>=<name>, and a name that cannot be given refuses asdetach-kept-name-takenorcopy-clone-name-collision, inimportanddrifttoo.importbinds the statements after aDETACHto the kept copy;driftplans a detach over kept copies, dropping those the target lacks and adding the target’s own after theDETACH;snapshotrenders a re-attached copy under its name; and dropping the detached table frees the copies’ names. A copy with a key facet retrofit does not model (DEFERRABLE,INCLUDE) stays adriftskip rather than being planned without it.- A
CopyTableclone, anExtractTablechild (one copying the source’s key, or a lookup child) and anUnjoinTablechild with a key start from their source’s data records as they stood at the op, as every migrated database holds them.snapshotandmigrate --from rootfill such a product where they rendered it empty;driftanddata capturecompare a capture of it against those rows, where every capture read as new; andop sealno longer refuses it asreference-rows-uncaptureduntil it is captured. A source record that cannot be rewritten into the product (a populatedCOPY, anUPDATE,DELETEorMERGE, or a body that does not repair to the op) is refused bysnapshotandmigrateasstale-dml-referenceunder the reason it names, marked stale indata list, and listed bydata checkas the newrouted-body-unroutablewarning.snapshotandmigratewarnrouted-floor-key-lostwhen a later capture of anExtractTablesurvivor drops a key the child’s rows still reference, which fails at apply.
Changed
validate --fail-on-apply-risknow also counts a column-type change (CastColumn) that rewrites the table, or scans it for an index or check, under a new code,apply-risk-window-table-rewrite, when the recovery retrofit can express (expand/contract for a rewrite, a drop and re-add for a scan) applies to it; a cast it cannot vouch for stays advisory. An existing user of the flag can see the gate fail on upgrade;migrateandvalidatewithout the flag are unchanged.- The refusal
arg-bindingis nowarg-shape: arguments that do not fit the verb together, such as-beside other sources or anAddForeignKeytarget given both ways. Its messages and hints are unchanged. Thereverse-policy-payloadhint for a missing parameter now readsadd {param}=, or revise reverse. - The
reverse-guardnote’svalidationvaluenot-valid-carrieris nownot-valid-as-check, and its JSON groupcarrier_columnis nowchecked_column: a NOT NULL a down returns as a NOT VALIDCHECK (<column> IS NOT NULL). Its hint names that CHECK where it said carrier. No alias is kept. - The command
retrofit data carryis nowretrofit data reanchor. Its report readsreanchored:,would reanchor:andnothing to reanchor, its JSON records are typereanchorwith outcomereanchored, and its stale-witness note isreanchor-witness-stale. The hintsretrofit-data-carry,retrofit-data-carry-again,retrofit-data-carry-op-seal,carry-lead,make-readable-carryandthen-finish-carryare nowretrofit-data-reanchor,retrofit-data-reanchor-again,retrofit-data-reanchor-op-seal,reanchor-lead,make-readable-reanchorandthen-finish-reanchor, and each names the new command. No alias is kept. - The refusal
carry-failed, whichop remove,op revise,op unsealandreconcileprint when the sweep after their write fails, is nowreanchor-failed; its hintfinish-carryis nowfinish-reanchor. The write still stands. - The skip kind
constraint-carrier-undecodableis nowconstraint-undecodable: retrofit could not decode its own stored form of a constraint. Its class (internal-defect), itsdrift-skip-internal-defectwarning and the exit status are unchanged. - The refusal
constraint-facet-uncarriedis nowconstraint-facet-unmodeled, andrule-custom:constraint-facet-uncarriedis nowrule-custom:constraint-facet-unmodeled: a declared constraint parameter retrofit’s model has no place for (enforced=false,period=true,without_overlaps=true), which replay would drop. Its message now readsreplay does not reproduce <param>=<value>; its hints are unchanged. - The surface export (
retrofit surface --as json) keys the hint variant of a code whose refusal prints another code’s hint wholeborrowed, with template{borrowed_hint}and slotborrowed_hint, in place ofcarried,{carried_hint}andcarried_hint. The codes it appends are unchanged. import --schema <name>leaves out a statement of any other schema, registered or not, and counts it ascross-schema. A schema other thanpublicrefused the whole run asschema-mismatch, andpublicwas left out uncounted. Theschema-mismatchcode is gone. A run that mints nothing for its schema failsimport-loss-bearing-skipswhen a skip lost something, exits 0 on a populated log, and otherwise refusesoplog-emptywith a hint naming the schemas it left out. Apg_dumpfile holding the target schema still refuses on a populated log asstate-source-on-populated-oplog.importdecides that a file’s primary-key drop and add, and the foreign keys dropped and re-added around them, are oneChangePrimaryKeyfrom the ops it has minted, not from a scan predicting the names PostgreSQL holds later in the file. A rotation folds where nothing between the drop and the add rebuilds the old key’s index or re-checks a moved foreign key, and the folded ops replay to the same state. Any other stays aDropPrimaryKeyand anAddPrimaryKey, with animport-change-primary-key-declinedwarning saying why. Re-importing a file can give a different op stream: more rotations fold on a table where a key name was taken, and one whose window rewrites the table stays apart. A key dropped under a foreign key still bound to it is refused by replay’s rule, as a split import refuses it. A rotation split across files stays a drop and an add, as before.
Fixed
driftno longer records a function whose signature names a type it set aside. A function with a parameter (of any mode), return,SETOForTABLEcolumn typed by aCREATE TYPEthe log lacks is afunction-type-unmintedskip, of class gap, in place of aCreateFunctionthat failed at apply. A function of the same name the log holds is not dropped.importof apg_dump --schema-onlyfile records a function whose string-literalLANGUAGE sqlbody, or%TYPEor%ROWTYPEdeclaration, reads a table, view or function the file creates later just after that statement, with a note naming the move.pg_dumpwrites such a function first and turnscheck_function_bodiesoff to restore it;migrate --from rootkept the file’s order, and PostgreSQL refused the function under the default setting. A laterLANGUAGE sqlfunction calling such a function moves after it too, aspg_dump’s name order often writes a callee first. A function stays where it is when any other statement it would pass names it, runs code or holds data, or when its caller cannot move.migrateno longer places aRunRawScriptwhose DO block holdsCOMMITorROLLBACK(the batched-backfill pattern) inside its transaction, where PostgreSQL failed the script with SQLSTATE 2D000 and rolled the file back. Under--envelope=ownthe window segments around the script and reportsmigrate-segmented-non-atomic;callerrefusesmigrate-tx-incompatible;independentemits the script with that advisory.--into-schematreats the copy the same way.migrate --downrefusesmigrate-non-reversibleunder every envelope when a window script’s reverse holdsCOMMITorROLLBACK, since the down never segments;op revertis not refused. A data body written as such a DO block now refusesmigrate-dml-unscreenedlike a bareCOMMIT;, and--allow-unscreened-dmladmits it with the segment count reported as a lower bound. Only transaction control in the parsed PL/pgSQL counts, not one in a string, a comment orEXECUTE.importkeeps aLANGUAGE cfunction’s link symbol (AS '$libdir/pgcrypto', 'pg_gen_salt'), andsnapshotandmigraterender both parts aspg_dumpprints them. Only the object file was kept, so PostgreSQL looked the function’s own name up as the symbol and refused it at apply, anddriftnever saw a changed symbol.- An authored
LANGUAGE internalfunction, such as a wrapper overint4pl, is rendered as a function.snapshotandmigratetook it for a trigger callback stub and left it out at exit 0 with no skip entry, anddriftplanned aReplaceFunctionfor it unchanged. Only a trigger function with an emptyLANGUAGE internalbody is a stub. snapshotwrites a key equal to another of the table’s keys, such as aPRIMARY KEYand aUNIQUEover the same columns added by separate statements, as its ownALTER TABLE ... ADDafter theCREATE TABLE. Inside theCREATE TABLEPostgreSQL merged the two, so the snapshot read back with one key, anddriftagainst a project’s own snapshot planned aDropUnique.op addrefuses aspayload-shapea comma-form--partition_boundwhose datums are bracketed or quoted (from=["2026-01-01 00:00:00+00"], the oplog’s display spelling), naming the JSON record form. The punctuation was stored in the datum: PostgreSQL ignored it at apply, butdriftread every bounded partition as re-bound and skipped it, andidentifyrefused.validaterefuses asrule-exists-and-alivea view, policy, triggerWHEN, standard-SQL function body or partition key that reads an element a later op mints, as a hand-ordered log can hold. It passed, andsnapshotat the reader’s cut left the reader out. A body PostgreSQL binds only when it runs, such as PL/pgSQL, is unaffected.validaterefuses aschange-primary-key-dangling-fkaChangePrimaryKeythat leaves a foreign key covered only by the unique index its old key was promoted from (USING INDEX). PostgreSQL drops that index with the key, somigrateemitted a foreign-key re-add that it refused.identifyanddriftread a column cast to its own type (lower((email)::text)on atextcolumn) as the bare column, as PostgreSQL stores it. Once aCastColumnmade such a cast a no-op, PostgreSQL dropped it from views and index keys:identifyfound no cut for a database retrofit had just built, anddriftplanned aReplaceView, or aDropIndexandCreateIndexof an unchanged index.drift’s index compare also folds a comparison label and a whole interval value asidentifydoes.- An accepted copy-table question in
driftlands the whole copy in one run. The copy follows the drops and renames that free the names its clones take, with what each of those must follow, where it refusedcopy-clone-name-collision. The target’s own indexes, checks, unique keys and foreign keys on the copy are planned with it, and a clone the target’s copy lacks, or names otherwise, is dropped or renamed. Each landed only on a second run. identify --expect lastandmigrate --from-dumprefusefrom-dump-working-copyfor a test bed holding a run thatreconcile --allow-sealed-rewritedemoted, after that run is sealed again. They answeredfrom-dump-no-matchwith adrifthint, and following it recorded a drop of the hotfix.op show --touchingandop revise --dry-runresolve a foreign key aChangePrimaryKeyre-minted, by its name or id, asop adddoes. They listed only the originalAddForeignKey, and refusedno-matchonce the re-mint was renamed.schema createrefuses asname-conflicta name that akind = "schema"[[exclude]]entry names, with a hint to remove the entry first. It wrote a[[schema]]entry that every later verb refused, and left an oplog behind.op infonames the lock five operators take:AddIdentityrewrites nothing (it reads the column once to seed the sequence),RenameIndextakesSHARE UPDATE EXCLUSIVEon the index, andRenameTrigger,RenameViewandRenameMaterializedViewtakeACCESS EXCLUSIVE.op addrefuses asno-matcha schema slot that names another registered schema (--schema a CreatePartition ... in=b), saying an op cannot yet place an element in another schema’s log. It bound an element of the target schema whose id began with those letters, at exit 0.importanddriftrecord a partition whose parent sits in another schema as a gap, so the run fails rather than losing the link. Read for the partition’s schema,CREATE TABLE b.c PARTITION OF a.pskips aspartition-parent-cross-schema, its own partitions and dependents ascross-skipped-partition-cross-schema-child, andpg_dump’sALTER TABLE ONLY a.p ATTACH PARTITION b.crecordsattach-partition-cross-schema-parent. Read for the parent’s schema,driftrecords thatATTACHasattach-partition-unresolved-child, asimportdoes. Both minted a table with no columns or read the link as the decisioncross-schema, and exited 0.importlists a statement of another registered schema once in its skip manifest and summary. ACREATE POLICY,ALTER POLICY,CREATE VIEWor rename was listed and counted twice.sequence-not-created-by-render’s hint names the column holding the sequence. Where another registered schema’s identity or serial column creates it, the hint says to apply that schema’s render first; it said to create the sequence, which made that render fail. A sequence another column of the same schema holds isOWNED BYthat column, not the first column drawing from it.op sealrefuses asrule-exists-and-alivea seal over a reference into another registered schema thatvalidatefinds dangling, such as a drop under another schema’s view, where the schema being sealed holds the reference or dropped its target. It sealed over it.import --sealanddrift --sealdo not run this check yet.import --forceof a source holding nothing for its schema and no data (aSET-only prelude, or a file wholly of another schema) refusesoplog-emptyand leaves the populated log as it was. It wrote the log empty first, then refused. A schema with data records getsoplog-emptyrather thanforce-corpus-orphan. To empty a log on purpose, runschema removeandschema create.driftrefuses asdrift-extension-moved, writing nothing, a source in which an extension sits in another schema than the log’s: one statedWITH SCHEMAelsewhere, or, in apg_dump -nfile that leaves the extension out, references respelled into its new schema. AfterALTER EXTENSION ... SET SCHEMAit recorded each respelled default, cast, check, view and index as a change at exit 0, and every script it rendered failed at apply. The hint names moving the extension back withALTER EXTENSION; retrofit cannot record a move yet.- Several refusals, hints and notes say what they mean, with codes, exits
and JSON fields unchanged.
op addsays a token names a table where the slot takes a column;identify --expectnames the nearest cut; therange-invertedhint ofmigrate --downnames--fromas the rollback target;CreatePartitionon a plain table says it is not partitioned, not that it is not live;apply-risk-exclusive-lock’s hint no longer says the runner pauses writers; and a count of one reads singular, where several messages printed1 tablesor1 bodies. --helpand the published command reference show a placeholder (<path>,<cut>,<opid>) for every flag value, where sixteen flags showed a name taken from an example.reconcilenamesfork-forced-order;op seal,data listanddata checkname which products start from their source’s rows (see Added) and whichop sealstill gates;data repin,op sealandmigratenamedata reanchorfirst for a revised or removed anchor; andvalidate --fail-on-apply-risknames a cast that scans the table for an index or check.
Log format
DetachPartitiongains three optional params, appended after its anchor halves:kept-ids, one fresh id per key, index or outbound foreign key the detached table keeps from its parent and the parent’s ancestors;kept-anchors, the ancestor element each id copies, recorded by the producer; andkept-names, the name each copy takes, given atop addas--kept-name <ancestor>=<name>. A log written before has none, and a replay of it mints no copy, as it did. An earlier release refuses a log holding them; regenerate it with this one.- A plain
LANGUAGE plpgsqlfunction body written ahead of a table it reads now records that read where the table is created, or renamed into the name.pg_dumpwrites functions first andimportkeeps the file’s order. The body then behaves as it does when the table comes first. ADropTableorDropColumnof an element it reads refuses asrule-no-live-references, naming the function, andsnapshotwrites a renamed table’s or column’s new name in the body. Earlier releases accepted the drop and kept the old name. A body reading a column the table never gets is listed as an opaque reader, as when the table comes first. A log holding such a drop is refused wherever it is replayed to be checked or rendered. To read it, replace the function (ReplaceFunction) or drop it (DropFunction) ahead of the drop, then regenerate the log or re-seal. The log’s bytes are unchanged. CopyTable,SplitTableandDetachPartitionrecord, beside each list of IDs they mint (column-ids,adjacent-ids), the element each ID was minted over (column-anchors,adjacent-anchors). Replay binds an ID to its element by that record, never by position, so reordering two ops that cannot reorder no longer moves an ID onto the wrong element. A record that does not match its source exactly refuses asderived-anchor-unmatched, naming the op, the list and each unmatched element; the verbs that replay it surface the refusal asstructural-op-invalid(validate),render-blocked-on-partial-forest(rendering,element,op revert,schema ref-tablesand thedataverbs that read the log),parse-error(op seal) orreplay-failed(data witness), each keeping thederived-anchor-unmatchedprefix. A log written before this release holds no anchors, so it refuses the same way wherever it is replayed past one of these ops: re-importthe database to regenerate the log. The retireddetach-id-list-aritycode is this one.CopyTableandSplitTableno longer requirecolumn-idsandadjacent-idsat parse: a line without one now parses, and replay refuses it asderived-anchor-unmatched, asDetachPartitiondoes. No log needs regenerating; a line that has both lists is unchanged. The refusal names anchors in the order the line records them, each with its table and name.
Known issues
- A project with several registered schemas is not supported yet. How a reference from one schema into another, a drop that another schema depends on, and the verbs that read more than one schema’s log should behave is still being designed, and each can go wrong today.
- Ending a column that a later
RunRawScriptbody names is refused only when the body is bound. For a body stored opaque,op removeorop reviseof the column, or a merge whose other side removed it, writes the log, andvalidateaccepts it.migratethen emits the script against a table without the column, so the migration fails whole at apply.validate --historyfails a pull request that adds such a body, unless the name sits only in a conditional branch or the body does not parse; it only advises on a body already on the target that the pull request leaves naming a removed column. Apply the migration to a scratch database in CI (retrofit migrate --from root --to last, or from the latest Seal). driftasks thereversalquestion only when the database holds something logged after the drop. A database that has not yet run the log’s newest drop reads as intent, and the plan re-adds the item with no question. Review a drift plan’s re-adds against the log before accepting them.- A released
transitionorcleansedata record, in a project that opts in to them, is protected only by its position in the log: an edit to one behind a Seal passesvalidate, andmigrateemits the edited bytes. Nor is a record the release shipped told apart from a branch’s backfill integrated behind it, which a database already past that point never runs. - Before 1.0 the oplog may gain operators and header keys. A build refuses
a log with a newer
retrofit-oplog-formator a key it does not know, so CI and every developer upgrade together.
[0.13.0]
Added
op revertrecords a rollback in the log. Aftermigrate --downhas rolled a database back, it appends the reversal that down computes: the ghost’s ops after the log’s last op, the reference leg’s records, a declared reverse as one raw script, and the data-loss notes. A database migrated from the log then gets the same result, and a later down knows what was undone;migratestill never writes the log. Each appended op recordsreason="undoes <opid>", naming the window op it undoes, unlessreason=<text>is given. Recording twice appends nothing, and recording after more ops land appends only the new reversal. The fork guard and thevalidategate run before the write, and--dry-runpreviews what would be appended. A revert that is not at the log’s tail refuses asrevert-not-at-tail, and one whose leading leg is off a Seal asrevert-leading-leg-unsealed, each with a hint naming the way out.migrate --downreads whatop revertrecorded and advances its range past it, so a window is not undone twice.- A reversal that re-creates a dropped table, column or other element (the
down’s ghost and
op revert) keeps the element’s ElementID when the re-create passes the same-element predicate, so the element’s history continues where a fresh id orphaned it. The mint is markedsame_element=reversal(see Log format), and--dry-runprints the mark.validaterefuses an unmarked mint of an id the log already knows, andop reviserefuses to edit a marked mint.op squashnets a drop and the mints that return its element as one group, andop sealreports a data loss a squash of its range would avoid. - A down over a
DropTableorDropColumnsays that the element it re-creates comes back without its rows, as are-created-emptyitem ofreverse-irreversible, beside the forward’sdata-loss-on-applyline. A paired table’s stand-in names each column that comes back empty. A forward window that re-creates a dropped element says it comes back empty too. schema database-at <cut>pins the cut the local database is at, in the self-ignored.retrofit/database-at;--clearremoves the schema’s line, and bare it prints the pin in force and where it came from.op add,importanddriftalso take the pin as--database-atorRETROFIT_DATABASE_AT, and judge apply-risk from it (see Changed). A pin file that exists but cannot be read refuses asio-errorrather than being rewritten.op add RenameTable,op add RenameColumn,import,driftandop reverttake--allow-readers-moved <old-name>, repeatable. It attests that the readers of the table or column with that qualified old name are handled by the time the rename applies, and withholdsapply-risk-rename-in-usefor that rename; every other rename still warns, and the warning’s hint names the flag. A value that answers no rename, including one a refused run never reached, is reported as having no effect.op add CastColumnraisesapply-risk-table-rewritewhen the cast makes PostgreSQL rewrite the table, naming the dependents rebuilt with it, a partition’s own and a name-less index included.op addandop reviserefuse asbody-resolvea name in a function, trigger, view or raw script body that neither the log nor the project’s declarations hold where the body binds it: a top-level script statement, an always-reachedDOstatement, aBEGIN ATOMICbody, a%TYPEor%ROWTYPEdeclaration, or a string-literal SQL body with no polymorphic argument. Where PostgreSQL binds the name only when the body runs (a PL/pgSQL statement, a polymorphic SQL body, a conditional branch), they warnbody-name-absentinstead. A name an operator took off the schema (MoveColumn,ExtractTable, a partition operator,DropTableand the like) is named in the refusal with that operator, and a held body citing it raisesbody-name-removed.importjudges every body at the end of its source, andvalidate --historyholds a range to the same check: a certain site fails, a conditional one isadvisory. Objects the log cannot see, such as an extension’s functions, are declared in[[external]]entries inretrofit.toml.op addwarnsschema-partition-tree-indexedfor aCreateIndexon a partition root or middle, and for aPartitionTableof a table that already holds an index:identifyskips an indexed partition tree, so it cannot place a database over one. Where that tree is the only difference left, thefrom-dump-no-match,from-dump-behindand--expectoutcomes say so and thatdriftfinds nothing else differing, and outside--expectsay to name the cut withmigrate --from. Outcomes, codes and exit statuses are unchanged.
Changed
op add,importanddriftjudge apply-risk over the window from the database’s starting state, asvalidate --fail-on-apply-riskdoes, rather than one op in isolation. The starting state is theschema database-atpin, else the latest Seal at or before the op, else root. A table created inside that window holds no rows when the database applies the op, so the empty-table codes (apply-risk-add-unique,apply-risk-add-not-nulland the like) are withheld on it, andapply-risk-rename-in-useis withheld on an old name the starting state does not hold. A line saying a code was withheld on an assumed starting state prints once per run. A schema in a fork keeps the per-op verdict.- A repeated
--schemaon any verb refuses asflag-invalid-value, where the last value won:op seal --schema a --schema bsealed onlyb.--from,--toand--atstill take the last value. validateand every verb that folds the registered schemas (import,drift,snapshot,identify,element list,element history) refuse two[[schema]]entries rooted at one SQL schema asname-conflict, naming both. Identical imports had bound a cross-schema view on some runs and refused it on others.identifyrefuses a log that breaks a replay rule with the rule’s own code and hint, asvalidate,migrateandsnapshotdo: arule-no-live-referenceslog gets thereplace-or-drop-functionrecourse where it was refusedrender-blocked-on-partial-forestwith no hint. A replay-classed entry this build cannot check refuses asunverifiable.op add CreatePartition payment public p1readspublicas the schema, asCreateTabledoes, where it boundname=publicandpartition_by=p1. A partition named like the schema spellsname=.op reviseof an op an unsealed Seal once covered says the op was sealed under that Seal, since unsealed, so a database placed there is a working copy. It said the op was never sealed and a database at the Seal was unaffected.- A
RunRawScriptbody kept opaque by a cast to a managed type gets a hint for the cast’s shape: drop a'literal'::app.kindcast where the literal meets a column of that type, or write one statement per arm of aCASEof string literals. schema createof a name already taken refuses with a hint namingretrofit init --config <dir>/retrofit.tomlfor a second project, andinitbelow a project it leaves alone notes that project on stderr.validateno longer reportsschema-stray-oplogfor an oplog inside a nested project, a directory with its ownretrofit.toml.- Wording: a scalar param spelled as a flag (
--referenced_table t) refusesunknown-keywith a hint spelling itreferenced_table=<value>; themissing-requiredhint gives the param’s help; theSplitTableandMergeTableexamples spell--condition=; theRunRawScriptsummary says when a body is bound;data scaffold --purposesays both values are experimental; and the run-onceRunRawScriptnotes ofsnapshotandidentifyread in the singular for one op.
Fixed
op unsealno longer refuses a log for a verdict the log already held before the unseal, such as a sealed drop a newer replay rule refuses. It reopens the tail so the op can be repaired and the log re-sealed; the other verbs still refuse the log until then.op unseal,op removeandop reviseunder--dry-runno longer say the write stands when the post-write sweep refuses: the hint says nothing was written.- A function whose argument,
OUT, table or return type is a managed type or a relation’s row type, stored unqualified (f(mood)), is emitted qualified inCREATE,DROP,ALTER ... RENAME,COMMENT ONand the restored grants.snapshotandmigraterendered it bare, so theCREATEfailed or bound a same-named type on thesearch_path. Built-in and unmanaged types stay as written, andidentifyresolves a dump’s qualified spelling against such a function. driftno longer plans aDropFunctionand aCreateFunctionfor an unchanged function whose signature pg_dump prints differently from the log: a managed type schema-qualified (SETOF, arrays, row types andOUTcolumns included), a long-form type (timestamp without time zone,int), or a typmod PostgreSQL discards (varchar(10),numeric(10,2)). NewCreateFunctionrecords store the folded spelling; sealed records keep their bytes and still pair.op addrefuses asrule-unique-name-inaCreateFunctionPostgreSQL would read as an existing function: one that differs only by a discarded typmod (f(varchar(20))besidef(varchar(10))) or by qualifying its own schema’s managed type (f(mood)besidef(public.mood)). Both were stored, andmigratethen issued a secondCREATE FUNCTIONthat failed.driftandidentifyno longer plan aReplaceFunction, or answerfrom-dump-no-match, for aLANGUAGE sqlbody the log rendered whose string literal sits under aCASE, an array subscript or anarray_to_stringseparator. TheCASEand subscript fix reaches CHECK, index predicate, view and policy bodies too.identifyplaces a database whose skipped run holds a function body that reads nothing (BEGIN RETURN 2; END) ahead of a laterChangePrimaryKey,CastColumn,AddEnumValueorRenameTable, and a body whose only mention of a renamed table is anEXECUTEstring. It answeredfrom-dump-no-matchand routed todrift.identifyon a dump that matches no cut no longer runs the full placement search over a log holdingRenameIndex, where one such op took nearly a minute.identifyno longer refuses the project’s own snapshot when[[exclude]]names a schema an extension is placed in, and no longer scans a large snapshot in quadratic time per candidate cut.- A log whose first op names a
prev=the file does not hold no longer panicsmigrate,snapshotoridentify: they replay it alone and judge its replay rules.validateandop sealstill refuse it asstructural-prev-unknown. With several registered schemas,snapshot,element listandmigrateno longer refuse such a log asunverifiablewith a hint naming a build that does not exist. import --forcethat re-mints a schema’s ids warns for each other registered schema whose bodies still name the old ids, andvalidatethen failsrule-exists-and-alive.snapshotrefuses such a body asinvalid-cut, naming its holder, where it wrote an<elem:...>placeholder into the SQL.DropColumnof a columnExtractTablederived (a rejoin column, or a survivor’s foreign key column) refuses under its live dependents as on any table, where it was admitted and left a snapshot with aPRIMARY KEYon a missing column; drop the foreign key and primary key first.AttachPartitionover such keys no longer failssnapshotwithinvalid-cut.op reviserefuses a changedRunRawScriptbody thatop addwould refuse (a bare statement, two statements, an empty body), in every form. It wrote them at exit 0, formigrateto emit outside the one-DOcontract.schema removeremoves the schema’s block wherever it sits inretrofit.toml, in every spelling the loader accepts. It deleted the oplog at exit 0 but left the block when another[[schema]]followed. A removal that cannot take every registration refuses before the oplog is deleted.schema removeandschema movehonor an absoluteoplog_root:removedeleted a path that did not exist and left the real oplog, andmoverefused asmove-failed.schema moveandschema enginerewrite the config line the loader reads, where a quoted or repeated key reported success and kept the old value.- The
apply-risk-recorded-unsealednote advisesretrofit validate --fail-on-apply-risk --from <cut>; the command it printed passed--schema, whichvalidatedoes not take, and exited 2. op infono longer saysDropGeneratedExpressionends an element: the column and its values stay.
Log format
- The 15 operators that create an element (
CreateTable,AddColumn,AddPrimaryKey,AddForeignKey,AddUnique,AddExclude,AddCheck,CreateIndex,CreateView,CreateMaterializedView,CreatePolicy,CreateType,CreateFunction,CreateTrigger,CreateExtension) take one optional derived key,same_element=reversal, on a mint that returns an element a drop ended under its own ElementID. A line without the key is unchanged; no log needs regenerating. A build that predates the key refuses a line that has it, by name. - A value outside the declared set of a derived enum (
same_element, andMoveColumn’sarm) now reads as an unknown key, sovalidatereports the line unverifiable and the verbs that render or write refuse it by key, where it was akind-mismatchparse error. An authored enum still refuses.
Known issues
- Ending a column that a later
RunRawScriptbody names is refused only when the body is bound. For a body stored opaque,op removeorop reviseof the column, or a merge whose other side removed it, writes the log, andvalidateaccepts it.migratethen emits the script against a table without the column, so the migration fails whole at apply.validate --historyfails a pull request that adds such a body, unless the name sits only in a conditional branch or the body does not parse; it only advises on a body already on the target that the pull request leaves naming a removed column. Apply the migration to a scratch database in CI (retrofit migrate --from root --to last, or from the latest Seal). identifyon a dump that matches no cut can take minutes on a log holding aMergeTable,JoinTable,UnjoinTableor partition operator, where a dump that matches answers in seconds.driftasks thereversalquestion only when the database holds something logged after the drop. A database that has not yet run the log’s newest drop reads as intent, and the plan re-adds the item with no question. Review a drift plan’s re-adds against the log before accepting them.- In a project with several registered schemas,
driftplans aDropForeignKeyfor an unchanged cross-schema foreign key thatimportrecorded, skips the file’s key asadd-foreign-key-cross-schema-unsupported, and exits 0. Review eachDropForeignKeyin such a plan with--dry-runbefore recording it. op add,op sealandmigrateread only their own schema’s log, so a drop in one registered schema is accepted, sealed and emitted while another registered schema depends on its target.validatefails such a drop when the dependent is a view, policy, function or other body in another schema, but accepts it when the dependent is a foreign key or a column typed by the dropped type. Apply the migration to a scratch database holding every schema before it lands.migraterefuses asemit-render-faileda body that names an element of another registered schema, thoughsnapshotrenders it. The detail reads as if the element were dropped; it is live in the other schema’s log. No cut renders it. Usesnapshotfor a fresh database.- A trigger whose
EXECUTE FUNCTIONnames a function in another registered schema imports with afunction-synthesized-stubfinding and renders the call without its schema, which fails at apply or binds a same-named function on the search path. Treat each such finding as a broken trigger. - In a project with several registered schemas,
validate --fail-on-apply-risk --from <opid>resolves the opid in every schema, so the schemas that do not hold it refuse asopid-not-found. Name the cut with an anchor, such as a Seal, instead. driftrecords aCreateFunctionwhose signature names a type it set aside, so the function fails at apply under--allow-capability-gaps. Record the type before runningdrift.migrate --from rootof a log imported from apg_dump --schema-onlyfile can create a function ahead of the table its string-literalLANGUAGE sqlbody or%TYPEdeclaration reads, which PostgreSQL refuses under the defaultcheck_function_bodies.snapshotof the same log applies. Usesnapshotfor a fresh database.- A released
transitionorcleansedata record, in a project that opts in to them, is protected only by its position in the log: an edit to one behind a Seal passesvalidate, andmigrateemits the edited bytes. Nor is a record the release shipped told apart from a branch’s backfill integrated behind it, which a database already past that point never runs. - Before 1.0 the oplog may gain operators and header keys. A build refuses
a log with a newer
retrofit-oplog-formator a key it does not know, so CI and every developer upgrade together.
[0.12.0]
Added
driftover a database that skipped a logged drop, as an out-of-order deploy does, asks areversalquestion where it re-added the item with no question. The plan line names the op it reverses and that op’s Seal. The answers arekeep, which an unanswered question takes and which leaves the log as it is, andaccept, which records the restore. A table, a column, a key, a constraint, a trigger and a rename back all take the question.- A
RunRawScriptDObody that names only elements its binder can follow is now recorded bound to them. A later rename renders through the body inmigrate,op show, the down andreconcile’s diff, andop removeof an element the body names refuses asrule-exists-and-alive. A body the binder cannot read stays opaque. - The script
migrateemits reads the privileges held on a table, view or column before a re-create drops it and restores them on the replacement, so a grant made outside the log stays. Columns moved byMoveColumn,ExtractTable,JoinTableorInlineTableare named as not restored. driftandimportwithout--sealprint oneapply-risk-recorded-unsealednote per run, naming the tables, columns and renames they recorded: apply-risk reads what the unsealed tail creates as new and empty, and a table recorded from a populated database is neither.- The docs gain a refusal codes page, generated from the binary’s code registry, with each code’s kind, its exit statuses and the commands that can print it.
Changed
migraterenders aReplaceViewthe old way, oneCREATE OR REPLACE VIEW, only when the view’s column names at the op’s input cut are a name prefix of the new list and every prefix column’s type is proven unchanged. Any other definition is a rebuild: it drops the view’s tracked readers and the view, creates the view with the new definition, restores its grants, and re-creates its triggers and readers, with a-- WARNING:line naming each reader that may fail and a-- post-deploy:REFRESH for each materialized view re-created empty. The rebuild takes ACCESS EXCLUSIVE on the view and each reader, andop addraisesapply-risk-view-rebuildfor it. A view whose prefix column reads another view’s or a materialized view’s output column now rebuilds even for a tail-column add, since the type cannot be proven. In the lendlog fixture the two opsb56bc83a6cafand1ed10791196frender as rebuilds where they rendered oneCREATE OR REPLACE VIEWbefore;driftand the down emit oneReplaceView, no longer aDropViewand aCreateView, so the view keeps its ElementID.validate --historygains theopaque-bodycheck: it fails when the pull request adds a body retrofit stores opaque (aRunRawScriptwith nobound_params=, a string-literal or unresolvedLANGUAGE sqlfunction body, or a view, materialized view or expression slot whose text names no element), since a later rename or drop leaves what it names stale and only a database apply checks it. Each finding names the line, itsoperator,slotandcause, and the rewrite. A pipeline that keeps applying the range to a database before it lands adds--allow-database-apply: the check then lists its findings asadvisoryand does not fail.op add,importandop reviserefuse nothing new; the cause finding they print for such a body names the check and the consent, andimportnow prints it for a string-literal function and a view whose query falls back.reconcileno longer applies a forced order that the run did not ask for. It refuses asfork-forced-order, exit 1, naming the flag that asks for the order and what the order overrides, and the JSON error gainsforced_orders. A barereconcilereports each forced order.migrateover aRenameColumn,RenameTable,RenameVieworRenameMaterializedViewthat a plainLANGUAGE plpgsqlfunction reads follows theALTERwith aCREATE OR REPLACE FUNCTIONunder the new name, since PostgreSQL never rewrites such a body. Where the same migration also renames a type in the function’s signature, it refuses and names the drop-before, create-after sequence.driftno longer records a column whose type it set aside. A column typed by aCREATE TYPEthe log lacks is acolumn-type-unmintedskip, of class gap, in place of anAddColumnorCastColumnthat failed at apply.driftraises a merge question only between versioned or snapshot names, so a table prefix such aswp_no longer makes every table one family. It asks an index rename only between equal definitions, and pairs rename questions the same way on every import of one file.importanddata captureprintpreamble: N statementsbefore their summary or verdict, so the summary is the last line of stdout. The text, the JSON placement and the counts are unchanged.validate --fail-on-apply-riskandmigrate’s counsel count a table created in the window as populated once aMergeTable, aMoveColumndown, a splice-band data body or a staged load gives it rows. A range that passed may now exit non-zero.spec_refinop infoand theSPECline ofop add <Op> --helpprint the Operators page URL, not a repository path. Help text names docs pages by title and says operator, not SMO.
Fixed
- A rename now follows, and a drop now counts, the references in a function
body that earlier builds left stale or did not see: quoted, three-part and
qualifier names in
LANGUAGE plpgsql,%typedeclarations, reads through a CTE or an aliased subquery,THENresults,INlists,VALUESrows, row-SETsources and subscripts.RETURN QUERYandRETURN NEXTbodies and assignments to a parameter now bind. A read the binder cannot resolve makes the body opaque, where it had bound a shadowing variable or nothing. - A
BEGIN ATOMICbody’sINSERT,UPDATE,DELETEandMERGEstatements now bind, with eachMERGEclause scoped as PostgreSQL scopes it.driftno longer proposes aReplaceFunctionfor an unchanged one on every run. - A trigger’s
UPDATE OFlist and a bareWHENor parameterDEFAULTnow follow renames. TheUPDATE OFcolumns are quoted, anddriftorders a column drop after the triggers that list it. CreateViewandReplaceViewexpand a*inside a CTE, aFROMsubquery and every arm of a set operation, not only the top-level leftmost one. A rename that makes a derived relation’s read ambiguous now skips the view insnapshot, named, rather than writing one PostgreSQL rejects.migrate --downnow re-creates a droppedLANGUAGE plpgsqlfunction and a dropped trigger under the target’s names, orders a re-created function at its own position, drops a window-created reader ahead of the column it reads, and renders a bound script’s bare relations under the declared search path or refuses.- A reference floor crossing its declaration’s removal now applies as a
counted row delta when no key identifies it, writes a
body=nonefloor as its empty state’s delta, deletes a dropped NULL key NULL-safely, and folds a partition child’s columns by its root’s ids. A keyless floor had duplicated its rows at every apply, at exit 0. CopyTableandSplitTableclones of keys and indexes now render named. PostgreSQL had picked a numbered name when the derived one was held, and a laterDropIndexorRenameIndexthen hit the wrong object.importkeeps%TYPEon a function’s argument or return type and keeps a managed type named like a built-in’s internal name (public.bool). A function’sSETandRESETclauses fold as PostgreSQL’sproconfigdoes, and each element of itssearch_pathlist renders as its own literal.driftresolvesSET search_path FROM CURRENTasimportdoes.migrateparenthesizes a column default theDEFAULTslot cannot read bare, such asnow() AT TIME ZONE 'utc'.importmoves a drop past a later statement of the same file that clears theLANGUAGE plpgsqlreader blocking it, as PostgreSQL applies it.ops upgrade-sql-bodieskeeps a function’s volatility, parallel safety,SECURITY DEFINER,SET,COST,STRICT,ROWSandLEAKPROOF, which it had reset to their defaults while reporting success. A body given toop revisenow binds at the line’s cut, as one given toop adddoes.driftorders a merge or join question after a surviving foreign key’s drop, names the operator on a structural question’s text line where it printed<?>, and reports an earlier op that replay refuses under that op’s own code.validate --historyno longer failsfate-kepton a revise of a revision the base holds or on a removal whose replacer stays on the chain. Itscollapse-parentfinding names the revert and a recovery that completes after a hand edit rewires a successor.validatereports its other findings beside a seal digest mismatch.reconcilenames a remedy for a displaced identity generation, and describes a partition bound as the DDL spells it and an owned sequence by the name its column renders.identifyon a dump matching no cut no longer searches every placement on a log holding aCopyTable,SplitTableorExtractTable, or on a dump of an indexed partition tree. One such dump had run past 30 minutes.op shownotes the sealed ops its default scope hides, so a touching query over sealed history no longer reads as no history.data listmarks an orphaned record asdata checkrefuses it.- A refusal or hint no longer names a verb that refuses in turn, a flag the
user did not pass, or a generic cause.
op addandop reviselist the accepted keys for an unknownkey=value,importof onlyCREATE SCHEMAnamesschema create, a stream source prints as a bare-in recipes, a malformed foreign key target refuses under a registered code, and-vfolds identical repair sites into one line with a count. import --helpsays sources are never re-sorted and a glob expands lexically (V10beforeV2), with--from-filefor any other order.drifton a 4,651-op dry run takes 31 s, where it took 3,619 s.validateon a log whose tables churn columns grows about 8x at 4x the operations, where it grew 16.2x. A three-filereconcileover a 2,311-op log peaks at 262 MB, where it peaked at 1.82 GB.
Log format
ReplaceViewaccepts any new column list: replay no longer refuses a narrowing or a reorder, and theview-replace-supersetcode leaves the catalog, the registry and the rule census. A sealedReplaceViewwrites the same bytes and replays as before; the render now rebuilds the view whereCREATE OR REPLACE VIEWcannot take the change. An older build refuses such a line asview-replace-superset; upgrade it to read the log.- A
RunRawScriptwhose bound body names a quoted relation or column (app."Account","isActive") is now bound, where earlier builds kept it opaque. Its bound sentinel writes each quote of the name doubled, and the line gains a new producer-onlybound_quoted=[body]besidebound_params=, listing the body params that hold such a sentinel. An older build refuses the line by that unknown key, and itsvalidaterefuses it asbody-bound-edited; upgrade it to read the log. A rename of such a name to one that needs quoting now renders it quoted instead of refusingbody-render-unquotable. A line without a quoted name is unchanged, so a log needs no regeneration. - A bound body whose sentinel was hand-edited malformed now loads whether or
not
bound_quoted=lists it, and replay refuses it asbody-bound-edited; earlier in this release it refused at load asbody-bound-unmarked. - A plain
LANGUAGE plpgsqlfunction body now records the tables, views and columns it reads, as anansi-sqlbody does, sosnapshotwrites it with a renamed element’s new name. ADropTable,DropVieworDropColumnof an element such a body still reads refuses asrule-no-live-references, naming the function, atop add, where earlier releases accepted it. A log holding one is refused by that code wherever it is replayed to be checked or rendered (validate,snapshot,migrate,data check,data listanddata witnessamong them);op showandschema enginestill read it.data checkstill reads the sealed part of a log whose only such drop sits after its last Seal. The log’s bytes are unchanged. To read such a log, replace the function with a body that no longer reads the element (ReplaceFunction) or drop it (DropFunction) ahead of the drop, then regenerate the log or re-seal. - A trigger’s
WHENand a function parameter’sDEFAULTnow record the managed functions they call, as PostgreSQL does. ADropFunctionof a function either still calls refuses asrule-no-live-references, naming the trigger or the function, atop add, where earlier releases accepted it and the migration failed at apply. A log holding one is refused by that code wherever it is replayed to be checked or rendered. The log’s bytes are unchanged. To read such a log, drop the trigger (DropTrigger), or replace the calling function without the call (ReplaceFunction) or drop it (DropFunction), ahead of the drop, then regenerate the log or re-seal. A call to one of two overloads of a name, or an unqualified call whose name a built-in shares, still records nothing. RunRawScriptgains the optional, producer-writtenbound_params=[body, reverse_body]param, written after the reversal params and beforereason. A build without it refuses a line that holds it as an unknown key; no log written today holds it, so nothing needs regenerating.- A body param whose text holds the bound-script form
"<bound:…>"but thatbound_params=does not list refuses at load asbody-bound-unmarked, whatever the operator. A log written by retrofit never holds one. Restore the marker, or replace each"<bound:…>"with the name it stands for. - Replay refuses more drops PostgreSQL refuses, as
rule-no-live-references, atop addand wherever a log holding one is replayed: aDropFunctionwhile a partition key calls the function; aDropColumnorMoveColumnof a column a trigger’sUPDATE OFlist names; aDropTablewhile a column is typed by its row type or a body names that type; and a drop of what aBEGIN ATOMICbody’s DML statement, a subscript or a%typedeclaration reads. The log’s bytes are unchanged. To read such a log, clear the reference ahead of the drop, then regenerate the log or re-seal. - A body that names a dropped element refuses as
rule-exists-and-alive, for a view, materialized view, function, policy, triggerWHEN, column default, generation expression or partition key expression. Only a merge that orders the drop first reaches this, andreconcile --cleannow refuses that order asfork-no-clean. - An index, or a
PRIMARY KEY,UNIQUEorEXCLUDEconstraint, named as another relation or index of its schema refuses asrule-unique-name-in. It failed at apply withrelation ... already exists. validateandop sealrefuse aprev=cycle asstructural-prev-cycleand a danglingprev=asstructural-prev-unknown, exit 1, wherever in the file they stand. A merge resolution or a rebase can leave either.- A
CopyTableorSplitTablethat clones a key or index whose name cannot be derived refuses ascopy-clone-name-collision, naming the workaround of dropping it first and re-adding it after. TheEXCLUDEcase had refused asrule-exists-and-alive.
Known issues
- Ending a column that a later
RunRawScriptbody names is refused only when the body is bound. For a body stored opaque,op removeorop reviseof the column, or a merge whose other side removed it, writes the log, andvalidateaccepts it.migratethen emits the script against a table without the column, so the migration fails whole at apply.validate --historyfails a pull request that adds such a body. A pipeline that passes--allow-database-applyshould apply the migration to a scratch database (retrofit migrate --from root --to last, or from the latest Seal). identifyon a dump that matches no cut can take minutes on a log holding aMergeTable,JoinTable,UnjoinTableor partition operator, where a dump that matches answers in seconds.driftasks thereversalquestion only when the database holds something logged after the drop. A database that has not yet run the log’s newest drop reads as intent, and the plan re-adds the item with no question. Review a drift plan’s re-adds against the log before accepting them.- In a project with several registered schemas,
driftplans aDropForeignKeyfor an unchanged cross-schema foreign key thatimportrecorded, skips the file’s key asadd-foreign-key-cross-schema-unsupported, and exits 0. Review eachDropForeignKeyin such a plan with--dry-runbefore recording it. op add,validate,op sealandmigrateread only their own schema’s log, so a drop in one registered schema is accepted, sealed and emitted while another registered schema depends on its target. Apply the migration to a scratch database holding every schema before it lands.migraterefuses asemit-render-faileda body that names an element of another registered schema, thoughsnapshotrenders it. The detail reads as if the element were dropped; it is live in the other schema’s log. No cut renders it. Usesnapshotfor a fresh database.- A trigger whose
EXECUTE FUNCTIONnames a function in another registered schema imports with afunction-synthesized-stubfinding and renders the call without its schema, which fails at apply or binds a same-named function on the search path. Treat each such finding as a broken trigger. driftrecords aCreateFunctionwhose signature names a type it set aside, so the function fails at apply under--allow-capability-gaps. Record the type before runningdrift.- A released
transitionorcleansedata record, in a project that opts in to them, is protected only by its position in the log: an edit to one behind a Seal passesvalidate, andmigrateemits the edited bytes. Nor is a record the release shipped told apart from a branch’s backfill integrated behind it, which a database already past that point never runs. - Before 1.0 the oplog may gain operators and header keys. A build refuses
a log with a newer
retrofit-oplog-formator a key it does not know, so CI and every developer upgrade together.
[0.11.0]
Added
op info --as json, for one operator or the--dump, gives each operator averbsobject: forop_add,import,drift,snapshotandmigrate, whether that verb ships the operator. Each value isavailable,unavailable, orsyntax-unavailablefor an operator this build cannot spell.
Changed
- A fresh text-mode
snapshotloads only the rows a record states as its table’s whole content (areplacesection, or a referencerestate) between the tables and the constraints, as before, and fires no trigger on them. Every other record (anappendordeltasection, a seedrestate) now runs in a second-- retrofit:snapshot-data v1band after the whole schema, beforeCOMMIT, with constraints, indexes and triggers live, asmigrate --from rootruns it, so its rows fire the enabled triggers and meet the foreign keys in authored order. Each band re-seats its own tables’ sequences. A note,snapshot-state-under-trigger, names each state table and triggermigratewould fire; a warning,data-statement-order-fk, names a foreign key the order meets (a state ahead of its parent’s statements, or a child’s statements ahead of its parent’s) with the recourse.--as=json,--data-onlyand the banner opid are unchanged. migratewarnsdata-statement-order-fkfor the same two orders in its trailing seed band, where every foreign key is live: a stagedreplacetable whose key reaches a parent only the band’s later records supply, and a child’s records ahead of its parent’s. The warning names the table, its parent, the key and the recourse; a key declaredINITIALLY DEFERREDraises neither, except under--envelope=independent. The script is unchanged.- The
drift plan:header counts changes, between removals and questions:drift plan: N additions, M removals, C changes, Q questions. A change is any planned op the header does not count as an addition or a removal, such as a rename, a cast or a NOT NULL flip, so a plan made only of those no longer reads0 additions, 0 removals. The JSONplan-headerrecord gains achangesfield. transitionandcleansedata records are experimental. Every verb refuses them unlessretrofit.tomlopts in ([experimental] purposes = ["transition", "cleanse"]), and warns on every use once it does: a released one is protected only by its position in the log. Write a backfill as aDOblock to keep it under a Seal’s digest. Without the key,import --purposeanddata scaffold --purposerefuse to mint either, andvalidate,data check,data list,data witness,data repin,migrateandsnapshotrefuse a corpus holding one, asexperimental-not-enabled, naming each file and its purpose. With it, each verb reads and mints as before and warnsexperimental-purposeonce per such file. The table is project-wide; an entry naming anything else refuses asconfig-invalid. A retrofit that predates the table refuses the whole file (unsupported table [experimental]).migrate --downreports a row loss its render chose (data-loss-on-apply-render-chosen) as a warning, asdata-loss-on-applyis, not as a note after the other warnings.migrate --allow-unreached-dml, removed in 0.10.0, refusesflag-retiredand says to drop the flag, instead of failing as an unknown flag.op show --touchingresolves a function, trigger, policy, extension or materialized view, by name, signature or element id, aselement historydoes.- Refusals that pointed at generic advice now name their own recovery:
a bare token in a payload record names the record’s fields (and
ExtractTable’s
--into); a body calling an overloaded function says no spelling binds it; a data file refused only for its record marker line leads with the marker grammar;reconcile --cleanover a demotion whose other order replays clean names that order’s flags before--accept-dirty;schema engine <name> <version>names--schema. - A skipped sequence names itself on drift’s skip line and
skippedevent, and identify’s residual skip lines no longer end in a space. after-traffic-outcomeon aRunRawScript, whose declared reverse is opaque, says the down may refuse or may drop the rows written since, rather than that it does.snapshot --help-alland docs/snapshot.md say that triggers and policies follow the data band, so a snapshot build runs no trigger on its seed rows whilemigrate --from root --labeldoes.
Fixed
- The
drift plan:header counts the creation of a policy, function, trigger or extension as an addition and its drop as a removal, as it does for a table or a view; it had counted them as changes.--show=removalsnow hides those creations and--show=additionsthose drops. migrateno longer emits a script PostgreSQL refuses with55006(pending trigger events) when an op writes a table holding aDEFERRABLE INITIALLY DEFERREDconstraint trigger and the same transaction then alters that table, asAddNotNull … fill=does. Under--envelope ownit refuses asmigrate-deferred-trigger-pending, naming the trigger, the table and the ops; undercallerthe script and adeferred-trigger-owedwarning name theSET CONSTRAINTS <schema>.<trigger> IMMEDIATE;the enclosing transaction can run first. A--downis treated the same way.migratenow warns, asdeferred-trigger-pending, where a data record writes a table holding aDEFERRABLE INITIALLY DEFERREDconstraint trigger and a later op of the same transaction alters that table, which PostgreSQL refuses with55006. The script has a-- retrofit:line above that op, and the warning names theSET CONSTRAINTS <schema>.<trigger> IMMEDIATE;the record can end with.op seal,import --sealanddrift --sealnow refuse, asdata-file-anchor-unresolved(exit 2), while the schema’s data corpus holds a record whoseanchor-op=no longer names an op, asdata checkdoes, with the same detail anddata repin’s recourse. Before, the seal was written,migraterefused the release it cut, and repairing the record neededop unsealfirst, sincedata repinrefuses a sealed target. Regenerating a log under a committed data file, orop reviseorop removeof the op a record anchors on, leaves such a record.data checknow refuses an oplog whoseretrofit-oplog-formatline is older or newer than this build reads, asoplog-format-too-oldoroplog-format-too-new, exit 1. Before, it reported no findings and exited 0 on a log every other verb refuses, so a check run alone passed it. On a sealed log,validatenow refuses the same line by the same code and the parser’s own hint, notsealed-oplog-unparsablewith advice to repair the line.op add PartitionTable,CreatePartitionandAttachPartitionnow warnschema-partition-bound-timestamptz-zonelesswhen a range or list bound on atimestamptzkey has no UTC offset (from=2024-01-01). PostgreSQL reads such a bound in the applying session’s time zone, soidentifyanddriftcannot compare it with pg_dump’s output. The op still appends; the warning names the zoned spelling (2024-01-01 00:00:00+00).identify’s refusal detail now gives each skip’s hint under it, so a zoneless bound reads as needing an offset rather than as unsupported partitioning.import --dry-runnaming a schema that is not registered no longer creates it. The run still previews the import, then leaves the project as it found it: no<schema>.oplog, no[[schema]]row, nodefault_schemapin, and no newoplog_rootdirectory. Previewing such an import no longer needs aretrofit schema createfirst.snapshot --include-drop=scopednow drops the views and materialized views it writes, before the tables, so it re-applies to a database that already holds a schema with a view over one of its tables. Before, the script stopped atDROP TABLEbecause the view depended on the table, and rolled back. An object the prologue does not drop, such as a function or a view outside the snapshot, still fails the apply when it depends on a table or view the prologue drops.importofDROP MATERIALIZED VIEW IF EXISTSnaming a materialized view the import does not hold is now a counted no-op that exits 0, asDROP VIEW IF EXISTSis. Before, it was a capability gap and exited 2.IF EXISTSnaming a table, view or index still exits 2, since PostgreSQL refuses it as not a materialized view.snapshot --include-drop=scopedof a schema with a foreign key now applies to an empty database. The prologue’s foreign key drop readsALTER TABLE IF EXISTS … DROP CONSTRAINT IF EXISTS …, so it does nothing when the table or its schema is absent. Before, it failed withschema "<name>" does not existorrelation "<name>" does not existand the script rolled back.
Log format
- A data file whose header says
purpose=transitionorpurpose=cleanserefuses asexperimental-not-enabledon every verb that reads it, unlessretrofit.tomlopts in with[experimental] purposes = ["transition", "cleanse"]. The file’s bytes and the oplog are unchanged; add the key to read such a log as 0.10.0 did, with a warning per file.
Known issues
- Ending a column that a later
RunRawScriptbody names is not flagged:op removeorop reviseof the column, or a merge whose other side removed it, writes the log, andvalidateaccepts it.migratethen emits the script against a table without the column, so the migration fails whole at apply. To catch it before landing, have CI apply the migration to a scratch database (retrofit migrate --from root --to last, or from the latest Seal). identifyon a dump that matches no cut can take minutes, where a dump that matches answers in seconds. In one measured case, a history of about 120 operations, a dump matching no cut took 137 s against 0.57 s for one that matched.driftover a database that skipped a migration the log records as a drop, as an out-of-order deploy or a migration-folder tool running an older pending migration does, re-adds the dropped column as anAddColumn, with no question and no note. Review a drift plan’s re-adds against the log before accepting them.- A released
transitionorcleansedata record, in a project that opts in to them, is protected only by its position in the log: an edit to one behind a Seal passesvalidate, andmigrateemits the edited bytes. Nor is a record the release shipped told apart from a branch’s backfill integrated behind it, which a database already past that point never runs. - Before 1.0 the oplog may gain operators and header keys. A build refuses
a log with a newer
retrofit-oplog-formator a key it does not know, so CI and every developer upgrade together.
[0.10.0]
Added
- This change log. CI requires a “Log format” entry for any change to the
oplog catalog golden, the serializer or the seal’s leaf-byte goldens. A
serializer edit that leaves the format unchanged may say so with a
Log-format: unchangedcommit trailer instead. - Each release archive carries a man page per command and the bash, zsh
and fish completion scripts, and the Homebrew cask installs them, so
man retrofit-migrateworks afterbrew install. migrateandsnapshottake--allow-unnamed-label <label>(repeatable). It attests that a labelled dataset was left unnamed on purpose, such as a test-onlyfixtureslabel on a long-lived target. Theapply-risk-data-exclusionswarning no longer counts that dataset, and its hint names this flag beside--label. Each value names one exact label, so a dataset added later still warns. An attestation that answers nothing prints a note that it had no effect.CopyTableandSplitTablecarry eachEXCLUDEconstraint of their source onto every table they create, under the name PostgreSQL gives it there (<table>_<keys>_excl);migrateadds it after the table’sLIKE.ExtractTable,JoinTableandUnjoinTablemove anEXCLUDEunder its own name onto the table that receives its key columns, and refuse one whose keys would land on two tables, naming it.MergeTabledrops each source’sEXCLUDEwith the source, and a merged row that breaks one onintofails the apply. A copy refuses anEXCLUDEwhose name it cannot derive, or two that would take one name.InlineTablecarries anEXCLUDEconstraint where it refused a table holding one. The survivor keeps its own, and a source’s moves onto the survivor under its own name, re-keyed onto the survivor’s key columns, as aUNIQUEdoes.migratechecks the rows the re-added constraint would hold by its own operators and refuses, naming it, when a padded row, or on the lookup arm two survivor rows sharing a folded row, would conflict, before anything is dropped.migrate --downof anExtractTableover a table holding anEXCLUDEnow inverts, re-adding it on the table it came from.
Log format
AddUniquetakes an optionalnulls_not_distinct=truefor aUNIQUE NULLS NOT DISTINCT, whichimportnow carries in every spelling where it skipped the key asunique-constraint, anddriftwhere it skipped it asadd-unique-unmodeled-facet.snapshotandmigratewriteUNIQUE NULLS NOT DISTINCT (…).driftrecreates a key that differs from the target only by it (DropUnique, thenAddUnique), andidentifytells the two apart. Replay refuses the key besidebacking_index, sinceUSING INDEXtakes the index’s NULLS treatment, and akeepover such a key refuses withreverse-policy-not-null: the NULLs it writes still collide. AUNIQUEthat also carriesINCLUDE,WITH (…), a tablespace or an access method still skips. A build without the key refuses a log that holds it, naming the key. A log that skipped such a key keeps the skip; re-import the statement to carry it.- The
columnslist ofCreateView,ReplaceView,CreateMaterializedViewandReplaceMaterializedViewpercent-escapes each name’s%,,and;(%25,%2C,%3B), so a quoted column name holding a comma, such as"revenue, usd", stays one column where it split into two.import,driftandmigratewrite the escaped form;op add’scolumns=takes it too. A list whose names hold none of the three is unchanged, as is its content hash. A log that holds a literal%25,%2Cor%3Bin a view column name now reads it as%,,or;; re-import that view. No other log needs regenerating. CreateTriggertakes optionalconstraint=true,deferrable=trueandinitially_deferred=truefor aCREATE CONSTRAINT TRIGGER, whichimportanddriftnow carry where they skipped it astrigger-unsupported. ItsFROMform still skips astrigger-unsupported. Replay refuses a constraint trigger that is notAFTER,FOR EACH ROW, on a table and without transition tables,initially_deferredwithoutdeferrable, either one on a plain trigger, andRenameTriggerof a constraint trigger. Its name is also one of its table’s constraint names: aCHECKor key of that name refuses withrule-unique-name-in, and an unnamed one takes the next free derived name.importrecordsALTER TRIGGER … RENAMEof one as the gapconstraint-trigger-rename-unsupported, and refuses a trigger created over one’s name, or one created over a plain trigger’s (constraint-trigger-replace-invalid).snapshotandmigratewriteCREATE CONSTRAINT TRIGGERwith itsDEFERRABLEclause, andmigratecounts it among the constraints a deferred key’s name must not share. A build without the keys refuses a log that holds them, naming the key. No log needs regenerating.CreateFunctiontakes an optionalprocedure=truefor aCREATE PROCEDURE, whichimportanddriftnow carry where they skipped it ascreate-procedure-unsupported. A procedure shares the functions’ names and signatures; replay refusesreturns,volatility,parallel_safety,strict,cost,rowsorleakproofon one, and a trigger that executes one.snapshotandmigratewriteCREATE OR REPLACE PROCEDURE, and name it as aPROCEDUREinDROP,ALTER … RENAMEandCOMMENT ON, each argument type markedIN.importreads thePROCEDUREandROUTINEforms ofDROP,ALTERandCOMMENT ON; with theFUNCTIONform, a name written without an argument list names the one routine of that name and kind, of any arity. It refuses what PostgreSQL refuses: an unmarked argument list two procedures answer to, or a name without a list that more than one routine answers to (routine-name-not-unique), and a statement naming a routine of the other kind, aCREATE OR REPLACEacross kinds or a trigger on a procedure (routine-kind-mismatch). A build without the key refuses a log that holds it, naming the key. No log needs regenerating.CreateFunctionandReplaceFunctiontake optionalrows=(a function’sROWS n, kept as written, likecost=) andleakproof=.importanddriftcarry both fromCREATE FUNCTION, where they skipped the function asfunction-rows-unsupportedorfunction-leakproof-unsupported, andimportfolds anALTER FUNCTION … ROWSor… LEAKPROOFinto aReplaceFunction;driftstill skips a targetALTER FUNCTION, as it does… COST. A build without them refuses a log that holds them, naming the key. No log needs regenerating.- A log holding one of the PostgreSQL 18 surfaces (
AddColumn generated_kind=virtual,enforced=false,period=true,without_overlaps=true,SetConstraintEnforced,SetConstraintInherit) now failsvalidatewithsyntax-not-available, and no verb renders it. Regenerate the log without them, or revise each op to its PostgreSQL 17 form (generated_kind=stored,enforced=true,period=false,without_overlaps=false). RunRawScripttakesreverse_noop=<reason>andreverse_blocked=<reason>besidereverse_body, at most one per line, and a newDeclareReversalline (target=,supersedes=, one of those arms) declares a sealed script’s reversal without changing a sealed byte. A build without them refuses a log that holds them, naming the key or operator. No log needs regenerating.reconcile --base --ours --theirsreads an input that has noengine-versionheader, and merges the key like any other: an absent key is a value. Every other command still refuses such a log, and no log written before needs regenerating.reconcile --base --ours --theirskeeps a side’s own followers’prev=bytes when the other side removes the line they follow: the removed line stays in that side’s place as its tombstone. No log needs regenerating.- The three-file mode’s refusal hints name the inputs by role (ours, theirs) and its own re-run, never a flag it refuses.
reconcile --base --ours --theirsrefusesfork-write-writefor a value both sides change that no block order decides (one side revising a base op in place, the other setting the same property);--keepor--swapwrites the stated outcome. No log needs regenerating.- The conflict-marker and duplicate-opid refusals name the three-file mode
(
reconcile --base --ours --theirs --result) with its inputs by role, and a partition-scoped column facet set against its parent’s recursing op refusesfork-write-writenaming<partition>.<column>. No log needs regenerating. reconcile --base --ours --theirswrites--swap’s order with a block’s tombstones beside the block they follow, so--swapover (base, ours, theirs) writes what--keepover (base, theirs, ours) writes, and reads a line a side holds as the base does through the other side’s revision of its predecessor as unchanged. No log needs regenerating.AddIdentity,DropIdentityandSetIdentityGenerationtake an optionalpartition=, a partition beneath the column’s owner whose own identity the op adds, drops or changes in place on a PostgreSQL 15 or 16 pin.op addwrites it for a<partition>.<column>token there; from 17 on the op refuses on a partition’s column. No log needs regenerating.AddColumnandAddIdentitytake optionalidentity_start,identity_increment,identity_minvalue,identity_maxvalue,identity_cacheandidentity_cycle=truefor an identity’s sequence options, whichimportnow carries on an inline identity column,ADD COLUMNandADD GENERATEDwhere it skipped them asidentity-options-dropped. An option that restates PostgreSQL’s default for the column’s type and direction is dropped;AS,OWNED BY,RESTARTand a partition’s own identity still skip. Replay refuses an option without an identity (identity-option-without-identity), one that is not an integer (identity-option-not-integer), and a bound outside the column’s type,MINVALUEnot belowMAXVALUE, aSTARToutside them,INCREMENT0 orCACHEbelow 1 (identity-option-out-of-range).snapshotandmigratewrite the options in the identity’s sequence list;driftcarries the target’s options onto an identity it adopts and reportsidentity-options-differfor a paired one whose options differ. A build without the keys refuses a log that holds them, naming the key. A log that skipped such options keeps the skip; re-import the statement to carry them.SetIdentityOptionschanges an identity column’s sequence options (ALTER COLUMN … SET START WITH | INCREMENT BY | [NO] MINVALUE | [NO] MAXVALUE | CACHE | [NO] CYCLE). It takesidentity_start,identity_increment,identity_minvalue,identity_maxvalue,identity_cache,identity_no_minvalue,identity_no_maxvalueandidentity_cycle(falseisNO CYCLE); an option it does not name keeps its value.importwrites it for those forms, which it skipped asAlterTableSetIdentity(RESTARTstill skips), anddriftfor an identity whose options differ from the target’s, which it skipped asidentity-options-differ. A build without the operator refuses a log that holds it. No log needs regenerating.SetIdentityOptionstakes an optionalpartition=, asSetIdentityGenerationdoes, and a partition-scopedAddIdentitytakes theidentity_*options it refused: on a PostgreSQL 15 or 16 pin a partition’s own identity keeps its sequence options throughATTACHandDETACH, which dropped them.importwrites them for anADD GENERATEDorALTER COLUMN … SETon an attached partition, which it skipped asidentity-options-droppedorAlterTableSetIdentity, anddriftplans the scopedSetIdentityOptionsfor a partition whose own identity’s options differ from the target’s. A log that skipped such a statement keeps the skip; re-import it to carry the options.AddExcludeandDropExcludeadd and drop anEXCLUDEconstraint.AddExcludetakesmethod(btree,gist,spgistorhash), the key list asCreateIndexspells it (columns,key_expressions,key_opclasses,column_directions,nulls_ordering),key_operators(each key’sWITHoperator,&&orOPERATOR(schema.op)),name,deferrableandinitially.importwrites them forALTER TABLE … ADD CONSTRAINT … EXCLUDE, the table-levelEXCLUDEinsideCREATE TABLEandDROP CONSTRAINTof one, where it skipped the first asalter-table-otherand the second asinline-exclude-constraint. AnEXCLUDEwith aWHERE,INCLUDE,WITH (…),USING INDEX TABLESPACEor a keyCOLLATE, or on a partitioned table or a partition, still skips, now asexclude-constraintin both spellings.snapshotandmigratewrite it, andAddExcluderaisesapply-risk-add-exclude.InlineTable,PartitionTableandDepartitionTablerefuse a table that holds one, andAddExcluderefuses a partitioned table. A build without the operators refuses a log that holds them. A log that skipped anEXCLUDEkeeps the skip; re-import the statement to carry it.driftreads anEXCLUDEin both spellings and pairs it with the log’s by its keys, method, operators, operator classes, sort order and deferral, not by name. A difference in any of them plansDropExcludethenAddExclude, and a name-only difference plansRenameConstraint. An operator pg_dump qualifies by the search path’s first schema (OPERATOR(public.===)) compares equal to the bare===. A targetEXCLUDEwith a facetAddExcludecannot carry is anexclude-constraintskip and drops nothing.identifykeys the same facets, so two cuts differing only by anEXCLUDE’s operator are not one match. Before, drift reported everyEXCLUDEthe log held as anexclude-constraintskip and ignored the target’s.AddExcludetakes optionalinclude_columns,storage_paramsandpredicatefor anEXCLUDE’sINCLUDE (…),WITH (…)andWHERE (…), each spelled as onCreateIndex.importanddriftnow carry the three where they skipped the constraint asexclude-constraint, andsnapshotandmigratewrite them. An unnamedEXCLUDEwithINCLUDEtakes PostgreSQL’s name over its keys and itsINCLUDEcolumns (t_x_n_excl), whereimportheld its name over the keys alone. AWHEREthat does not resolve skips the whole constraint, since without it the constraint compares every row; aWITH (…)value the log cannot spell,USING INDEX TABLESPACEand a keyCOLLATEstill skip.driftplans a change to any of the three asDropExcludethenAddExclude, andDROP COLUMNof a column only theINCLUDEorWHEREnames drops the constraint with it, as PostgreSQL does. A build without the parameters refuses a log that uses them. A log that skipped such anEXCLUDEkeeps the skip; re-import the statement to carry it.PartitionTabletakes two optional lists,partition_parentsandpartition_keys, one entry per partition, to declare a tree of several levels; a single-level record is unchanged and carries neither. Nothing to regenerate: an existing log replays as it did.op show --framing=addwrites a partition bound or key as one JSON object, whichop addtakes back.- Every oplog names its form on a fourth header line,
# retrofit-oplog-form: mergeor# retrofit-oplog-form: compact. A log without the line is refused (oplog-form-header-missing), as is any other value (unknown-oplog-form). In the merge form, whichschema createwrites unless--form compactis given, a removed or revised operation stays in the file as a~tombstone or behind areplaces=list;op compactdrops those traces. Every read checks each live Seal over its chain, so a Seal whose digest no longer matches is refused. For an existing log, add# retrofit-oplog-form: compactas line 4, after# engine-version:, then runretrofit schema form mergeto keep the history from then on. op show’s runnable framings printCreateIndex’skey_opclassesas--key_opclass, not--key_opclasse. The log’s bytes are unchanged; re-runop showfor any copied--key_opclasseline.- A
# retrofit-*header key this build does not know is refused on every verb that loads the oplog (unknown-key,writes-blocked-on-unknownsorrender-blocked-on-unknownsby the verb’s role), naming the key and its line, instead of being carried through opaquely. Run a build that knows the header; do not delete it. ExtractTabledeclares the ruleextract-partitioned-source: an extract that moves every live column refuses a partitioned parent or a partition as its source. A log holding such an extract no longer replays. Regenerate the log, runningDepartitionTableorDetachPartitionbefore the extract, or leave a parent’s partition key on it so the source survives.AttachPartitionends the partition’s own columns into its parent’s, andDetachPartitionmints the detached table’s columns and CHECKs afresh, recording their IDs in its new optionalcolumn-idsandadjacent-ids.SetDefault,DropDefault,AddNotNull,DropNotNullandSetColumnCommenttake an optionalpartition(partition=<table>, or the on-column spelled<partition>.<column>) that scopes the op to one partition of the column’s family. New refusals:partition-not-beneath-owner,partition-not-null-inherited,partition-identity-absent,detach-id-list-arityandattach-partition-check-mismatch. A log whose ops name an attached partition’s own columns, or a detached table’s columns by their earlier IDs, no longer replays: regenerate any log that holds anAttachPartition.
Fixed
migrate --downover a window that drops a table holding anEXCLUDEconstraint, or merges such a table into another, re-creates the table with eachEXCLUDEunder its name. It refused the windowmigrate-non-reversiblewhere the same window over aUNIQUEinverted. AnEXCLUDEaCopyTableorSplitTablecloned still refuses. The down of anExtractTablethat inverts as anInlineTablestill refuses a table that holds anEXCLUDE, and now names the constraint.reconcile <fork-selector> --cleanon a fork nested in another fork’s block closes that fork alone. It also hung the enclosing fork’s other block onto the result, closing the enclosing fork in an order nothing checked, and when that block’s lines came first it moved the nested fork’s ops under it. The enclosing fork now stays a fork, which the nextreconcile --cleanwrites or refusesfork-no-clean. Thefork-nestedrefusal and the bare report’s next step name the nested fork’s selector to close first, where they said no reconcile invocation could resolve it.op add ChangePrimaryKeyrequiresname=for the new key, asAddPrimaryKeydoes, unless it setsbacking_index, anddriftandimportgive a rotated-in key PostgreSQL’s<table>_pkey. An unnamed key was added bare, so the name PostgreSQL gave it depended on where an earliermigratewindow ended, and a later window could drop or rename a name the database did not hold. A log that already holds an unnamedChangePrimaryKeystill reads.migratereports a seed record no--labelnamed asnot-selectedeven when it is anchored behind--from. It used to print adml-behind-windowwarning for it, and--allow-unnamed-labelnoted that it had no effect, so afixturesdataset kept warning on every later window. A selected record behind the window still warnsdml-behind-window.- The
dml-chain-near-misswarning a data mint raises compares a new chain only with existing chains of its own band (reference, seed or history).data capture --label fixtureson a table that also holds acleansechain no longer warns that a label drift forks the chain, or suggests re-running with the other chain’s label-set: a chain of another band is never the one a capture supersedes. data capture’s verdict line printsnot comparable:once before the reason instead of twice, anddata capture --dry-runrehearses a table it would capture aswould capture:, notwould captured:.migrate’sdata-replace-prologueanddata-replace-cascadewarnings say a replaced table “is replaced by its full observed state” andreplacing <table>, not “cleared” and “clearing”: a keyed replace upserts the observed rows and deletes only the keys the state lacks, and never empties the table.op add CreateIndex --helpspells the per-key operator-class flag--key_opclass, whichop addandop reviseaccept beside--key_opclasses. The misspelling--key_opclasseis refused.initagainst an existing.gitattributesprints a note naming each line it appends, instead of changing the file silently.- A data file’s record marker line that leaves out a required token
(
kind=,seq=or thesrc=provenance) is refused naming the missing token, not as carrying an “unrecognized or mis-ordered” one. Every marker-line refusal (G-e) now spells the marker grammar as its remedy, and the shared repair hint names the data file;data checkand the other corpus readers no longer append a line about a “sidecar”. identifyandmigrate --from-dumprefusefrom-dump-schema-absentfor a dump of a database that holds neither the (non-public) schema nor any of its objects, and point atmigrate --from root. They matched such a dump to theCreateSchemacut, somigrate --from-dumpemitted a script withoutCREATE SCHEMAthat failed when applied.identifyandmigrate --from-dumplist the schema facets that did not pair (cut only:anddump only:lines in the detail) when the dump matches no cut butdriftwould plan nothing against the nearest one. Thefrom-dump-no-matchhint no longer sends the user to adrift --dry-runthat reports 0 ops, and thefrom-dump-behindhint no longer says such a divergence was made outside the log; both offer the nearest cut by hand withmigrate --from.retrofit completion <shell>prints the shell’s completion script. The root’s help listed it, but running it was refused as an unknown command, and the scripts’ completion requests were refused the same way.- Every verb that loads the oplog refuses an operator this build does not
know, a
~tombstone included, naming the operator and the line, with the hint “upgrade the CLI to one that knows the operator; do not delete or edit the line”. Theopverbs andvalidaterefuse it asunknown-operator, except thatvalidatekeepssealed-oplog-unparsablefor one inside a sealed range. A renamed or split operator keeps its re-import or rename remedy. validatereports an unknown key once, naming its line, the key and the rules it leaves unverifiable, instead of oneunverifiablerow per rule for every op of the operator. No hint says to hand-fix the op. It also reports a header key this build does not know, and aretrofit-model-versionnewer than the build, as unverifiable (exit 2); an unsealed log with a newer model-version no longer passes.reconcileexits 2, not 1, for a file only a newer build can read: an unknown operator, a too-newretrofit-oplog-formatorretrofit-model-version, an unknown header, and (as before) an unknown key. The JSONcodetells them apart; a conflict marker, a repeated header or a refused fork stays exit 1.reconcile, theopverbs andop revise -irefuse an oplog that exists but cannot be read (for example mode 000) withoplog-unreadable(exit 2), printingcannot read oplogand the read error, instead ofoplog-not-found(“no oplog”) or a barecommand-error.validaterefuses an oplog holding an op that needs a newer engine version than itsengine-versionpin, withunsupported-on-engine-version(exit 2), asmigrateandsnapshotdo. A log whose pin was lowered by hand, as in a resolved header merge conflict, no longer passes the CI gate.op sealandidentifyrefuse the same log with the sameunsupported-on-engine-version(exit 2):op sealwrites no Seal over it andidentifyresolves no cut.op infodrops the note that said they did not check the engine-version gate.validate,migrateand the other commands that read the log refuse aCreatePartitioncarrying bothpartition_columnsandpartition_expression, as they already refused it onCreateTableandPartitionTable. Such a sub-partition used to replay, andmigraterendered its key from the expression alone, dropping the columns. ACreateTablecarrying both with nopartition_byis refused too, rather than read as a plain table with its key dropped.op addand the commands that read the log refuse anExtractTablethat moves every live column of a partitioned parent or a partition, withextract-partitioned-source. Such an extract ended the source while a parent’s partitions stayed live, or while a partition’s parent still named it, and the renderedDROPdiverged the database from the log.
Changed
- Every
own-envelopemigratefile now carries an open-transaction check: a head ahead ofBEGIN;and a recheck after theON_ERROR_STOPguard. This enforces theowncontract, that the file owns its transaction. Applied inside a transaction that is already open (psql --single-transaction, a runner’s own transaction, psqlAUTOCOMMIT off, a single-Sync batch), the file refuses by name,own-envelope-transaction-open(SQLSTATE25001), before any DDL. That includes cells that happened to work before, such as a lone unsegmented file underpsql --single-transaction, and cells that applied but lost the file’s atomicity at exit 0. The check needsUSAGEonplpgsql. Everyownfile’s bytes change, so a checksum recorded over an earlierownfile (Flyway’s, Liquibase’s) no longer matches a re-emit. - PostgreSQL 18 is a supported engine version, like 15, 16 and 17, at the
PostgreSQL 17 syntax level: every verb writes and reads the SQL it does at
17. The PostgreSQL 18 syntax refuses
syntax-not-available(exit 2) at every engine version:AddColumn generated_kind=virtual,enforced=falseonAddForeignKeyandAddCheck,period=trueonAddForeignKey,without_overlaps=trueonAddPrimaryKey,AddUniqueandChangePrimaryKey, and theSetConstraintEnforcedandSetConstraintInheritoperators.op addandop reviserefuse it, andvalidate,op seal,identify,snapshotandmigraterefuse a log that holds it, anddriftrefuses to append to one.op infomarks these values and operators as not available. migraterefuses a forward window whose reference record is superseded by a laterreplaceof its table while a step between them reads the table (dml-superseded-in-window, exit 2). Before, such a window exited 0 and the dependent step ran on rows the pruned record would have filled, so anAddNotNullcould fail at apply.data captureanddriftwarn with the same code when the floor they mint makes such a window, thesnapshotcapture hint names the split and the version, andop sealrefuses a range that would seal the pair.migraterenders aPartitionTableof several levels as one nestedPARTITION OFtree behind aLOCK TABLE ... ACCESS EXCLUSIVEand a guard that refuses, before any change, rows the declared tree does not route or that aCHECKof the window rejects. ADepartitionTablerebuild opens with the same lock, and the target of one must be the root: a leaf or middle refusesdepartition-target-is-partition. Its warnings now walk every descendant, naming each partition-owned index, constraint, policy and trigger (an inherited copy is not named), each partition’s stamped DEFAULT, NOT NULL and comment, and each kept serial sequence the rebuild drops; they say the down across it refuses for now. The generated-column recompute warning says the same.ExtractTableof a column that a partition-owned element or a middle’s partition key names refuses, as it did for the source’s own.op infoshows, for each rule, what retrofit does to an op that breaks it at two moments: OP ADD (op addand the other commands that write an op) and READING THE LOG (migrate,snapshot,validate,op seal,identify), eachrefusesorlets through.--as jsonkeepswhenand addsenforcement: {"append": ..., "replay": ...}, derived from it.op infosays READING THE LOGrefusesforpartition_cols_xor_expronCreateTable,CreatePartitionandPartitionTable, and forview-replace-supersetonReplaceView: the commands that read the log refuse an op that breaks them, whichop infosaid they let through.--as jsonreports these rules withwhen: append+replay,enforcement.replay: refusesandbinding: binding, no longerauthor-time-convenience.MergeTable’s rules listMergeShapeonsources: the refusal it already raised, when a source’s columns differ frominto’s or a source has a nextval default or no live column, now has a declaration.- Advisory messages end without a period. The second sentence of
data-motion-partition-not-pg-declarativeand ofmove-column-down-refused, the advice, moved to the hint. Amigrate --downscript’sreverse-unrecordedline ends; the database sits at <opid>. The five witness warnings on a record the down reverts state their cause instead of repeating their code, anddata-motion-source-consumed,dml-body-partition-dependent,data-only-single-statement-autocommitand the two drift-skip warnings are reworded in plain terms. data-loss-on-applymessages end without a period too: the reasonsop add,migrateandmigrate --downbuild for a drop, a cast, a join, an unjoin, a matview replace and a lookup extract or inline. Drift’s extract-recognition diagnostics read “an ExtractTable”.- The Advisory codes page shows every form of a message that varies with a field, each led by the value that selects it, and follows a message that is only a placeholder with its detail and hint.
Removed
migrate --allow-unreached-dmland itsdml-ahead-of-windowwarning. A data record anchored after--tois outside the window and is now omitted without a refusal, a warning or a flag, as a reference or seed record past--toalready was. Promoting an environment that sits behind the tail no longer needs the flag; drop it from scripts, where it is now an unknown flag.dml-anchor-unresolvednow means only an anchor the oplog does not hold, fixed withretrofit data repin.LayerSeparationandNotSealedno longer appear in any operator’s rules: retrofit never checked either. Layer separation is a property of the oplog’s slots (docs/concepts.md, “The oplog”); the sealed region is guarded bysealed-region-write.
Known issues
- Ending a column that a later
RunRawScriptbody names is not flagged:op removeorop reviseof the column, or a merge whose other side removed it, writes the log, andvalidateaccepts it.migratethen emits the script against a table without the column, so the migration fails whole at apply. To catch it before landing, have CI apply the migration to a scratch database (retrofit migrate --from root --to last, or from the latest Seal). identifyon a dump that matches no cut can take minutes, where a dump that matches answers in seconds. In one measured case, a history of about 120 operations, a dump matching no cut took 137 s against 0.57 s for one that matched.driftover a database that skipped a migration the log records as a drop, as an out-of-order deploy or a migration-folder tool running an older pending migration does, re-adds the dropped column as anAddColumn, with no question and no note. Review a drift plan’s re-adds against the log before accepting them. A fix is planned after v0.10.0.- A released
transitionorcleansedata record is protected only by its position in the log: an edit to one behind a Seal passesvalidate, andmigrateemits the edited bytes. Nor is a record the release shipped told apart from a branch’s backfill integrated behind it, which a database already past that point never runs. - Before 1.0 the oplog may gain operators and header keys. A build refuses
a log with a newer
retrofit-oplog-formator a key it does not know, so CI and every developer upgrade together. Planned after this release:op record;Archive,UnarchiveandPurge;AnnotateandRetract; erasing a released record; renaming a schema; possibly a marker for experimental operators.