Troubleshooting: recover the Google BigQuery changelog after a JVM crash

Liquibase batches all DATABASECHANGELOG (DBCL) and DATABASECHANGELOGHISTORY (DBCLH) entries are held in memory and written as a single bulk INSERT at the end of each command. If the JVM is killed (SIGINT, SIGKILL or OOM) before the flush completes, physical tables may already exist in the database while the tracking tables show no record of those changesets. Liquibase exits with code 1 when the flush fails due to OOM. On SIGKILL or SIGINT, the exit code is OS-controlled.

1. Verify the inconsistency. Run status or update. All changesets will appear as unrun and Liquibase will immediately fail with table already exists errors.

2. Identify the cutoff. Find the last Running Changeset: line in the console output or log file. This is the last changeset that executed before the crash.

3. Add a recovery tag. In your changelog, add a tagDatabase changeset with tag="recovery-point" immediately after the last successfully executed changeset. See the code example below.

4. Restore tracking. Run changelog-sync-to-tag --tag=recovery-point to update DBCL for exactly those changesets. See the command example below.