Policy check roll out best practices

Enable your candidate checks at a non-blocking severity and let them run for a full release cycle. Change nothing else. You are collecting data, not enforcing yet.

What you learn:

• Which rules fire constantly against existing code. Some of those hits are real technical debt. Some mean the rule is wrong for your organization. You cannot tell which from the rule name alone.

• Roughly how much remediation blocking would cause. If a rule would have blocked 40% of last month's changes, it needs a conversation before it blocks anything.

• Which rules never fire. Either they're protecting against something that doesn't happen here, or they're misconfigured. Both are worth knowing.

At the end of the cycle, decide what to do with each rule. Keep it, fix its configuration, or disable it. Only then start blocking. A rule that goes straight to blocking without this pass is a rule you will be asked to disable in an incident, which is the worst possible time to evaluate it.

The same rule should not behave identically everywhere. A DROP TABLE is unremarkable in a build environment that gets recreated nightly and should stop a production deployment cold.

Give each environment its own governance and tighten as changes are promoted. In the web application, that is a separate assignment per environment, selected by its ID in each pipeline. When you run from files, keep a separate checks settings file per environment:

The shape matters more than the specific rows. Nothing blocks in dev, the important structural rules block by QA, and the irreversible-damage rules block in production. Naming conventions stay advisory everywhere. Blocking on style produces resentment without producing safety. In severity terms, warn is INFO, which returns exit code 0, and block is any severity that returns a nonzero exit code.

Store settings files in the central governance repository, not in each application repo. Assignments live in the web application, so they need no file at all.

Liquibase has two check scopes, and the distinction determines where a check can run at all.

Changelog-scoped checks analyze changesets and SQL. They need no database connection, so they run in seconds on any runner. Put these in pre-merge, as a branch-protection check. That is the cheapest feedback in the whole pipeline.

Database-scoped checks analyze objects in a live database, so they need a connection. They cannot be a merge gate, because a pull request has no database. Run them post-deployment or on a schedule.

Trying to run a database-scoped check in a pull request is a common configuration mistake in a policy rollout, and the symptom depends on the scope setting. With the default scope the check silently never runs and the job passes. With --checks-scope=database and no database connection the run errors. If a check works locally and fails in CI, check its scope first.

By default, liquibase checks run executes only changelog-scoped checks. Specify --checks-scope=changelog,database to include them alongside your changelog checks.

Failing a build only helps if the developer knows what to do next.

• Point at the report. Set liquibase.reports.enabled=true so every run produces a policy checks report. For changelog-scoped checks it names the changeset that triggered the rule and shows its content. Enough to fix without asking anyone.

• Write specific rule descriptions. When you customize or create a check, its message is what a developer sees at 5pm on a release day. "Violates naming policy" is not enough. "Table names must be lowercase with underscores. Rename CustomerOrders to customer_orders" is.

• Name an owner. Every blocking rule needs someone who can answer "is this rule right?" and grant an exception. An unanswerable block becomes a disabled rule.

The technical rollout is the easy half.

Announce before you block. Tell teams which rules are going from warn to block, and when. A blocked release that nobody was warned about costs you more goodwill than the rule saves.

Review what's firing, every cycle. A rule that fires constantly is either finding a real systemic problem worth a project, or it's wrong. Both need action. Neither resolves itself.

Revisit the rule set periodically. Standards change, platforms change, and a check set nobody has looked at in a year is a check set people have learned to work around.

Watch the trend, not just the events. The Policy Checks dashboard under Monitor shows violations over time, which is how you tell "we have a training gap" from "one team had a bad week."