Use Quality Checks in Automation: Severity and Exit Code

Before deploying code to your databases, it is important to detect and correct any non-compliant code. When you run quality checks on a changelog file manually, you might see CLI warning messages from triggered checks. For example, the quality checks triggered for failing to find a comment, label, or context in the changesets will leave CLI messages.

When you deploy Liquibase jobs automatically, you may not directly review the CLI or log file output. Therefore, your automation tools need an exit code configured for your specific needs and an assessment of the checks' severity to determine if the job moves forward or stops.

Note: When multiple checks are triggered, Liquibase returns the highest exit code of all the triggered checks. This exit code can be detected and processed by automation tools with the echo $? command for Linux or the echo ? command for Windows. Also, if you use quality checks in a flow file, Liquibase 4.24.0 returns the highest exit code specified.

Setting quality checks' severity and exit code

Setting quality checks' severity in an automation process may look as follows:

  1. Run the liquibase checks show command to see the list of available quality checks. Each quality check has a default exit code of 0, which corresponds to the severity level of INFO.
  2. Customize the severity level of the needed checks to set the exit code value.
  3. Configure your automation tools to continue or stop jobs when specific exit codes are returned.

Note: Liquibase triggers a check when a condition is met. For example, "SQL does follow a specific pattern", or "changesets do not have comments." You decide if you want to set conditions that catch missing or existing points and determine your severity levels accordingly.

An example of customizing the severity level of a quality check

Tip: For best results, specify all commands and parameters in the --kebab-case format in the CLI. If your preference is camelCase, it also works in the CLI.

To customize a quality check's severity and exit code:

  1. Run the following example command:

    2. liquibase checks customize --check-name=ModifyDataTypeWarn

  1. Follow the CLI instructions to set the severity level:

    2. Set the Severity to return a code of 0-4 when triggered (current: '<currentvalue>'. options: 'INFO' (0) 'MINOR' (1), 'MAJOR' (2), 'CRITICAL' (3), BLOCKER' (4) )

  1. Change the default severity of INFO in the CLI output by entering one of the below words or their associated number:

    • MINOR or 1
    • MAJOR or 2
    • CRITICAL or 3
    • BLOCKER or 4

Example: If you set the severity to CRITICAL, you will see 3 in the show output and as the returned exit code when triggered.

The default checks-settings.config file or the custom checks settings file you specify when running the customize command changes the ModifyDataTypeWarn check to have a new severity value in a corresponding number.

Executing Quality Checks against changelog.xml

Executing all checks because a valid Liquibase Pro license was found!

Checks completed validation of the changelog and found the following issues:

Changeset ID:       1::triggerGrantCheck
Changeset Filepath: changelog.xml
Check Name:         Warn on Detection of 'GRANT' Statements (SqlGrantWarn)
Check Severity:     CRITICAL (Return code: 3)
Message:            A statement granting privileges to a role or user was
					detected in your changelog. To prevent unintended access to
					the database or data it is recommended that the granting and
					revocation of privileges be carefully reviewed.

Changeset ID:       1::triggerGrantCheck
Changeset Filepath: changelog.xml
Check Name:         Warn on Detection of grant that contains 'WITH ADMIN OPTION' (SqlGrantAdminWarn)
Check Severity:     INFO (Return code: 0)
Message:            A statement granting privileges to a role or user with the
					ADMIN OPTION was detected in your changelog. Granting
					privileges to users or roles with the ADMIN OPTION can lead
					to security issues.  We recommend this changeset be
					carefully reviewed.

Changesets Validated:
  ID: 1::triggerGrantCheck; Author: Liquibase Pro User; File path: changelog.xml

run against each changeset:
  Warn on Detection of 'GRANT' Statements
  Warn on Detection of 'REVOKE' Statements
  Warn when 'DROP TABLE' detected
  Warn when 'DROP COLUMN' detected
  Warn when 'MODIFY <column>' detected
  Check Table Column Count
  Warn when 'TRUNCATE TABLE' detected
  Warn on Detection of grant that contains 'WITH GRANT OPTION'
  Warn on Detection of grant that contains 'WITH ADMIN OPTION'
  Changesets Must Have a Label Assigned
  Changesets Must Have a Context Assigned
  Changesets Must Have a Comment Assigned

liquibase.command.CommandFailedException: run command exited with an error code of 3

Related links