variables and include feature

Liquibase flow files can include variables from external YAML files and set them as globalVariables directly in the file. These variables are set up in the flow file's header section before you define a flow file's stages.

Note: The flow feature requires Liquibase 4.17.0+.

In Liquibase 4.24.0+, variables you define in a flow file are shared across "nested" flow files and can be passed through multiple flow files.

Structure

Default variable values

You can set default values for variables by setting the value after ::. These can be set in global variables or stage variables. With optional arguments in place such as labels or contexts, a Flow file will detect the null argument value and it won't pass an empty attribute in the command.

Example: CHANGELOG_FILE: "${changelogFile::null}"

In this example, if the user passes a value to ${changelogFile}, Liquibase sets CHANGELOG_FILE to that value. Otherwise, Liquibase sets the variable CHANGELOG_FILE to null by default.

globalVariables

globalVariables must be defined at the top of the Flow File with the syntax shown in the example Flow File. Each Action specified in the Flow File can reference a globalVariable as a command argument. This saves you time and keeps your Flow File tidy.

Define a global variable:

offlineRefUrl: "offline:postgres?snapshot=refSnapshot.json"

Use a global variable you have defined:

cmdArgs: { url: "${offlineRefUrl}"}

Reference System Environment Variables through globalVariables

globalVariables can reference System Environment Variables to provide values within a Flow file. Within the Flow file example below, notice that the command echoes "Running Liquibase from ${LIQUIBASE_HOME}". This allows you to change environment variables as desired without changing the Flow file.

Note:  Place the environment variable you want to reference between the parenthesis: "Running Liquibase from ${INSERT_ENVIRONMENT_VARIABLE_HERE}"

Copy 
stages:
    systemEnvironmentVariableSample:
        actions:
        - type: shell
            command: echo "Running Liquibase from ${LIQUIBASE_HOME}"

stageVariables

stageVariables work exactly like globalVariables except they are declared inside of each stage group. If you have the same variable declared as a globalVariable, the stageVariables are applied, not the globalVariable. For example, if you have a changelog globalVariable but need to specify a different changelog, you can apply that via a stageVariable. stageVariables always override globalVariables.

All variables, especially labels and context, must be specified in the Flow File with quotations to operate properly. If you want to run all changesets with or without labels, you must leave the label names quotes empty so you do not have to list all labels individually.

Copy
Label Example 1
Default:
    stageVariables:
      LABELNAMES: ""
Copy
Label Example 2
#
# Run the update
#
- type: liquibase
  command: update
  cmdArgs: {labels: "${LABELNAMES}"}

Include configured YAML files within a Flow File

Flow Files can include references to other YAML files with configuration by using the include section of the file. You need only create a variable name for the referenced YAML file and then reference the file title so that the Flow File can locate it.

The include variable should be in the following format:

username: actual-Username

The key is username and the value is actual-Username. To use the values defined in the included file in the Flow File, you need to reference the namespace and the key. For example:

cmdArgs: {username: "${username}"}

When the flow command executes, it will read the included file, locate the username key and substitute the value for that key into the Liquibase action. In the case of the key:value example here, the substitution results in ${postgresNamespace.user} becoming actual-Username. The Liquibase action executes using actual-Username as the user property.

Example of Flow File include functionality:

include:
- postgresNamespace: postgres-vars.yaml

To enable the functionality, you will then add the created variable name to the cmdArgs section or your changelog.

cmdArgs: { url: "${postgresNamespace.url}", username: "${postgresNamespace.user}", password: "${postgresNamespace.password}", changelog-file: "${postgresNamespace.changelogFile}"}

Example Flow File

Includes cmdArgs, globalArgs, and a referenced include file (YAML)

##########           LIQUIBASE FLOW FILE               ##########
##########  learn more http://docs.liquibase.com/flow  ##########

## NOTE: This is an advanced example flowfile, compared to the other sample at examples/liquibase.flowfile.yaml
#### HOW TO USE THIS FILE:
#### example for CLI: liquibase flow --flow-file=liquibase.advanced.flowfile.yaml
#### example for ENV Var: LIQUIBASE_FLOW_FLOW_FILE=liquibase.advanced.flowfile.yaml

## Advanced options show in this file include:
#### non-default name of 'liquibase.advanced.flowfile.yaml' (use by setting flowfile property to this name)
#### use of 'include' to inject namespaced yaml files of key: val variables
#### use of globalVariables and stageVariables
#### use of globalArgs and cmdArgs
#### use of property substitution
#### use of a nested flowfile (in this case in the endStage, but could be elsewhere)
#### use of if: conditional which allows a -type: shell or -type: liquibase command to run
###### In the example below, we set an environment variable LIQUIBASE_CURRENT_TARGET, such as 'export LIQUIBASE_CURRENT_TARGET=dev'
###### This could be determined dynamically, of course, from the build tools, bu tthis is simpler for this example "if:" conditional
#### use of shell commands in a -type: shell block.
######  command: bash -c "the shell command || and its chained commands && go in the quotes"
########
#### POTENTIAL use of environment variables:
######  DATETIME STAMP
######## In this file, you could replace ${FLOWVARS.THISDATE} with an env var, such as ${LIQUIBASE_THISDATE} set via .bash_profile
######## for example 'export LIQUIBASE_THISDATE=$( date +'%Y-%m-%dT%H-%M-%S' )'

## Bring in and namespace an external file with yaml 'key: val' pairs for use in this file
## The variables will be used as ${namespace.variablename}, seen in this example as ${FLOWVARS.PROJNAME}
include:
  FLOWVARS: liquibase.flowvariables.yaml

## Set up some global variables for property substitution in ANY stage
globalVariables:
  DIRNAME: "./${FLOWVARS.PROJNAME}_${FLOWVARS.THISDATE}"
  STATUSFILE: "status.txt"
  UPDATELOG: "update.log"
  HISTORYFILE: "history.txt"

## Start of the stages.
stages:

  ## A prep stage. There can be more than one stage if desired.
  stage-prep:

    actions:
      - type: shell
        command: bash -c "mkdir -p ${DIRNAME}"

  ## Another stage.
  stage-dowork:

    ## set up vars for property substitution in THIS stage only
    stageVariables:

      VERBOSESTATE: TRUE

    actions:
      # Do a validate command
      - type: liquibase
        command: validate

      # Tell me what is pending a deployment
      - type: shell
        command: bash -c "liquibase --show-banner false --outputfile ./${DIRNAME}/${STATUSFILE} status --verbose ${VERBOSESTATE}"

          # This is the structured way to setup a liquibase command, if you dont want to run it as one 'bash -c' command
          #- type: liquibase
          #  command: status
          #  globalArgs:
          #    outputfile: "${DIRNAME}/${STATUSFILE}"
          #    showbanner: false
          #  cmdArgs: {verbose: "${VERBOSESTATE}"

      # And then save a version in detail, if env var LIQUIBASE_FILE_OUTPUT == 1
      - type: shell
        command: bash -c "echo 'LIQUIBASE_ env vars ' && env | grep 'LIQUIBASE_' "

      - type: liquibase
        ## if this var LIQUIBASE_CURRENT_TARGET is "dev", then the updatesql will run
        if: "${LIQUIBASE_CURRENT_TARGET} == dev"
        command: updatesql
        globalArgs: {outputfile: "${DIRNAME}/${UPDATELOG}"}

      - type: shell
        ## if this var LIQUIBASE_CURRENT_TARGET is not "dev", then the message will be displayed
        if: "${LIQUIBASE_CURRENT_TARGET} != dev"
        command: echo "No output files created. Set env var LIQUIBASE_CURRENT_TARGET to dev to trigger file creation."

      # Quality Checks for changelog
      - type: liquibase
        command: checks run
        cmdArgs: {checks-scope: changelog}

      # Run update
      - type: liquibase
        command: update

      # Quality Checks for database
      - type: liquibase
        command: checks run
        cmdArgs: {checks-scope: database}

      # Create a history file
      - type: liquibase
        command: history
        globalArgs: {outputfile: "${DIRNAME}/${HISTORYFILE}"}

## The endStage ALWAYS RUNS.
## So put actions here which you desire to perform whether previous stages' actions succeed or fail.
## If you do not want any actions to ALWAYS RUN, simply delete the endStage from your flow file.

endStage:
  actions:
    - type: liquibase
      ## Notice this is a flow command in a flow file, and it called a 'nested' flowfile, which in this case lives in the same dir, but could be elsewhere
      command: flow
      cmdArgs: {flowfile: liquibase.endstage.flow}

Example endStage Flow File

In this example, we run some final commands in an endStage file. The endStage always runs, so you can use it for processes that you want to run after every deployment. For example, see the following example of content in liquibase.endstage.flow:

##########           LIQUIBASE FLOW FILE               ##########
##########  learn more http://docs.liquibase.com/flow  ##########

## NOTE: This example flowfile is called from the examples/liquibase.advanced.flowfile.yaml file
## While it could be run on its own, this file is designed to show that flow-files can be decomposed
## into separate files as makes sense for your use cases.

stages:
  cleanuptheDB:
    actions:

      # Clear out the database
      - type: liquibase
        command: dropAll

      # Check that database is empty by seeing what is ready to be deployed
      - type: liquibase
        command: status
        cmdArgs: {verbose: TRUE}

## The endStage ALWAYS RUNS. 
## So put actions here which you desire to perform whether previous stages' actions succeed or fail.
## If you do not want any actions to ALWAYS RUN, simply delete the endStage from your flow file,
## as it has been deleted here in this liquibase.endStage.flow file.

Note: You can also keep all your actions in a single flow file. However, using a main flow file to call on other flow files is a modular approach that can keep you organized.

Related links