Remote File Path Rules

Below are all of the remote file path rules that apply to Liquibase using remote files. Review the example use cases and instructions to ensure it is the right avenue for your project.

Reference files with Liquibase CLI commands

You can reference files with Liquibase CLI commands depending on the file type and location. For more information, see Liquibase commands for remote files.

Remote Files referencing other remote files

A file in one remote file can reference a file in a different remote filebucket. In this example, a Flow file is located in a S3 bucket named myBucket.

flow --flow-file=s3://myBucket/liquibase.s3.flowfile.yaml

Below, we see a snippet of Flow File action code, which references a nested flow file in a separate S3 bucket.

Note: You could use the same example with Azure Blob Storage by replacing the flowfile cmdArgs with an Azure path. For example flow --flow-file=az://myContainer/liquibase.flowfile.yaml

Copy

endStage:
  actions:
    - type: liquibase
      command: flow
      cmdArgs: {flowfile: "s3://dbateam/resources/liquibase.endstage.flowfile.yaml"}

Use multiple locations in the same command

Resources are often stored in different locations and Liquibase allows you to find those separate locations in one command.

For example, Let's say the checks-settings.conf you need is in the DBA’s resources bucket and the flowfile.yaml is in the DevOps Orchestration Team’s bucket. Simultaneously, the changelog is stored locally while you are in dev, but then is relocated to the deploy subdirectory of the DevOps team. You should put the direct full path on the liquibase.properties file as shown below.

Note: The file exceptions to this rule are changelog files, snapshot files, or native executor configuration files which must be split over searchpath.

Example of multiple S3 locations called on in the same command:

liquibase --search-path=s3://devopsteam/deploy --changelog-file=mychangelog.sql flow --flow-file=s3://devopsteam/orchestration/liquibase.production.flowfile.yaml

Example of multiple Azure Blob Storage locations called on in the same command:

liquibase --search-path=az://devopsteam/deploy --changelog-file=mychangelog.sql flow --flow-file=az://devopsteam/orchestration/liquibase.production.flowfile.yaml

Absolute path

An absolute path is set on a file's property in the CLI, liquibase.properties file, or any standard method. It must include the full, direct path to the remote file to find the file successfully. Absolute paths can be set for any file type except changelog files, snapshot files, or native executor configuration files. When an absolute path to a file is provided, Liquibase uses the file specified in the absolute path without checking search-path.

The absolute path to the flow file is located in the S3 bucket named myBucket or the Azure blob named myContainer.

liquibase flow --flow-file=s3:/myBucket/my.flowfile.yaml

liquibase flow --flow-file=az:/myContainer/my.flowfile.yaml

Note: Paths for changelog files, snapshot files, and native executor configuration files must be split over a relative path using search-path.

Relative path

Relative Path is enabled with a combination of setting search-path and a relative path on file’s property, which are combined to form the complete remote file path. Relative Path is the only way to successfully find changelog files, snapshot files, or native executor configuration files.

In this example, search-path is set to an S3 bucket:

liquibase --search-path=s3://myBucket/demo update --changelog-file=example-changelog.sql

Liquibase will look in the s3://myBucket/demo folder for the file titled:example-changelog.sql.

In this example, search-path is set to an Azure blob:

liquibase --search-path=az://myContainer/demo update --changelog-file=example-changelog.sql

Liquibase will look in the az://myContainer/demo folder for the file titled:example-changelog.sql.

For more information, see How Liquibase Finds Files: Liquibase Search Path and Use --search-path with remote files.

Sync S3 files to use locally

You may need to work offline and with local docker databases with the same operational files located in your S3 bucket. To do this, you can download S3 files to work with them locally. In some cases, you will need to download an entire directory or a single file. To learn how to download objects from S3, see AWS: sync. It is important to note that Liquibase will not automatically sync the local files with S3 files. To learn how to sync local files with S3, see AWS: sam sync. Files must be in the same directory structure as they are on S3 to use them locally.