Remote File Path Rules

Below are all of the remote file path rules that apply to Liquibase using S3. Review the example use cases and instructions to ensure it is the right avenue for your project.

Reference S3 files with Liquibase CLI commands

You can reference S3 files with Liquibase CLI commands depending on the file type and location. For more information, see Reference S3 Files with Liquibase Commands.

S3 files referencing other S3 files

A file in one S3 bucket can reference a file in a different S3 bucket. In this example, a Flow file is located in a S3 bucket named myS3bucket.

flow --flow-file=s3://myS3bucket/liquibase.s3.flowfile.yaml

Below, we see a snippet of Flow File action code, which references a nested flow file in a separate S3 bucket.

Copy

endStage:
  actions:
    - type: liquibase
      command: flow
      cmdArgs: {flowfile: "s3://dbateam/resources/liquibase.endstage.flowfile.yaml"}

Use multiple S3 locations in the same command

Resources are often stored in different S3 locations and Liquibase allows you to find those separate locations in one command.

For example, Let's say the checks-settings.conf you need is in the DBA’s resources bucket and the flowfile.yaml is in the DevOps Orchestration Team’s bucket. Simultaneously, the changelog is stored locally while you are in dev, but then is relocated to the deploy subdirectory of the DevOps team. You should put the direct full path on the liquibase.properties file as shown below.

Note: The file exceptions to this rule are changelog files, snapshot files, or native executor configuration files which must be split over searchpath.

Example of multiple S3 locations called on in the same command:

liquibase --search-path=s3://devopsteam/deploy --changelog-file=mychangelog.sql flow --flow-file=s3://devopsteam/orchestration/liquibase.production.flowfile.yaml

Absolute path

An absolute path is set on a file's property in the CLI, liquibase.properties file, or any standard method. It must include the full, direct path to the S3 file to find the file successfully. Absolute paths can be set for any file type except changelog files, snapshot files, or native executor configuration files. When an absolute path to a file is provided, Liquibase uses the file specified in the absolute path without checking search-path.

The absolute path to the flow file is located in S3 bucket named MyS3Bucket.

liquibase flow --flow-file=s3:/MyS3Bucket/my.flowfile.yaml

Note: Paths for changelog files, snapshot files, and native executor configuration files must be split over a relative path using search-path.

Relative path

Relative Path is enabled with a combination of setting search-path and a relative path on file’s property, which are combined to form the complete S3 path to the file. Relative Path is the only way to successfully find changelog files, snapshot files, or native executor configuration files.

In this example, search-path is set to an S3 bucket:

liquibase --search-path=s3://myS3bucket/demo update --changelog-file=example-changelog.sql

Liquibase will look in the s3://myS3bucket/demo folder for the file titled:example-changelog.sql.

For more information, see How Liquibase Finds Files: Liquibase Search Path and Use search-path with S3.

Sync S3 files to use locally

You may need to work offline and with local docker databases with the same operational files located in your S3 bucket. To do this, you can download S3 files to work with them locally. In some cases, you will need to download an entire directory or a single file. To learn how to download objects from S3, see AWS: sync. It is important to note that Liquibase will not automatically sync the local files with S3 files. To learn how to sync local files with S3, see AWS: sam sync. Files must be in the same directory structure as they are on S3 to use them locally.