S3

Note: This feature is available in Liquibase Pro v4.17.0 and later.

Liquibase integrates the use of Amazon S3 to store and retrieve data from anywhere. Extending Liquibase to use remote file locations enables secure, centralized file management. Liquibase files include changelogs, snapshots, SQL files, liquibase.properties files, native executor configuration files, checks settings configuration files, and flow files. Remote file location access allows you to build a reusable repository of Liquibase files. It also removes the need for all developers to have local copies of Liquibase files, which ensures files are always accurate.

Set up S3 with Liquibase

  1. Download liquibase-s3-extension-<version>.jar and put it in the liquibase/lib directory. If you use Maven, add this dependency to your pom.xml file:
    2. <dependency>
    <groupId>org.liquibase</groupId>
    <artifactId>liquibase-s3-extension</artifactId>
    <version>1.1.3</version>
</dependency>

  2. Configure your AWS keys as local environment variables or in your AWS configuration files. You can use these secure credentials instead of a traditional username and password. You must set the following keys:

    • AWS_REGION
    • AWS_ACCESS_KEY_ID
    • AWS_SECRET_ACCESS_KEY
    • AWS_SESSION_TOKEN (optional)

    For example, if you use AWS configuration files, you can set this in your config file (~/.aws/config):

    [default]
region=us-east-2

    In your shared credentials file (~/.aws/credentials), you can use temporary values from your AWS access portal. For example:

    [default]
aws_access_key_id=AKIAIOSFODNN7EXAMPLE
aws_secret_access_key=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
aws_session_token=IQoJb3JpZ2luX2IQoJb3JpZ2luX2IQoJb3JpZ2luX2IQoJb3JpZ2luX2IQoJb3JpZVERYLONGSTRINGEXAMPLE
  3. Optional: If you usually store your Liquibase Pro license key in a defaults file which you want to store on S3, set up your license in an environment variable, in the CLI, or with another standard property-setting method.

    4. Set the environment variable:

    LIQUIBASE_LICENSE_KEY=Enter Key Here

    Export the key in the CLI:

    export LIQUIBASE_LICENSE_KEY="Enter Key Here"

    Note: A Liquibase Pro license key is needed to access remote S3 files. Therefore, the license key will not be accessible if only stored in a remote S3 defaults file. It is best practice to set up an environment variable that contains the key.

    Once the AWS keys and Liquibase Pro license key are configured, you are ready to begin using S3 with Liquibase.

Create changelog

You can create a basic Liquibase project, including a changelog, with the init project command:

liquibase init project
  --project-dir=s3://mybucket//init
  --project-guide=off
  --changelog-file=dbchangelog.sql

Note: Liquibase creates the check-settings file locally unless you provide the absolute path with the project-dir attribute, as shown in this example. For more information, see Use the Checks Settings Configuration File.

Tip: When using init project, you cannot give attributes to changelogs. This prevents you from using a relative path with the changelog. To solve this, use the --changelog-file property in the liquibase.properties file to name the changelog file from the CLI when using init project in a non-interactive way. See also: Remote File Path Rules.

If you want to use multiple changelogs in your new project, see Design Your Liquibase Project for best practices on structure and organization.

Run flow file

In the CLI, create a flow file with the flow command. Then you can run the flow file using S3:

liquibase
  --changelog-file=example-changelog.sql
  --search-path=s3://example/demo
  --defaults-file=s3://example/demo/liquibase.properties
flow
  --flow-file=s3://example/liquibases3/liquibase.s3.advanced.flowfile.yaml

For a list of commands you can use with S3, see Reference S3 Files with Liquibase Commands. For more information about the search-path property, see Use --search-path with S3.

Related videos

Learn more about Flow in Liquibase Pro

Learn how to use Flow files with S3

Related links