Using Liquibase with Amazon DynamoDB Pro

Amazon DynamoDB is a fully managed NoSQL database service. For an overview of the Liquibase Proextension for DynamoDB, see Liquibase Amazon DynamoDB Pro Extension Overview. For more information, see the Amazon DynamoDB Documentation.

Verified database versions

• Cloud

Prerequisites

1. Introduction to Liquibase: Dive into Liquibase concepts.
2. Install Liquibase: Download Liquibase on your machine.
3. Ensure Java is installed: Liquibase requires Java to run. If you used the Liquibase Installer, Java is included automatically. Otherwise, you must install Java manually.
4. Get Started with Liquibase: Learn how to use Liquibase with an example database.
5. Design Your Liquibase Project: Create a new Liquibase project folder and organize your changelogs.
6. How to Apply Your Liquibase Pro License Key: If you use Liquibase Pro, activate your license.

Install drivers

CLI users

To use Liquibase and Amazon DynamoDB, you must download the JAR file that contains the Liquibase DynamoDB extension.

To use the Liquibase CLI, place your JAR file(s) in the liquibase/lib directory.

Maven users

To use Liquibase with Maven, you must instead include the driver JAR(s) as a dependency in your pom.xml file. Using this information, Maven automatically downloads the driver JAR from Maven Central when you build your project.

<dependency>
    <groupId>org.liquibase.ext</groupId>
    <artifactId>liquibase-commercial-dynamodb</artifactId>
    <version>1.0.0</version>
</dependency>

Implement Amazon DynamoDB

1. Configure your AWS keys as local environment variables or in your AWS configuration files. You can use these secure credentials instead of a traditional username and password. You must set the following keys:

   • AWS_ACCESS_KEY_ID
   • AWS_SECRET_ACCESS_KEY
   • AWS_SESSION_TOKEN (optional)

   Note: The Liquibase DynamoDB extension automatically detects the value of AWS_REGION from your database connection URL, so you don't have to specify it in your config file (~/.aws/config). For an example connection URL, see the Establish a connection section.

   In your shared credentials file (~/.aws/credentials), you can use temporary values from your AWS access portal. For example:

   [default]
   aws_access_key_id=AKIAIOSFODNN7EXAMPLE
   aws_secret_access_key=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
   aws_session_token=IQoJb3JpZ2luX2IQoJb3JpZ2luX2IQoJb3JpZ2luX2IQoJb3JpZ2luX2IQoJb3JpZVERYLONGSTRINGEXAMPLE

2. Ensure your AWS user account has Identity and Access Management (IAM) permissions to make the following API actions:

"dynamodb:CreateTable",
"dynamodb:DeleteTable",
"dynamodb:DescribeTable",
"dynamodb:ListTables",
"dynamodb:UpdateTable",
"dynamodb:BatchGetItem",
"dynamodb:BatchWriteItem",
"dynamodb:ConditionCheckItem",
"dynamodb:DeleteItem",
"dynamodb:GetItem",
"dynamodb:PutItem",
"dynamodb:UpdateItem",
"dynamodb:PartiQLDelete",
"dynamodb:PartiQLInsert",
"dynamodb:PartiQLSelect",
"dynamodb:PartiQLUpdate",
"dynamodb:Query",
"dynamodb:Scan",
"dynamodb:ListTagsOfResource",
"dynamodb:TagResource",
"dynamodb:UntagResource"

You may want to add more permissions. For a list of permission keys, see Actions, resources, and condition keys for Amazon DynamoDB.

3. (Optional) If you want to make operations with the sseSpecification feature, you must also have the following permissions:

"kms:DescribeKey",
"kms:CreateGrant"

4. (Optional) If you want to use AWS Secrets Manager, you must also have the following permissions:

"secretsmanager:DescribeSecret",
"secretsmanager:GetSecretValue",
"secretsmanager:ListSecrets",
"secretsmanager:ListSecretVersionIds"

5. (Optional) If you want to use changelogs from AWS S3, you must also have the following permissions:

"s3:PutObject",
"s3:GetObject",
"s3:ListBucket",
"s3:ListBucketMultipartUploads",
"s3:ListMultipartUploadParts"

6. (Optional) Specify custom values for Liquibase Parameters for Amazon DynamoDB Pro. This is only necessary if you do not want to use the default Liquibase values. You can set these in your command line (CLI), in your liquibase.properties file, or as environment varibles. There are two groups of parameters:
   1. Those that specify the initial billing modes and read/write capacity for Liquibase tracking tables: DATABASECHANGELOG and DATABASECHANGELOGLOCK.
   2. Those that specify the number of polling attempts, time between attempts, total timeouts, and timeout behavior for the DynamoDB waiter to use when running create, update, and delete operations through Liquibase.

Establish a connection

1. Ensure your Amazon DynamoDB database is configured. For more information, see Amazon: Setting Up DynamoDB.

2. Ensure your Liquibase Pro license key is specified. For example, in a liquibase.properties file (defaults file):

   licenseKey: zQl8kNZjZgSp9LvqWQFAtGwiHrpg97UyAfQrNSiJQBCDH8FQPdDzANUpIe4Bj3CZA2IXgDBaoYZFvMw0E/s4JcECB3/A6jO+...

3. Specify the database URL in the liquibase.properties file (defaults file), along with other properties you want to set a default value for. Liquibase does not parse the URL. You can either specify the full database connection string or specify the URL using your database's standard connection format:
   • Cloud: url: https://dynamodb.<region>.amazonaws.com
   • Local: url: dynamodb://localhost:8000

Note: For a list of AWS regions and other parameters that you can use in your URL, see AWS service endpoints. For example, url: https://dynamodb.us-west-2.amazonaws.com.

Tip: To apply a Liquibase Pro key to your project, add the following property to the Liquibase properties file: licenseKey: <paste code here>

Test your connection

1. Create a text file called changelog (.xml, .yaml, or .json) in your project directory and add a changeset.

• YAML example
• JSON example
• XML example

databaseChangeLog:
  - changeSet:
      id: 1
      author: your.name
      changes:
        - createDynamoTable:
            tableName: Music3YAML
            billingMode: PROVISIONED
            tableClass: STANDARD
            deletionProtectionEnabled: false
            keySchema:
              hashAttributeName: Artist
              rangeAttributeName: SongTitle
            attributeDefinitions:
              - attributeDefinition:
                  attributeName: Artist
                  attributeType: S
              - attributeDefinition:
                  attributeName: SongTitle
                  attributeType: S
              - attributeDefinition:
                  attributeName: Album
                  attributeType: N
            provisionedThroughput:
                readCapacityUnits: 5
                writeCapacityUnits: 5
            localSecondaryIndexes:
                - localSecondaryIndex:
                    indexName: localSecondaryIndex
                    keySchema:
                      hashAttributeName: Artist
                      rangeAttributeName: Album
                    projection:
                        projectionType: INCLUDE
                        nonKeyAttributes:
                          - attributeName: SongTitle
            globalSecondaryIndexes:
                - globalSecondaryIndex:
                    indexName: SongTitleGlobalIndex
                    keySchema:
                      hashAttributeName: Artist
                      rangeAttributeName: SongTitle
                    projection:
                        projectionType: INCLUDE
                        nonKeyAttributes:
                          - attributeName: Album
                    provisionedThroughput:
                        readCapacityUnits: 5
                        writeCapacityUnits: 5
            streamSpecification:
              streamEnabled: true
              streamViewType: NEW_IMAGE
            sseSpecification:
              enabled: true
              sseType: KMS
            tags:
              - key: tag
                value: tagValue
              - key: tag2
                value: tagValue2
      rollback:
        deleteDynamoTable:
          tableName: Music3YAML

{
  "databaseChangeLog": [
    {
      "changeSet": {
        "id": "1",
        "author": "your.name",
        "changes": [
          {
            "createDynamoTable": {
              "tableName": "Music3JSON",
              "billingMode": "PROVISIONED",
              "tableClass": "STANDARD",
              "deletionProtectionEnabled": false,
              "keySchema": {
                "hashAttributeName": "Artist",
                "rangeAttributeName": "SongTitle"
              },
              "attributeDefinitions": [
                {
                  "attributeDefinition": {
                    "attributeName": "Artist",
                    "attributeType": "S"
                  }
                },
                {
                  "attributeDefinition": {
                    "attributeName": "SongTitle",
                    "attributeType": "S"
                  }
                },
                {
                  "attributeDefinition": {
                    "attributeName": "Album",
                    "attributeType": "N"
                  }
                }
              ],
              "provisionedThroughput": {
                "readCapacityUnits": 5,
                "writeCapacityUnits": 5
              },
              "localSecondaryIndexes": [
                {
                  "localSecondaryIndex": {
                    "indexName": "localSecondaryIndex",
                    "keySchema": {
                      "hashAttributeName": "Artist",
                      "rangeAttributeName": "Album"
                    },
                    "projection": {
                      "projectionType": "INCLUDE",
                      "nonKeyAttributes": [
                        {
                          "attributeName": "SongTitle"
                        }
                      ]
                    }
                  }
                }
              ],
              "globalSecondaryIndexes": [
                {
                  "globalSecondaryIndex": {
                    "indexName": "SongTitleGlobalIndex",
                    "keySchema": {
                      "hashAttributeName": "Artist",
                      "rangeAttributeName": "SongTitle"
                    },
                    "projection": {
                      "projectionType": "INCLUDE",
                      "nonKeyAttributes": [
                        {
                          "attributeName": "Album"
                        }
                      ]
                    },
                    "provisionedThroughput": {
                      "readCapacityUnits": 5,
                      "writeCapacityUnits": 5
                    }
                  }
                }
              ],
              "sseSpecification": {
                "enable": true,
                "sseType": "KMS"
              },
              "streamSpecification": {
                "streamEnabled": true,
                "streamViewType": "NEW_IMAGE"
              },
              "tags": [
                {
                  "tag": {
                    "key": "tag",
                    "value": "tagValue"
                  }
                },
                {
                  "tag": {
                    "key": "tag2",
                    "value": "tagValue2"
                  }
                }
              ]
            }
          }
        ],
        "rollback": [
          {
            "deleteDynamoTable": {
              "tableName": "Music3JSON"
            }
          }
        ]
      }
    }
  ]
}

<databaseChangeLog
    xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:dynamodb="http://www.liquibase.org/xml/ns/pro-dynamodb"
    xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog  http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-latest.xsd
    http://www.liquibase.org/xml/ns/pro-dynamodb http://www.liquibase.org/xml/ns/pro-dynamodb/liquibase-pro-dynamodb-latest.xsd">

    <changeSet id="1" author="your.name">
        <dynamodb:createDynamoTable tableName="Music3XML" billingMode="PROVISIONED" tableClass="STANDARD" deletionProtectionEnabled="false">
            <dynamodb:keySchema hashAttributeName="Artist" rangeAttributeName="SongTitle"/>
            <dynamodb:attributeDefinitions>
                <dynamodb:attributeDefinition attributeName="Artist" attributeType="S"/>
                <dynamodb:attributeDefinition attributeName="SongTitle" attributeType="S"/>
                <dynamodb:attributeDefinition attributeName="Album" attributeType="N"/>
            </dynamodb:attributeDefinitions>
            <dynamodb:provisionedThroughput readCapacityUnits="5" writeCapacityUnits="5"/>
            <dynamodb:localSecondaryIndexes>
                <dynamodb:localSecondaryIndex indexName="localSecondaryIndex">
                    <dynamodb:keySchema hashAttributeName="Artist" rangeAttributeName="Album"/>
                    <dynamodb:projection projectionType="INCLUDE">
                        <dynamodb:nonKeyAttributes attributeName="SongTitle"/>
                    </dynamodb:projection>
                </dynamodb:localSecondaryIndex>
            </dynamodb:localSecondaryIndexes>
            <dynamodb:globalSecondaryIndexes>
                <dynamodb:globalSecondaryIndex indexName="SongTitleGlobalIndex">
                    <dynamodb:keySchema hashAttributeName="Artist" rangeAttributeName="SongTitle"/>
                    <dynamodb:projection projectionType="INCLUDE">
                        <dynamodb:nonKeyAttributes attributeName="Album"/>
                    </dynamodb:projection>
                    <dynamodb:provisionedThroughput readCapacityUnits="5" writeCapacityUnits="5"/>
                </dynamodb:globalSecondaryIndex>
            </dynamodb:globalSecondaryIndexes>
            <dynamodb:tags>
                <dynamodb:tag key="tag" value="tagValue"/>
                <dynamodb:tag key="tag2" value="tagValue2"/>
            </dynamodb:tags>
            <dynamodb:streamSpecification streamEnabled="true" streamViewType="NEW_IMAGE"/>
            <dynamodb:sseSpecification enabled="true" sseType="KMS"/>
        </dynamodb:createDynamoTable>

        <rollback>
            <dynamodb:deleteDynamoTable tableName="Music3XML"/>
        </rollback>
    </changeSet>

</databaseChangeLog>

2. Navigate to your project folder in the CLI and run the Liquibase status command to see whether the connection is successful:

liquibase status --changelog-file=<changelog.xml>

Note: You can specify arguments in the CLI or keep them in the Liquibase properties file.

If your connection is successful, you'll see a message like this:

4 changesets have not been applied to <your_connection_url>
Liquibase command 'status' was executed successfully.

3. Then execute these changes to your database with the update command:

liquibase update --changelog-file=<changelog.xml>

If your update is successful, Liquibase runs each changeset and displays a summary message ending with:

Liquibase: Update has been successful.
Liquibase command 'update' was executed successfully.

4. From a database UI tool, ensure that your database contains the Music3<format> object you added along with the DATABASECHANGELOG table and DATABASECHANGELOGLOCK table.

Now you're ready to start making deployments with Liquibase!

Supported features

To see a list of supported Liquibase features in Amazon DynamoDB, see the Liquibase Amazon DynamoDB Pro Extension Overview.

You can use most Liquibase commands in Amazon DynamoDB except database inspection commands and commands that generate SQL.

For examples of Change Types and parameters for Amazon DynamoDB, see the following pages:

• Liquibase Change Types for Amazon DynamoDB Pro
• Liquibase Parameters for Amazon DynamoDB Pro

Troubleshooting

As of the Liquibase DynamoDB extension version 1.1.0, if you use Maven, you cannot specify Liquibase Parameters for Amazon DynamoDB Pro directly in your pom.xml file. For example, the following syntax does not work in your POM:

<liquibase.dynamodb.waiter.create.maxAttempts>-1</liquibase.dynamodb.waiter.create.maxAttempts>

Instead, you must specify them in a liquibase.properties file that you reference in your POM. For example:

<plugin>
  <groupId>org.liquibase</groupId>
  <artifactId>liquibase-maven-plugin</artifactId>
  <configuration>
    <propertyFile>src/main/resources/liquibase/liquibase.properties</propertyFile>
  </configuration>
<plugin>

Then, in your liquibase.properties file, you can specify the DynamoDB parameter(s). For example:

liquibase.dynamodb.waiter.create.maxAttempts: -1

Related links

• Getting started with DynamoDB
• Liquibase Commands
• Workflows