Liquibase Amazon DynamoDB Pro Extension Overview

Amazon DynamoDB is a fully managed, flexible-schema NoSQL database service. You can use Amazon DynamoDB to create a database table that can store and retrieve any amount of data, and serve any level of request traffic. Amazon DynamoDB automatically spreads the data and traffic for the table over a sufficient number of servers to handle the request capacity specified by the customer and the amount of data stored. For more information, see the Amazon DynamoDB Documentation.

You can deploy Amazon DynamoDB changes in YAML, JSON, and XML changelogs using the Amazon DynamoDB Extension for Liquibase Pro. However, because Amazon DynamoDB does not use SQL, you cannot use SQL changelogs or Liquibase commands that generate SQL output. Instead, you can specify some statements in PartiQL (a SQL-compatible query language) using the partiql and partiqlFile Change Types. You can also use Liquibase DynamoDB Change Types to create, update, and delete tables and indexes. For a list of supported features, see the "Features" section.

Getting started tutorial

To learn how to install, configure, and use the Liquibase Amazon DynamoDB extension with your instance of Amazon DynamoDB, see Using Liquibase with Amazon DynamoDB Pro. This page contains driver download links, AWS permissions guidance, and a sample changelog to use for a test deployment.

Verified database versions

Liquibase Amazon DynamoDB extension version	Liquibase Pro version required	Amazon DynamoDB versions verified
1.1.0	4.28.0+	Cloud
1.0.0	4.26.0+

For release notes, see Amazon DynamoDB Extension Release Notes.

Features

The Liquibase Amazon DynamoDB extension supports many Liquibase Open Source and Liquibase Pro features, including:

• Targeted updates and targeted rollbacks (per changeset and per update)
• Liquibase Flow Files
• Structured Logging
• Secrets Management
• Using resources stored on AWS S3
• Some Policy Checks

Amazon DynamoDB billing modes

Amazon DynamoDB has two pricing models for the number of read and write operations you perform on tables in your database every second:

• On-demand (PAY_PER_REQUEST): only pay for requests you actually make
• Provisioned (PROVISIONED): you pay for a specific maximum number of requests per second

For more information, see Read/write capacity mode.

Liquibase connects to the DynamoDB API to let you configure the billing mode of each table in your database. You can configure the billing mode for the Liquibase Tracking Tables separately.

Amazon DynamoDB waiters

When you deploy a change, Liquibase submits a request and then DynamoDB processes it. Internally, DynamoDB uses waiters to ensure that objects are in the correct state before other code can modify them.

Asynchronous DynamoDB operations can take a long time, such as increases to a table's provisioned capacity. Synchronous operations are typically faster, such as simple updates to data in a table. Liquibase does not control the sequence or timing of DynamoDB operations.

Liquibase runs changesets sequentially. You can use Liquibase waiter parameters to specify how long Liquibase should wait for DynamoDB to respond while deploying a change. If Liquibase times out, it moves onto the next changeset, even though DynamoDB may not have finished.

It may be necessary to set higher wait times to give DynamoDB flexibility while making complex changes; or lower wait times to ensure that a single time-consuming changeset does not delay your whole CI/CD process. However, if Liquibase skips a changeset that other changesets depend on, the dependent changesets may fail. You can disable waiters entirely, but this can also lead to failures.

For a list of Liquibase DynamoDB waiter parameters, see: Supported parameters.

Note: Liquibase does not use Amazon DynamoDB waiters when you create, update, or delete indexes.

Amazon DynamoDB objects

DynamoDB has a different structure than relational SQL databases. Liquibase connects to the DynamoDB API to run operations on your database. The following are objects your DynamoDB database contains:

• Tables: a group of items. Normalization is not enforced.
  • Items: similar to a record (row) in a relational database, but every item can have its own unique attributes.
  • Attributes: similar to a field (column) in a relational database, formatted as key-value pairs.
• Primary keys: DynamoDB assigns a unique primary key to every item in every table in your database, but is otherwise schemaless.
  • Partition keys: a non-composite primary key that lets you query items in your tables.
  • Sort keys: differentiates items that share a partition key. When combined with a partition key, a sort key is a composite primary key.
• Secondary indexes: lets you query the data in the table using an alternate key, in addition to queries against the primary key.
  • Global secondary indexes: indexes with a partition key and sort key that can be different from those on the table.
  • Local secondary indexes: indexes with the same partition key as the table, but a different sort key.
• Streams: near-real time records that capture data modifications in DynamoDB

For more information, see Core components of Amazon DynamoDB.

Supported Change Types

Change Types in the Liquibase DynamoDB extension mirror actions in the DynamoDB API and have similar attributes. Normal Liquibase Change Types are not supported in Amazon DynamoDB. You must use the following Change Types in your changelog:

• Tables:
  • createDynamoTable
  • deleteDynamoTable
  • updateDynamoTable
  • updateTableProvisionedThroughput
• Indexes:
  • createGlobalSecondaryIndex
  • deleteGlobalSecondaryIndex
  • updateGlobalSecondaryIndex
• PartiQL:
  • partiql
  • partiqlFile

For more information, see Liquibase Change Types for Amazon DynamoDB Pro.

Supported parameters

You can use all normal Liquibase Parameters with Amazon DynamoDB.

To configure DynamoDB-specific behavior, such as the billing mode for the Liquibase Tracking Tables and the behavior of the DynamoDB waiter, you can set these Liquibase parameters in the CLI, in flow files, in your liquibase.properties file, or as environment variables:

• Tracking tables:
  • --dynamodb-tracking-tables-billing-mode
  • --dynamodb-tracking-tables-provisioned-throughput-read-capacity-units
  • --dynamodb-tracking-tables-provisioned-throughput-write-capacity-units
• Waiters:
  • For configuration:
    • --dynamodb-waiters-enabled (overrides all other waiter settings)
    • --dynamodb-waiters-fail-on-timeout
  • For logging:
    • --dynamodb-waiters-log-notification-enabled
    • --dynamodb-waiters-log-notification-interval
  • For creating objects:
    • --dynamodb-waiter-create-fixed-delay-backoff-strategy-duration
    • --dynamodb-waiter-create-max-attempts
    • --dynamodb-waiter-create-total-timeout
  • For deleting objects:
    • --dynamodb-waiter-delete-fixed-delay-backoff-strategy-duration
    • --dynamodb-waiter-delete-max-attempts
    • --dynamodb-waiter-delete-total-timeout
  • For updating objects:
    • --dynamodb-waiter-update-fixed-delay-backoff-strategy-duration
    • --dynamodb-waiter-update-max-attempts
    • --dynamodb-waiter-update-total-timeout

For more information, see Liquibase Parameters for Amazon DynamoDB Pro.

Supported commands

Liquibase supports the following commands for Amazon DynamoDB:

• All update and rollback commands, except for those that generate SQL output
• The connect command works as of Liquibase Amazon DynamoDB Pro Extension v1.1
• All change tracking and utility commands, except for execute-sql and those that generate SQL output
• The flow and flow validate commands
• All other commands, unless otherwise specified in the Limitations section.

For more information, see Liquibase Commands.

Supported policy checks

Liquibase supports the following policy checks for Amazon DynamoDB:

Checks for the changelog:

• ChangesetCommentCheck
• ChangesetContextCheck
• ChangesetLabelCheck
• OneChangePerChangeset
• RequireChangesetIDisUUID
• RollbackRequired
• SqlSelectStarWarn
• SqlUserDefinedPatternCheck
• userDefinedCommentCheck
• UserDefinedContextCheck
• UserDefinedLabelCheck

For more information, see Changelog Policy Checks.

Limitations

• DynamoDB uses a distributed data model to partition data to run on different servers. Read operations are eventually consistent by default, which may increase the latency of some Liquibase operations.
• The Liquibase Amazon DynamoDB extension does not support any normal Liquibase Change Types. You must use the Change Types specified in the Supported Change Types section.
• The Liquibase Amazon DynamoDB extension does not support the following commands:
  • Any command that generates SQL output, such as update-sql and rollback-sql
  • Database inspection commands: diff, diff-changelog, generate-changelog, snapshot, snapshot-reference
  • The execute-sql command
• The partiql and partiqlFile Change Types do not support JSON PartiQL queries
  • You can still specify non-JSON PartiQL queries in Liquibase JSON changelogs
• If you use Maven, you cannot specify Liquibase Parameters for Amazon DynamoDB Pro directly in your pom.xml file. Instead, you must specify them in a liquibase.properties file that you reference in your POM. For more information, see Using Liquibase with Amazon DynamoDB Pro.

Related links

• Developer Guide: What is Amazon DynamoDB?
• API Reference: Amazon DynamoDB Actions