What is the Liquibase Amazon DynamoDB Secure Extension?

Features

The Liquibase Secure AWS extension supports many Liquibase Open Source and Liquibase Secure features, including:

• Targeted updates and targeted rollbacks (per changeset and per update)

• Liquibase Flow Files

• Structured Logging

• Secrets Management

• Using resources stored on AWS S3

• Some Policy Checks

Amazon DynamoDB billing modes

Amazon DynamoDB has two pricing models for the number of read and write operations you perform on tables in your database every second:

• On-demand (PAY_PER_REQUEST): only pay for requests you actually make

• Provisioned (PROVISIONED): you pay for a specific maximum number of requests per second

For more information, see Read/write capacity mode.

Liquibase connects to the DynamoDB API to let you configure the billing mode of each table in your database. You can configure the billing mode for the Liquibase tracking tables separately.

Amazon DynamoDB waiters

When you deploy a change, Liquibase submits a request, which DynamoDB processes. Internally, DynamoDB uses waiters to ensure that objects are in the correct state before other code can modify them.

Asynchronous DynamoDB operations, such as increasing a table's provisioned capacity, can take a long time. Synchronous operations, such as simple updates to data in a table, are typically faster. Liquibase does not control the sequence or timing of DynamoDB operations.

Liquibase runs changesets sequentially. You can use Liquibase waiter parameters to specify how long Liquibase should wait for DynamoDB to respond while deploying a change. If Liquibase times out, it moves onto the next changeset, even though DynamoDB may not have finished.

It may be necessary to set higher wait times to give DynamoDB flexibility while making complex changes, or lower wait times to ensure that a single time-consuming changeset does not delay your whole CI/CD process. However, if Liquibase skips a changeset that other changesets depend on, the dependent changesets may fail. You can disable waiters entirely, but this can also lead to failures.

For a list of Liquibase DynamoDB waiter parameters, see: Supported parameters.

Note: Liquibase does not use Amazon DynamoDB waiters when you create, update, or delete indexes.

Amazon DynamoDB transactions

Starting in Liquibase 5.2, partiql and partiqlFile changesets execute as atomic DynamoDB transactions by default. There are no partial writes, and no manual recovery is required.

Liquibase has two limits per changeset: 100 items and 4 MB aggregate payload size. If a changeset exceeds either limit, Liquibase stops before executing anything. Split the changeset or set runInTransaction="false" on the changeset to opt out of transaction wrapping.

<changeSet id="bulk-load-products" author="your.name" runInTransaction="false"> <dynamodb-pro:partiqlFile path="bulk-insert-products.partiql" splitStatements="true" endDelimiter=";"/> </changeSet>

Note: DDL operations (createDynamoTable, deleteDynamoTable, updateDynamoTable, and index changes) cannot participate in DynamoDB transactions and execute immediately. If you mix DDL and DML in a single changeset and the DML transaction fails, the DDL changes cannot be rolled back. Keep DDL and DML in separate changesets.

For retry behavior after a failed DATABASECHANGELOG write, see DynamoDB transaction retry behavior.

Amazon DynamoDB objects

DynamoDB has a different structure than relational SQL databases. Liquibase connects to the DynamoDB API to run operations on your database. The following are objects your DynamoDB database contains:

Tables: a group of items. Normalization is not enforced.

• Items: similar to a record (row) in a relational database, but every item can have its own unique attributes.

• Attributes: similar to a field (column) in a relational database, formatted as key-value pairs.

Primary keys: DynamoDB assigns a unique primary key to every item in every table in your database, but is otherwise schemaless.

• Partition keys: a non-composite primary key that lets you query items in your tables.

• Sort keys: differentiates items that share a partition key. When combined with a partition key, a sort key is a composite primary key.

Secondary indexes: lets you query the data in the table using an alternate key, in addition to queries against the primary key.

• Global secondary indexes: indexes with a partition key and sort key that can be different from those on the table.

• Local secondary indexes: indexes with the same partition key as the table, but a different sort key.

Streams: near-real-time records that capture data modifications in DynamoDB

For more information, see Core components of Amazon DynamoDB.

Change Types in the Liquibase Secure AWS extension mirror actions in the DynamoDB API and have similar attributes. Normal Liquibase Change Types are not supported in Amazon DynamoDB. You must use the following Change Types in your changelog:

Tables

• createDynamoTable

• deleteDynamoTable

• updateDynamoTable

• updateTableProvisionedThroughput

Indexes

• createGlobalSecondaryIndex

• deleteGlobalSecondaryIndex

• updateGlobalSecondaryIndex

PartiQL

• partiql

• partiqlFile

For more information, see Liquibase Change Types for Amazon DynamoDB Secure.

Supported parameters

You can use all normal Liquibase parameters with Amazon DynamoDB.

To configure DynamoDB-specific behavior, such as the billing mode for the Liquibase tracking tables and the behavior of the DynamoDB waiter, you can set the following Liquibase parameters in the CLI, in flow files, in your liquibase.properties file, or as environment variables:

Tracking tables

• --dynamodb-tracking-tables-billing-mode

• --dynamodb-tracking-tables-provisioned-throughput-read-capacity-units

• --dynamodb-tracking-tables-provisioned-throughput-write-capacity-units

Waiters

For configuration:

• --dynamodb-waiters-enabled (overrides all other waiter settings)

• --dynamodb-waiters-fail-on-timeout

For logging:

• --dynamodb-waiters-log-notification-enabled

• --dynamodb-waiters-log-notification-interval

For creating objects:

• --dynamodb-waiter-create-fixed-delay-backoff-strategy-duration

• --dynamodb-waiter-create-max-attempts

• --dynamodb-waiter-create-total-timeout

For deleting objects:

• --dynamodb-waiter-delete-fixed-delay-backoff-strategy-duration

• --dynamodb-waiter-delete-max-attempts

• --dynamodb-waiter-delete-total-timeout

For updating objects:

• --dynamodb-waiter-update-fixed-delay-backoff-strategy-duration

• --dynamodb-waiter-update-max-attempts

• --dynamodb-waiter-update-total-timeout

For more information, see Liquibase parameters for Amazon DynamoDB Secure.

Supported commands

Liquibase supports the following commands for Amazon DynamoDB:

• All update and rollback commands, except for those that generate SQL output

• The connect command works as of Liquibase Amazon DynamoDB Secure Extension v1.1

• All change tracking and utility commands, except for execute-sql and those that generate SQL output

• The flow and flow validate commands

• All other commands, unless otherwise specified in the Limitations section.

For more information, see Liquibase Commands.

Supported policy checks

Liquibase supports the following policy checks for Amazon DynamoDB:

Checks for the changelog:

• ChangesetCommentCheck

• ChangesetContextCheck

• ChangesetLabelCheck

• OneChangePerChangeset

• RequireChangesetIDisUUID

• RollbackRequired

• SqlSelectStarWarn

• SqlUserDefinedPatternCheck

• UserDefinedContextCheck

• UserDefinedLabelCheck

For more information, see Changelog policy checks.

Limitations

• DynamoDB uses a distributed data model to partition data to run on different servers. By default, read operations are eventually consistent, which may increase the latency of some Liquibase operations.

• The Liquibase Secure AWS extension does not support any normal Liquibase.

• Change types. You must use the Change types specified in the Supported Change Types section.

The Liquibase Secure AWS extension does not support the following commands:

• Any command that generates SQL output, such as update-sql and rollback-sql.

• Database inspection commands: diff, diff-changelog, generate-changelog, snapshot, snapshot-reference.

• The execute-sql command.

The partiql and partiqlFile Change Types do not support JSON PartiQL queries. You can still specify non-JSON PartiQL queries in Liquibase JSON changelogs.

If you use Maven, you cannot specify Liquibase parameters for Amazon DynamoDB Secure directly in your pom.xml file. Instead, you must specify them in a liquibase.properties file that you reference in your POM.

For more information, see Using Liquibase with Amazon DynamoDB Secure.

Verified database versions