Using Liquibase with BigQuery

Google BigQuery is a fully managed analytics data warehouse. For more information, see BigQuery Documentation.

Supported versions

  • 2.13.6

Verification level

Compatible: This database is widely reported by the community to work with Liquibase. Automated testing may verify partial Liquibase Open Source and Liquibase Pro functionality. The Liquibase support team provides best-effort usage support to Commercial users.

Note: A database's verification level indicates how well it works with different features in Liquibase and across different products, such as Liquibase Open Source and Liquibase Pro. For more information, see Liquibase Database Tutorials.

Prerequisites

Install drivers

To use Liquibase and BigQuery, you need two JAR files:

Place your JAR file(s) in the liquibase/lib directory. Read more: Adding and Updating Liquibase Drivers.

If you use Maven, you must include the driver JAR as a dependency in your pom.xml file. Read more: Configuring Liquibase Attributes in your Maven POM File.

<dependency>
    <groupId>com.google.cloud</groupId>
    <artifactId>google-cloud-bigquery</artifactId>
    <version>2.13.6</version>
</dependency>
<dependency>
    <groupId>org.liquibase.ext</groupId>
    <artifactId>liquibase-bigquery</artifactId>
    <version>4.17.2</version>
</dependency>

Test your connection

  1. Ensure your BigQuery database is configured. See BigQuery Quickstarts for more information. For example, you can run a query of a sample table in BigQuery using the bq command-line tool:
  2. bq show bigquery-public-data:samples.shakespeare
  3. Specify the database URL in the Liquibase properties file. Liquibase does not parse the URL. You can either specify the full database connection string or specify the URL using your database's standard JDBC format:

  4. url: jdbc:bigquery://https://googleapis.com/bigquery/v2:443/<dbname>;ProjectId=<STR>;OAuthType=<INT>;

    Specify the name of your database in place of dbname. Specify the ID of your BigQuery project as the value of ProjectId. Specify your BigQuery authentication method as the value of OAuthType:

    OAuthType Description Notes Example
    0 Google Services Account

    You must also specify the options OAuthServiceAcctEmail and OAuthPvtKeyPath in your url property. For more information, see:

    jdbc:bigquery://https://googleapis.com/bigquery/v2:443/myDatabase;
    ProjectId=myProject;
    OAuthType=0;
    OAuthServiceAcctEmail=lbtest@bq123.iam.gserviceaccount.com;
    OAuthPvtKeyPath=C:\path\serviceKey.p12;
    1 Google User Account

    Requires your user account credentials. For more information, see:

    jdbc:bigquery://https://googleapis.com/bigquery/v2:443/myDatabase;
    ProjectId=myProject;
    OAuthType=1;
    2 Google Authorization Server Access Token

    You must also specify the options OAuthAccessToken, OAuthRefreshToken, OAuthClientId, and OAuthClientSecret in your url property. For more information, see:

    jdbc:bigquery://https://googleapis.com/bigquery/v2:443/myDatabase;
    ProjectId=myProject;
    OAuthType=2;
    OAuthAccessToken=a25c7cfd36214f94a79d;
    OAuthRefreshToken=2kl0Qvuw9qt4abia54qga5t97;
    OAuthClientId=22n6627g243322f7;
    OAuthClientSecret=cDE+F2g3Hcjk4K5lazM;
    3 Application Default Credentials

    For more information, see:

    jdbc:bigquery://https://googleapis.com/bigquery/v2:443/myDatabase;
    ProjectId=myProject;
    OAuthType=3;

    For more information, see Simba JDBC Authentication Driver Configuration Options.

    Tip: To apply a Liquibase Pro key to your project, add the following property to the Liquibase properties file: liquibaseProLicenseKey: <paste code here>

  1. Create a text file called changelog (.xml, .sql, .json, or .yaml) in your project directory and add a changeset.
  2. Navigate to your project folder in the CLI and run the Liquibase status command to see whether the connection is successful:
  3. liquibase --username=test --password=test --changelog-file=<changelog.xml> status

    Note: You can pass arguments in the CLI or keep them in the Liquibase properties file.

  4. Inspect the SQL with the update-sql command. Then make changes to your database with the update command.
  5. liquibase --changelog-file=<changelog.xml> update-sql
    liquibase --changelog-file=<changelog.xml> update
  6. From a database UI tool, ensure that your database contains the test_table you added along with the DATABASECHANGELOG table and DATABASECHANGELOGLOCK table.

Related links