Create a Python Virtual Environment

When you write Custom Policy Checks, you can access the Liquibase API by using Liquibase Python Modules. Liquibase installs these modules for you when you download the Liquibase Checks extension, along with a few natively supported Python modules.

Liquibase is a Java application, so it uses the GraalPy runtime on the back-end to execute all Python scripts for custom policy checks. Liquibase runs all Python scripts in a built-in local Python virtual environment. Virtual environments help you isolate your projects so that you can have different versions of modules in each one.

This tutorial shows you how to create a custom Python virtual environment instead of using the one Liquibase provides.

Uses

You can create and run custom policy checks exlusively using the built-in virtual environment and modules that come with the Liquibase Checks extension.

However, you must create a custom virtual environment if you want to:

  • Write policy checks using some external Python modules. Python installs its standard library when you download it. Additionally, the Liquibase Checks JAR comes with several Liquibase modules as well as a few external modules (simplejson, sqlparse, and urllib3). However, if you create a virtual environment, you can install any modules you want, including the built-in modules with the liquibase-checks-python package.
  • Ensure your IDE recognizes the Liquibase API. By default, your IDE does not know to look in a Java JAR file for Python-language libraries. As a result, your IDE can't provide in-IDE documentation or auto-complete using the built-in virtual environment, making development more tedious. However, if you create a custom virtual environment, your IDE can recognize any modules you install, like the Liquibase Python modules. If you don't use an IDE, this makes no difference.

To create your own virtual environment, you must install the GraalPy binary and then use the Python pip tool to install all external modules you want to use in your virtual environment (including Liquibase modules).

Prerequisites

Create your own virtual environment

  1. Download and extract the latest version of GraalPy on your machine. You must use the GraalPy runtime to create the virtual environment, not the Python runtime.
  2. Update your PATH environment variable to include the path to the GraalPy /bin directory you just installed.
  3. In your command line, invoke GraalPy to run the venv module to create a virtual environment in the directory you specify:
    4. graalpy -m venv <path_to_venv>

    Your command line won't display anything, but this creates some scripts. You may have to wait a few seconds for the command to execute before you can type in the terminal again.

    Note: Some terminals may display: "WARNING: Unable to create a system terminal, creating a dumb terminal (enable debug logging for more information)." You can ignore this.

Install Python modules

By default, your virtual environment has no modules installed. To install them, follow these steps:

  1. Activate the virtual environment:
    • Windows CMD and PowerShell: <path_to_venv>\Scripts\activate
    • Windows GitBash: source <path_to_venv>/Scripts/activate
    • Unix: source <path_to_venv>/bin/activate
  2. Configure your IDE to recognize your custom virtual environment.
  3. Install modules and packages using pip:
    4. pip install liquibase-checks-python simplejson sqlparse urllib3

    When complete, the terminal displays:

    Successfully installed liquibase-checks-python-<x.y.z> simplejson-<x.y.z> sqlparse-<x.y.z> urllib3-<x.y.z>

    Note: You must install liquibase-checks-python to access the Liquibase API, and you must install sqlparse to run the Liquibase modules.

  4. Deactivate the virtual environment:
    • Windows GitBash, Windows PowerShell, and Unix: deactivate
    • Windows CMD: exit

Configure Liquibase to use your virtual environment

  1. Use the Liquibase parameter --script-python-executable-path to provide Liquibase the path to the GraalPy Python executable in your virtual environment:
    • Command line at runtime: --script-python-executable-path=<path_to_venv>\Scripts\python.exe
    • Defaults file (liquibase.properties): liquibase.script.python.executablePath=<path_to_venv>\Scripts\python.exe
    • Environment variable: LIQUIBASE_SCRIPT_PYTHON_EXECUTABLE_PATH=<path_to_venv>\venv\Scripts\python.exe
    • Command line at runtime: --script-python-executable-path=<path_to_venv>/bin/python
    • Defaults file (liquibase.properties): liquibase.script.python.executablePath=<path_to_venv>/bin/python
    • Environment variable: LIQUIBASE_SCRIPT_PYTHON_EXECUTABLE_PATH=<path_to_venv>/bin/python
  2. In your Python file, ensure that any imports of modules come from your custom virtual environment instead of the built-in virtual environment in the Liquibase Checks extension JAR. Replace import <module> with from liquibase_checks_python import <module>. For example:
    3. # import Liquibase modules containing useful functions
from liquibase_checks_python import liquibase_utilities as lb
import sys

# define reusable variables
obj = lb.get_database_object() # database object to examine
status = lb.get_status() # Status object of the check

# write check logic
if lb.is_table(obj): # checks if the current object is a table
    status.fired = True # indicates that the custom check has been triggered
    status.message = "No tables allowed!" # message for Liquibase to return when check is triggered
    sys.exit(1) # halt execution of the script

# default return code
False

You have now successfully configured your virtual environment for Liquibase policy checks.

If you restart your IDE, you should also see auto-complete prompts for Liquibase Python modules.

Run custom policy checks with a virtual environment

If you haven't yet run any Liquibase policy checks commands to configured your Python script as a check, follow the steps on Create a Custom Policy Check.

If you've already configured your custom policy check, you can now run it using your new virtual environment. For example:

  • Windows:
    • liquibase --script-python-executable-path=<path/to/venv>/Scripts/python.exe checks run --check-name=MySweetPythonCheck --checks-scripts-enabled=true --checks-scope=database
  • Linux/macOS:
    • liquibase --script-python-executable-path=<path/to/venv>/bin/python checks run --check-name=MySweetPythonCheck --checks-scripts-enabled=true --checks-scope=database

If you specified --script-python-executable-path in your liquibase.properties file or as an environment variable, you don't need to specify it at runtime.

Note: If you don't specify a custom virtual environment to use, Liquibase runs your script in the built-in virtual environment.

Troubleshooting

liquibase_checks_python module not found

Your IDE may not display correct syntax highlighting for your Liquibase module imports if you haven't configured your virtual environment correctly, or haven't configured your IDE to recognize your virtual environment.

You may also receive this error if --script-python-executable-path is set to the Python executable in your custom virtual environment but you haven't correctly installed your modules:

Error while executing script 'python_scripts/test_python_check.py': ModuleNotFoundError: No module named
'liquibase_checks_python' line:  2

To fix this, ensure that you:

  1. Install the correct version of GraalPy. In Liquibase 4.30.0 and earlier, Linux users must install GraalPy 24.0.0 (not the latest version). In Liquibase 4.31.0 and later, you can use the latest version of GraalPy.
  2. Configure your IDE to recognize Python-language files.
  3. Configure your IDE to use the GraalPy Python executable in your custom virtual environment.
  4. Activate your virtual environment. Then import the liquibase-checks-python package and any other Python modules you want to use.
    5. pip install liquibase-checks-python simplejson sqlparse urllib3
  5. Restart your IDE to see any changes.

Next steps