In part 1 we looked at the Dataform CompilationResult and WorkflowInvocation objects. Let's follow up with a quick look at some other API endpoints that I find useful.
Workspace management - delete_workspace and create_workspace
delete_workspace is essential for Dataform instances used by large teams as it allows workspaces to be automatically deleted post-merge, saving the bi-annual "can everyone please let me know which workspaces they're finished with?" message. This can be integrated into CI/CD tools such as GitHub Actions.
from google.cloud import dataform_v1
client = dataform_v1.DataformClient()
workspace_path = client.workspace_path(
PROJECT_ID,
REGION,
REPOSITORY_ID,
WORKSPACE_ID,
)
try:
operation = client.delete_workspace(name=workspace_path)
There's also create_workspace - very useful for automatically creating a workspace if a code change is made on a new branch outside of the Dataform UI.
Git operations - push_git_changes and pull_git_changes
Another set of useful endpoints that allow code changes to be automatically applied outside of the Dataform UI:
from google.cloud import dataform_v1
client = dataform_v1.DataformClient()
workspace_path = client.workspace_path(
project_id,
region,
repository_id,
workspace_id,
)
# ------------------------------------------------------------
# 1. Pull latest changes from Git into the workspace
# ------------------------------------------------------------
print("Pulling latest Git changes into workspace...")
pull_request = dataform_v1.PullGitChangesRequest(
name=workspace_path
)
pull_operation = client.pull_git_changes(request=pull_request)
print("Pull initiated:", pull_operation.operation.name)
# Optional: wait for completion (simplified polling)
pull_result = pull_operation.result()
print("Pull completed successfully")
# ------------------------------------------------------------
# (Optional) 2. Make programmatic changes here
# ------------------------------------------------------------
# e.g. update config blocks, inject variables, generate models, format SQL, etc.
print("Applying automated workspace changes...")
# ------------------------------------------------------------
# 3. Push workspace changes back to Git
# ------------------------------------------------------------
print("Pushing workspace changes to Git...")
push_request = dataform_v1.PushGitChangesRequest(
name=workspace_path
)
push_operation = client.push_git_changes(request=push_request)
push_result = push_operation.result()
print("Push completed successfully")
print("Workspace successfully synced with Git")
