The Pulumi Automation API lets you manage your infrastructure programmatically, treating your infrastructure as code within your application, not just in standalone Pulumi projects.
Let’s see it in action. Imagine you have a Pulumi project that deploys a simple AWS S3 bucket.
import pulumi
from pulumi_aws import s3
# Create an S3 bucket
bucket = s3.Bucket("my-app-bucket")
# Export the bucket name
pulumi.export("bucket_name", bucket.id)
Now, you want to deploy this from another Python application using the Automation API. You’d typically set this up in a main.py file in your application’s root.
from pulumi_automation import LocalWorkspace, StackArgs, Config
def run_pulumi_deploy():
# Define the path to your Pulumi project directory
project_dir = "./my-pulumi-project" # Assuming your Pulumi project is in a subdirectory
# Create a local workspace
workspace = LocalWorkspace(project_dir=project_dir)
# Set up the stack. If it doesn't exist, it will be created.
stack, _ = workspace.create_stack(
stack_name="dev",
stack_args=StackArgs(
# Optional: Set project configuration values
config=Config({"aws:region": "us-east-1"})
)
)
# Select the stack to work with
workspace.select_stack(stack_name="dev")
# Set any necessary secrets or config values for the stack
stack.set_config("aws:region", "us-east-1") # Example of setting config
# Perform a Pulumi update
print("Performing Pulumi update...")
try:
# The up() method performs 'pulumi up'
result = stack.up(on_output=lambda msg: print(f"Pulumi Output: {msg}"))
print(f"Deployment successful. Stack outputs: {result.outputs}")
except Exception as e:
print(f"Pulumi deployment failed: {e}")
# Perform a Pulumi destroy (optional, for cleanup)
# print("Performing Pulumi destroy...")
# try:
# destroy_result = stack.destroy(on_output=lambda msg: print(f"Pulumi Destroy Output: {msg}"))
# print(f"Destruction successful. Destroy outputs: {destroy_result.outputs}")
# except Exception as e:
# print(f"Pulumi destruction failed: {e}")
if __name__ == "__main__":
run_pulumi_deploy()
This code sets up a LocalWorkspace pointing to your Pulumi project directory. It then creates or selects a stack named "dev", configures it (e.g., setting the AWS region), and runs pulumi up via the stack.up() method. The on_output callback lets you see the real-time output from the Pulumi engine as it executes.
The core problem the Automation API solves is bridging the gap between your application’s runtime and your infrastructure provisioning. Instead of having a separate CI/CD pipeline trigger pulumi up from a shell script, you can integrate infrastructure deployment directly into your application’s lifecycle. This is powerful for scenarios like:
- Application Self-Service: An application can provision its own required infrastructure (databases, queues, etc.) upon startup or during specific events.
- Dynamic Infrastructure: Spin up ephemeral environments for testing or development on demand.
- Complex Workflows: Orchestrate infrastructure changes as part of a larger application deployment or business process.
- Internal Developer Platforms (IDPs): Build custom portals or tools that allow developers to manage their infrastructure without direct CLI access.
Internally, the LocalWorkspace object acts as a client to the Pulumi engine. When you call stack.up(), stack.destroy(), or stack.preview(), the Automation API serializes these commands and their associated configuration and sends them to the Pulumi engine running in the specified project_dir. The engine then executes the Pulumi program, interacts with the cloud provider, and reports the results back to your application.
The StackArgs object is where you define initial stack configurations, like setting project-level config values that will be applied when a new stack is created. The stack.set_config(key, value) method, on the other hand, sets configuration values specifically for an existing stack. This includes sensitive values, which can be set using stack.set_secrets(key, value), though for production, you’d typically manage secrets through Pulumi’s secret management features or external secret stores.
The result object returned by stack.up() contains valuable information, including result.outputs, which is a dictionary of the stack’s exported outputs, just like you’d see from a pulumi up command in the CLI.
One of the most subtle yet powerful aspects of the Automation API is its ability to manage multiple stacks from a single application instance. You can create, select, and operate on different stacks (e.g., dev, staging, prod) all within the same running Python program. Each stack maintains its own state and configuration independently. This allows for fine-grained control and parallel management of your infrastructure deployments without the need for separate Pulumi projects or complex environment switching.
The next step after managing stacks programmatically is often integrating this into a larger application lifecycle, such as triggering deployments based on code commits or user actions within a web interface.