You can write your entire cloud infrastructure in Python, and it’s not just a wrapper around cloud APIs.
Let’s see how Pulumi manages a simple S3 bucket in Python.
import pulumi
import pulumi_aws as aws
# Create an AWS resource (S3 bucket)
bucket = aws.s3.Bucket("my-bucket")
# Export the name of the bucket
pulumi.export("bucket_name", bucket.id)
This code defines an S3 bucket. When you run pulumi up, Pulumi talks to AWS, sees you want a bucket named "my-bucket," and creates it. If the bucket already exists, Pulumi knows that too and does nothing. It’s not just blindly calling cloud APIs; it’s maintaining state and understanding the desired versus actual infrastructure.
The core problem Pulumi solves is managing infrastructure as code, but with the power of a general-purpose programming language. Instead of declarative YAML or JSON, you use Python (or TypeScript, Go, C#, Java). This means you get loops, conditionals, functions, classes, and the vast ecosystem of Python libraries. You can generate infrastructure dynamically, abstract common patterns into reusable functions or classes, and even perform complex computations to determine resource properties.
Internally, Pulumi works by creating a resource graph. When you define resources like aws.s3.Bucket, Pulumi doesn’t immediately call AWS. Instead, it records that you want an S3 bucket with certain properties. When you define another resource that depends on the first (e.g., an S3 bucket object), Pulumi knows this dependency and builds a graph. This graph is then sent to the Pulumi engine, which orchestrates the creation, update, or deletion of resources in the correct order, ensuring dependencies are met. The engine communicates with the cloud providers via their APIs, but it uses the resource graph and its own state management to ensure idempotency and correctness.
The pulumi.export statements are how you get information out of your infrastructure stack. These are values that Pulumi will display after a successful deployment, like the ID of the bucket, its ARN, or a website endpoint. You can reference these exports in other stacks or use them in your applications.
When you define a resource, you’re not just providing static values. You can use Python’s capabilities to make your infrastructure dynamic. For instance, imagine you need to create multiple S3 buckets based on a list of environments:
import pulumi
import pulumi_aws as aws
environments = ["dev", "staging", "prod"]
buckets = {}
for env in environments:
bucket = aws.s3.Bucket(f"my-{env}-bucket")
buckets[env] = bucket
pulumi.export(f"{env}_bucket_name", bucket.id)
# You can also access properties of created resources
pulumi.export("first_bucket_arn", buckets["dev"].arn)
This loop dynamically creates three S3 buckets, one for each environment. The naming is programmatic, and you can export each bucket’s name and even the ARN of the first one. This level of programmatic control is a significant departure from purely declarative approaches.
The core of Pulumi’s state management is a backend service that stores the desired and actual state of your infrastructure. This backend can be Pulumi’s managed service, or you can self-host it using options like an S3 bucket, Azure Blob Storage, or Google Cloud Storage. When you run pulumi up, the Pulumi CLI compares your desired state (defined in your code) with the current state stored in the backend. It then calculates a diff and generates an execution plan, which is what you see before confirming the changes.
One common pitfall is how Pulumi handles resource replacement versus in-place updates. For many resource properties, changing them will result in an in-place update (e.g., changing the lifecycle rules of an S3 bucket). However, for certain fundamental properties, like changing the bucket_name of an S3 bucket after it’s been created, Pulumi cannot perform an in-place update. In such cases, Pulumi will propose to delete the old resource and create a new one with the updated property. This is a critical distinction because deleting a resource like an S3 bucket will also delete its contents. Pulumi will always prompt you to confirm such destructive operations, but understanding why a replacement is proposed is key to avoiding accidental data loss.
The next step in mastering Pulumi is understanding how to manage multiple stacks and the concept of component resources for better organization.