Pulumi’s S3 backend is the default and most common way to store your infrastructure’s state, but it’s not just a passive dump of JSON; it’s an active component that safeguards your deployments.

Let’s see it in action. Imagine you have a simple Pulumi program to create an S3 bucket:

import pulumi
import pulumi_aws as aws

bucket = aws.s3.Bucket("my-app-bucket")

pulumi.export("bucket_name", bucket.id)

When you run pulumi up, Pulumi first checks if a state file exists in the configured S3 bucket. If it does, it reads the current state of your infrastructure. Then, it compares this state with your desired state defined in your code. If there are differences, it calculates a plan to bring your infrastructure in line with your code. After you approve the plan, Pulumi executes the changes and then writes the new state back to the S3 bucket. This state file is crucial; it’s Pulumi’s single source of truth for your managed resources.

The core problem the S3 backend solves is managing the lifecycle and consistency of your infrastructure’s state. Without a robust backend, coordinating changes across teams, recovering from failures, or even just knowing what infrastructure you actually have deployed would be a nightmare. The S3 backend provides:

  • Durability: S3 is designed for 99.999999999% durability, meaning your state is incredibly safe.
  • Availability: S3 is highly available, ensuring you can access your state when you need to deploy.
  • Concurrency Control: It uses S3’s object locking (specifically, versioning and conditional writes) to prevent multiple users or automated systems from corrupting the state file during concurrent updates.
  • Encryption: State files can be encrypted at rest in S3, protecting sensitive configuration data.

Here’s a breakdown of the key levers you control:

  1. Bucket Name: This is the primary identifier for your state. You set this during pulumi login or by configuring the backend explicitly.

    pulumi stack init dev --secret # Initializes a stack
    pulumi config set s3:bucket my-pulumi-state-bucket # Configures the S3 backend
    

    The bucket must exist and be accessible by your Pulumi CLI.

  2. Region: The AWS region where your S3 bucket resides. Pulumi needs to know this to interact with the bucket.

    pulumi config set aws:region us-east-1
    
  3. Prefix (Optional): You can use a prefix within the bucket to organize state files for different projects or environments. This is highly recommended for larger organizations.

    pulumi config set s3:prefix my-project/dev/
    

    This would store your state file as my-project/dev/state.json within your S3 bucket.

  4. Encryption: You can specify server-side encryption for your state files.

    # Using SSE-S3 (default managed encryption)
    pulumi config set s3:sse s3
    
    # Using SSE-KMS (KMS-managed encryption)
    pulumi config set s3:sse kms
    pulumi config set s3:kmsKeyId arn:aws:kms:us-east-1:123456789012:key/your-kms-key-id
    

    This ensures that even if someone gains access to your S3 bucket, the state file contents remain encrypted.

  5. Versioning: Enabling versioning on your S3 bucket is critical for the S3 backend’s concurrency control and recovery mechanisms. When Pulumi writes a new state, it leverages S3’s versioning to create a new version of the state.json object. This allows Pulumi to atomically update the state and also provides a history for rollbacks if needed. You can enable this via the AWS console or CLI:

    aws s3api put-bucket-versioning --bucket my-pulumi-state-bucket --versioning-configuration Status=Enabled
    

    Pulumi relies on this to ensure that if two pulumi up operations happen concurrently, only one successfully writes its state, and the other will fail, prompting a re-evaluation.

The state.json file itself is not meant to be human-readable or editable directly. It contains a complex, versioned representation of all the resources Pulumi manages, including their physical IDs, properties, dependencies, and outputs. When Pulumi performs an update, it compares the current state.json with the desired state from your code, generates a diff, and then applies the changes. After a successful update, it uploads a new version of state.json to S3. This versioning is key to preventing race conditions; if two pulumi up commands run simultaneously, the second one to try and write will fail because the object version it expects to overwrite won’t match the current version in S3, forcing it to re-read the state and re-evaluate.

If you were to delete your Pulumi stack’s state file from S3, Pulumi would treat it as if no infrastructure had ever been deployed. The next pulumi up would then attempt to provision all resources defined in your program from scratch, which is generally not the desired outcome and can lead to duplicate resources if they already exist outside of Pulumi’s management.

Want structured learning?

Take the full Pulumi course →