S3 Server Access Logging is surprisingly absent from most default AWS configurations, leaving a huge blind spot for understanding who’s touching your buckets and how.

Let’s see it in action. Imagine a simple scenario: you’ve got a public S3 bucket for serving static assets, and you want to know if anyone’s actually hitting it.

# First, enable logging for your source bucket
aws s3api put-bucket-logging \
    --bucket my-public-assets \
    --bucket-logging-status '{
        "LoggingEnabled": {
            "TargetBucket": "my-s3-access-logs",
            "TargetPrefix": "assets-log/"
        }
    }'

# Now, let's simulate some access to the public bucket
# (This would typically be a user's browser or an application)
aws s3api get-object \
    --bucket my-public-assets \
    --key index.html \
    --request-payer requester # If your bucket is requester pays

A few minutes later, you’ll see log files appearing in your my-s3-access-logs bucket, under the assets-log/ prefix.

The core problem S3 Server Access Logging solves is providing an audit trail for requests made to your S3 buckets. Without it, you’re flying blind on:

  • Who is accessing your data: IP addresses, user agents, and sometimes even authenticated AWS principals.
  • What operations are being performed: GET, PUT, DELETE, HEAD, etc.
  • When the access occurred: Timestamps with millisecond precision.
  • Where the requests originated from: Geographic location based on IP.
  • How much data was transferred: Bytes sent and received.

This information is crucial for security analysis, operational troubleshooting, and even billing.

How it Works Internally:

When you enable server access logging, S3 starts writing log records for every request made to the source bucket. These records are batched and delivered to a specified target bucket. The target bucket can be the same as the source bucket, but it’s best practice to use a separate bucket for security and organization. You can also specify a prefix within the target bucket to organize logs by date, application, or any other criteria.

The log files are delivered in Apache common log format (or a slightly modified version). They are delivered approximately hourly, though there can be delays. Each log file contains records for requests made during a specific time window.

Key Configuration Levers:

  1. TargetBucket: The name of the S3 bucket where log files will be stored. This bucket must exist and have appropriate permissions for S3 to write to it.
  2. TargetPrefix (Optional): A prefix that S3 will add to log file names. This is incredibly useful for organizing logs, for example, my-logs/assets/ or my-logs/application-a/.
  3. Permissions on the Target Bucket: The target bucket needs a bucket policy that grants s3:PutObject permission to the S3 service principal (logging.s3.amazonaws.com). This is automatically handled if you use the AWS CLI or SDK to enable logging, but it’s worth verifying.
  4. Permissions on the Source Bucket: The source bucket doesn’t need special permissions for logging itself, but the principal that enables logging does.

Example Bucket Policy for Target Bucket:

If your target bucket is named my-s3-access-logs and your source bucket is my-public-assets, the policy on my-s3-access-logs should look something like this:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "GrantS3ServerAccessLogging",
            "Effect": "Allow",
            "Principal": {
                "Service": "logging.s3.amazonaws.com"
            },
            "Action": "s3:PutObject",
            "Resource": "arn:aws:s3:::my-s3-access-logs/*",
            "Condition": {
                "StringEquals": {
                    "s3:x-amz-acl": "bucket-owner-full-control"
                }
            }
        },
        {
            "Sid": "GrantS3ServerAccessLoggingAcl",
            "Effect": "Allow",
            "Principal": {
                "Service": "logging.s3.amazonaws.com"
            },
            "Action": "s3:PutObjectAcl",
            "Resource": "arn:aws:s3:::my-s3-access-logs/*",
            "Condition": {
                "StringEquals": {
                    "s3:x-amz-acl": "bucket-owner-full-control"
                }
            }
        }
    ]
}

The StringEquals condition on s3:x-amz-acl is important for ensuring the logs are owned by the bucket owner, not the logging service.

Analyzing the Logs:

Once logs start accumulating, you can analyze them using various tools:

  • AWS Athena: Query logs directly in S3 using SQL. You’ll need to define a table schema that matches the log format.
  • AWS CloudWatch Logs: Ingest logs into CloudWatch Logs for real-time monitoring and alarms.
  • Custom Scripts: Download logs and parse them with Python, Perl, or other scripting languages.
  • Third-Party Tools: Many security and log management solutions integrate with S3 logs.

When you enable logging for a bucket, S3 doesn’t retroactively log past requests; it only starts logging from the moment the configuration is applied.

This system is foundational for understanding your S3 traffic, and the next logical step is often integrating these logs into a more comprehensive security information and event management (SIEM) system.

Want structured learning?

Take the full S3 course →