S3 object tagging is how you assign arbitrary key-value pairs to your S3 objects, and it’s surprisingly powerful for both cost allocation and managing object lifecycles.
Let’s see it in action. Imagine you’ve got a bucket for storing user-uploaded images. Some users are on a free tier, others on a paid tier. You want to track costs per user and eventually move older images from paid users to cheaper storage.
Here’s a bucket with a few objects:
aws s3api list-objects-v2 --bucket my-user-uploads --prefix images/
Output:
{
"Contents": [
{
"Key": "images/user-a/photo1.jpg",
"LastModified": "2023-10-26T10:00:00.000Z",
"ETag": "\"abcdef123456\"",
"Size": 102400,
"StorageClass": "STANDARD"
},
{
"Key": "images/user-b/photo2.png",
"LastModified": "2023-10-26T11:00:00.000Z",
"ETag": "\"ghijkl67890\"",
"Size": 204800,
"StorageClass": "STANDARD"
},
{
"Key": "images/user-a/profile.jpg",
"LastModified": "2023-10-25T09:00:00.000Z",
"ETag": "\"mnopqr789012\"",
"Size": 51200,
"StorageClass": "STANDARD"
}
]
}
Now, let’s add tags to these objects to track the user and their tier.
For images/user-a/photo1.jpg:
aws s3api put-object-tagging --bucket my-user-uploads --key images/user-a/photo1.jpg --tagging '{"TagSet": [{"Key": "User", "Value": "user-a"}, {"Key": "Tier", "Value": "free"}]}'
For images/user-b/photo2.png:
aws s3api put-object-tagging --bucket my-user-uploads --key images/user-b/photo2.png --tagging '{"TagSet": [{"Key": "User", "Value": "user-b"}, {"Key": "Tier", "Value": "paid"}]}'
For images/user-a/profile.jpg:
aws s3api put-object-tagging --bucket my-user-uploads --key images/user-a/profile.jpg --tagging '{"TagSet": [{"Key": "User", "Value": "user-a"}, {"Key": "Tier", "Value": "free"}]}'
After tagging, if you run list-objects-v2 again, you won’t see the tags directly in the output. You need to specifically request them:
aws s3api list-objects-v2 --bucket my-user-uploads --prefix images/ --output json --query 'Contents[].{Key:Key, Tags:Tags}' --metadata-directive COPY --tagging '{"TagSet": [{"Key": "User", "Value": "user-a"}, {"Key": "Tier", "Value": "free"}]}'
Wait, that’s not quite right. The list-objects-v2 command doesn’t directly show tags. To see tags, you’d typically use get-object-tagging for individual objects, or enable S3 Inventory to report tags for all objects.
Let’s enable S3 Inventory to get a report that includes tags.
First, create a bucket for inventory reports:
aws s3api create-bucket --bucket s3-inventory-reports-bucket --region us-east-1
Then, configure the inventory:
aws s3api put-bucket-inventory-configuration --bucket my-user-uploads --id UserTierInventory --inventory-configuration '{
"Id": "UserTierInventory",
"IsEnabled": true,
"Destination": {
"S3BucketDestination": {
"BucketArn": "arn:aws:s3:::s3-inventory-reports-bucket",
"Format": "CSV",
"Prefix": "inventory/"
}
},
"Schedule": {
"Frequency": "Daily"
},
"IncludedObjectVersions": "All",
"OptionalFields": ["Tag"]
}'
After the inventory runs (it takes up to 24 hours for the first run), you’ll get a CSV file in s3://s3-inventory-reports-bucket/inventory/. One of the columns will be tags. This is how you’d aggregate costs.
To allocate costs, you need to configure S3 server access logging to include the tags. However, standard S3 server access logs don’t include object tags. The primary way to allocate costs based on tags is through AWS Cost Allocation Tags. You first enable the tags you used ( User and Tier) in the AWS Billing console under "Cost Allocation Tags." Once enabled, S3 usage (storage, requests, etc.) will appear on your Cost and Usage Reports (CUR) with these tags. You can then filter your bills by User and Tier to see how much each user or tier is costing you.
Now, for lifecycle management. You want to move older objects from paid users to cheaper storage. Let’s say you want to move objects older than 30 days from Tier: paid to S3 Infrequent Access (IA).
Go to your bucket my-user-uploads in the AWS console, select "Management" tab, and click "Create lifecycle rule."
Rule Name: PaidUserInfrequentAccess
Scope: Choose "Apply to all objects in the bucket" (for simplicity here, but you could filter by prefix).
Lifecycle rule actions:
- Transition:
- Transition current versions of objects between storage classes:
- Storage class: S3 Standard-Infrequent Access
- Days after object creation: 30
- Filter:
- Filter by object tags:
- Key:
Tier - Value:
paid - Operator:
Equals
- Key:
- Filter by object tags:
- Transition current versions of objects between storage classes:
This rule tells S3: "If an object has the tag Tier with the value paid, and it’s been in the bucket for 30 days or more, transition it from its current storage class (likely STANDARD) to S3 Standard-IA."
The most surprising thing about S3 object tagging, especially when considering its use with lifecycle policies, is that the lifecycle policy itself doesn’t directly "read" the tags. Instead, you configure the lifecycle rule to filter which objects it applies to based on those tags. The S3 service then processes objects that match the filter criteria and the time-based conditions.
This means you can have a single lifecycle rule that targets many objects across different prefixes, as long as they share a common tag. For example, a rule could target all objects tagged with Project: Alpha, regardless of where they are stored within the bucket.
The next step in managing your S3 data often involves setting up S3 Replication to copy objects to another region for disaster recovery, and you can use object tags to control which objects are replicated and to which destination buckets.