S3 Storage Lens reveals that your organization’s S3 usage is far more dynamic and less predictable than you think.
Imagine you’re looking at your S3 buckets, and you want to understand how they’re being used across your entire organization – who’s putting what in, how much it’s costing, and where the bottlenecks might be. S3 Storage Lens is the tool for that. It’s not just a dashboard; it’s a way to get a bird’s-eye view of your S3 activity, aggregated across all your accounts and buckets, without you having to build custom reports.
Let’s see it in action. First, you need to enable Storage Lens. This is usually done at the management account level for AWS Organizations.
aws organizations enable-all-features
aws organizations enable-aws-service-access --service-principal storage-lens.amazonaws.com
aws organizations enable-organizations-from-aws-service-access --service-principal storage-lens.amazonaws.com
Once enabled, Storage Lens starts collecting data. You can then navigate to the S3 console, find "Storage Lens," and start exploring. You’ll see metrics like total storage by bucket, by region, by storage class, and even by access pattern (e.g., infrequent access vs. frequent access).
Here’s a sample of what you might see in the dashboard:
- Total Storage: 1.2 PB across 500 buckets.
- Top Regions:
us-east-1(400 TB),eu-west-2(350 TB). - Storage Class Distribution: Standard (60%), Intelligent-Tiering (25%), Glacier (15%).
- Buckets by Size: 10 buckets > 100 TB, 100 buckets 10-100 TB, 400 buckets < 10 TB.
This gives you immediate insights. For instance, if you see a huge spike in Standard storage that you didn’t expect, you can drill down. Storage Lens allows you to create custom dashboards with specific metrics and filters relevant to your needs. You can also set up export to S3, so you can analyze the raw data with tools like Athena or QuickSight.
The core problem Storage Lens solves is visibility and cost optimization. Without it, understanding your S3 footprint across multiple accounts is a manual, often incomplete, process. You might not realize that a development team is using a massive amount of Standard storage for logs that could be moved to Intelligent-Tiering, or that a particular application is generating an exorbitant number of PUT requests, driving up costs. Storage Lens surfaces these "gotchas" by providing a unified view of usage patterns and potential cost savings.
Internally, Storage Lens uses AWS’s internal telemetry to gather metrics. It’s designed to be non-intrusive, meaning it doesn’t impact the performance of your S3 operations. The data is aggregated and anonymized where appropriate, providing you with actionable insights without exposing sensitive information. You control what data is collected and how it’s presented through the dashboard and export configurations.
A key aspect is its ability to track object-level metrics, which are typically not available through standard S3 metrics. This includes metrics like object age, versioning status, and encryption status, offering a granular understanding of your data’s lifecycle and security posture. For example, you can identify a large number of un-transitioned objects within Intelligent-Tiering, indicating a potential misconfiguration or a need to re-evaluate your lifecycle policies.
The most surprising thing is how often Storage Lens reveals entirely unexpected usage patterns. You might assume your data is mostly static, but Storage Lens can show you that a significant portion is actively being written to and read from, even if it’s in infrequent access tiers, which can have cost implications. It also highlights the effectiveness of your lifecycle policies by showing the actual movement of data between storage classes.
The next logical step after gaining organizational-wide insights is to implement automated cost-saving measures based on these findings.