💾Amazon S3 Files and ZeroFS Offer New POSIX Filesystems
Your file storage just got more flexible with S3 Files and ZeroFS
TL;DR
Amazon introduces S3 Files and ZeroFS, two new POSIX filesystems backed by object storage. Both aim to optimize performance and cost for different use cases, with S3 Files preserving file identity in S3 and ZeroFS offering packing, compression, and encryption.
Amazon has unveiled S3 Files and ZeroFS, two new POSIX filesystems built on top of Amazon S3. The key difference lies in how they handle data: S3 Files keeps files as individual objects within S3 buckets, while ZeroFS packs multiple files into segments for better compression and encryption. If you're dealing with large datasets or need advanced storage features like client-side encryption, these new options could be a game-changer. For instance, if your team uses AWS-managed high-performance storage in S3 Files, you'll see faster active data access compared to ZeroFS's separate paths for file data and metadata. The pricing model varies based on the amount of data stored in high-performance storage tiers and the frequency of reads and writes.
Key Points
S3 Files preserves file identity as individual objects in buckets; metadata and active data reside on AWS-managed high-performance tier.
ZeroFS packs multiple files into segments for compression and encryption, with separate paths for data and metadata until reaching object storage.
Cost model includes S3 storage plus charges for high-performance storage tiers, file access, and synchronization in S3 Files.
Cold reads trigger imports of directory metadata and small files below the import threshold (128 KiB by default) into high-performance storage.
AWS reports resident bytes and inodes via CloudWatch; actual costs depend on residency time, compression rate, I/O size, and fsync frequency.
Why It Matters
If you're running large datasets or need advanced features like client-side encryption, S3 Files and ZeroFS offer new ways to optimize storage. For instance, if your team uses AWS-managed high-performance storage in S3 Files, you'll see faster active data access compared to ZeroFS's separate paths for file data and metadata. However, the cost model varies significantly based on residency time and I/O patterns.
Frequently Asked Questions
Why does this matter?
If you're running large datasets or need advanced features like client-side encryption, S3 Files and ZeroFS offer new ways to optimize storage. For instance, if your team uses AWS-managed high-performance storage in S3 Files, you'll see faster active data access compared to ZeroFS's separate paths for file data and metadata. However, the cost model varies significantly based on residency time and I/O patterns.
What happened?
Amazon introduces S3 Files and ZeroFS, two new POSIX filesystems backed by object storage. Both aim to optimize performance and cost for different use cases, with S3 Files preserving file identity in S3 and ZeroFS offering packing, compression, and encryption.
Comments
Be the first to comment
Enjoyed this article?
Get it daily. 7am. Free. Reads in 5 minutes.
Join 2,068 builders reading daily.