SeaweedFS: a simple and highly scalable distributed file system

SeaweedFS is a simple and highly scalable distributed file system. There are two objectives: to store billions of files!, to serve the files fast!

SeaweedFS started as an Object Store to handle small files efficiently. Instead of managing all file metadata in a central master, the central master only manages volumes on volume servers, and these volume servers manage files and their metadata. This relieves concurrency pressure from the central master and spreads file metadata into volume servers, allowing faster file access (O(1), usually just one disk read operation).

There is only 40 bytes of disk storage overhead for each file's metadata. It is so simple with O(1) disk reads that you are welcome to challenge the performance with your actual use cases.

SeaweedFS's GitHub page

It's Apache-licensed and the code is, as usual, on GitHub.