GCS (or S3) as the storage?

[ashikaumanga]

First of all ,thank you for the implementing and open-sourcing this library. I am new to Rust and use this for my first Rust project.

My question is : can I use GCS (or S3) as the storage ? I am planing to mount GCS bucket using gcs-fuse and use it as the partition folder. Will there be any impact to atomicity..etc when using any of these object stores ?

[marvin-j97]

can I use GCS (or S3) as the storage

Out of the box no, only the file system is supported. If you use an abstraction like you plan to that has the same behaviour as a local file system, it may work, but may come with a performance trade-off, which may or may not be acceptable.

Will there be any impact to atomicity

Atomicity and Durability are achieved using the write-ahead journal which is kept in the journals/ folder.

I am planing to mount GCS bucket using gcs-fuse and use it as the partition folder.

I cannot really speak for the performance and guarantees of gcs-fuse. Their page says:

• "does not transfer object metadata": This is fine ✅
• "does not provide concurrency control for multiple writes to the same file": This is fine as long as you only have a single database accessing the bucket ✅
• "does not support hard links": This is fine ✅
• "Cloud Storage FUSE does not support file locking": This is fine ✅
• "directory renaming [is] not atomic": This is fine ✅
• "Cloud Storage FUSE has much higher latency than a local file system, and as such, should not be used as the backend for storing a database": This is probably not fine ⚠️ - but this may be a trade-off that may be acceptable.
• "After a successful fsync or a successful close, the contents of the inode are guaranteed to have been written to the GCS object with the matching name": This is important, otherwise a lot of atomic guarantees would be lost ✅

Let me know how it goes.

[ashikaumanga]

Thanks. My concerned was; usually these object-stores doesn't support file update. When a file needs to be updated , the entire file/object get replaced (GCS doesn't directly support in-place modifications of objects.) So wonder whether this might have any impact during compaction...etc

(But again, compaction read from existing SStable? files and write to a new file ; similar to external-merge-sort ? So if there are no updates to any existing files, it should be fine I assume ? )

[marvin-j97]

Exactly, that's where LSM-trees really shine. All segments (SSTables) are immutable. On compaction, some segments get written out to a new segment, which then becomes immutable as well. In fact, that's how Google Bigtable works. SSTables are stored in GFS (Colossus), and only cached on tablet servers. All writes are appended to a journal. That all only works because SSTables are immutable, and is very similar to using something like gcs-fuse, really.