Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

curvefs supports multiple s3 in one fs #2226

Open
Cyber-SiKu opened this issue Feb 3, 2023 · 0 comments
Open

curvefs supports multiple s3 in one fs #2226

Cyber-SiKu opened this issue Feb 3, 2023 · 0 comments
Labels
curvefs enhancement improve feature good first issue Good for newcomers

Comments

@Cyber-SiKu
Copy link
Contributor

Cyber-SiKu commented Feb 3, 2023

Description

Currently the data of curvefs is stored in S3 and one fs corresponds to one S3 bucket. The download bandwidth of the entire fs is limited by the download bandwidth of a single bucket.
In order to improve download bandwidth, a fs needs to support multiple S3 buckets. And it allows deployment in different S3 services (such as NOS, minio, OSS, etc.).

Proposed solution

Hash the S3 object into the S3 bucket, s3info Add a key to indicate which s3 bucket .
Add this key before the name of the s3 object to indicate the s3 bucket it belongs to.

@Cyber-SiKu Cyber-SiKu added enhancement improve feature GSOC2023 labels Feb 3, 2023
@wuhongsong wuhongsong added low low priority and removed GSOC2023 labels Feb 15, 2023
@ilixiaocui ilixiaocui added good first issue Good for newcomers and removed low low priority labels Jun 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
curvefs enhancement improve feature good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

4 participants