Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add new rados/striper package #1015

Merged
merged 9 commits into from
Aug 8, 2024
Merged

Conversation

phlogistonjohn
Copy link
Collaborator

Add a new rados/striper package that wraps Ceph's libradosstriper. The libradosstriper library builds on top of the librados library to support striping large "objects" over multiple RADOS objects.

Fixes: #1011

Checklist

  • Added tests for features and functional changes
  • Public functions and types are documented
  • Standard formatting is applied to Go code
  • Is this a new API? Added a new file that begins with //go:build ceph_preview
  • Ran make api-update to record new APIs

New or infrequent contributors may want to review the go-ceph Developer's Guide including the section on how we track API Status and the API Stability Plan.

The go-ceph project uses mergify. View the mergify command guide for information on how to interact with mergify. Add a comment with @Mergifyio rebase to rebase your PR when github indicates that the PR is out of date with the base branch.

@phlogistonjohn
Copy link
Collaborator Author

Looks like rados_striper_stat2 isn't that old. I guess that's why I made this PR a draft first ;-)

@phlogistonjohn phlogistonjohn changed the title [WIP] Add new rados/striper package Add new rados/striper package Jul 29, 2024
@phlogistonjohn phlogistonjohn marked this pull request as ready for review July 29, 2024 21:24
@anoopcs9 anoopcs9 added the API This PR includes a change to the public API of a go-ceph package label Aug 2, 2024
rados/striper/striper.go Outdated Show resolved Hide resolved
rados/striper/striper.go Outdated Show resolved Hide resolved
rados/striper/striper_test.go Show resolved Hide resolved
rados/striper/write_test.go Outdated Show resolved Hide resolved
Reorganize the RUN command in the Dockerfile that sets up distro based
repositories and ceph packages. Move the `&&` to the beginning of the
lines to highlight when commands begin (and end). Change the indent
level of items that are arguments to commands to further clarify
what is part of what.

Signed-off-by: John Mulligan <[email protected]>
Add libradosstriper devel and debug packages to the test container
image. This is in preparation for adding a future `rados/striper`
package to go-ceph that will wrap these APIs.

Signed-off-by: John Mulligan <[email protected]>
@phlogistonjohn phlogistonjohn force-pushed the jjm-rados-striper branch 2 times, most recently from a127aef to 9f3bea0 Compare August 2, 2024 14:57
@phlogistonjohn phlogistonjohn requested a review from anoopcs9 August 3, 2024 17:46
anoopcs9
anoopcs9 previously approved these changes Aug 4, 2024
Copy link
Collaborator

@anoopcs9 anoopcs9 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm.

Copy link
Collaborator

@ansiwen ansiwen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I only have one comment, but will not insist on it.

Otherwise I'm not sure I understand the striping concept here. Isn't it more like a bundle? What happens if you make the striped object smaller? Does the number of underlying objects change? Anyway, that is not related to this PR.

rados/striper/stat.go Show resolved Hide resolved
@phlogistonjohn
Copy link
Collaborator Author

Isn't it more like a bundle? What happens if you make the striped object smaller? Does the number of underlying objects change?

Maybe they're using the terms poorly, but AFAIK it just writes segments of the "virtual object" across many objects in rados. Any other questions, I think you'd have to ask the ceph team :-)

Start a new `rados/striper` package that wraps Ceph's libradosstriper.
The libradosstriper library builds on top of the librados library to
support striping large "objects" over multiple RADOS objects.

Fixes: ceph#1011

Signed-off-by: John Mulligan <[email protected]>
Add support for xattr APIs to the Striper object.

Signed-off-by: John Mulligan <[email protected]>
Add a package doc comment that also points out some things I learned
about how `rados` command line tool lists striped objects to preempt
being asked later on. :-)

Signed-off-by: John Mulligan <[email protected]>
Add rados/striper to the implements tool. It needs to be recognized
as a new C-API wrapping library.

Signed-off-by: John Mulligan <[email protected]>
@mergify mergify bot dismissed anoopcs9’s stale review August 7, 2024 18:22

Pull request has been modified.

Copy link
Collaborator

@anoopcs9 anoopcs9 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm still good with the current version.

Copy link
Collaborator

@ansiwen ansiwen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mergify mergify bot merged commit 5eb565f into ceph:master Aug 8, 2024
17 checks passed
@phlogistonjohn phlogistonjohn mentioned this pull request Aug 8, 2024
@phlogistonjohn phlogistonjohn deleted the jjm-rados-striper branch August 8, 2024 21:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
API This PR includes a change to the public API of a go-ceph package
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Rados Striping API Support
3 participants