Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Testing Kopia Backup to Local Volume Provider #8358

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

kaovilai
Copy link
Member

@kaovilai kaovilai commented Oct 28, 2024

Thank you for contributing to Velero!

Please add a summary of your change

Does your change fix a particular issue?

Fixes #(issue)

Please indicate you've done the following:

hackmd

@github-actions github-actions bot added the Area/Design Design Documents label Oct 28, 2024
@kaovilai kaovilai force-pushed the kopia-to-local-volume-provider branch from ebf7caa to f43f79b Compare October 28, 2024 20:06
@Lyndon-Li
Copy link
Contributor

@kaovilai
Some high level advices before you continue this PR:

  1. NFS support has several sub tasks, please clarify which sub task this PR is for.
  2. If this PR is for sub task Unified repo support file system store #6197, the unified repo backend already supports the NFS/local volume by FSBackend type. But it is not exposed to the upper level, neither tested, which could be the following tasks.
  3. As far as I understand, Velero's current implementations, including the repo part and data mover part, go away from the community's NFS plugin implementation(e.g., the one from replicated for Restic path), so the target is not to integrate to any of the community's plugin, but to develop Velero's native NFS support which complies with the high level designs, i.e., unified repo design, volume snapshot data mover design and VGDP micro service design, etc.
  4. For the same reason of 3, before we work on each of the sub tasks, it's better that we have a high level design to elaborate on the architecture, workflow and user scenarios based on the aforementioned designs, but it may not be mandatory.

@kaovilai
Copy link
Member Author

It is aimed at #6197 mentioned. I'm going to be doing some testing in this area so figured I use this PR to write some notes down for potential design discussions.

@kaovilai
Copy link
Member Author

Making smallest change possible so hoping to not have to replace the local volume provider (LVP) object store plugin yet and aiming solely at enabling kopia usage from within velero code by detecting LVP and using bsl info for kopia.

@kaovilai
Copy link
Member Author

kaovilai commented Oct 29, 2024

My goal is to enable ASAP testing of kopia performance via local volume as backup store. Once we have some ideas we can pursue long term vision around replacing LVP integration. So perhaps this design will just close and we'll open another.

@kaovilai kaovilai force-pushed the kopia-to-local-volume-provider branch from f43f79b to 7749c35 Compare October 31, 2024 19:56
Copy link

codecov bot commented Oct 31, 2024

Codecov Report

Attention: Patch coverage is 16.66667% with 10 lines in your changes missing coverage. Please review.

Project coverage is 59.17%. Comparing base (0224d99) to head (239cca0).

Files with missing lines Patch % Lines
pkg/repository/provider/unified_repo.go 9.09% 9 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #8358      +/-   ##
==========================================
- Coverage   59.19%   59.17%   -0.02%     
==========================================
  Files         369      369              
  Lines       39270    39280      +10     
==========================================
  Hits        23245    23245              
- Misses      14561    14570       +9     
- Partials     1464     1465       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@kaovilai
Copy link
Member Author

looks like there is already a filesystem store in the code so looking into that.

var backendStores = []kopiaBackendStore{
	{udmrepo.StorageTypeAzure, "an Azure blob storage", &backend.AzureBackend{}},
	{udmrepo.StorageTypeFs, "a filesystem", &backend.FsBackend{}},
	{udmrepo.StorageTypeGcs, "a Google Cloud Storage bucket", &backend.GCSBackend{}},
	{udmrepo.StorageTypeS3, "an S3 bucket", &backend.S3Backend{}},

@kaovilai
Copy link
Member Author

BSL needs to have backupLocation.Spec.Provider set to velero.io/fs or fs to trigger this path. For testing I will make modifications such that replicated LVP plugin is also considered equivalent to velero.io/fs and see if that allows kopia to work.

@kaovilai kaovilai force-pushed the kopia-to-local-volume-provider branch from 7749c35 to 56e6e71 Compare October 31, 2024 20:35
@kaovilai kaovilai changed the title Design: Kopia Backup to Local Volume Provider Testing Kopia Backup to Local Volume Provider Oct 31, 2024
@kaovilai kaovilai force-pushed the kopia-to-local-volume-provider branch from 56e6e71 to 9886256 Compare October 31, 2024 21:28
@kaovilai
Copy link
Member Author

replicated/lvp will handle pvc creation and mounting to nodeagents and hopefully the spice flows.

@kaovilai kaovilai force-pushed the kopia-to-local-volume-provider branch from 9886256 to 7448a03 Compare October 31, 2024 21:33
@weshayutin
Copy link
Contributor

woot! thanks for pushing on this @kaovilai

@kaovilai kaovilai force-pushed the kopia-to-local-volume-provider branch from 7e6045a to 239cca0 Compare December 14, 2024 08:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area/Design Design Documents
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants