Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add --blob-cache-dir arg use to generate raw blob cache and meta #1433

Merged
merged 1 commit into from
Oct 10, 2023
Merged

Add --blob-cache-dir arg use to generate raw blob cache and meta #1433

merged 1 commit into from
Oct 10, 2023

Conversation

zyfjeff
Copy link
Contributor

@zyfjeff zyfjeff commented Sep 25, 2023 •
edited
Loading

Relevant Issue (if applicable)

#1415

Details

generate blob cache and blob meta through the —-blob-cache-dir parameters,
so that nydusd can be started directly from these two files without
going to the backend to download. this can improve the performance
of data loading in localfs mode.

Types of changes

What types of changes does your PullRequest introduce? Put an x in all the boxes that apply:

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation Update (if none of the other choices apply)

Checklist

Go over all the following points, and put an x in all the boxes that apply.

  • I have updated the documentation accordingly.
  • I have added tests to cover my changes.

@zyfjeff zyfjeff requested a review from a team as a code owner September 25, 2023 06:24
@zyfjeff zyfjeff requested review from bergwolf, liubin and hsiangkao and removed request for a team September 25, 2023 06:24
@codecov
Copy link

codecov bot commented Sep 25, 2023 •
edited
Loading

Codecov Report

Merging #1433 (d9b7d2f) into master (6ea22cc) will decrease coverage by 0.12%.
The diff coverage is 12.50%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1433      +/-   ##
==========================================
- Coverage   46.40%   46.28%   -0.12%     
==========================================
  Files         123      123              
  Lines       38745    38872     +127     
  Branches    38745    38872     +127     
==========================================
+ Hits        17979    17992      +13     
- Misses      19794    19906     +112     
- Partials      972      974       +2
Files Coverage Δ
builder/src/compact.rs 0.00% <ø> (ø)
builder/src/core/blob.rs 25.00% <40.00%> (-0.30%) ⬇️
builder/src/lib.rs 52.76% <50.00%> (-0.26%) ⬇️
builder/src/core/node.rs 41.53% <14.28%> (-0.22%) ⬇️
builder/src/tarball.rs 62.84% <45.45%> (ø)
builder/src/directory.rs 0.00% <0.00%> (ø)
builder/src/stargz.rs 0.00% <0.00%> (ø)
src/bin/nydus-image/main.rs 41.25% <22.00%> (-0.34%) ⬇️
builder/src/core/context.rs 44.00% <2.10%> (-3.95%) ⬇️

... and 1 file with indirect coverage changes

@zyfjeff zyfjeff changed the title Add --blob-meta and --blob-cache arg use to generate raw blob cache and meta Wip: Add --blob-meta and --blob-cache arg use to generate raw blob cache and meta Sep 25, 2023
@zyfjeff zyfjeff force-pushed the support-generate-blobcache branch from d6f568c to 8f42e29 Compare September 25, 2023 07:12
@zyfjeff zyfjeff changed the title Wip: Add --blob-meta and --blob-cache arg use to generate raw blob cache and meta Add --blob-meta and --blob-cache arg use to generate raw blob cache and meta Sep 25, 2023
@zyfjeff zyfjeff force-pushed the support-generate-blobcache branch 5 times, most recently from 274cfba to 7c11850 Compare September 27, 2023 07:35
imeoer
imeoer reviewed Sep 27, 2023
View reviewed changes
smoke/tests/blobcache_test.go Outdated Show resolved Hide resolved
smoke/tests/blobcache_test.go Outdated Show resolved Hide resolved
src/bin/nydus-image/main.rs Outdated Show resolved Hide resolved
@zyfjeff zyfjeff force-pushed the support-generate-blobcache branch from 7c11850 to 3950e33 Compare October 9, 2023 09:59
@zyfjeff zyfjeff changed the title Add --blob-meta and --blob-cache arg use to generate raw blob cache and meta Add --blob-cache-dir arg use to generate raw blob cache and meta Oct 9, 2023
@zyfjeff zyfjeff force-pushed the support-generate-blobcache branch 2 times, most recently from 163432f to 1a0660f Compare October 9, 2023 10:09
jiangliu
jiangliu reviewed Oct 9, 2023
View reviewed changes
builder/src/core/blob.rs Outdated Show resolved Hide resolved
jiangliu
jiangliu reviewed Oct 10, 2023
View reviewed changes
builder/src/lib.rs Outdated Show resolved Hide resolved
@zyfjeff zyfjeff force-pushed the support-generate-blobcache branch 2 times, most recently from ab10dd0 to 0f4ff08 Compare October 10, 2023 02:56
imeoer
imeoer reviewed Oct 10, 2023
View reviewed changes
src/bin/nydus-image/main.rs Outdated Show resolved Hide resolved
@zyfjeff zyfjeff force-pushed the support-generate-blobcache branch from 0f4ff08 to 0a8d855 Compare October 10, 2023 03:16
@zyfjeff zyfjeff force-pushed the support-generate-blobcache branch 3 times, most recently from 9a7bbaa to 81764ab Compare October 10, 2023 08:19
@zyfjeff zyfjeff force-pushed the support-generate-blobcache branch 2 times, most recently from 611761c to a787562 Compare October 10, 2023 09:24
@zyfjeff
Copy link
Contributor Author

zyfjeff commented Oct 10, 2023

nydus-unit-test-coverage failed due to network issues, and it had nothing to do with my changes.

@zyfjeff @jiangliu
Add --blob-cache-dir arg use to generate raw blob cache and meta
d9b7d2f 
generate blob cache and blob meta through the —-blob-cache-dir parameters,
so that nydusd can be started directly from these two files without
going to the backend to download. this can improve the performance
of data loading in localfs mode.

Signed-off-by: zyfjeff <[email protected]>
@jiangliu jiangliu force-pushed the support-generate-blobcache branch from a787562 to d9b7d2f Compare October 10, 2023 10:07
@imeoer imeoer merged commit 9ab1ec1 into dragonflyoss:master Oct 10, 2023
23 of 24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jiangliu jiangliu jiangliu approved these changes

@imeoer imeoer imeoer approved these changes

@bergwolf bergwolf Awaiting requested review from bergwolf bergwolf is a code owner automatically assigned from dragonflyoss/nydus-reviewers

@liubin liubin Awaiting requested review from liubin liubin is a code owner automatically assigned from dragonflyoss/nydus-reviewers

@hsiangkao hsiangkao Awaiting requested review from hsiangkao hsiangkao is a code owner automatically assigned from dragonflyoss/nydus-reviewers

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@zyfjeff @imeoer @jiangliu