Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

import/get: escape non-pattern LFS path parts #10416

Merged
merged 1 commit into from
May 13, 2024
Merged

import/get: escape non-pattern LFS path parts #10416

merged 1 commit into from
May 13, 2024

Conversation

sisp
Copy link
Contributor

@sisp sisp commented May 7, 2024

I've fixed a bug related to passing literal/non-pattern path (parts) to the LFS prefetching function from scmrepo which would interpret them as Unix filename patterns and result in incorrect path filtering results. Now, the literal/non-path path (parts) are escaped.

See iterative/scmrepo#355 (comment) for the preceding discussion.

/cc @shcheklein

Copy link

codecov bot commented May 7, 2024

Codecov Report

All modified and coverable lines are covered by tests βœ…

Project coverage is 90.50%. Comparing base (a7c0f6e) to head (59ea6c6).
Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main   #10416   +/-   ##
=======================================
  Coverage   90.50%   90.50%           
=======================================
  Files         502      503    +1     
  Lines       38982    39004   +22     
  Branches     5622     5626    +4     
=======================================
+ Hits        35279    35300   +21     
- Misses       3006     3007    +1     
  Partials      697      697           

β˜” View full report in Codecov by Sentry.
πŸ“’ Have feedback on the report? Share it here.

Copy link
Member

@shcheklein shcheklein left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks @sisp ! can we expand / add test for this?

@sisp
Copy link
Contributor Author

sisp commented May 11, 2024

@shcheklein Done. WDYT?

@skshetry skshetry requested a review from shcheklein May 12, 2024 08:22
@@ -44,3 +46,76 @@ def test_lfs_prefetch(tmp_dir, dvc, scm, mocker):
with dvc.switch(rev):
lfs_prefetch(dvc.dvcfs, ["foo"])
mock_fetch.assert_called_once()


def test_download_lfs_prefetch_directory(tmp_dir, dvc, scm, mocker):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should these test belong to something like test_download.py?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good. I wasn't sure what the test structure under tests/func/ is. I've moved the tests as you suggested.

)


@pytest.mark.skipif(os.name == "nt", reason="invalid directory name syntax on Windows")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are any of special (to be escaped) symbols allowed on NT? (make a simpler test that runs there as well?)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. I've checked Microsoft docs and the relevant characters are * and ?. I've refactored the tests to test each subpattern case independently and skip only those on Windows that are incompatible. WDYT?

Copy link
Member

@shcheklein shcheklein left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks @sisp . Just a few questions re the test. Thanks a lot.

@sisp
Copy link
Contributor Author

sisp commented May 12, 2024

Thanks for your feedback, @shcheklein! Would you mind another review? πŸ™

tests/func/test_download.py Outdated Show resolved Hide resolved
@shcheklein
Copy link
Member

thanks @sisp !

@shcheklein shcheklein merged commit 5bfacec into iterative:main May 13, 2024
20 checks passed
@sisp sisp deleted the lfs/glob-escape branch May 13, 2024 17:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants