Skip to content
This repository has been archived by the owner on Jul 5, 2022. It is now read-only.

GS: Versioning scenario should include a page about directories #24

Open
iesahin opened this issue Feb 26, 2021 · 1 comment
Open

GS: Versioning scenario should include a page about directories #24

iesahin opened this issue Feb 26, 2021 · 1 comment

Comments

@iesahin
Copy link
Contributor

iesahin commented Feb 26, 2021

Currently dvc add is told in context of files only in track-a-file-or-directory.md

We need to provide directory examples and describe how they differ from files and how their members and content are tracked with examples.

@iesahin
Copy link
Contributor Author

iesahin commented Mar 14, 2021

I removed this content from S&P scenario, this should be put elsewhere.


How are directories cached and tracked?

By default, DVC stores the outputs defined in stage add in the cache
similar to other data. data/prepared directory is defined as an output in
prepare stage, and the hash value is:

grep -A 1 'path: data/prepared' dvc.lock{{execute}}

and this points us to .dvc/cache/f1/b1d214c4cc7a3efdb200410227b975.dir file.

It's a JSON file that lists all individual elements of the directory with
their hash values.

cat .dvc/cache/f1/b1d214c4cc7a3efdb200410227b975.dir | jq{{execute}}

For example we see that the individual hash value of train.tsv as
fcebfd4c6f1645ac4987d39f1c5cf610 and check its content

.dvc/cache/fc/ebfd4c6f1645ac4987d39f1c5cf610{{open}}.

Note also that DVC adds /prepared to .gitignore to prevent output data
files to be committed in Git.

You can review changes in the repository from the Source Control tab of VS Code.

Finally we are registering dvc.yaml, dvc.lock and .gitignore to Git and
complete this step.

git add dvc.yaml dvc.lock data/.gitignore
git commit -m "Configured prepare stage"
```{{execute}}

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant