Add a dev mode that allows for local storing of config files and logs #1682

ratchek · 2023-04-04T17:35:17Z

Description

Allows vorta to be called with a command-line flag that will make it use a directory in the project tree to store all the settings, logs, and cache.
Also allows for a custom directory path allowing for multiple "configuration" folders at once.

Related Issue

Fixes #1162

Motivation and Context

This can be used to prevent the vorta instance that a developer is working on from accessing the configuration files that they have set up for their personal backups.

How Has This Been Tested?

Just been tested in the terminal by me.

Screenshots (if appropriate):

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

I have read the CONTRIBUTING guide.
My code follows the code style of this project.
My change requires a change to the documentation.
I have updated the documentation accordingly.
I have added tests to cover my changes.
All new and existing tests passed.

I provide my contribution under the terms of the license of this repository and I affirm the Developer Certificate of Origin.

This does nothing for now except echo that it was called to terminal.

This adds tooltips to the settings as well as a 'info' button that shows the tooltip when hovering over it. * Add tooltips to settings. * src/vorta/store/models.py (SettingsModel): Add `tooltip` column. * src/vorta/store/migrations.py (run_migrations): Create `tooltip` column. * src/vorta/store/connection.py (init_db): Populate `tooltip` column. Increase `SCHEMA_VERSION`. * src/vorta/views/misc_tab.py (MiscTab.populate): Set tooltip of checkbox widgets. * src/vorta/store/settings.py : Add tooltips and update label of `override_mount_permissions` * Add *help* button to settings. * src/vorta/assets/icons/help-about.svg: Add info icon. * src/vorta/views/partials/tooltip_button.py: Implement `ToolTipButton`. * src/vorta/views/misc_tab.py: Add `ToolTipButton` for each setting with a tooltip. Add `set_icons` and connect it to palette change. * tests/test_misc.py (test_autostart): Update test. --------- Co-authored-by: real-yfprojects <[email protected]>

Fixes borgbase#1463. This makes the code count the database rows for the current profile only. Previously one could run a backup without any sources when one had sources configured in a different profile. * src/vorta/borg/create.py (BorgCreateJob.prepare)

This creates the option to use the --development or -D flags to launch vorta in development mode which stores all settings, cache, temp, and log files in the project tree instead of in the default places for the system. Allows for working on the project without it pulling in the settings from the system-wide copy of vorta that you use for your actual backups.

…he name

real-yfprojects · 2023-04-04T18:33:23Z

Note how you can write Fixed #1162 to close the issue when merging the PR.

real-yfprojects · 2023-04-04T18:38:03Z

There are some commits in the branch diff that do not belong to this PR. Can you fix that?

real-yfprojects

Thank you very much for taking on this issue. That will be a very useful development feature.

src/vorta/__main__.py

src/vorta/utils.py

src/vorta/__main__.py

ratchek · 2023-04-04T20:25:33Z

There are some commits in the branch diff that do not belong to this PR. Can you fix that?

Could you elaborate on that please? This is the first time I'm using git in a collaborative manner. I take it you mean that there are commits arising from me merging in work that other people did? How do I go about fixing that?

This does nothing for now except echo that it was called to terminal.

This creates the option to use the --development or -D flags to launch vorta in development mode which stores all settings, cache, temp, and log files in the project tree instead of in the default places for the system. Allows for working on the project without it pulling in the settings from the system-wide copy of vorta that you use for your actual backups.

…he name

real-yfprojects · 2023-04-05T07:37:08Z

I take it you mean that there are commits arising from me merging in work that other people did? How do I go about fixing that?

Yes, usually these commits aren't shown in the branch diff. Here they are because you somehow rewrote the commits making you the committer. I managed to fix this by running

git rebase -i master

And dropping the commits in question since they are already included in the master branch. I forced pushed the fixed branch, so you will have to update your local copy of it.

ratchek · 2023-04-13T17:18:09Z

Hi, this i a silly question, but you mentioned I made a mess of the git commits, so I figured I'd ask to make sure it doesn't happen again. When I'm pulling in changes from the vorta master repo into my local dev branch, should I be using merge or rebase?

Basically, I'm not sure exactly what my workflow should be in regards to making sure that I stay on top of changes happening in the main repo.

real-yfprojects · 2023-04-13T18:05:19Z

This is how I do it:

Keep master branch in sync with upstream

git checkout master
git pull upstream

Now you can either merge master into the feature branch. This what many people do. Its the easiest method:

git checkout feature-branch
git merge master

However I prefer keeping clean and linear git histories. I therefore rebase my feature branch onto master:

git checkout feature-branch
git rebase master

The rebase will also remove all merge commits of master that occur after the diverging.

I would be interested how you managed to get branch you had before. My only guess was that you might have merged origin/master or upstream/master and than rebased onto master which wasn't in sync with the remote branches.

ratchek · 2023-04-19T14:31:52Z

Ok, that's pretty much exactly what I do most of the time. The problem is, I think whenever I get merge conflicts (and I think that's where I git those weird results last time - trying to resolve merge conflicts).

Sidenote:
When I resolve merge conflicts and commit the changes, what should I be putting in the commit message? Just a generic "merge changes from upstream"? That seems redundant and unhelpful, but I don't know what the alternative might be.

Changes based on the discussion here: borgbase#1682 Biggest change is that src/config is rewritten to initialize the config variables while the program is running, so you need to import the whole module every time you use variables/constants from it. This was done to avoid duplicating code responsible for setting up the directory in which config data will be stored.

real-yfprojects · 2023-04-20T05:40:50Z

Ok, that's pretty much exactly what I do most of the time. The problem is, I think whenever I get merge conflicts (and I think that's where I git those weird results last time - trying to resolve merge conflicts).

Merge conflicts are usually resolved by the merge commit itself without rewriting any previous commits. Only few commands rewrite git history.

When I resolve merge conflicts and commit the changes, what should I be putting in the commit message? Just a generic "merge changes from upstream"?

That's fine. The merge commit will have the commits with helpful descriptions as a parent. Mostly we squash PRs anyway.

src/vorta/application.py

ratchek · 2023-05-11T20:18:42Z

So, I just realized that now there's this issue, where everything works fine, unless you call vorta with the --development flag and pass it a custom directory using relative paths. So for example, if you do vorta --development ./dev-mode. In that particular scenario, the link to logs in the misc tab doesn't work. The logs exist and they're in the correct folder, but the link doesn't work.
I would suspect the call to config on line 37 in views/misc_tab.py is happening before the actual modification of the config values, however if you supply the full path when you're running the program (for example vorta --development /home/username/path/to/project/folder/dev-mode), then everything works fine. Is it possible that the string interpolation is not unpacking the full path correctly?

ratchek · 2023-05-11T20:26:19Z

Is it possible that the string interpolation is not unpacking the full path correctly?

Yup. That was it. Fixed it. Should be good now.

src/vorta/__main__.py

src/vorta/application.py

src/vorta/__main__.py

real-yfprojects

Nice! Can you add type hints and docstrings?

real-yfprojects · 2023-05-28T15:51:16Z

src/vorta/utils.py

+    parser.add_argument(
+        '--development',
+        '-D',
+        nargs='?',
+        const=DEFAULT_DIR_FLAG,
+        help='Start vorta in a local development environment. '
+        'All log, config, cache, and temp files will be stored within the project tree. '
+        'You can follow this flag with an optional path and it will store the files in the provided location.',
+    )


Can you specify a metavar so that it doesn't say

usage: vorta [-h] [--version] [--daemonize] [--create PROFILE] [--development [DEVELOPMENT]]

when running vorta --help?

real-yfprojects · 2023-05-29T20:54:38Z

Nice work, thanks!

ratchek · 2023-05-29T21:45:38Z

Nice work, thanks!

Thank you so so much for guiding me through this and being so patient.

m3nu · 2023-05-30T06:23:23Z

That's fine. I see you mostly changed XXX to config.XXX. Reminds me a bit of Django now.

Updating the docs would be important too. So people actually use this feature as intended.

real-yfprojects · 2023-05-31T06:22:09Z

Updating the docs would be important too. So people actually use this feature as intended.

Follow up issue: borgbase/vorta.borgbase.com#42

…borgbase#1682) Allows vorta to be called with the command-line flag `--development` or `-D` that will make it use a directory in the project tree to store all the settings, logs, and cache. This default directory will be called `.dev_config` and placed in the projects root. Also allows for a custom directory path allowing for multiple "configuration" folders at once. This can be used to prevent the vorta instance that a developer is working on from accessing the configuration files that they have set up for their personal backups. * .gitignore : Add `.dev_config`. * src/vorta/utils.py (parse_args): Add `--development` flag. The default will be `DEFAULT_DIR_FLAG`. * src/vorta/utils.py : Add `DEFAULT_DIR_FLAG`. * src/vorta/config.py : Add methods for populating the config directories exposed by this module. * src/vorta/__main__.py (main): Handle `--development` flag and update config directories if its specified. * Access config constants through the `config` module instead of importing them directly with `from .config import`. --------- Co-authored-by: yfprojects <[email protected]>

ratchek and others added 11 commits February 22, 2023 17:05

Add development flag to options.

7bd4900

This does nothing for now except echo that it was called to terminal.

Merge branch 'master' into dev_mode

5e1b601

Fix before merge

fac82a0

Modify the config directory to a hidden directory and add config to t…

4d2f969

…he name

Merge branch 'master' into dev_mode

073e6b1

Add support for custom dev dirs

3ef5e54

Change the default dev config dir in gitignore

28d9340

Clean up print statements used for debugging

2b0b756

real-yfprojects reviewed Apr 4, 2023

View reviewed changes

ratchek added 6 commits April 5, 2023 09:25

Add development flag to options.

097cac7

This does nothing for now except echo that it was called to terminal.

Change the default dev config dir in gitignore

a2d7e75

Modify the config directory to a hidden directory and add config to t…

382eb70

…he name

Add support for custom dev dirs

e7ba167

Clean up print statements used for debugging

8e4a5a1

real-yfprojects force-pushed the dev_mode branch from 2b0b756 to 8e4a5a1 Compare April 5, 2023 07:32

Merge branch 'dev_mode' of github.com:ratchek/vorta into dev_mode

93c2515

Add DEFAULT_DIR_FLAG

949da74

ratchek added 2 commits April 19, 2023 14:19

ratchek added 2 commits April 19, 2023 18:36

Update logger to import config properly

2784172

Initialize as system wide when running through application.py

330ca1c

real-yfprojects reviewed Apr 29, 2023

View reviewed changes

src/vorta/application.py Outdated Show resolved Hide resolved

src/vorta/application.py Outdated Show resolved Hide resolved

src/vorta/application.py Outdated Show resolved Hide resolved

src/vorta/application.py Outdated Show resolved Hide resolved

ratchek added 3 commits May 9, 2023 20:38

Merge upstream

8fa0772

Fix typo and loading config

0198ce8

Initialize config values in config module

a3238a4

Resolve paths when passed to dev mode

5e651ad

real-yfprojects reviewed May 12, 2023

View reviewed changes

src/vorta/__main__.py Outdated Show resolved Hide resolved

src/vorta/application.py Show resolved Hide resolved

src/vorta/__main__.py Outdated Show resolved Hide resolved

Shift path resolution to config

5f1d6c7

real-yfprojects reviewed May 12, 2023

View reviewed changes

ratchek added 2 commits May 23, 2023 16:27

Merge branch 'master' into dev_mode

3761bcc

Add docstrings and typing

6224e88

real-yfprojects reviewed May 28, 2023

View reviewed changes

ratchek added 2 commits May 29, 2023 10:36

Merge branch 'master' into dev_mode

1814782

Add metavar for better help message

b6578e6

real-yfprojects approved these changes May 29, 2023

View reviewed changes

real-yfprojects requested a review from m3nu May 29, 2023 20:54

m3nu approved these changes May 30, 2023

View reviewed changes

real-yfprojects merged commit 2cb9afd into borgbase:master May 30, 2023

real-yfprojects mentioned this pull request May 31, 2023

Update contributing guide with information about -D flag. borgbase/vorta.borgbase.com#42

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add a dev mode that allows for local storing of config files and logs #1682

Add a dev mode that allows for local storing of config files and logs #1682

ratchek commented Apr 4, 2023 •

edited by real-yfprojects

Loading

real-yfprojects commented Apr 4, 2023 •

edited

Loading

real-yfprojects commented Apr 4, 2023

real-yfprojects left a comment

ratchek commented Apr 4, 2023

real-yfprojects commented Apr 5, 2023

ratchek commented Apr 13, 2023 •

edited

Loading

real-yfprojects commented Apr 13, 2023

ratchek commented Apr 19, 2023

real-yfprojects commented Apr 20, 2023

ratchek commented May 11, 2023

ratchek commented May 11, 2023

real-yfprojects left a comment

real-yfprojects May 28, 2023

real-yfprojects commented May 29, 2023

ratchek commented May 29, 2023

m3nu commented May 30, 2023

real-yfprojects commented May 31, 2023

Add a dev mode that allows for local storing of config files and logs #1682

Add a dev mode that allows for local storing of config files and logs #1682

Conversation

ratchek commented Apr 4, 2023 • edited by real-yfprojects Loading

Description

Related Issue

Motivation and Context

How Has This Been Tested?

Screenshots (if appropriate):

Types of changes

Checklist:

real-yfprojects commented Apr 4, 2023 • edited Loading

real-yfprojects commented Apr 4, 2023

real-yfprojects left a comment

Choose a reason for hiding this comment

ratchek commented Apr 4, 2023

real-yfprojects commented Apr 5, 2023

ratchek commented Apr 13, 2023 • edited Loading

real-yfprojects commented Apr 13, 2023

ratchek commented Apr 19, 2023

real-yfprojects commented Apr 20, 2023

ratchek commented May 11, 2023

ratchek commented May 11, 2023

real-yfprojects left a comment

Choose a reason for hiding this comment

real-yfprojects May 28, 2023

Choose a reason for hiding this comment

real-yfprojects commented May 29, 2023

ratchek commented May 29, 2023

m3nu commented May 30, 2023

real-yfprojects commented May 31, 2023

ratchek commented Apr 4, 2023 •

edited by real-yfprojects

Loading

real-yfprojects commented Apr 4, 2023 •

edited

Loading

ratchek commented Apr 13, 2023 •

edited

Loading