Choosing and managing environments for tools

[jkiviluo]

First noting that it is already possible to ship projects with environments - they just need to be in the project folder (taking lot of space) and they will work only for one architecture.

This replaces issues #1499 and #2196

Relates to #2554 (version control of workflows - after the 2nd stage in this issue, it would be easier for the users to update the tools to the new versions according to the project level settings).

Relates to #2794 (users will need to make sure they have a working environment for the tools in a workflow, but we can make it easier and more transparent with the changes in this issue)

What if a tool spec / tool is missing: #2958.

This is written as for Python, but Julia tools need a similar update of course. I suppose executables are mostly immune to this.

Target functionality is as follows (later these changes are broken into stages - we want to do something else between those stage):

Toolbox level:

• Defines default interpreter/console for new tools, with three radio button options:
  • Use Toolbox interpreter (recommended)
  • Drop-down list to choose interpreter; also button to add new interpreters (dialog)
  • Drop-down list to choose kernel; also button to make new kernel (dialog to choose from which interpreter)

Project level:

• This is just developer mode convenience functionality and saves to local project settings (version control of workflows will address tool versions at project level)
• Defines interpreter/console for new Python tools in the project, with three radio button options:
  • Use Toolbox interpreter (recommended for Python3 scripts that uses only packages that Toolbox already has)
  • Drop-down list to choose interpreter; also button to add new interpreters (dialog)
  • Drop-down list to choose kernel; also button to make new kernel (dialog to choose from which interpreter)

Tool specification level:

• Radio button to requirements.txt (1st priority to make functional), pyproject.toml (2nd priority), setup.py (3rd priority), environment.yml (distant 4th priority)
• If none is given, displays a warning that no source for environment is defined - tool may not function on different computer

Tool level:

• Note that this is the primary level where things are really defined. Toolbox and project levels are just helpers to have initial defaults (after something has been defined in tool, those defaults do not apply anymore).
• Show current interpreter/kernel path (already there)
• Show if tool requirements are not met by the current environment [later]
  • Find matching environment automatically and suggest to use that [later]
  • Warn if the tool specification does not have information how to setup the environment [later]
• Dialog to change the environment
  • Those three options again
• Make new environment using requirements.txt/pyproject.toml/setup.py/environment.yml

This could be implemented in three stages:

[ ] First stage (before user / developer mode update):

• Add 'use Python interpreter of Spine Toolbox' option to toolbox level (for clarity - at the moment this is assumed when no interpreter is selected in the text box)
• Do not yet change underlying functionality (current assumption: if tool does not have interpreter chosen, then use this default)
• Move choosing the environment from tool specification to tool (dialog to choose the environment with those three options)
  • As there is no better information, tool uses the Toolbox environment until user has chosen an environment
• Add more explanation (what is environment, requirements.txt etc., interpreter, kernel, console, ...) behind '?' button in the tool icon next to the environment selection

[ ] Second stage (after user / developer mode update):

• Tool specification points to the environment source (requirements.txt and pyproject.toml for starters). Consider Poetry etc. if they are a better option and maybe would allow to handle all options right away.
• Tool: Make new environment using requirements.txt etc.
• Change the logic: toolbox level settings just give defaults for new tools. Tools will no longer use Toolbox defaults when nothing is chosen - they will use current Spine Toolbox environment. Not sure if this requires some migration management (and not fullly sure if this could be done already in the first stage without causing headache to users - if it would not, then would be good to do it in the first stage already).
• If not too hard: check and show if the tool has a functional environment

[ ] Third stage (let's see when can be done):

• Make project level
  • We should probably have File --> Settings and File --> Project settings. That should then include current project description too.
• Tool specification/making new environment: Add rest of environment sources. At least setup.py. But environment.yml may be something we don't want to support (because requires conda).
• Automatic matching of requirements (tool level)