Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add plugins to conda-based installer #21490

Open
mrclary opened this issue Nov 3, 2023 · 10 comments · May be fixed by #22862
Open

Add plugins to conda-based installer #21490

mrclary opened this issue Nov 3, 2023 · 10 comments · May be fixed by #22862
Assignees
Milestone

Comments

@mrclary
Copy link
Contributor

mrclary commented Nov 3, 2023

Per last developer meeting, we propose:

This will provide the opportunity for these plugins to be tested in the wild in the absence of development releases for the plugins and a plugin manager.
In the future, a plugin manager will allow users to install development releases of plugins for testing (alpha, beta, release-candidate).

@mrclary mrclary added this to the v6.0alphaX milestone Nov 3, 2023
@mrclary mrclary self-assigned this Nov 3, 2023
@mrclary
Copy link
Contributor Author

mrclary commented Nov 3, 2023

@jitseniesen, I don't remember what the third plugin was...
Also, are these plugins compatible with Spyder 6.x?

@jitseniesen
Copy link
Member

@jitseniesen, I don't remember what the third plugin was...

spyder-line-profiler

Also, are these plugins compatible with Spyder 6.x?

No. I believe you have discovered this yourself in the mean time. Is it sufficient if the master branch on github is compatible with Spyder 6.x or do we need a version on conda-forge that is compatible?

@mrclary
Copy link
Contributor Author

mrclary commented Nov 3, 2023

Preferably a version on conda-forge should be available. Otherwise, conda packages will have to be built locally when the installer is created.

@jitseniesen
Copy link
Member

I can put a pre-release version of the plugin which is compatible with Spyder 6 on conda-forge. That's a bit of work, so I would prefer to do it for just one plugin (e.g., spyder-unittest) initially so that we can test everything out. Would that work for you?

@mrclary
Copy link
Contributor Author

mrclary commented Nov 6, 2023

I can put a pre-release version of the plugin which is compatible with Spyder 6 on conda-forge. That's a bit of work, so I would prefer to do it for just one plugin (e.g., spyder-unittest) initially so that we can test everything out. Would that work for you?

Let's talk about it at our next meeting. Ideally, I'd like to understand what the update strategy is and then work within that. Even if we build conda packages of plugins on Spyder's CI for nightly builds (they can be cached), I just want to make sure that we have a well thought out plan so that we don't have uncertainty moving forward.

@jitseniesen
Copy link
Member

I did not appreciate that the plan is to add the plugins to the "standard" self-contained installer. I somehow was working under the impression that we would create a new variant, like the "lite" (sic) installer, with the external plugins and that the standard installer would remain as it is now. That would soften the impact of any issues caused by the external plugins, but of course having yet another variant of the installer adds complexity and costs more resources.

I think the idea to release the external plugins on conda-forge just before Spyder and have the external plugin depend on a specific version of Spyder is workable with good communication, but it is not a long term solution. Maybe do this for 6.0.x and plan for another system for 6.1.x? For context, Spyder 5.0.0 and 5.1.0 were four months apart.

@mrclary
Copy link
Contributor Author

mrclary commented Nov 10, 2023

I think the idea to release the external plugins on conda-forge just before Spyder and have the external plugin depend on a specific version of Spyder is workable with good communication, but it is not a long term solution.

I agree that this is not a long-term solution. Having a plugin manager is the best solution, and external plugins could be released on their own schedule, effectively independent from Spyder. In the mean time, I'm fairly confident that the worst extent of the issues caused by the external plugins will be a non-working plugin.

I think you're correct that releasing external plugins on conda-forge immediately prior to a Spyder release is workable, just like we do for spyder-kernels and friends. I propose the following:

  • Change the external plugin's constraint on Spyder only as needed for the plugin, and do so only with commits to the appropriate branch; no release is necessary. The only requirement here is that the final commit on the branch shows a constraint on Spyder that allows the current Spyder dev version, e.g. 6.0.0a3.dev0.
  • Nightly builds of Spyder's installer will build a local conda package of the external plugin. If the previous requirement is met, then the build should be successful.
  • New releases of the external plugin can be made whenever the maintainer deems appropriate, as Spyder does not have a dependency on the plugin.
  • Immediately prior to a Spyder release, a compatible version of the external plugin should be available on conda-forge. This may be a stable or non-stable release of the plugin. The only requirement here is that a conda solve will be able to locate the appropriate (latest) plugin version.

@jitseniesen
Copy link
Member

Change the external plugin's constraint on Spyder only as needed for the plugin [...]

I am not sure I understand. The plugin's constraint is only listed in setup.py, e.g. here: https://github.com/spyder-ide/spyder-unittest/blob/7b614af02af2de09b181bfa84ed47df96cedf87b/setup.py#L40. Do you want to change this to spyder==6.0.0.dev0?

Nightly builds of Spyder's installer will build a local conda package of the external plugin.

Does this mean that we need to put a Conda recipe somewhere? Or will the recipe from conda-forge be used? Buildling spyder-notebook requires an extra step to compile the JavaScript, which may complicate things.

@mrclary
Copy link
Contributor Author

mrclary commented Nov 13, 2023

Change the external plugin's constraint on Spyder only as needed for the plugin [...]

I am not sure I understand. The plugin's constraint is only listed in setup.py, e.g. here: https://github.com/spyder-ide/spyder-unittest/blob/7b614af02af2de09b181bfa84ed47df96cedf87b/setup.py#L40. Do you want to change this to spyder==6.0.0.dev0?

I would not use spyder==6.0.0.dev0

Nightly builds of Spyder's installer will build a local conda package of the external plugin.

Does this mean that we need to put a Conda recipe somewhere? Or will the recipe from conda-forge be used? Buildling spyder-notebook requires an extra step to compile the JavaScript, which may complicate things.

I will use the existing feedstocks. I did notice that I had to compile the javascript on my local machine when installing spyder-notebook from source. I think this should also be incorporated into the feedstock. Nevertheless, it will be cached on CI so it will only get compiled/built if spyder-notebook changes (i.e. new commits).

At present, here is the Spyder version and its constraints in the plugins (on their respective master branch):

  • Spyder = 6.0.0a3.dev295
  • spyder-notebook: spyder >=5.4.3,<6 -> spyder >=6.0.0.dev0,<7
  • spyder-unittest: spyder >=5.4.1,<7 -> spyder >=6.0.0.dev0,<7
  • spyder-line-profiler: spyder>=6.0.0.dev0,<7 -> no change

spyder-notebook must raise the upper constraint to <7 in order to be compatible with Spyder 6 and included in the conda-based installer build. I recommend raising the lower constraint (since there are breaking changes), but this is optional. I also recommend raising the lower constraint in spyder-unittest. spyder-line-profiler does not require any changes.

The above recommendations should allow successful nightly builds of the conda-based installer.

When we release a version of Spyder, for example the next 6.0.0a3, we should have compatible versions of the plugins available on conda-forge immediately prior to the release because the release workflow will not build conda packages for the plugins. If a plugin release is not stable (alphas, betas, ...), then the appropriate conda channel will need to be specified in the workflow.

@jitseniesen
Copy link
Member

Okay, I changed the lower constraint on spyder-unittest in commit spyder-ide/spyder-unittest@7b921ae so that you can integrate it in the installer. It is not clear to me how this will be used, but I guess I will see when the package is built

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants