Add support for maintaining local registries #45
Conversation
This new method is intended to be used interactively to manage private repositories.
|
Thank you for making this much needed refactor! You still need to route the users request to this new register method. This happens here: https://github.com/JuliaComputing/Registrator.jl/blob/9023c098db39b5343eaed4f0f6050b934d4161a3/src/Server.jl#L803 Maybe you can have a |
|
I was intending for this to be used interactively at the REPL like the mockup in this discourse comment. It's not clear to me what the workflow should be for someone running a Registrator server, and wanting to locally tag releases. Could you elaborate? |
* Switch to git add --all to fix a problem when the registry directory
is relative.
* Don't try to print the empty packages section using TOML.print
because it discards empty sections.
|
In that case, its fine. I was thinking (as an example) the registry is on a distributed file system that an org would access for its project. Registrator would listen to GitHub comments on the package repo and just make commits accordingly on the registry. |
|
Looks like this PR will be perfect for use in a CI script to allow organizations to register versions and create private registries in platform-independent manner. |
| @debug("Verifying package name and uuid in deps") | ||
| for (k, v) in pkg.deps | ||
| u = string(v) | ||
| if haskey(registry_data["packages"], u) |
There was a problem hiding this comment.
Where does registry_data come from? I can't see it defined in this function.
| throw(err) | ||
| end | ||
| else | ||
| err = "Error in `[deps]`: Package '$k' with UUID: $u not found in registry or stdlib" |
There was a problem hiding this comment.
It looks like this only works if all non-stdlib dependencies are within the same registry. It would be good to support other scenarios, in particular the case of a private registry that allows external dependencies both in stdlib and the General registry.
| length(remote_name) > 1 && throw("Repository has multiple remotes.") | ||
| # TODO: Handle this case? | ||
| remote_name[1] == "" && throw("Repository does not have a remote.") | ||
| return read(`$git remote get-url $(remote_name[1])`, String) |
There was a problem hiding this comment.
This should chomp the ending newline? (readchomp can be used directly if the String argument is removed.)
|
I guess this PR is in a rather bad state of conflicts at this point and that local registry maintenance will need a retake when other things settle. However, those who need that functionality now are welcome to try https://github.com/GunnarFarneback/Registrator.jl, which adds some finishing touches on top of this PR. It's obviously an evolutionary dead end though. |
|
@GunnarFarneback, your repo is quite useful, many thanks. Would love to see this "official" version finished off. |
|
I really like this! This would be very useful to us. I'm planning to run a small custom registry for a physics collaboration (public, actually, but the packages are way too application-specific for Julia general), and I have the impression that this PR would make it much easier to maintain without running and maintaining a Registrator server. |
|
I just started using the version by @GunnarFarneback and liked it very much
so this still needs a couple of fixes and for this it would be great to have only one Registrator in order to go forward here where should I propose such fixes? |
|
let me highlight that this is a super crucial package for adoption of julia background: I just read up about how to deal with a development status and the result is that you better use a private registry if you want to work with private packages on several computes or with several people. Specifically, just using Hence getting this package Registrator ready seems crucial for everyone to develop their own packages. I hope it gets some priority soon. Next to having no Linter in Juno, this is one of the most unsupported parts of julia while still being super crucial for adoption (at least in my opinion). |
Best option would be to fix up this PR and get it into a merge-worthy state. Want to give it a try @schlichtanders? |
|
Note: I think most of the registry handling functionality has moved to https://github.com/JuliaRegistries/RegistryTools.jl so this PR should probably target that instead. Note also https://github.com/JuliaRegistries/RegistryTools.jl was moved out such that it could be used to rewrite https://github.com/JuliaLang/PkgDev.jl so the functionality you are after might already exist in PkgDev. |
|
@davidanthoff can probably answer whether or not this functionality ("support for maintaining local registries") currently exists in |
|
There were a number of complaints about the quality of Registrator and that it was "not fit for open source". It's quite difficult to keep it running as a production service and satisfy all the things people want to do with it as an open source project. Rather than keep trying to do that—and apparently failing—this package should be effectively considered legacy now. Julia Computing is still running an instance of it to help manage General registrations, but the project you should contribute to and use for your registries is RegistryTools. |
|
thanks for the clarification. Unfortunately, https://github.com/JuliaRegistries/RegistryTools.jl is not fully documented yet it seems. |
I am willing to merge bugfix PRs to my fork for the short term but clearly the long-term future is elsewhere. It's difficult to guess the scope of RegistryTools but if this functionality doesn't fit there in its entirety, the best solution is probably to start a new package for the purpose, which likely would be a rather thin wrapper around RegistryTools. |
|
https://github.com/JuliaLang/PkgDev.jl in principle supports local registries, we use it with one. There are two issues right now with PkgDev.jl:
Other than that I much prefer the workflow of it vs the JuliaRegistrator story, because it also handles increasing version numbers in |
|
Glad to see this thread is getting picked up again. The biggest drawback I see with the current PkgDev story is that it is Github-specific. Would be great if we can separate out the GH-specific parts so that it would work with other git hosting providers if needed. Would be great to either additionally support Gitlab or just generic git (which removes the "create pull request" part). I can take a look at this when I have time to see what it would take to contribute this change. What such a change be welcome/desired? |
YES, very much! I think we should first get the GitHub story more stable, and start to have some test story, but in principle I would love to see support for other hosting solutions as well. Could you maybe open an issue over at PkgDev.jl for this discussion? Probably best to have that over there. |
The short term ended early. That fork is now discontinued.
The functionality of the fork is instead available in LocalRegistry, which so far has added some minor improvements, off-loaded a small part of the code to |
|
@davidanthoff Thanks for the enthusiasm -- I just added JuliaLang/PkgDev.jl#164 for this. @GunnarFarneback -- This is awesome. Thank you for putting this together. I had something private that relied on your old defunct branch for the purpose. I'll try out your new LocalRegistry package soon to see if it works for us. |
|
@GunnarFarneback so I guess bug-reports like #45 (comment) should be going to LocalRegistry? |
|
Yes, although I just might have fixed those particular ones already. |
|
Between RegistryTools, PkgDev, and LocalRegistry (now registered) I think this can be closed. |
|
LocalRegistry has been very reliable for me, thanks so much for contributing it! |
I'd like to second that, LocalRegistry is very very helpful! |
|
Likewise, I should have followed up earlier -- LocalRegistry is working great for us and covering our needs on this front. |
This seems to work, but I'm not very attached to the way everything is done here. If you would prefer it in a separate package (apart from the refactoring), then feel free to say so.
Todo: