Inadequate build system

[zcalusic]

I was about to test and hopefully later document restic-server, only to find out that I have no idea how to build it in its current form. :(

Further inspection shows that sources are definitely weird, they're importing restic package as if though it is a standard library package, which is of course not true. Opening any source file in the editor, and it complains that it can't find the library. The source is not ready to be edited with an editor well equipped with various small tools really helpful for Go programming. For example I don't remember when was the last time I added import statement manually, it's the editor's job. ;)

OK, so far I've blindly run go run build.go as suggested in the README, noticing that it is also not optimal, because it rebuilds the binary even when nothing in the source has changed. And leaves the binary right there, instead of putting it in the proper place (which is $GOPATH/bin in most if not all cases). But, that is what go build does, it is brute force. Go install is the elegant and clever one, but it can't be used at all in the current state.

Finally, there's a vendor folder, but it is once again not following Go vendor folders guidelines, it's something different, don't know why or for what purpose.

So, enough of problem noticing, let's switch to question(s). :)

What is stopping us to architecture restic source like all other Go projects?

• have cmd/restic, cmd/restic-server etc...
• be able to build like go install ./cmd/restic or go install ./cmd/... to build them all
• go install is good, because it knows when to build, and when not to (saves energy, time...)
• go install will also put the binary in the proper place, together with all other binaries (which is already in $PATH, no need to mv after every build)
• make vendor folder be a regular Go vendor folder
• fix import paths everywhere, so that source files can be edited with powerful editors which understand errors and immensely speed up development
• more importantly fix source paths to be proper Go programs, there's definitely NO restic package in the Go standard library!

I'll proceed with restic-server testing by extracting it from the restic tree and fixing import paths, so that I can build it the regular way. But, if I then encounter any bugs, and fix them, the patches produced will no longer apply cleanly to the restic tree, and we have a problem. :(

Basically, what I'm suggesting with this rather long text, is to lower the bar for people who would be willing to contribute to the project, by making it similar to all other Go projects, not so different in so many details that standard tools and common knowledge useful for hundreds of other Go projects fails. And I'm willing to help in this regard, if help is needed.

[fd0]

Hey, thanks for raising this issue and offering your help. I'm wondering: Didn't you find the hint in README.md about CONTRIBUTING.md and especially the section about the development environment? At the end of this section it is explained how to set GOPATH (even automatically with direnv) to use the standard Go tools and editor integrations...

To be honest, at first I was quite irritated that you found the restic build/development environment "inadequate", but I suspect that you just did not find the documentation, so I think we need to improve it :)

By the way: running gb build also builds the restic REST server...

[zcalusic]

Believe it or not, I found it half an hour ago. I'm not happy with so many additional rules for building, but I'll give it a try. I have a feeling I'll need to send first patch for restic soon, so I currently concentrate on that task.

[zcalusic]

OK, it seems that after some extra commands I can edit files. But, I'll also need to be extra careful to NOT launch editor from panel or similar, because it won't work that way. Only from windows where I have changed $GOPATH. This is the first time I have to tamper with $GOPATH, and I really don't like it. I guess I'll try to live with that fact, only because restic is such a great project. 👍

But, I would really ask if you could revisit the way restic is built. I don't know if all these changes are because gb build tool is used, or for some other reason, but it feels quite cumbersome and complicated. Imagine if every Go project used its own $GOPATH. Basically, one would spend day switching $GOPATH. :(

I've since forked restic-server and the list of rules to compile it start and end with "go install". That's it, simple, elegant, effective. To put this in perspective, I counted more that 1000 Go projects/libraries in my (one & only :)) $GOPATH. Only 3 of them are inelegant in a way that they are rebuilt every time: grobi, inspeqtor & restic. And I already mentioned that this is the first time ever I'll edit $GOPATH.

[fd0]

In my opinion, gb is a great tool which simplifies many things. I encourage you to give it a try and read on the reasons why it was implemented. Many editors (e.g. vim-go) already include support for it out of the box.

Also, I think having a separate GOPATH for each project is better than one big GOPATH, so it seems we disagree on this one ;)

However, I can follow your argument that for new developers it is more complicated to get started with restic if they don't yet know how gb works. I'd like to hear your ideas on how to improve the documentation for new developers.

At the moment, we're using two different build systems:

• For developers: gb
• For end-users: build.go

I don't think that either system is "inadequate". Therefore I'm going to close this issue for now.

I know that the text I wrote in this comment may sound a bit negative, but please let me assure you that I value your feedback, and I'm open to (technical) arguments regarding a potential switch back to the default go tool. By the way: Integrating restic into a (global) GOPATH is a separate discussion.