Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bundle install fails on windows-latest after using bundle config #13

Closed
mojavelinux opened this issue Jan 30, 2020 · 36 comments
Closed

Comments

@mojavelinux
Copy link

If I use bundle config to set bundler options (e.g., bundle config --local without docs), the subsequent call to bundle install fails. The action gives no reason why it failed.

##[error]Process completed with exit code 1.

This does not happen on ubuntu-latest.

@mojavelinux
Copy link
Author

In contrast, passing options directly to bundle install works fine:

bundle --path .bundle/gems

However, that results in warnings about using persistent flags with bundle install.

@eregon
Copy link
Member

eregon commented Jan 30, 2020

Could you link to a log failing because of this? And/or share a workflow triggering it?

@mojavelinux
Copy link
Author

There's literally nothing in the log. You can find a workflow example here: https://github.com/asciidoctor/asciidoctor-pdf/blob/master/.github/workflows/ci.yml#L21 (I'm currently messing with it, so I could be changing).

@mojavelinux
Copy link
Author

2020-01-30T11:23:35.9402420Z Running `bundle install --jobs 3 --retry 3 --verbose` with bundler 2.1.4
2020-01-30T11:23:36.4473629Z Fetching https://github.com/prawnpdf/prawn-table.git
2020-01-30T11:23:39.8952804Z NOTE: Gem::Specification#rubyforge_project= is deprecated with no replacement. It will be removed on or after 2019-12-01.
2020-01-30T11:23:39.9130110Z ##[error]Process completed with exit code 1.
2020-01-30T11:23:39.9155841Z Cleaning up orphan processes

So the process is just dying.

@mojavelinux
Copy link
Author

mojavelinux commented Jan 30, 2020

This may or may not having anything to do with this action. I really have no idea because it's giving absolutely no information when it dies. Maybe that's GitHub Actions suppressing the output. Maybe that's Windows. If this issue only serves as a warning to others not to use bundle config on Windows runners, I would be okay with that. We can only fix what we can fix.

@mojavelinux
Copy link
Author

mojavelinux commented Jan 30, 2020

For completeness, here are the warnings we get when using flags instead of bundle config.

[DEPRECATED] The `--path` flag is deprecated because it relies on being remembered across bundler invocations, which bundler will no longer do in future versions. Instead please use `bundle config set path '.bundle/gems'`, and stop using this flag
[DEPRECATED] The `--without` flag is deprecated because it relies on being remembered across bundler invocations, which bundler will no longer do in future versions. Instead please use `bundle config set without 'docs lint coverage'`, and stop using this flag

@eregon
Copy link
Member

eregon commented Jan 30, 2020

As long as the error doesn't happen during the uses: eregon/use-ruby-action@v1 step, I think it is unlikely it has anything to do with this action.
An easy way to make sure would be to use https://github.com/actions/setup-ruby and see if it fails too.

BTW, could you use eregon/use-ruby-action@v1 and not eregon/[email protected] as recommended in the README? That way you'll get bug fixes, and old releases are not guaranteed to work.

I'm currently messing with it, so I could be changing

y can be used to point to a specific version of a file in GitHub

A link to a log would help me understand better.
I found https://github.com/asciidoctor/asciidoctor-pdf/runs/416999064?check_suite_focus=true
That seems clearly a bug of either GHA or Bundler 2.

I would recommend to not install bundler explicitly, unless you really need a specific version. This action already provides a version of bundle for all Rubies (see https://github.com/eregon/use-ruby-action#bundler).

@slonopotamus
Copy link

That way you'll get bug fixes, and old releases are not guaranteed to work.

Oh, I didn't know that it works that way.

@eregon
Copy link
Member

eregon commented Jan 30, 2020

@slonopotamus That's documented in https://github.com/eregon/use-ruby-action#versioning

@MSP-Greg
Copy link
Collaborator

MSP-Greg commented Jan 30, 2020

@mojavelinux

I'll have a look at this later today. I tried the following locally with ruby master on Windows:

PS C:\Greg\GitHub\asciidoctor-pdf> bundle config --local path .bundle/gems
PS C:\Greg\GitHub\asciidoctor-pdf> bundle config --local without docs lint coverage
PS C:\Greg\GitHub\asciidoctor-pdf> bundle --jobs 3 --retry 3

See fix at asciidoctor/asciidoctor-pdf#1531

There's always been some quirky things with PowerShell in CI. How STDERR output is handled is a configurable item. Same with EXIT codes.

@eregon
Copy link
Member

eregon commented Jan 30, 2020

Thank you for looking at this @MSP-Greg, it's much appreciated and I had no idea what's going on here. I'll close this issue because I don't think this action can do anything about it, it seems a GitHub Actions/PowerShell issue.

I think it's best to discuss in the PR: asciidoctor/asciidoctor-pdf#1531
Maybe a workaround would be to use shell: bash for multi-line scripts.
That might be useful as I expect many Ruby developers are more familiar with bash than PowerShell, and it allows to use a common syntax on all 3 OS.

@eregon eregon closed this as completed Jan 30, 2020
@slonopotamus
Copy link

use shell: bash for multi-line scripts

That explodes with JRuby on Windows.

@eregon
Copy link
Member

eregon commented Jan 30, 2020

That explodes with JRuby on Windows.

@slonopotamus Can you show an example log where that happens?

@eregon
Copy link
Member

eregon commented Jan 30, 2020

BTW, I'm happy to improve the README based on the discussion in the PR.

@slonopotamus
Copy link

That explodes with JRuby on Windows.

@slonopotamus Can you show an example log where that happens?

https://github.com/slonopotamus/epubcheck-ruby/runs/407569369?check_suite_focus=true#step:4:23

@eregon
Copy link
Member

eregon commented Jan 30, 2020

Interesting.
I noticed the bin/jruby: line 155: [: C:\Program: binary operator expected as well, but it didn't seem fatal. It was fixed in JRuby master, but not yet released: jruby/jruby#6007 (comment)

I'm not sure what

/usr/bin/sh: -c: line 0: unexpected EOF while looking for matching `"'
/usr/bin/sh: -c: line 1: syntax error: unexpected end of file

comes from.
It looks like it happens from bundle exec rake.

@slonopotamus
Copy link

slonopotamus commented Jan 30, 2020

I didn't investigate it very much. Just added a mark my brain: jruby+windows+bash = no fun, use native shell instead. There are so many more issues with jruby on windows, this is a very small of them.

@mojavelinux
Copy link
Author

As long as the error doesn't happen during the uses: eregon/use-ruby-action@v1 step, I think it is unlikely it has anything to do with this action.

True, though I'm kind of exploring if it has to do with how Ruby is installed.

could you use eregon/use-ruby-action@v1 and not eregon/[email protected] as recommended in the README

Yes, ignore that. We're actively testing specific versions. Once our build is stable, we'll switch to the major.

I would recommend to not install bundler explicitly, unless you really need a specific version. This action already provides a version of bundle for all Rubies

Didn't realize that. Yes, that's what we want.

Maybe a workaround would be to use shell: bash for multi-line scripts.

Is the multi-line script what is causing the problem? That's what I'm trying to determine.

@mojavelinux
Copy link
Author

BTW, I'm happy to improve the README based on the discussion in the PR.

If we arrive at a conclusion as to what causes the exit code 1, I think we should do that. I am still not seeing the reason why we get an exit code 1 when using bundle config vs when not using bundle config. Unless it's the multiple run lines.

@eregon
Copy link
Member

eregon commented Jan 30, 2020

Is the multi-line script what is causing the problem? That's what I'm trying to determine.

Could you try with individual commands to see if that would help?
@MSP-Greg was saying in the PR it might be related to multiple lines + PowerShell settings about how warnings (STDERR output) are treated.

@eregon
Copy link
Member

eregon commented Jan 30, 2020

Let's reopen until we figure it out better, sorry for the early close.

@eregon eregon reopened this Jan 30, 2020
@MSP-Greg
Copy link
Collaborator

Sorry, not sure where to discuss. I'm not sure what is causing this. The warning are coming from RubyGems. I don't know if the command is exiting with a code other than 0, and I know that I've seen Windows CI get confused over STDERR output, especially using PowerShell.

With some AppVeyor build scripts for Ruby, I had to pipe STDERR to STDOUT, otherwise the script stopped.

IOW, this is not an issue with this repo or asciidoctor-pdf. It's somewhere within the space of Actions, PowerShell, Bundler, & RubyGems. The fact that I can't repo it locally doesn't help...

@eregon
Copy link
Member

eregon commented Jan 30, 2020

Re JRuby on Windows, it seems to work better if we remove the bin/jruby Bash script (and therefore always use the .exe).
Compare https://github.com/eregon/use-ruby-action/runs/418152773?check_suite_focus=true which warns for gem/bundle steps and https://github.com/eregon/use-ruby-action/runs/418163739?check_suite_focus=true which doesn't warn.
I'll do that change.

@MSP-Greg
Copy link
Collaborator

Re JRuby on Windows, it seems to work better if we remove the bin/jruby Bash script (and therefore always use the .exe).

Sorry, I haven't really looked at JRuby on Windows. But, unless the bash script does something, I'd use the exe. There is no script with mswin or mingw builds for Ruby itself. All the gem bin files are .bat, .cmd, or .ps1. Windows normally needs the extension to know how to launch the file...

eregon added a commit to ruby/ruby-builder that referenced this issue Jan 30, 2020
eregon added a commit to ruby/ruby-builder that referenced this issue Jan 30, 2020
* The Bash launcher causes warnings, see
  ruby/setup-ruby#13 (comment)
* Add tests when using an absolute path to jruby.
@eregon
Copy link
Member

eregon commented Jan 30, 2020

I removed the Bash launcher on JRuby, it seems better now.

@mojavelinux
Copy link
Author

Using separate steps fixes the problem.

https://github.com/mojavelinux/asciidoctor-pdf/actions/runs/33167078

I see no warnings though, which is very strange. While we work to pinpoint the cause of the exit 1, perhaps recommending using separate steps is a reasonable tradeoff until then.

@eregon
Copy link
Member

eregon commented Jan 30, 2020

In that run, the existing Bundler is used, which might change something, but probably not.
It seems far more likely the problem is indeed PowerShell with multiple lines.

The full warning is:

Fetching https://github.com/prawnpdf/prawn-table.git
NOTE: Gem::Specification#rubyforge_project= is deprecated with no replacement. It will be removed on or after 2019-12-01.
Gem::Specification#rubyforge_project= called from D:/a/asciidoctor-pdf/asciidoctor-pdf/.bundle/gems/ruby/2.7.0/bundler/gems/prawn-table-515f2db29486/prawn-table.gemspec:19.

And in https://github.com/asciidoctor/asciidoctor-pdf/runs/416845301 it's cut after the first warning line:

Fetching https://github.com/prawnpdf/prawn-table.git
NOTE: Gem::Specification#rubyforge_project= is deprecated with no replacement. It will be removed on or after 2019-12-01.

So it really seems PowerShell just aborts on STDERR output, for multi-line scripts, as @MSP-Greg said.

I think this would be highly valuable to report to GitHub Actions as a bug when using PowerShell with multiple lines. Could you report it?

@mojavelinux
Copy link
Author

Thanks for that assessment, @eregon. That seems to be it!

I agree, this should go upstream. I'm not very familiar with the GitHub Actions project. Do you know where it should be filed?

@MSP-Greg
Copy link
Collaborator

I just tried the following in a step, and it passed:

- name: ruby test
  run: |
    ruby -e "STDERR.puts 'Causes PS exit?'"
    ruby -v

@eregon
Copy link
Member

eregon commented Jan 30, 2020

I think https://github.community/t5/GitHub-Actions/bd-p/actions is a good place to report it.

A bit of searching led me to
https://help.github.com/en/actions/automating-your-workflow-with-github-actions/workflow-syntax-for-github-actions#exit-codes-and-error-action-preference
And $ErrorActionPreference = 'stop' is according to https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_preference_variables?view=powershell-7#erroractionpreference:

Determines how PowerShell responds to a non-terminating error, an error that doesn't stop the cmdlet processing. For example, at the command line or in a script, cmdlet, or provider, such as the errors generated by the Write-Error cmdlet.
You can use a cmdlet's ErrorAction common parameter to override the preference for a specific command.
The valid values are as follows:
...
Stop: Displays the error message and stops executing.

And "the Write-Error cmdlet" seems similar as writing to STDERR.
Sounds related at least.

@eregon
Copy link
Member

eregon commented Jan 30, 2020

@MSP-Greg Could you post the log of the workflow run?

@mojavelinux
Copy link
Author

In that run, the existing Bundler is used, which might change something, but probably not.

Nope, it didn't change anything. It works either way when I use separate steps.

@eregon
Copy link
Member

eregon commented Jan 30, 2020

I can reproduce the failure in GitHub Actions with:
https://github.com/eregon/use-ruby-action/runs/418285035?check_suite_focus=true

    - name: Test
      run: |
        gem install bundler
        bundle config --local path .bundle/gems
        ruby -e 'STDERR.puts %{NOTE: warning}; STDERR.puts 42'
        echo after

Removing the bundle config makes it pass:
https://github.com/eregon/use-ruby-action/runs/418285586?check_suite_focus=true

    - name: Test
      run: |
        gem install bundler
        ruby -e 'STDERR.puts %{NOTE: warning}; STDERR.puts 42'
        echo after

Not installing latest bundler also passes:
https://github.com/eregon/use-ruby-action/runs/418287337?check_suite_focus=true

    - name: Test
      run: |
        bundle config --local path .bundle/gems
        ruby -e 'STDERR.puts %{NOTE: warning}; STDERR.puts 42'
        echo after

Installing latest bundler and bundle config but no STDERR print passes too:
https://github.com/eregon/use-ruby-action/runs/418294467?check_suite_focus=true

    - name: Test
      run: |
        gem install bundler
        bundle config --local path .bundle/gems
        echo after

So somehow gem install bundler + bundle config --local path .bundle/gems + STDERR output cause the failure? Maybe recent Bundler does something strange to the shell?

@MSP-Greg
Copy link
Collaborator

What version (Bundler or Ruby) passed? I'll check this locally...

@eregon
Copy link
Member

eregon commented Feb 8, 2020

I'm not sure what's wrong here, but I already spent too much time debugging weird Windows shell bugs.
Please report the issue to GitHub Actions, for example at https://github.community/t5/GitHub-Actions/bd-p/actions

This is clearly not a bug of this action.

@eregon eregon closed this as completed Feb 8, 2020
@mojavelinux
Copy link
Author

As far as I'm concerned, this is a solved issue either by:

  1. Using a separate step or
  2. Setting $ErrorActionPreference = 'Continue'

It's just not a hill I want to die on trying to solve otherwise.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

4 participants