Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce include_dir for use with gyp in a scalar context #766

Closed

Conversation

lovell
Copy link
Contributor

@lovell lovell commented Jul 14, 2020

This change reverts PR #757, adds a new include_dir scalar property and suggests the deprecation of the ambiguous include string-as-array property.

This is required due to the currently-documented approach of using the single include path in an array/list context instead of a scalar context with gyp.

-  'include_dirs': ["<!@(node -p \"require('node-addon-api').include\")"],
+  'include_dirs': ["<!(node -p \"require('node-addon-api').include_dir\")"],

The use of a gyp list/array context just happens to work when a path is absolute, hence no one spotting this ambiguity before, but shell expansion can fail on Windows when paths are relative and a gyp file contains multiple entries in its include_dirs directive.

This problem, which only affects v3.0.1 on Windows, will manifest itself as fatal error C1083: Cannot open include file: 'napi.h': No such file or directory.

This PR provides an overall approach that is more consistent with that used by nan - https://github.com/nodejs/nan#usage

"include_dirs" : ["<!(node -e \"require('nan')\")"]

Modules wishing to add support for filesystem paths containing whitespace will need to use this new include_dir property in the correct and newly-documented gyp scalar context.

Deprecate use of `include` in an gyp array context, which
happens to work when paths are absolute, but can fail on Windows
when paths are relative and a gyp file contains multiple entries
in its `include_dirs` directive.

This change corrects documentation and tooling, adds support for
relative paths (e.g. those containing whitespace) in a backwards
compatible manner and makes the approach holistically consistent
with that used by nan.
@mhdawson
Copy link
Member

@lovell, did we just break windows for existing users with 3.0.1 or just not fix it on windows?

@lovell
Copy link
Contributor Author

lovell commented Jul 15, 2020

node-addon-api v3.0.1 breaks node-gyp based installation for Windows users where a module that depends on it has defined multiple include_dir entries. Apologies for not picking this up in my testing.

As far as I can tell, the ability to support both filesystem paths containing whitespace and Windows users isn't possible with the <!@(... approach, hence this PR to revert and provide backwards compatibility via a new property to use with the preferred <!(... approach.

@mhdawson
Copy link
Member

mhdawson commented Jul 15, 2020

@lovell understood, thanks for the quick reply.

Would it be possible to add some tests to this PR which will test the broken case as well as the ones you were fixing? That way our CI will be able to validate it works across platforms (we have windows, linux etc. in that) My bad for not looking for tests in the first PR as we probably would have caught the issue.

@NickNaso we should get some messaging out to tell people not to use 3.0.1 and that a 3.0.2 will be coming soon. I think we should try to get a 3.0.2 out by end of next week which either reverts the breaking change or incorporates this PR if it is ready/landed.

@mhdawson
Copy link
Member

@NickNaso I also meant to ask if you wanted to send out the messaging via twitter and I think also a message on the README.md (if so let me know when you do and I'll retweet what you send out) or if you'd like somebody else to take a look at doing that?

@lovell
Copy link
Contributor Author

lovell commented Jul 15, 2020

I notice there's a Windows CI environment defined in an Appveyor config file, but this isn't run for PRs and doesn't appear to actually work as-is.

I attempted to modernise it but there is a compilation failure in the existing tests:

  addon_data.cc
  arraybuffer.cc
..\addon_data.cc(71): error C2248: 'Napi::Env::DefaultFini': cannot access private member declared in class 'Napi::Env' [C:\projects\node-addon-api\test\build\binding.vcxproj]
  c:\projects\node-addon-api\napi-inl.h(357): note: see declaration of 'Napi::Env::DefaultFini'
  C:\projects\node-addon-api\napi.h(168): note: see declaration of 'Napi::Env'

Is there an example of a successful Windows CI config/build/log as I can't quite work out how the tests are currently passing on Windows?

UPDATE: Appveyor was defaulting to VS2015. Updating to VS2017 allows the tests to compile - I'll submit a separate PR to re-enable the use of Appveyor testing.

@mhdawson
Copy link
Member

The regular CI runs are here: https://ci.nodejs.org/view/x%20-%20Abi%20stable%20module%20API/job/node-test-node-addon-api-LTS%20versions/

But they use our jenkins infra as opposed to an external service.

@mhdawson
Copy link
Member

With this as a windows example: https://ci.nodejs.org/job/node-test-node-addon-api-new/nodes=win-vs2019/2465/console, but seems you'd already figured out the issues, and @lovell thanks for looking at fixing up that file in a separate PR.

vladimiry added a commit to vladimiry/ElectronMail that referenced this pull request Jul 18, 2020
* windows-os specific error ("keytar" dependency compiling process)
* we lock "keytar/[email protected]" transient dependency for now since recently released "[email protected]" is broken, see nodejs/node-addon-api#766
* TODO revert when "node-addon-api" releases v3.0.2
@mhdawson
Copy link
Member

@lovell are you working on some tests? @NickNaso mentioned he might be able to add some tests to the PR if you were not able to get to it.

@lovell
Copy link
Contributor Author

lovell commented Jul 23, 2020

I had a quick look at how we might be able to reproduce this using the existing tests but haven't spotted an obvious solution yet. Nick, if you have an idea then please go ahead.

The underlying cause is a gyp bug/feature relating to path-mangling on Windows (in my experience >80% of gyp problems affect only Windows). Passing it an include_dirs value of ..\node_modules\node-addon-api in an array context appears to result in a mis-conversion to /I"..\node_modulesnode-addon-api".

@NickNaso
Copy link
Member

Hi everybody,
I have some ideas and I will work on that in this weekend.

@mhdawson
Copy link
Member

@NickNaso just wondering if you've had a chance to take a look at this one?

@NickNaso
Copy link
Member

@mhdawson sorry for the delay I would complete the work in this weekend.

@mhdawson
Copy link
Member

@NickNaso thanks.

@mhdawson
Copy link
Member

mhdawson commented Aug 24, 2020

Update from @NickNaso through email, -> just back from vacation, working on it.

Copy link
Member

@mhdawson mhdawson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, tests are going to be added through -> #808

mhdawson pushed a commit that referenced this pull request Sep 4, 2020
Introduce `include_dir` for use with gyp in a scalar context

Deprecate use of `include` in an gyp array context, which
happens to work when paths are absolute, but can fail on Windows
when paths are relative and a gyp file contains multiple entries
in its `include_dirs` directive.

This change corrects documentation and tooling, adds support for
relative paths (e.g. those containing whitespace) in a backwards
compatible manner and makes the approach holistically consistent
with that used by nan.

PR-URL: #766
Reviewed-By: Michael Dawson <[email protected]>
@mhdawson
Copy link
Member

mhdawson commented Sep 4, 2020

Landed as f27623f

@mhdawson mhdawson closed this Sep 4, 2020
@lovell
Copy link
Contributor Author

lovell commented Sep 4, 2020

Thanks for merging, thank you both for your continued maintenance of this module, and with apologies for temporarily breaking things for Windows users.

@lovell lovell deleted the introduce-include_dir-as-gyp-scalar branch September 4, 2020 20:35
mhdawson pushed a commit that referenced this pull request Sep 10, 2020
Superlokkus pushed a commit to Superlokkus/node-addon-api that referenced this pull request Nov 20, 2020
Introduce `include_dir` for use with gyp in a scalar context

Deprecate use of `include` in an gyp array context, which
happens to work when paths are absolute, but can fail on Windows
when paths are relative and a gyp file contains multiple entries
in its `include_dirs` directive.

This change corrects documentation and tooling, adds support for
relative paths (e.g. those containing whitespace) in a backwards
compatible manner and makes the approach holistically consistent
with that used by nan.

PR-URL: nodejs#766
Reviewed-By: Michael Dawson <[email protected]>
Superlokkus pushed a commit to Superlokkus/node-addon-api that referenced this pull request Nov 20, 2020
devDefiWeb added a commit to devDefiWeb/electron-mail-app that referenced this pull request May 28, 2022
* windows-os specific error ("keytar" dependency compiling process)
* we lock "keytar/[email protected]" transient dependency for now since recently released "[email protected]" is broken, see nodejs/node-addon-api#766
* TODO revert when "node-addon-api" releases v3.0.2
kevindavies8 added a commit to kevindavies8/node-addon-api-Develop that referenced this pull request Aug 24, 2022
Introduce `include_dir` for use with gyp in a scalar context

Deprecate use of `include` in an gyp array context, which
happens to work when paths are absolute, but can fail on Windows
when paths are relative and a gyp file contains multiple entries
in its `include_dirs` directive.

This change corrects documentation and tooling, adds support for
relative paths (e.g. those containing whitespace) in a backwards
compatible manner and makes the approach holistically consistent
with that used by nan.

PR-URL: nodejs/node-addon-api#766
Reviewed-By: Michael Dawson <[email protected]>
kevindavies8 added a commit to kevindavies8/node-addon-api-Develop that referenced this pull request Aug 24, 2022
Marlyfleitas added a commit to Marlyfleitas/node-api-addon-Development that referenced this pull request Aug 26, 2022
Introduce `include_dir` for use with gyp in a scalar context

Deprecate use of `include` in an gyp array context, which
happens to work when paths are absolute, but can fail on Windows
when paths are relative and a gyp file contains multiple entries
in its `include_dirs` directive.

This change corrects documentation and tooling, adds support for
relative paths (e.g. those containing whitespace) in a backwards
compatible manner and makes the approach holistically consistent
with that used by nan.

PR-URL: nodejs/node-addon-api#766
Reviewed-By: Michael Dawson <[email protected]>
Marlyfleitas added a commit to Marlyfleitas/node-api-addon-Development that referenced this pull request Aug 26, 2022
wroy7860 added a commit to wroy7860/addon-api-benchmark-node that referenced this pull request Sep 19, 2022
Introduce `include_dir` for use with gyp in a scalar context

Deprecate use of `include` in an gyp array context, which
happens to work when paths are absolute, but can fail on Windows
when paths are relative and a gyp file contains multiple entries
in its `include_dirs` directive.

This change corrects documentation and tooling, adds support for
relative paths (e.g. those containing whitespace) in a backwards
compatible manner and makes the approach holistically consistent
with that used by nan.

PR-URL: nodejs/node-addon-api#766
Reviewed-By: Michael Dawson <[email protected]>
wroy7860 added a commit to wroy7860/addon-api-benchmark-node that referenced this pull request Sep 19, 2022
johnfrench3 pushed a commit to johnfrench3/node-addon-api-git that referenced this pull request Aug 11, 2023
Introduce `include_dir` for use with gyp in a scalar context

Deprecate use of `include` in an gyp array context, which
happens to work when paths are absolute, but can fail on Windows
when paths are relative and a gyp file contains multiple entries
in its `include_dirs` directive.

This change corrects documentation and tooling, adds support for
relative paths (e.g. those containing whitespace) in a backwards
compatible manner and makes the approach holistically consistent
with that used by nan.

PR-URL: nodejs/node-addon-api#766
Reviewed-By: Michael Dawson <[email protected]>
johnfrench3 pushed a commit to johnfrench3/node-addon-api-git that referenced this pull request Aug 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants