Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

deps: delete deps/v8/third_party/zlib #47493

Closed
wants to merge 4 commits into from

Conversation

kvakil
Copy link
Contributor

@kvakil kvakil commented Apr 10, 2023

We currently have two copies of Chromium's zlib: one in deps/zlib and another in deps/v8/third_party/zlib. This has a couple of disadvantages:

  1. There is an additional cost to keeping both dependencies up-to-date, and in fact they were already out-of-sync (see the refs).

  2. People who compile with --shared-zlib (i.e. distro maintainers) will probably not be thrilled to learn that there is still a copy of zlib inside.

  3. It's aesthetically unpleasing.

This diff (per discussion in the refs) centralizes on deps/zlib and deletes deps/v8/third_party/zlib. That requires changing the include headers in two files in deps/v8/src, which was pretty straightforward.

When the user requests compiling with a shared zlib build, we still need to compile the contents of deps/zlib/google. This is not ideal but it's also not a change in behavior: prior to this diff those files were being compiled in the deps/v8/third_party/zlib version.

I tested this on Linux with the default build and a --shared-zlib build. I checked that the shared-zlib build dynamically linked zlib according to ldd, and that the regular build did not. I would appreciate if the reviewers could suggest some other build configurations to try.

Note to reviewers: the first commit does the changes, the second commit actually deletes the directory.

Refs: #47145
Refs: #47157

@nodejs-github-bot
Copy link
Collaborator

Review requested:

  • @nodejs/gyp
  • @nodejs/v8-update

@nodejs-github-bot nodejs-github-bot added build Issues and PRs related to build files or the CI. needs-ci PRs that need a full CI run. v8 engine Issues and PRs related to the V8 dependency. labels Apr 10, 2023
@kvakil kvakil force-pushed the unifyZlibApproachOne branch from 6ba7074 to f3d3302 Compare April 10, 2023 01:20
@Trott
Copy link
Member

Trott commented Apr 10, 2023

It appears that this breaks the build on Windows.

@bnoordhuis
Copy link
Member

D:\a\node\node\deps\zlib\google\compression_utils_portable.h(19,10): fatal error C1083: Cannot open include file: 'zlib.h': No such file or directory [D:\a\node\node\deps\zlib\zlib_google.vcxproj]

That happens because you changed includes in deps/v8/src from:

#include "third_party/zlib/google/compression_utils_portable.h"

To:

#include "compression_utils_portable.h"

You'll need to add deps/zlib to the right include_dirs directives in tools/v8_gypfiles/v8.gyp and possibly other build files in said directory.

As a rule we don't accept V8 patches that haven't been merged upstream but deleting 45,000+ lines of code in exchange for a two-line diff seems like a worthwhile tradeoff.

If Windows weren't a concern, I'd suggest symlinking deps/v8/third_party/zlib to deps/zlib but yeah, Windows...

@targos
Copy link
Member

targos commented Apr 11, 2023

Can you please make a standalone commit for the changes in deps/v8 and common.gypi? We must be able to cherry-pick it easily during V8 updates.

As a rule we don't accept V8 patches that haven't been merged upstream but deleting 45,000+ lines of code in exchange for a two-line diff seems like a worthwhile tradeoff.

+1. I'll adapt the V8 update script to stop adding zlib after this lands.

@targos
Copy link
Member

targos commented Jun 14, 2023

@kvakil is there a chance you'll continue this PR? I think it may fix a problem introduced with V8 11.5: #48456 (comment)

kvakil added 3 commits June 22, 2023 16:21
This is the first of a series of patches. This patch is contains changes
to the existing zlib.gyp file to allow it to be used by our v8.gyp.

---

We currently have two copies of Chromium's zlib: one in deps/zlib and
another in deps/v8/third_party/zlib. This has a couple of disadvantages:

1. There is an additional cost to keeping both dependencies up-to-date,
   and in fact they were already out-of-sync (see the refs).

2. People who compile with --shared-zlib (i.e. distro maintainers) will
   probably not be thrilled to learn that there is still a copy of zlib
   inside.

3. It's aesthetically unpleasing.

This diff (per discussion in the refs) centralizes on deps/zlib and
deletes deps/v8/third_party/zlib.

When the user requests compiling with a shared zlib build, we still need
to compile the contents of deps/zlib/google. This is not ideal but it's
also not a change in behavior: prior to this diff those files were
being compiled in the deps/v8/third_party/zlib version.

I tested this on Linux with the default build and a --shared-zlib build.
I checked that the shared-zlib build dynamically linked zlib according
to ldd, and that the regular build did not. I would appreciate if the
reviewers could suggest some other build configurations to try.

Refs: nodejs#47145
Refs: nodejs#47157
This is the second of a series of patches. This patch is contains changes
to V8 and its build system to switch to our other copy of zlib.
Previous patches in the series make it such that this directory is no
longer needed. This commit actually does the deletion.
@kvakil kvakil force-pushed the unifyZlibApproachOne branch from f3d3302 to 8dc576e Compare June 22, 2023 16:23
@kvakil
Copy link
Contributor Author

kvakil commented Jun 22, 2023

You'll need to add deps/zlib to the right include_dirs directives in tools/v8_gypfiles/v8.gyp and possibly other build files in said directory.

thanks. Fixed in the latest iteration.

If Windows weren't a concern, I'd suggest symlinking deps/v8/third_party/zlib to deps/zlib but yeah, Windows...

If we really want to avoid the V8 patch, it seems possible to do this via a gyp action to "generate" the correct directory structure (via symlink on POSIX systems and just copying on Windows systems).


Can you please make a standalone commit for the changes in deps/v8 and common.gypi? We must be able to cherry-pick it easily during V8 updates.

I did my best here, but I'm not really sure how to achieve this. If every commit needs to be independently buildable, then I think I need to make the changes to deps/v8 and tools/v8_gypfiles/v8.gyp at the same time. But I tried my best to make it smaller: the second commit in this patch series is now just the changes to V8-related files.

Also happy to give the gyp idea mentioned above a try if floating the V8 patch is harder.


Updated the current patch series. Main changes:

  • should build on windows now
  • should work with cross-compilation now

@kvakil kvakil added the request-ci Add this label to start a Jenkins CI on a PR. label Jun 22, 2023
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Jun 22, 2023
@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

Copy link
Member

@targos targos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thank you!

@targos
Copy link
Member

targos commented Jun 24, 2023

The problem is that the V8 changes break V8 CI

@nodejs-github-bot
Copy link
Collaborator

@kvakil
Copy link
Contributor Author

kvakil commented Jun 25, 2023

Hm. Perhaps we can cp -r deps/zlib deps/v8/third_party/zlib (or symlink) in tools/make-v8.sh, and that might be enough? Will give it a go tomorrow.

@nodejs-github-bot
Copy link
Collaborator

Symlink the actual directory in.
@targos
Copy link
Member

targos commented Jun 26, 2023

I suppose we also need to adapt BUILD.gn (for the include dir)

@kloczek
Copy link

kloczek commented Oct 24, 2023

It appears that this breaks the build on Windows.

So what is the problem on Windows with installing system zlib + its devel resources before start building node? 🤔

@aduh95
Copy link
Contributor

aduh95 commented May 11, 2024

This needs a rebase.

@aduh95 aduh95 added the stalled Issues and PRs that are stalled. label May 11, 2024
Copy link
Contributor

This issue/PR was marked as stalled, it will be automatically closed in 30 days. If it should remain open, please leave a comment explaining why it should remain open.

Copy link
Contributor

Closing this because it has stalled. Feel free to reopen if this issue/PR is still relevant, or to ping the collaborator who labelled it stalled if you have any questions.

@github-actions github-actions bot closed this Jun 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build Issues and PRs related to build files or the CI. needs-ci PRs that need a full CI run. stalled Issues and PRs that are stalled. v8 engine Issues and PRs related to the V8 dependency.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants