Reduce memory usage of eachMapping w/ loop #5
Conversation
Avoid creating arrays of intermediate values to pass to `aCallback`, by refactoring the .map + .forEach combination to a single loop that that creates the options object and immediately passes it to aCallback. This reduces memory usage, because engine no longer has to hold all the options objects in memory, each of the created objects is free to be gcd once the loop continues to next item. When the looped-over array `mappings` is very large (e.g. >40k items), the operation may even crash with out of memory exception without this patch.
|
After upgrading to Webpack 5, our largeish application's full production build started throwing out of memory exceptions, even after replacing After this patch, our build passes fine and takes about 20% less memory. |
|
Pinging @7rulnik because you are not in the Watch-list of this repo 🙂 |
|
@noppa hi and thanks! I will take a look into it on next week |
There was a problem hiding this comment.
@noppa sorry for such slow response :(
Great catch!
BTW webpack doesn't use source-map library anymore since https://github.com/webpack/webpack/releases/tag/v5.47.0. But it still makes sense for PostCSS for example!
Avoid creating arrays of intermediate values to pass to
aCallback, byrefactoring the .map + .forEach combination to a single loop that that
creates the options object and immediately passes it to aCallback.
This reduces memory usage, because engine no longer has to hold all the
options objects in memory, each of the created objects is free to be gcd
once the loop continues to next item.
When the looped-over array
mappingsis very large (e.g. >40k items),the operation may even crash with out of memory exception without this
patch.